Children psychological state assessment method and system

Abstract

The invention relates to the technical field of psychological state assessment, in particular to a child psychological state assessment method and system, and the method comprises the steps: collecting data: respectively establishing data sets for different age groups; physiological data, behavior data and language data of children are collected; data preprocessing: carrying out data cleaning on the collected data, and removing repeated and false data; feature extraction: performing feature extraction on the collected data by adopting a dynamic feature extraction mode, and constructing a feature vector; constructing a model: constructing a psychological state evaluation model based on a convolutional neural network, performing decision-making layer fusion based on a multi-modal fusion algorithm, and outputting an evaluation result; model training: training the model by using the data of the training set and the verification set until the performance index of the model meets a preset condition; performing psychological state evaluation by using the trained model and outputting a result; according to the invention, data processing and a machine learning algorithm are utilized to evaluate the psychological state of the child, and a more accurate evaluation result is provided.

Description

Technical Field

[0001] The present invention relates to the technical field of psychological state assessment, and in particular to a method and system for assessing the psychological state of children. Background Art

[0002] The importance of children's mental health cannot be ignored. It is related to the current growth quality of children and their future development prospects. The mental health status of children is crucial to the formation of a sound personality. A healthy mental state helps children develop positive qualities such as self-confidence, courage, and optimism. On the contrary, mental health problems may cause children to develop bad personality traits such as inferiority, loneliness, and pessimism. Mentally healthy children are more likely to concentrate, have higher learning efficiency, and develop intelligence more rapidly. Mental health problems may interfere with children's cognitive abilities, affect learning outcomes and intellectual development. Based on this, the assessment of children's mental state is particularly important. Conventional mental state assessment methods are conducted in the form of questionnaires. For children, especially young children, their ability to understand questionnaires is low. The simple questionnaire form assessment results have large errors and cannot objectively obtain children's mental state. In order to solve the above problems, the present invention proposes an assessment method using data processing and machine learning algorithms to assess children's mental state, intervene in children with mental state problems as early as possible, and promote children's physiological and mental health development. Summary of the invention

[0003] The present invention is proposed in view of the above-mentioned background problems.

[0004] Therefore, the problem to be solved by the present invention is how to use the evaluation method of data processing and machine learning algorithm to evaluate the psychological state of children, intervene in children with psychological problems as early as possible, and promote the physiological and mental health development of children.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: In a first aspect, a method for evaluating a child's mental state is provided, comprising the following specific steps: S1, data collection: establish data sets for different age groups; collect children's physiological data, behavioral data and language data; S2, data preprocessing: clean the collected data and remove duplicate and false data; S3, feature extraction: extract features from the collected data using dynamic feature extraction to construct feature vectors; S4, model building: building a psychological state assessment model based on a convolutional neural network, performing decision-making layer fusion based on a multimodal fusion algorithm, and outputting the assessment results; S5, model training: Use the data of the training set and the validation set to train the model until the performance indicators of the model meet the preset conditions; S6, use the trained model to evaluate the psychological state and output the results.

[0006] Preferably, in step S1, the user's heart rate data, sleep data, and daily behavior activity and language performance data are collected through a wearable device; Wearable devices include wristbands, smart watches, and cameras with voice recording functions; By wearing a bracelet or smart watch on the hand, the user's heart rate data can be detected in real time and uploaded to the cloud database through the built-in communication module; The user's daily behavior and language data are recorded through a camera with language recording function; the collected data is uploaded to the cloud database through the built-in communication module; The collected data are used to construct multiple data sets based on time series according to different types.

[0007] Preferably, in step S2, Python's Pandas library is used to clean the data to remove missing values ​​and duplicate data to improve the accuracy of subsequent model training.

[0008] Preferably, in step S3, a sliding window method is used to extract features from the time series data; Set the window size to w , the step length is s , then the eigenvector F The calculation is done using the following formula: ; In the formula, F is the feature vector, x(t) Represents at a point in time t The collected feature values; x(t+s) Indicates at a point in time t after s The feature values ​​collected in time units; x(t+w) Indicates at a point in time t after w The feature values ​​collected within the window range; For each time window w , calculate the following dynamic features: average value: ; variance: ; Maximum value: ; Minimum: ; Volatility: ; Rate of change: ; For each time window w Construct the feature vector: ; Combine the feature vectors of all time windows into a feature matrix; ; In the formula, n is the number of feature points in the corresponding time window; N is the number of different types of data sets; M is the number of time windows; x i Indicates i Time windows w The eigenvector of Represents the average value in the corresponding time window; x n Indicates n Time windows w The eigenvector of F NM Representative N Item in the data set M The feature vector of the time window; The feature matrix is ​​used as input features and deep learning is performed through the learning model constructed in step S4.

[0009] Preferably, step S3 further includes performing feature layer fusion on all constructed feature vectors; Taking time series as nodes, data of different modalities are fused to align feature vectors.

[0010] Preferably, in step S4, a training model of a multi-layer CNN architecture is designed, wherein the input is a multimodal feature matrix; Specifically include: Input layer, used to receive feature matrix; Convolutional layer 1, including 32 filters of size 3x3; Activate layer 1, using the ReLU function; Pooling layer 1, using 2x2 maximum pooling; Convolutional layer 2, including 64 filters of size 3x3; Activate layer 2, using the ReLU function; Pooling layer 2, using 2x2 maximum pooling; Fully connected layer: includes 128 neurons; The output layer uses Sigmoid or Softmax function to output the psychological state evaluation results; Use the weighted fusion method to fuse the feature matrices of different modes: ; In the formula, It is the multimodal feature matrix after feature layer fusion; N is the number of different types of data sets; is the weight coefficient of each mode, and the specific value needs to be verified through experiments; Representative N The feature matrix is ​​synthesized from the feature vectors of all time windows in the item dataset.

[0011] Preferably, in step S5, the Adam optimizer is used to adjust the ratio of the training set to the validation set to 9:1, and no less than 19 rounds of training are performed; after each round of training, a new validation set is randomly generated for verification; and it is ensured that the model achieves an accuracy of no less than 85% and a recall of no less than 80% on the validation set and the test set.

[0012] The second aspect of the present invention provides a child psychological state assessment system, based on the above-mentioned child psychological state assessment method, including a data acquisition module, a data preprocessing module, a feature extraction module, a recognition model and an output module; The data collection module includes a wearable device or a camera capable of recording language. The camera capable of recording language is divided into a fixed camera and a wearable portable camera. The data collection module is used to collect physiological data, behavioral data and language data of children. The data preprocessing module is used to perform preprocessing steps on the collected raw data to improve the accuracy of the data; The feature extraction module is used to perform feature extraction steps on the processed data to generate feature vectors and feature matrices; The recognition model is used to input the feature matrix and then calculate to obtain the recognition result; The output module outputs the recognition results in a visual form.

[0013] A third aspect of the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned child psychological state assessment method when executing the computer program.

[0014] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-mentioned child psychological state assessment method.

[0015] The beneficial effects of the present invention are as follows: the present invention collects a certain amount of user data; performs data preprocessing, feature extraction and other steps based on the data, uses a multimodal fusion algorithm to perform a fusion at the feature layer based on the time series, then trains through a deep learning algorithm, and performs a secondary fusion at the decision layer to output psychological state results of at least two dimensions; it is convenient for users to intervene according to the psychological state assessment results. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 Flowchart of the method for assessing children's mental status.

[0018] Figure 2 This is a structural diagram of the child mental state assessment system. DETAILED DESCRIPTION

[0019] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0020] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0021] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0022] Embodiment 1: The present invention provides a method for evaluating a child's psychological state, comprising the following specific steps: S1, data collection: establish data sets for different age groups; collect children's physiological data, behavioral data and language data; In step S1, the user's heart rate data, sleep data, daily behavior activity and language performance data are collected through the wearable device; Wearable devices include wristbands, smart watches, and cameras with voice recording functions; By wearing a bracelet or smart watch on the hand, the user's heart rate data can be detected in real time and uploaded to the cloud database through the built-in communication module; The user's daily behavior and language data are recorded through a camera with language recording function; the collected data is uploaded to the cloud database through the built-in communication module; The collected data are used to construct multiple data sets based on time series according to different types.

[0023] S2, data preprocessing: clean the collected data and remove duplicate and false data; In step S2, Python's Pandas library is used to clean the data and remove missing values ​​and duplicate data to improve the accuracy of subsequent model training.

[0024] S3, feature extraction: extract features from the collected data using dynamic feature extraction to construct feature vectors; In step S3, a sliding window method is used to extract features from the time series data; Set the window size to w , the step length is s , then the eigenvector F The calculation is done using the following formula: ; In the formula, F is the feature vector, x(t) Represents at a point in time t The collected feature values; x(t+s) Indicates at a point in time t after s The feature values ​​collected in time units; x(t+w) Indicates at a point in time t after w The feature values ​​collected within the window range; For each time window w , calculate the following dynamic features: average value: ; variance: ; Maximum value: ; Minimum: ; Volatility: ; Rate of change: ; For each time window w Construct the feature vector: ; Combine the feature vectors of all time windows into a feature matrix; ; In the formula, n is the number of feature points in the corresponding time window; N is the number of different types of data sets; M is the number of time windows; x i Indicates i Time windows w The eigenvector of Represents the average value in the corresponding time window; x n Indicates n Time windows w The eigenvector of F NM Representative N Item in the data set M The feature vector of the time window; The feature matrix is ​​used as input features and deep learning is performed through the learning model constructed in step S4.

[0025] Step S3 also includes performing feature layer fusion on all constructed feature vectors; Taking time series as nodes, data of different modalities are fused to align feature vectors.

[0026] S4, model building: building a psychological state assessment model based on a convolutional neural network, performing decision-making layer fusion based on a multimodal fusion algorithm, and outputting the assessment results; In step S4, a training model of a multi-layer CNN architecture is designed, wherein the input is a multimodal feature matrix; Specifically include: Input layer, used to receive feature matrix; Convolutional layer 1, including 32 filters of size 3x3; Activate layer 1, using the ReLU function; Pooling layer 1, using 2x2 maximum pooling; Convolutional layer 2, including 64 filters of size 3x3; Activate layer 2, using the ReLU function; Pooling layer 2, using 2x2 maximum pooling; Fully connected layer: includes 128 neurons; The output layer uses Sigmoid or Softmax function to output the psychological state evaluation results; Use the weighted fusion method to fuse the feature matrices of different modes: ; In the formula, It is the multimodal feature matrix after feature layer fusion; N is the number of different types of data sets; is the weight coefficient of each mode, and the specific value needs to be verified through experiments; Representative N The feature matrix is ​​synthesized from the feature vectors of all time windows in the item dataset.

[0027] S5, model training: Use the data of the training set and the validation set to train the model until the performance indicators of the model meet the preset conditions; in step S5, use the Adam optimizer to adjust the ratio of the training set to the validation set to 9:1, and perform no less than 19 rounds of training; after each round of training, randomly generate a new validation set for verification; ensure that the model achieves an accuracy of no less than 85% and a recall of no less than 80% on the validation set and the test set.

[0028] S6, use the trained model to evaluate the psychological state and output the results.

[0029] The following uses a specific case to introduce this solution in detail: A certain number of children of different ages were given smart watches and cameras with voice recording functions. The heart rate data and sleep data of the children within a certain period of time were obtained through smart watches, and the behavior data and language data were obtained through cameras with voice recording functions. The data were observed for 30 consecutive days and summarized. The age groups were divided into the following categories: Under 3 years old, 3-6 years old, 6-12 years old, 12-15 years old; (1) No less than 50 children are selected as samples in each age group; a separate data storage unit is set up for each child and an unchangeable identification code is configured; the user data of a specified child is accessed through the identification code, and the data storage unit is encrypted through a hash algorithm to avoid privacy leakage; in a child's data storage unit, data of different modalities are time-calibrated and divided into several time series data sets to improve the accuracy of subsequent feature recognition.

[0030] For example, heart rate data can reflect the degree of psychological changes of users in different situations. For example, when a user is nervous or anxious, the heart rate will be higher than when the user is in a resting state. Sleep data can be used to determine whether children have a good schedule based on sleep duration and the proportion of deep sleep. It is generally believed that a good schedule helps maintain children's mental health.

[0031] Behavior data and language data can be identified through pre-trained models to identify children's specific behaviors, such as pushing, fighting, insulting others, snatching, social behavior, etc. The above behaviors can be used to determine the adaptive and maladaptive behaviors of children; common ones are: Adaptive behavior is manifested in good social skills, emotional management, problem-solving skills, etc., and the ability to adapt to environmental changes.

[0032] Maladaptive behaviors manifest themselves in withdrawal, aggression, over-dependence, social avoidance, etc., which may cause difficulties for children's social interaction and learning.

[0033] The aforementioned behavior and language recognition through video and voice already has relatively mature and highly accurate algorithms. The innovation point of this application is not here, so it will not be repeated in this solution.

[0034] (2) Use Python's Pandas library to clean the collected data set and remove missing values ​​and duplicate data to improve the accuracy of subsequent model training.

[0035] (3) Using dynamic feature extraction to extract features from the preprocessed data and construct feature vectors; Take the heart rate data of a child within 30 minutes as an example, as follows: Table 1 A child's heart rate data for 30 minutes Time Heart rate (bpm) Time Heart rate (bpm) Time Heart rate (bpm) 1 80 11 85 21 85 2 82 12 89 22 87 3 85 13 90 23 88 4 88 14 92 24 89 5 90 15 91 25 90 6 87 16 90 26 91 7 84 17 88 27 92 8 82 18 86 28 93 9 81 19 84 29 94 10 83 20 83 30 95 Set the window size W to 5 and the step size S to 1 For window 1 (1-5min) Then the heart rate data: 80,82,85,88,90 average value: ; variance: ; Maximum value: 90; Minimum value: 80; Volatility: ; Rate of change: ; The corresponding eigenvector is: ; According to the above method, different types of data of each child are processed to obtain corresponding feature vectors; Then the feature vectors are fused once at the feature layer; By completing a fusion at the feature layer, we can identify the psychological changes of users when they have certain special behaviors; for example, in children's social activities, they occasionally snatch toys during play. It is generally believed that snatching phenomena have the following four different psychological states: 1. Curiosity and novelty Children are full of curiosity about the things around them, and they are eager to explore the unknown world. When they see toys in other children's hands, they may have a strong desire to explore because of the novelty and attraction of the toys, and want to snatch them to satisfy their curiosity.

[0036] 2. Self-awareness and possessiveness Children gradually develop self-awareness as they grow up. They begin to realize that they are independent individuals and have the concepts of "mine" and "yours". However, since their sense of property rights has not yet fully matured, they may regard other people's toys as their own, thus developing a strong possessive desire. This possessive desire drives them to snatch other people's toys to satisfy their own needs and desires.

[0037] 3. Social skills In the process of socializing, children may take a direct and simple approach such as snatching toys to get what they want due to lack of effective communication skills and ways of expression.

[0038] 4. Competitive mentality They may also be influenced by a competitive mentality, thinking that grabbing toys will make them stand out from the crowd and become the center of attention.

[0039] When the snatching behavior occurs, it will be accompanied by other changes in body movements, language emotions and heart rate. After the feature layer is fused, it is easier to find the correlation between different modal features based on the time series. For example, when children snatch toys out of curiosity and novelty, they may show less aggressiveness in body movements, say words such as "I want" in language, and their heart rate remains relatively stable; when children snatch toys out of self-awareness and possessiveness, they may show less aggressiveness in body movements, say words such as "this is mine" and "not yours" in language, and their heart rate rises slightly; when children snatch toys out of social skills, they may show stronger aggression in body movements, and may be relatively silent in language expression, and their heart rate rises slightly; when children snatch toys out of competitive psychology, they may show stronger aggression in body movements, and may express words such as "take it" and "not for you" in language expression, and their heart rate rises more obviously. Based on this, it is easier to find the correlation between body movements, language expression and heart rate by extracting features, which is convenient for identifying children's psychological state.

[0040] (4) Model construction: Construct a psychological state assessment model based on a convolutional neural network, perform decision-making layer fusion based on a multimodal fusion algorithm, and output the assessment results; In step S4, a training model of a multi-layer CNN architecture is designed, wherein the input is a multimodal feature matrix; Specifically include: Input layer, used to receive feature matrix; Convolutional layer 1, including 32 filters of size 3x3; Activate layer 1, using the ReLU function; Pooling layer 1, using 2x2 maximum pooling; Convolutional layer 2, including 64 filters of size 3x3; Activate layer 2, using the ReLU function; Pooling layer 2, using 2x2 maximum pooling; Fully connected layer: includes 128 neurons; The output layer uses Sigmoid or Softmax function to output the psychological state evaluation results; Use the weighted fusion method to fuse the feature matrices of different modes: ; In the formula, It is the multimodal feature matrix after feature layer fusion; Representative N The feature matrix synthesized from the feature vectors of all time windows in the item dataset; N is the number of different types of data sets; is the weight coefficient of each mode, and the specific value needs to be verified through experiments. For example, for children under 3 years old, their language expression ability is poor, and there are more unconscious behaviors in their behavior data, so the weight coefficient of the corresponding feature matrix is ​​lowered accordingly; for children aged 3-6, the weight coefficient of the corresponding feature matrix is ​​increased accordingly.

[0041] (5) Model training: Use the data from the training set and the validation set to train the model until the performance indicators of the model meet the preset conditions. In step S5, use the Adam optimizer to adjust the ratio of the training set to the validation set to 9:1, and perform no less than 19 rounds of training. After each round of training, randomly generate a new validation set for verification. Ensure that the model achieves an accuracy of no less than 85% and a recall of no less than 80% on the validation set and the test set.

[0042] (6) Use the trained model to evaluate the psychological state and output the results.

[0043] The output results can be different according to different classifiers, and users can make further choices.

[0044] 1. Classification by behavioral performance Adaptive behavior: manifested as good social skills, emotional management, problem-solving skills, etc., and the ability to adapt to environmental changes.

[0045] Maladaptive behavior: manifested as withdrawal, aggression, over-dependence, social avoidance, etc., which may cause difficulties for children's social interaction and learning.

[0046] 2. Classification by mental health status Normal mental state: Children's emotions and behaviors are within the normal range and they can adapt to life and learning.

[0047] Mild psychological problems: such as mild anxiety, inattention, etc., usually have little impact on daily life.

[0048] Moderate psychological problems: such as depression, anxiety, etc., require attention and may require intervention.

[0049] Serious psychological problems: such as self-harming behavior, severe social disorders, etc., require professional psychological treatment and support.

[0050] 3. Classification by psychological characteristics Introversion: Tends to be introspective, which may manifest as shyness, social avoidance, etc.

[0051] Extroversion: tends to be social, lively, active, and easy to build relationships with others.

[0052] Anxious type: Often feel nervous and worried, and may feel uncomfortable with new environments and social situations.

[0053] Depressive type: manifested as low mood, lack of interest and motivation, which may affect study and life.

[0054] 4. Classification by emotion regulation ability Good emotion regulation skills: Ability to effectively identify and manage one's own emotions and adapt to changes in the environment.

[0055] Poor emotion regulation ability: Difficulty in identifying and managing emotions, prone to emotional outbursts, affecting social interaction and learning.

[0056] The output results are the two closest psychological states, such as introversion-mild depression and anxiety-mild depression. According to the output results, psychological intervention or treatment can be carried out conveniently.

[0057] In addition, in this embodiment, the video recording device with voice recording function can also be replaced by a fixed video recording device; for example, a fixed camera is installed in a campus classroom to collect the behavior and language data of all students in the class in their study and life within a certain period of time; after using the above method for feature recognition, the deep learning algorithm is trained and the corresponding recognition results are output.

[0058] Embodiment 2: The second aspect of the present invention provides a child psychological state assessment system, based on the above-mentioned child psychological state assessment method, including a data acquisition module, a data preprocessing module, a feature extraction module, a recognition model and an output module; The data collection module includes a wearable device or a camera capable of recording language, including a fixed and wearable portable camera; used to collect physiological data, behavioral data and language data of children; The data preprocessing module is used to perform preprocessing steps on the collected raw data to improve the accuracy of the data; The feature extraction module is used to perform feature extraction steps on the processed data to generate feature vectors and feature matrices; The recognition model is used to input the feature matrix and then calculate to obtain the recognition result; The output module outputs the recognition results in a visual form.

[0059] Embodiment 3: This embodiment also provides a computer device suitable for the child psychological state assessment method, including a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the child psychological state assessment method proposed in the above-mentioned embodiment 1 and the child psychological state assessment system proposed in embodiment 2.

[0060] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0061] Embodiment 4: This embodiment provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the child psychological state assessment method described in Embodiment 1 are implemented.

[0062] The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination of them, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable red-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, disk or optical disk.

[0063] In summary, the present invention collects a certain amount of user data; performs data preprocessing, feature extraction and other steps based on the data, uses a multimodal fusion algorithm to perform a fusion at the feature layer based on the time series, then trains through a deep learning algorithm, and performs a second fusion at the decision layer to output psychological state results in at least two dimensions, so that users can intervene according to the psychological state assessment results.

[0064] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for assessing a child's psychological state, characterized in that: The specific steps include: S1, data collection: establish data sets for different age groups; collect children's physiological data, behavioral data and language data; S2, data preprocessing: clean the collected data and remove duplicate and false data; S3, feature extraction: extract features from the collected data using dynamic feature extraction to construct feature vectors; S4, model building: building a psychological state assessment model based on a convolutional neural network, performing decision-making layer fusion based on a multimodal fusion algorithm, and outputting the assessment results; S5, model training: Use the data of the training set and the validation set to train the model until the performance indicators of the model meet the preset conditions; S6, use the trained model to evaluate the psychological state and output the results.

2. The method for evaluating the psychological state of children according to claim 1, characterized in that: In step S1, the user's heart rate data, sleep data, daily behavior activity and language performance data are collected through the wearable device; Wearable devices include wristbands, smart watches, and cameras with voice recording functions; By wearing a bracelet or smart watch on the hand, the user's heart rate data can be detected in real time and uploaded to the cloud database through the built-in communication module; The user's daily behavior and language data are recorded through a camera with language recording function; the collected data is uploaded to the cloud database through the built-in communication module; The collected data are used to construct multiple data sets based on time series according to different types.

3. The method for evaluating the psychological state of children according to claim 1, characterized in that: In step S2, Python's Pandas library is used to clean the data and remove missing values ​​and duplicate data to improve the accuracy of subsequent model training.

4. The method for evaluating the psychological state of children according to claim 1, characterized in that: In step S3, a sliding window method is used to extract features from the time series data; Set the window size to w , the step length is s , then the eigenvector F The calculation is done using the following formula: ； In the formula, F is the feature vector, x(t) Represents at a point in time t The collected feature values; x(t+s) Indicates at a point in time t after s The feature values ​​collected in time units; x(t+w) Indicates at a point in time t after w The feature values ​​collected within the window range; For each time window w , calculate the following dynamic features: average value: ; variance: ; Maximum value: ; Minimum: ; Volatility: ; Rate of change: ; For each time window w Construct the feature vector: ; Combine the feature vectors of all time windows into a feature matrix; ； In the formula, n is the number of feature points in the corresponding time window; N is the number of different types of data sets; M is the number of time windows; x i Indicates i Time windows w The eigenvector of Represents the average value in the corresponding time window; x n Indicates n Time windows w The eigenvector of F NM Representative N Item in the data set M The feature vector of the time window; The feature matrix is ​​used as input features and deep learning is performed through the learning model constructed in step S4.

5. The method for evaluating the psychological state of children according to claim 1, characterized in that: Step S3 also includes performing feature layer fusion on all constructed feature vectors; Taking time series as nodes, data of different modalities are fused to align feature vectors.

6. The method for evaluating the child's psychological state according to claim 4, characterized in that: In step S4, a training model of a multi-layer CNN architecture is designed, wherein the input is a multimodal feature matrix; Specifically include: Input layer, used to receive feature matrix; Convolutional layer 1, including 32 filters of size 3x3; Activate layer 1, using the ReLU function; Pooling layer 1, using 2x2 maximum pooling; Convolutional layer 2, including 64 filters of size 3x3; Activate layer 2, using the ReLU function; Pooling layer 2, using 2x2 maximum pooling; Fully connected layer: includes 128 neurons; The output layer uses Sigmoid or Softmax function to output the psychological state evaluation results; Use the weighted fusion method to fuse the feature matrices of different modes: ; In the formula, It is the multimodal feature matrix after feature layer fusion; N is the number of different types of data sets; is the weight coefficient of each mode, and the specific value needs to be verified through experiments; Representative N The feature matrix is ​​synthesized from the feature vectors of all time windows in the item dataset.

7. The method for evaluating a child's psychological state according to claim 1, wherein: In step S5, the Adam optimizer is used to adjust the ratio of the training set to the validation set to 9:1, and no less than 19 rounds of training are performed; after each round of training, a new validation set is randomly generated for verification; and the model is ensured to achieve an accuracy of no less than 85% and a recall of no less than 80% on the validation set and the test set.

8. A child psychological state assessment system, based on the child psychological state assessment method according to any one of claims 1 to 7, characterized in that: It includes data acquisition module, data preprocessing module, feature extraction module, recognition model and output module; The data acquisition module includes a wearable device or a camera capable of recording speech. The camera capable of recording speech is divided into a fixed type and a wearable portable camera. Used to collect children's physiological data, behavioral data and language data; The data preprocessing module is used to perform preprocessing steps on the collected raw data to improve the accuracy of the data; The feature extraction module is used to perform feature extraction steps on the processed data to generate feature vectors and feature matrices; The recognition model is used to input the feature matrix and then calculate to obtain the recognition result; The output module outputs the recognition results in a visual form.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the child psychological state assessment method described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the child psychological state assessment method according to any one of claims 1 to 7 are implemented.

Images