A method, device and equipment for monitoring and training children with ADHD

Abstract

The present invention is applicable to the field of psychology-assisted testing and training technology, and provides a method, device and equipment for monitoring and training children with ADHD. The child ADHD monitoring and training device includes: a task module, a data acquisition module, a data processing module, a monitoring module and a training module; the task module is used for test evaluation tasks and executive function training tasks for the subject. The present invention realizes comprehensive monitoring of children with ADHD under task state by performing fusion calculation on multimodal fusion sequences, analyzes individual characteristics through multi-dimensional perception of individuals, and provides scientific and effective solutions for personalized intervention. In addition, the present invention adopts a computer-assisted system (EFTS) for executive function training, establishes an executive function feedback path, improves the executive function defects of children with ADHD, and realizes the integration of testing and training.

Description

Technical Field

[0001] The present invention belongs to the technical field of psychology-assisted testing and training, and in particular relates to a method, device and equipment for monitoring and training children with attention deficit hyperactivity disorder. Background Art

[0002] The pathological mechanism of ADHD is complex, making it difficult for clinicians to monitor ADHD patients accurately and objectively. At present, doctors mainly rely on clinical observation and rating scales for monitoring, such as DSM-5, ADHD Rating Scale for Parents (ADHDDS-P), and ADHDDS-T. However, the scale is inherently subjective and its accuracy is unreliable. Some researchers have proposed monitoring based on operational neuropsychological tests, represented by the Continuous Performance Test (CPT), which combines CPT tasks with computer software technology. By recording the correct rate, error rate, missing response rate, reaction time, and answer time of the subjects in responding to specific signals during the test, the subjects' attention deficit, hyperactivity, and impulsive components are quantified. However, the operation is difficult and requires meticulous data analysis by professional doctors. In addition, some researchers use actigraphy to monitor the activity level of the subjects. The actigraphy recorder can record the amount of limb movement during this period of time in frames of several seconds to several minutes. The test results of the actigraphy combined with CPT cannot be highly correlated with the ADHD rating scale. In addition, the wearing method and the device itself are also prone to inaccuracy and inconvenience.

[0003] The intervention of ADHD mainly includes cognitive behavioral intervention, drug intervention, neuromodulation, neurofeedback and other means, but most of them require the full participation of professional doctors. ADHD is a chronic neurodevelopmental mental disorder that requires a combination of family training and hospital training. Currently, most R&D institutions focus on the development of hospital equipment and ignore the development of family training devices.

[0004] Therefore, it is necessary to design a convenient and non-intrusive integrated device for testing and training children's ADHD. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide a monitoring and training device for children with ADHD, which can solve the deficiencies of the above-mentioned prior art and construct a convenient and non-intrusive integrated testing and training method for family monitoring and rehabilitation needs.

[0006] The embodiment of the present invention is implemented as follows: a child ADHD monitoring and training device, the child ADHD monitoring and training device comprising: a task module, a data acquisition module, a data processing module, a monitoring module and a training module;

[0007] The task module is used to establish a test evaluation task and an executive function training task for the subject; wherein the test evaluation task includes testing the subject's physical activity index, attention index, and digital evaluation scale; the executive function training task includes training the subject's physical activity index and concentration index;

[0008] The data acquisition module is used to acquire the dynamic pressure data and time-series expression data of the subject during the test evaluation task or the functional training task;

[0009] The data processing module is used to fuse the dynamic pressure data with the time-series expression data to obtain a multi-modal fusion sequence;

[0010] The monitoring module is used to input the multimodal fusion sequence and the digital assessment scale into the trained LSTM prediction model for classification, so as to obtain the ADHD quantitative assessment and classification results of the subject;

[0011] The training module is used to establish and execute a training feedback path for the subject based on the multimodal fusion sequence and its regression result in the trained LSTM prediction model.

[0012] Furthermore, the child ADHD monitoring and training device also includes: an input module, a display module;

[0013] The input module is used to receive the input of the test subject performing the test evaluation task or performing the functional training task;

[0014] The display module is used to display the test evaluation task or perform the functional training task, and cooperate with the input module to advance the test evaluation task or perform the functional training task.

[0015] Furthermore, the data processing module includes a pressure segmentation calculation unit, a time sequence expression calculation unit and a feature splicing unit.

[0016] The pressure segmentation calculation unit segments and extracts features from the dynamic pressure data to obtain an action feature sequence;

[0017] The time-series expression calculation unit pre-processes the time-series expression data to obtain a one-dimensional expression sequence, and inputs the sequence into a convolutional neural network to obtain an expression feature sequence;

[0018] The feature splicing unit obtains a multimodal fusion sequence after fusing the action feature sequence and the expression feature sequence.

[0019] Another object of the embodiment of the present invention is to provide a child ADHD monitoring and training device, the child ADHD monitoring and training device comprises a sensor device and a human-computer interaction integrated machine, the sensor device and the human-computer interaction integrated machine are communicatively connected;

[0020] The human-machine interactive integrated machine is pre-installed with the above-mentioned children's ADHD monitoring and training device;

[0021] The sensor device is used to collect the dynamic pressure data and time-series expression data of the subject, and transmit them to the data acquisition module of the children's ADHD monitoring and training device.

[0022] Further, the sensing device includes a camera, a force sensor and a test part, and two or more force sensors are arranged on the test part to collect pressure data generated by the testee on the test part to obtain dynamic pressure data;

[0023] The camera device is arranged around the test part and is directed toward the subject on the test part to collect micro-expressions of the subject to obtain time-series expression data.

[0024] Furthermore, the force sensor adopts a flexible array force sensitive sensor, and the camera device adopts a depth camera.

[0025] Furthermore, the testing section at least includes a testing table and a testing chair matching the testing table.

[0026] Furthermore, the sensing device also includes a communication module and a power supply module. The communication module can establish communication between the sensing device and the human-machine interaction integrated machine, and communication between the human-machine interaction integrated machine and a remote device. The power supply module is used to provide electrical energy of a set power.

[0027] Another object of the embodiment of the present invention is to provide a method for monitoring and training children with ADHD, the method comprising the following steps:

[0028] Establishing test evaluation tasks and executive function training tasks for the subjects; wherein the test evaluation tasks include testing the subject's physical activity index, attention index, and digital evaluation scale; the executive function training tasks include training the subject's physical activity index and concentration index;

[0029] Acquiring dynamic pressure data and time-series expression data of the subject while performing the test assessment task or the functional training task;

[0030] The dynamic pressure data and the time series expression data are fused to obtain a multimodal fusion sequence;

[0031] Inputting the multimodal fusion sequence and the digital assessment scale into the trained LSTM prediction model for classification, and obtaining the ADHD quantitative assessment and classification results of the subject;

[0032] According to the multimodal fusion sequence and its regression result in the trained LSTM prediction model, a training feedback path for the subject is established and executed.

[0033] Furthermore, the step of fusing the dynamic pressure data with the time series expression data to obtain a multimodal fusion sequence specifically includes:

[0034] Segmenting and feature extracting the dynamic pressure data to obtain an action feature sequence;

[0035] Preprocessing the time series expression data to obtain a one-dimensional expression sequence, and inputting the sequence into a convolutional neural network to obtain an expression feature sequence;

[0036] The action feature sequence and the expression feature sequence are fused to obtain a multimodal fusion sequence.

[0037] Compared with the prior art, the child ADHD monitoring and training device provided by the embodiment of the present invention obtains the child's pressure change, activity and other physical activity indicators in the task state through the set data acquisition module, and obtains the child's expression data in the task state, and then the multimodal fusion data is fused and calculated through the data processing module, combined with the monitoring module and the training module, to achieve comprehensive monitoring of ADHD children in the task state, through the multi-dimensional perception of the individual, the individual characteristics are analyzed, and a scientific and effective solution is provided for personalized intervention. In addition, the present invention uses a computer-aided system (EFTS) for executive function training, establishes a training feedback path, improves the executive function defects of children with ADHD, and realizes the integration of testing and training, which is convenient and has strong anti-interference ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A structural diagram of a child ADHD monitoring and training device provided by an embodiment of the present invention;

[0039] Figure 2 A structural block diagram of a child ADHD monitoring and training device provided by an embodiment of the present invention;

[0040] Figure 3 A flowchart of a method for monitoring and training children with ADHD provided by an embodiment of the present invention;

[0041] Figure 4 A schematic diagram of a method for monitoring and training children with ADHD provided by an embodiment of the present invention;

[0042] Figure 5A structural block diagram of another device for monitoring and training children's attention deficit hyperactivity disorder provided by an embodiment of the present invention;

[0043] Figure 6 is a structural block diagram of a data processing module in one embodiment;

[0044] Figure 7 A flowchart of monitoring and evaluating children's ADHD in one embodiment;

[0045] Figure 8 A flowchart of a training assessment for children with ADHD in one embodiment;

[0046] Fig. 9 FIG. 4 is a block diagram of the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0048] Figure 1 An application environment diagram of a child ADHD monitoring and training device provided by an embodiment of the present invention, which is also a structural diagram of a child ADHD monitoring and training device; Figure 2 This is a structural block diagram of a child ADHD monitoring and training device provided by an embodiment of the present invention. The child ADHD monitoring and training device can be Figure 1 The application environment is implemented; specifically comprising: a task module 100, a data acquisition module 200, a data processing module 300, a monitoring module 400 and a training module 500;

[0049] The task module 100 is used to establish a test evaluation task and an executive function training task for the subject; wherein the test evaluation task includes testing the subject's physical activity index, attention index, and digital evaluation scale; the executive function training task includes training the subject's physical activity index and concentration index;

[0050] The data acquisition module 200 is used to acquire the dynamic pressure data and time-series expression data of the subject during the test evaluation task or the functional training task;

[0051] The data processing module 300 is used to fuse the dynamic pressure data with the time-series expression data to obtain a multi-modal fusion sequence;

[0052] The monitoring module 400 is used to input the multimodal fusion sequence and the digital assessment scale into the trained LSTM prediction model for classification, so as to obtain the ADHD quantitative assessment and classification results of the subject;

[0053] The training module 500 is used to establish and execute a training feedback path for the subject based on the multimodal fusion sequence and its regression result in the trained LSTM prediction model.

[0054] In this embodiment, the subject information includes basic information of the subject, physical activity indicators and attention indicators. Children are subjects of ADHD. The physical activity indicators of the subject include whether to leave, pressure distribution, activity level and other indicators. The attention indicators include attention bias, expression change and other indicators. When conducting psychological tests, the gender and age of the child are recorded as the basic information of the subject. Thereafter, the ADHD child classification sample library established can be compared with conventional physical activity indicators. In the ADHD child classification sample library, the ADHD type of the child can be determined based on empirical data (or by professional physicians based on physical activity indicators (whether to leave, pressure distribution, activity level and other indicators) and attention indicators (attention bias, expression change and other indicators)). The ADHD type can include normal, attention deficit type, hyperactivity-impulsive type, mixed ADHD, etc., and then ADHD classification standards can be established. The test tool can adopt the assessment scale commonly used by ADHD patients (i.e., ADHD assessment scale), and use it after digital processing.

[0055] In one example of this embodiment, a classification sample library for children with ADHD can be established in advance. Usually, the classification sample library for children with ADHD has a typical classification representation of ADHD types. Those skilled in the art can determine the ADHD type (normal, attention deficit type, hyperactivity-impulsive type, mixed type ADHD) of the child based on the subject's physical activity indicators (whether to leave, pressure distribution, activity level, etc.) and attention indicators (attention bias, expression changes, etc.) and establish ADHD classification standards; then build an LSTM prediction model based on the ADHD classification standards, and train it to obtain a trained LSTM prediction model for the calculation of multimodal fusion sequences. When establishing a task, you can directly use the classification in the ADHD type, associate its corresponding physical activity indicators and attention indicators with the ADHD classification standards, and digitize and export them.

[0056] In an example of this embodiment, the digital assessment scale can be digitized based on the assessment scale commonly used by ADHD patients, and then developed based on the Schulte grid to implement a digital scale based on a touch screen tablet.

[0057] In an example of this embodiment, the test evaluation task can be divided into three levels of difficulty: simple mode, normal mode, and difficult mode; the simple mode is to try to press the corresponding numbers in sequence under the guidance of colors; the normal mode is not to display colors; the difficult mode is to add color interference, and the color appears on other numbers of the correct number to interfere with the answer.

[0058] In one example of this embodiment, the task module 100 may include a test evaluation task establishment unit and a function training task establishment unit;

[0059] The test assessment task establishment unit can digitize the assessment scale commonly used by ADHD patients and generate a digital scale developed based on the Schulte grid; the digital scale can be displayed on a touch tablet, a touch display or a human-computer interaction all-in-one machine, etc.

[0060] The functional training task establishment unit can design individual or combined training according to multiple indicators of executive function (planning, working memory, impulse control, inhibition, set shifting, psychological flexibility, etc.), thereby determining the alarm threshold S of the device, establishing an executive function feedback path, and providing visual, auditory and tactile feedback to achieve functional training for the subject.

[0061] It is understandable that the test evaluation task establishment unit and the functional training task establishment unit can be implemented by a computer program, and its working principle can be referred to Figure 7 , Figure 8 ; Figure 7 A flowchart of monitoring and evaluating children's ADHD in one embodiment; Figure 8 This is a flowchart of training and evaluating children with ADHD in an embodiment; users can freely choose assessment tasks and training tasks of different difficulty levels for monitoring and evaluation, and rehabilitation training; the outputs corresponding to the two units can be used as inputs for the LSTM prediction model. The LSTM prediction model can select a classification problem or a regression problem according to the specific usage scenario (monitoring / training), that is, when the monitoring function is used, the output is the ADHD child type, and when the training function is used, the output is the concentration value.

[0062] In an example of this embodiment, the training module 500 is used to establish and execute the training feedback path of the subject according to the multimodal fusion sequence and its regression results in the trained LSTM prediction model; specifically, the training feedback path of the subject can be constructed by the monitoring data of the monitoring module and the pre-determined constraints, and the constraints can be the relevant alarm thresholds, which are adjustable. In one case, when the executive function training task is established, individual or combined training is carried out according to multiple indicators of executive function (planning, working memory, impulse control, inhibition, set shifting, psychological flexibility, etc.), and professional physicians provide guidance to determine the alarm threshold S of the device; specifically, it can be the pressure perception threshold (or flexible seat alarm threshold) and the concentration alarm threshold.

[0063] This embodiment obtains physical activity indicators such as pressure changes and activity levels of children in a task state through a data acquisition module, as well as facial expression data of children in a task state, and obtains multimodal fusion data after processing; then the multimodal fusion data is fused and calculated through a data processing module, and combined with a monitoring module and a training module, a comprehensive monitoring of children with ADHD in a task state is achieved, and individual characteristics are analyzed through multi-dimensional perception of individuals, providing a scientific and effective solution for personalized intervention. In addition, a computer-aided system (EFTS) is used for executive function training, a training feedback path is established, the executive function defects of children with ADHD are improved, and the integration of testing and training is achieved, which is convenient and has strong anti-interference ability.

[0064] like Figure 5 As shown, in one embodiment, the child ADHD monitoring and training device further includes: an input module 600, a display module 700;

[0065] The input module 600 is used to receive the input of the test subject to perform the test evaluation task or perform the functional training task;

[0066] The display module 700 is used to display the test evaluation task or perform the functional training task, and cooperate with the input module to advance the test evaluation task or perform the functional training task.

[0067] In this embodiment, the input module can be a wired data transmission interface or protocol; or a wireless data transmission interface or protocol, which realizes touch input conversion transmission, keyboard input conversion transmission or audio and video input conversion transmission; it is mainly used for inputting basic information such as age and gender of the subject.

[0068] The display module may be a program or executable instruction developed based on the Schulte grid, and can display the digitized scale after execution based on a touch screen tablet to facilitate testing and training.

[0069] In an example of this embodiment, the combination of the input module and the display module can be a conventional touch screen drive unit, specifically a touch screen driver that complies with the HID standard. Through this drive unit, not only the transmission of input can be realized, but also the display of input and output can be realized. When performing test evaluation tasks or functional training tasks, a digital scale based on the Schulte grid is displayed. If input heat exchange display is required, the drive unit can perform table testing and advancement until the test or training is completed.

[0070] In an example of this embodiment, the data processing module can cooperate with the input module and the data acquisition module respectively to realize the input or acquisition of test results, dynamic pressure data and time-series expression data of test evaluation tasks or functional training tasks.

[0071] like Figure 6 As shown, in one embodiment, the data processing module 300 includes a pressure segmentation calculation unit 310, a time sequence expression calculation unit 320 and a feature splicing unit 330.

[0072] The pressure segmentation calculation unit 310 segments and extracts features from the dynamic pressure data to obtain an action feature sequence;

[0073] The time-series expression calculation unit 320 pre-processes the time-series expression data to obtain a one-dimensional expression sequence, and inputs it into a convolutional neural network to obtain an expression feature sequence;

[0074] The feature concatenation unit 330 performs a fusion process on the action feature sequence and the expression feature sequence to obtain a multi-modal fusion sequence.

[0075] In this embodiment, specifically, it is assumed that the acquisition or sampling period of the dynamic pressure data is the same as that of the time-series expression data, and the period is set to T, where T is a constant value; the time-series expression image is acquired according to the sampling period T, and the image shape is corrected by affine transformation, and the time-series expression data is obtained after digital processing, and then expanded to obtain the expression one-dimensional sequence (I1, I2, ...I n ); segment the dynamic pressure data according to the sampling period T to obtain a segmented pressure sequence (F1, F2, ... F n );

[0076] Then use the trained convolutional neural network to train the one-dimensional sequence of expressions (I1, I2, ...I n ) performs multiple convolution operations to obtain the user's facial expression feature sequence Z I =(Z I1 , Z I2 , …Z In ); for the segmented pressure sequence (F1, F2, ... Fn ) extracts basic mechanical features and obtains the action feature sequence Z F =(Z F1 , Z F2 '…Z Fn );Use feature splicing to combine the action feature sequence Z F Fusion of the expression feature sequence Z I , we get T multimodal fusion sequences, which can be expressed as a multimodal fusion vector:

[0077] Z IF ={(Z I1 , Z F1 ), (Z I2 , Z F2 ), …(Z IT , Z FT )}.

[0078] In an example of an embodiment, based on the LSTM prediction model, assuming that at the jth sampling moment, the original input is (Z Ij , Z Fj ), after modal gating, the fused input is x j , x j Satisfies the following formula (1):

[0079] x j =concate(f(W Z Z Ij +V Z Z Fj +Q Z h j-1 )[0]·Z Ij ,f(W Z Z Ij +V Z Z Fj +Q Z h j-1 )[1]·Z Fj )(1);

[0080] In formula (1): W Z ,V Z and Q Z are the expression features, action features and network parameters corresponding to the hidden layer in the fusion gated network layer of the LSTM prediction model, f is the sigmoid activation function, h is the j-1 is the hidden layer state output at sampling time j-1 in the LSTM network; the output layer of the LSTM network outputs y at sampling time j j for:

[0081] y j =f(Wx j +Qj-1 ) (2);

[0082] In formula (2), W and Q are the network parameters corresponding to the integrated input and hidden states in the output layer.

[0083] In one embodiment, the LSTM prediction model can select the classification function or regression function according to the specific usage scenario (monitoring / training). When the monitoring function is used, y j The output is the ADHD child type. When using the training function, y j The output is the concentration value.

[0084] In an example of an embodiment, the concentration value output by the LSTM prediction model is y i ;y i =(y1,y2,…y n ), compare the output concentration value with the preset alarm threshold S, when y i ≥S, it means that the user is highly focused at this moment and no feedback is needed; when y i When ≤S, it means that the user's concentration is not high at this moment, and tactile feedback, visual and auditory feedback are needed to remind the subject to concentrate; in this way, a training feedback channel for the subject is established.

[0085] like Figure 4 As shown, in an example of an embodiment, according to the specific usage scenario, before the dynamic pressure data and the time series expression data are input into the LSTM prediction model, the data processing is similar; after the dynamic pressure data is acquired through the data acquisition module, data preprocessing is performed, for example: noise reduction, normalization, segmentation, to obtain a segmented pressure sequence, and then the segmented pressure sequence is feature extracted to obtain an action feature sequence; after acquiring the time series expression data, data preprocessing is performed to obtain a one-dimensional expression sequence, the one-dimensional expression sequence is input into the convolutional neural network for multiple convolution operations to obtain an expression feature sequence, and then the action feature sequence and the expression feature sequence are feature spliced ​​to obtain a multimodal fusion sequence; the multimodal fusion sequence is input into the LSTM prediction model (that is, the time series prediction model), and according to different usage scenarios, the ADHD type (under the monitoring task state) or the concentration value (under the training task state) is output.

[0086] In one embodiment, the monitoring module 400 can perform modal gate operation on the expression feature sequence and the action feature sequence, assign different dynamic weights to the features of different modalities, realize the dynamic fusion of the expression features and the action features, input the multimodal fusion sequence into the LSTM prediction model for classification, and obtain the ADHD type of the child; the training module can evaluate the activity status of the ADHD child online according to the regression results of the LSTM prediction model, and adjust the visual, auditory, and tactile feedback in real time.

[0087] like Figure 1 As shown, in another embodiment, a child ADHD monitoring and training device, the child ADHD monitoring and training device comprises a sensor device and a human-computer interaction all-in-one machine 2, the sensor device and the human-computer interaction all-in-one machine 2 are communicatively connected;

[0088] The human-machine interactive integrated machine 2 is pre-installed with the children's ADHD monitoring and training device as described above;

[0089] The sensor device is used to collect the dynamic pressure data and time-series expression data of the subject, and transmit them to the data acquisition module of the children's ADHD monitoring and training device.

[0090] In an example of this embodiment, the child ADHD monitoring and training device may specifically include the above-mentioned task module 100, data acquisition module 200, data processing module 300, monitoring module 400 and training module 500;

[0091] The task module 100 is used to establish a test evaluation task and an executive function training task for the subject; wherein the test evaluation task includes testing the subject's physical activity index, attention index, and digital evaluation scale; the executive function training task includes training the subject's physical activity index and concentration index;

[0092] The data acquisition module 200 is used to acquire the dynamic pressure data and time-series expression data of the subject during the test evaluation task or the functional training task;

[0093] The data processing module 300 is used to fuse the dynamic pressure data with the time-series expression data to obtain a multi-modal fusion sequence;

[0094] The monitoring module 400 is used to input the multimodal fusion sequence and the digital assessment scale into the trained LSTM prediction model for classification, so as to obtain the ADHD quantitative assessment and classification results of the subject;

[0095] The training module 500 is used to establish and execute a training feedback path for the subject based on the multimodal fusion sequence and its regression result in the trained LSTM prediction model.

[0096] The sensing device includes a camera, a force sensor and a test part, and two or more force sensors are arranged on the test part to collect pressure data generated by the testee on the test part to obtain dynamic pressure data;

[0097] The camera device is arranged around the test part and is directed toward the subject on the test part to collect micro-expressions of the subject to obtain time-series expression data.

[0098] In this example, the force sensor is a flexible array force sensitive sensor 5, and the camera device is a depth camera 1.

[0099] As mentioned above, the physical activity indicators in the test evaluation task can be collected or obtained through the flexible array force sensitive sensor 5, the attention indicator can be obtained through the depth camera 1, and the digital evaluation scale is stored in the human-computer interaction all-in-one machine 2, which can be selected, modified, displayed and executed through the human-computer interaction all-in-one machine 2; the physical activity indicators and concentration indicators in the execution function training task are also collected or obtained through the flexible array force sensitive sensor 5 and the depth camera 1 respectively, and feedback is provided through the human-computer interaction all-in-one machine 2, which can be visual and sound feedback, and tactile feedback can be provided through the matching flexible array force sensitive sensor 5.

[0100] Among them, the flexible array force sensitive sensor 5 adopts 27 points / cm 2 , the sampling frequency is 100Hz, and the response time is 2ms. The camera device can also use a conventional camera lens, in conjunction with a corresponding image processing algorithm to achieve the collection of facial micro-expressions. The flexible array force-sensitive sensor 5 used can be flexibly deformed, on the one hand, it can fit with the seat body 10, and on the other hand, it can better adapt to the pressure collection of the legs, buttocks, back and waist of the subject.

[0101] In an example of this embodiment, the test section at least includes a test table 3 and a test chair 4 matching the test table 3 .

[0102] In this example, the test seat 4 includes a base 9, a detachable and liftable seat body 10 arranged on the base 9, and a box 6 is arranged on the side of the seat body 10. Some functional modules can be placed in the box 6, such as: a communication module 7 and a power supply module 8, etc.

[0103] In an example of this embodiment, the depth camera 1 and the human-computer interaction integrated machine 2 are arranged on the test table 3; the depth camera 1 is arranged above the test table 3 through a telescopic rod, and the length of the telescopic rod can be freely adjusted according to the sitting height of the child, and the human-computer interaction integrated machine 2 is placed in the middle of the desktop of the test table 3, see Figure 1 The test seat 4 limits the sitting position of the child (or the subject) to face the test table 3. The test seat 4 cooperates with the flexible array force-sensitive sensor 5 to collect the user's action data during the task, such as pressure data; the depth camera is used to collect the time-series expression data of the child during the task. The human-computer interaction all-in-one machine 2 is connected to the flexible array force-sensitive sensor 5 and the depth camera 1 respectively, and the multi-modal data is fused and calculated. After that, the ADHD of the test child is evaluated and trained through the trained LSTM prediction model.

[0104] In an example of this embodiment, the sensing device also includes a communication module 7 and a power supply module 8. The communication module 7 can establish communication between the sensing device and the human-computer interaction integrated machine 2, and communication between the human-computer interaction integrated machine 2 and a remote device. The power supply module 8 is used to provide electrical energy of a set power.

[0105] Among them, the power supply module 8 adopts a lithium-ion battery with a nominal voltage of 3.7v; the communication module 7 includes a WIFI module, an Ethernet port, a serial port and a USB port. The WIFI module and the Ethernet port are used for data communication in a local area network or the Internet, and the serial port and the USB port communicate data with the human-computer interaction all-in-one machine through data acquisition and transmission components; the remote device can be a cloud, a mobile device, a mobile base station, etc., to realize the interconnection of all things and increase the user experience of this device.

[0106] In an example of this embodiment, the camera device is variably arranged in front of the test part, and can be specifically fixed by one or more telescopic rods, and the camera device can be variably arranged by the one or more telescopic rods.

[0107] like Figure 3 As shown, in another embodiment, a method for monitoring and training children with ADHD includes the following steps S101-S109:

[0108] S101, establishing a test evaluation task and an executive function training task for the subject; wherein the test evaluation task includes testing the subject's physical activity index, attention index, and digital evaluation scale; the executive function training task includes training the subject's physical activity index and concentration index;

[0109] S103, obtaining dynamic pressure data and time-series expression data of the subject during the process of performing the test evaluation task or the functional training task;

[0110] S105, fusing the dynamic pressure data with the time series expression data to obtain a multimodal fusion sequence;

[0111] In this embodiment, the step of fusing the dynamic pressure data with the time-series expression data to obtain a multimodal fusion sequence specifically includes:

[0112] Segmenting and feature extracting the dynamic pressure data to obtain an action feature sequence;

[0113] Preprocessing the time series expression data to obtain a one-dimensional expression sequence, and inputting the sequence into a convolutional neural network to obtain an expression feature sequence;

[0114] The action feature sequence and the expression feature sequence are fused to obtain a multimodal fusion sequence.

[0115] S107, inputting the multimodal fusion sequence and the digital assessment scale into the trained LSTM prediction model for classification, and obtaining the ADHD quantitative assessment and classification results of the subject;

[0116] S109: Establish and execute a training feedback path for the subject based on the multimodal fusion sequence and its regression result in the trained LSTM prediction model.

[0117] In an example of an embodiment, the concentration value output by the LSTM prediction model is y i ;y i =(y1,y2,…y n ), compare the output concentration value with the preset alarm threshold S, when y i ≥S, it means that the user is highly focused at this moment and no feedback is needed; when y i When ≤S, it means that the user's concentration is not high at this moment, and tactile feedback, visual and auditory feedback are needed to remind the subject to concentrate; in this way, a training feedback channel for the subject is established.

[0118] In one example of an embodiment, Figure 7 A flowchart of monitoring and evaluating children's ADHD in one embodiment; Figure 8 A flowchart of training assessment for children with ADHD in one embodiment.

[0119] In the first case of this example, according to actual usage, the test assessment task can be divided into three levels of difficulty: simple mode, normal mode and difficult mode; the simple mode is to try to press the corresponding numbers in sequence under the guidance of colors; the normal mode is not to display colors; the difficult mode is to add color interference, and the color appears on other numbers of the correct number to interfere with the test answer.

[0120] The test and evaluation tasks are mainly implemented through the above-mentioned children's ADHD monitoring and training equipment. The specific usage process is as follows:

[0121] Turn on the power and wait for the device to be initialized. The depth camera 1, human-machine interaction integrated machine 2, flexible array force sensor 5 and other devices will enter the working state.

[0122] After the user logs in to the human-computer interaction integrated machine 2, he or she will understand the assessment process and answering rules according to the voice broadcast and demonstration pictures, and select the difficulty of the test assessment task;

[0123] Before answering the questions, the flexible pressure sensing of the flexible array force-sensitive sensor 5 and the micro-expression calculation based on the depth camera 1 are synchronously started;

[0124] The user answers the question by pointing out the positions of the 1 to 25 Arabic numerals in the grid displayed by the human-computer interaction integrated machine 2 in the order of 1 to 25 with his fingers;

[0125] The human-machine interactive integrated machine 2 performs quantitative evaluation and classification of the user through fusion calculation based on the task results, various physical activity indicators during the task (obtained through the flexible array force-sensitive sensor 5) and attention indicators (obtained through the depth camera 1);

[0126] After the evaluation is completed, the human-computer interaction integrated machine 2 will automatically save the evaluation results, and the evaluation results will be presented to the user in the form of a report to provide a basis for personalized intervention.

[0127] In the second case of this example, according to the conventional ADHD classification, ADHD types can be divided into: normal, attention deficit type, hyperactive-impulsive type, and mixed type. When conducting rehabilitation training through functional training tasks, the specific usage process is as follows:

[0128] Turn on the power and wait for the device to initialize;

[0129] After the user logs in to the human-computer interaction integrated machine 2, he or she will learn the training process and answering rules according to the voice broadcast and demonstration pictures, and select the training task and training difficulty;

[0130] Before answering the questions, the flexible pressure sensing of the flexible array force-sensitive sensor 5 and the micro-expression calculation based on the depth camera 1 are synchronously started;

[0131] The user answers the question by pointing out the positions of the 1 to 25 Arabic numerals in the grid displayed by the human-computer interaction integrated machine 2 in the order of 1 to 25 with his / her finger;

[0132] When the user's physical activity index exceeds the set stress alarm threshold, or the concentration index exceeds the set concentration alarm threshold, the user will be prompted through visual, auditory and tactile feedback to correct his posture and concentrate. Long-term training can improve the executive function defects of children with ADHA.

[0133] After the training is completed, the human-computer interaction integrated machine 2 will automatically save the evaluation results or record the training records. The human-computer interaction integrated machine 2 will give a training evaluation of this training based on the alarm frequency, various physical activity indicators and attention indicators during the task execution process, and the training evaluation will be presented to the user in the form of a report.

[0134] Among them, the user's stress alarm threshold and concentration alarm threshold can be determined under the guidance of a professional physician based on multiple indicators of executive function (planning, working memory, impulse control, inhibition, set shifting, psychological flexibility, etc.), thereby establishing an executive function feedback pathway (i.e., the subject's training feedback pathway) and providing visual, auditory and tactile feedback.

[0135] In one embodiment, for a trained LSTM prediction model, during the process of training the LSTM prediction model, an optimizer can be used to adaptively adjust relevant parameters of the LSTM prediction model, such as adaptively adjusting parameters of the loss function and activation function, so as to complete the training of the LSTM prediction model faster and more accurately.

[0136] It is to be understood that the optimizer includes but is not limited to stochastic gradient descent optimizer (SGD), adaptive gradient optimizer (AdaGrad), exponentially weighted moving average optimizer (RMSProp). In a preferred embodiment, the optimizer used is stochastic gradient descent optimizer (SGD).

[0137] like Fig. 9 As shown, in another embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the above-mentioned child ADHD monitoring and training method:

[0138] S101, establishing a test evaluation task and an executive function training task for the subject; wherein the test evaluation task includes testing the subject's physical activity index, attention index, and digital evaluation scale; the executive function training task includes training the subject's physical activity index and concentration index;

[0139] S103, obtaining dynamic pressure data and time-series expression data of the subject during the process of performing the test evaluation task or the functional training task;

[0140] S105, fusing the dynamic pressure data with the time series expression data to obtain a multimodal fusion sequence;

[0141] S107, inputting the multimodal fusion sequence and the digital assessment scale into the trained LSTM prediction model for classification, and obtaining the ADHD quantitative assessment and classification results of the subject;

[0142] S109: Establish and execute a training feedback path for the subject based on the multimodal fusion sequence and its regression result in the trained LSTM prediction model.

[0143] The embodiment provides a monitoring and training device for children with ADHD. The data acquisition module is set to obtain physical activity indicators such as pressure changes and activity levels in the child's task state, and the expression data of the child in the task state. Then, the multimodal fusion data is fused and calculated by the data processing module. The monitoring module and the training module are combined to realize comprehensive monitoring of children with ADHD in the task state. Through the multi-dimensional perception of the individual, the individual characteristics are analyzed to provide a scientific and effective solution for personalized intervention. In addition, the present invention uses a human-computer interaction all-in-one machine 2, namely a computer-aided system (EFTS), to perform executive function training, establish a training feedback path, improve the executive function defects of children with ADHD, realize the integration of testing and training, and is convenient and has strong anti-interference ability.

[0144] Fig. 9 The internal structure diagram of a computer device in an embodiment is shown. The computer device includes a processor, a memory, a network interface, an input device and a display screen connected through a system bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement the child ADHD monitoring and training method. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can execute the child ADHD monitoring and training method. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0145] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0146] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0147] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A child ADHD monitoring and training device, characterized in that: The children's ADHD monitoring and training device comprises: a task module, a data acquisition module, a data processing module, a monitoring module and a training module; The task module is used to establish a test evaluation task and an executive function training task for the subject; wherein the test evaluation task includes testing the subject's physical activity index, attention index, and digital evaluation scale; the executive function training task includes training the subject's physical activity index and concentration index; The data acquisition module is used to acquire the dynamic pressure data and time-series expression data of the subject during the test evaluation task or the functional training task; The data processing module is used to fuse the dynamic pressure data with the time-series expression data to obtain a multi-modal fusion sequence; The monitoring module is used to input the multimodal fusion sequence and the digital assessment scale into the trained LSTM prediction model for classification, so as to obtain the ADHD quantitative assessment and classification results of the subject; The training module is used to establish and execute a training feedback path for the subject based on the multimodal fusion sequence and its regression result in the trained LSTM prediction model.

2. The children's ADHD monitoring and training device according to claim 1, characterized in that: The child ADHD monitoring and training device also includes: an input module and a display module; The input module is used to receive the input of the test subject performing the test evaluation task or performing the functional training task; The display module is used to display the test evaluation task or perform the functional training task, and cooperate with the input module to advance the test evaluation task or perform the functional training task.

3. The children's ADHD monitoring and training device according to claim 1, characterized in that: The data processing module includes a pressure segmentation calculation unit, a time sequence expression calculation unit and a feature splicing unit. The pressure segmentation calculation unit segments and extracts features from the dynamic pressure data to obtain an action feature sequence; The time-series expression calculation unit pre-processes the time-series expression data to obtain a one-dimensional expression sequence, and inputs the sequence into a convolutional neural network to obtain an expression feature sequence; The feature splicing unit obtains a multimodal fusion sequence after fusing the action feature sequence and the expression feature sequence.

4. A child ADHD monitoring and training device, characterized in that: The children's ADHD monitoring and training device comprises a sensor device and a human-machine interactive integrated machine, wherein the sensor device is communicatively connected to the human-machine interactive integrated machine; The human-computer interaction integrated machine is pre-installed with the child ADHD monitoring and training device as described in any one of claims 1 to 3; The sensor device is used to collect the dynamic pressure data and time-series expression data of the subject, and transmit them to the data acquisition module of the children's ADHD monitoring and training device.

5. The children's ADHD monitoring and training device according to claim 4, characterized in that: The sensing device includes a camera, a force sensor and a test part, wherein two or more force sensors are arranged on the test part to collect pressure data generated by the testee on the test part to obtain dynamic pressure data; The camera device is arranged around the test part and is directed toward the subject on the test part to collect micro-expressions of the subject to obtain time-series expression data.

6. The children's ADHD monitoring and training device according to claim 5, characterized in that: The force sensor adopts a flexible array force sensitive sensor, and the camera device adopts a depth camera.

7. The children's ADHD monitoring and training device according to claim 5, characterized in that: The testing section at least includes a testing table and a testing chair matching the testing table.

8. The children's ADHD monitoring and training device according to claim 5, characterized in that: The sensing device further includes a communication module and a power supply module. The communication module can establish communication between the sensing device and the human-machine interactive integrated machine, and communication between the human-machine interactive integrated machine and a remote device. The power supply module is used to provide electrical energy of a set power.

9. A method for monitoring and training children with ADHD, characterized in that: The children's ADHD monitoring and training method comprises the following steps: Establishing test evaluation tasks and executive function training tasks for the subjects; wherein the test evaluation tasks include testing the subject's physical activity index, attention index, and digital evaluation scale; the executive function training tasks include training the subject's physical activity index and concentration index; Acquiring dynamic pressure data and time-series expression data of the subject while performing the test assessment task or the functional training task; The dynamic pressure data and the time series expression data are fused to obtain a multimodal fusion sequence; Inputting the multimodal fusion sequence and the digital assessment scale into the trained LSTM prediction model for classification, and obtaining the ADHD quantitative assessment and classification results of the subject; According to the multimodal fusion sequence and its regression result in the trained LSTM prediction model, a training feedback path for the subject is established and executed.

10. The method for monitoring and training children with ADHD according to claim 9, characterized in that: The step of fusing the dynamic pressure data with the time-series expression data to obtain a multimodal fusion sequence specifically includes: Segmenting and feature extracting the dynamic pressure data to obtain an action feature sequence; Preprocessing the time series expression data to obtain a one-dimensional expression sequence, and inputting the sequence into a convolutional neural network to obtain an expression feature sequence; The action feature sequence and the expression feature sequence are fused to obtain a multimodal fusion sequence.

Images