Autism children emotion recognition and regulation method and system based on wearable device

Abstract

The application belongs to the field of autism children emotion regulation, and provides a method and system for autism children emotion recognition and regulation based on wearable devices, and the process is as follows: after the physiological data and facial expressions of autism children are input into a mixed multi-layer neural network after data processing, emotion recognition results are obtained; if negative emotions are recognized, the inflatable vest performs inflation action to relieve the bad mood of the children. The application uses multi-modal data to identify the emotions of autistic children, which is conducive to fully mining the real emotional characteristics of the children and improving the emotion recognition accuracy. At the same time, the application uses the sensitivity of physiological signals to emotional fluctuations to realize the functions of emotion warning and early intervention. In addition, the application systematically integrates various wearable devices, which can continuously, intelligently and effectively regulate the emotions of autism children.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of emotional regulation of autistic children, and particularly relates to the fields of wearable smart devices, deep learning algorithm application, physiological signal processing, deep pressure therapy (DPT), and the like, and in particular to an autistic child emotion recognition and regulation method and system based on a wearable device. TECHNICAL BACKGROUND

[0002] At the present stage, autistic children are often discovered when they have negative emotions through observation of the external manifestations of the patient's body and expression. This delay in emotional perception greatly hinders timely emotional intervention and seriously affects the rehabilitation process of the patient. Current research shows that physiological signals (such as skin electrical response and heart rate) can be identified at the initial stage of emotional fluctuation, and are more prospective in pre-emotion prediction.

[0003] At the same time, traditional intervention methods (such as support and accompaniment) usually intervene after negative emotions appear, and cannot achieve early intervention. In addition, although deep pressure therapy (DPT) based deep pressure inflatable vests have been proven to relieve autistic children's anxiety, anger and other negative emotions, manually inflated pressure vests lack intelligence and ease of use.

[0004] Therefore, based on the above problems, there is an urgent need for a more intelligent and more accurate autistic child emotion recognition and regulation method and system that can meet the special needs of autistic children and perform more timely and effective emotional monitoring and regulation. SUMMARY

[0005] The present application provides an autistic child emotion recognition and regulation method and system based on a wearable device to solve the problems and deficiencies of the prior art.

[0006] The present application solves the above technical problems by the following technical solutions:

[0007] The present application provides an autistic child emotion recognition and regulation method based on a wearable device, which includes the following steps:

[0008] Step S1, real-time acquisition of physiological signals of the current user within a certain time period, including skin electrical signals, skin temperature signals, heart rate signals, and blood volume pulse signals, and real-time acquisition of facial expression images of the current user within the corresponding time period;

[0009] Step S2, noise reduction filtering preprocessing operation on the acquired physiological signals, and preprocessing operation on the acquired facial expression images;

[0010] Step S3, the pre-processed physiological signal data and facial expression image data are input into a hybrid multi-layer neural network for feature extraction, emotion recognition classification, and output of emotion classification results;

[0011] Step S4, whether the emotion classification result in step S3 is a negative emotion is judged, if the emotion classification is a negative emotion, step S5 is performed, if the emotion classification is not a negative emotion, step S6 is performed;

[0012] Step S5, whether the deep pressure inflatable vest is in an inflated state is judged, if the deep pressure inflatable vest is in an inflated state, step S8 is performed, if the deep pressure inflatable vest is not in an inflated state, step S7 is performed;

[0013] Step S6, whether the deep pressure inflatable vest is in an inflated state is judged, if the deep pressure inflatable vest is in an inflated state, step S9 is performed, if the deep pressure inflatable vest is not in an inflated state, the whole method process is repeated and executed from step S1, and the emotion state is continuously acquired and classified;

[0014] Step S7, the deep pressure inflatable vest performs an inflation action, and the whole method process is repeated and executed from step S1, and the emotion state is continuously acquired and classified until the emotion reaches a normal index, and the deep pressure inflatable vest is enabled to deflate;

[0015] Step S8, the inflation state of the deep pressure inflatable vest is continuously maintained, the whole method process is repeated and executed from step S1, and the emotion state is continuously acquired and classified until the emotion reaches a normal index, and the deep pressure inflatable vest is enabled to deflate;

[0016] Step S9, the deep pressure inflatable vest performs a deflation action, and the whole method process is repeated and executed from step S1, and the emotion state is continuously acquired and classified.

[0017] The application also provides a system for autism child emotion recognition and adjustment based on a wearable device, which comprises an acquisition module, a data analysis module, a wireless communication module and an adjustment module, the data analysis module is connected with the wireless communication module, the acquisition module is connected with the data analysis module and the wireless communication module respectively, the adjustment module is connected with the wireless communication module, and the wireless communication module is connected with a terminal;

[0018] The acquisition module is used for acquiring data reflecting the emotion state of a user in a certain time period in real time.

[0019] The data analysis module is used for pre-processing and analyzing the data collected by the acquisition module, realizing emotion category judgment and output.

[0020] A wireless communication module, including one or more of a network controller, a Bluetooth transceiver, a WiFi transceiver, and a mobile network transceiver, is used to transmit the collected data of the acquisition module to the data analysis module, and transmit the action instructions generated by the data analysis results to the adjustment module;

[0021] The adjustment module is used to provide timely emotional regulation measures when the data analysis module analyzes that the autistic child produces negative emotions. The adjustment module is triggered by the results of the data analysis module. When the data analysis module identifies and judges the potential negative emotional state, it activates or maintains the operation of the adjustment module. When the data analysis module identifies and judges the non-potential negative emotional state, it terminates the operation of the adjustment module.

[0022] The terminal is used to receive the data analysis results and the adjustment module action state from the wearable device, and provides an intuitive user interface to enable users to understand the received information.

[0023] Further, in order to ensure the collection effect of the data, the acquisition module includes a facial collection module and a physiological signal collection module, the facial collection module is connected with the data analysis module, and the physiological signal collection module is connected with the data analysis module;

[0024] The facial collection module is used to collect facial expression image data of the user;

[0025] The physiological signal collection module is used to collect physiological signal data of the user.

[0026] Further, in order to ensure the collection effect of the facial expression data, a high-definition camera is used in the facial collection module, the high-definition camera is connected with the data analysis module, and is used to acquire facial expression images of the user in real time.

[0027] Further, in order to ensure the collection effect of the physiological signal data, a health monitoring smart bracelet is used in the physiological signal collection module, and the health monitoring smart bracelet is connected with the data analysis module;

[0028] The health monitoring smart bracelet is used to monitor and acquire four physiological signals of skin electricity, skin temperature, heart rate and blood volume pulse of the user in real time.

[0029] Further, the data analysis module includes a data processing unit, a hybrid multi-layer neural network, and a command and dispatch unit;

[0030] The data processing unit is used to acquire time-frequency domain analysis features corresponding to the four physiological signals of skin electricity, skin temperature, heart rate and blood volume pulse, and to transform the resolution and depth of the collected facial expression images;

[0031] The mixed multi-layer neural network is used for feature extraction and emotion recognition on the pretreated data, and emotion classification judgment results are obtained, which can guarantee the processing effect of the data.

[0032] The command and dispatch unit is used for providing data or instructions for command, dispatch and coordination for the networked wearable devices and terminals.

[0033] Further, in order to guarantee the data transmission effect, a wireless Bluetooth transceiver, a WiFi transceiver and a mobile network transceiver are used in the wireless communication module for data communication and transmission.

[0034] Further, in order to guarantee the effect of negative emotion regulation, a deep pressure inflatable vest is used in the regulation module.

[0035] The deep pressure inflatable vest is a wearable vest based on deep pressure therapy (DPT) to relieve negative emotions such as anxiety. In the system, it is used to receive the inflation and deflation instructions of the data analysis module and execute or terminate the process of user emotion regulation.

[0036] Further, the data processing unit is used for low-pass filtering and denoising of the four physiological signals of skin electricity, skin temperature, heart rate and blood volume pulse input by the acquisition module, and processing and obtaining the related time-frequency domain analysis features, while converting the resolution and depth of the facial expression image.

[0037] Further, the mixed multi-layer neural network can guarantee the effect of emotion recognition. It uses LSTM to extract features of the time-frequency analysis features of the four physiological signals after data preprocessing, uses VGG-16 to extract features of the pretreated facial expression image, then uses convolution operation to transform the two parts of feature vectors to obtain fusion features, and finally uses attention mechanism to analyze and capture the relationship between the features, and finally outputs the emotion classification judgment results through the full connection layer.

[0038] Compared with the prior art, the present application has the following advantages:

[0039] Through the mutual cooperation of the acquisition module and the data analysis module, and the mutual cooperation of the two modules and the adjustment module, the wireless communication module and the terminal, real-time emotional monitoring and adjustment of the autistic child patient can be realized. The acquisition module can accurately capture the facial behavior and physiological state of the patient, and the data analysis module classifies the collected data to identify the possibility of the patient being in a negative emotional state. Once the system identifies a negative emotional state, the adjustment module cooperates with the wireless communication module and the terminal to take emotional adjustment measures in advance and in a timely manner through the deep pressure inflatable vest. This mutual cooperation monitoring mechanism not only improves the accuracy and efficiency of monitoring, but also provides more timely and effective emotional intervention support for the patient, which helps to better manage the emotional state and prevent emotional breakdown. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings described below are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0041] Figure 1 is a flow chart of a method for identifying and adjusting the emotions of autistic children based on a wearable device according to an embodiment of the present application.

[0042] Figure 2 is a structural block diagram of a system for identifying and adjusting the emotions of autistic children based on a wearable device according to an embodiment of the present application.

[0043] Figure 3 is a structural diagram of a mixed multi-layer neural network according to an embodiment of the present application.

[0044] Figure 4 is a structural schematic diagram of a deep pressure inflatable vest according to an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0046] As shown in Figure 1 , the present embodiment provides a method for identifying and adjusting the emotions of autistic children based on a wearable device, which comprises the following steps:

[0047] Step S1, real-time acquisition of physiological signals of the current user within a certain time period, including skin electrical signals, skin temperature signals, heart rate signals, blood volume pulse signals, and real-time acquisition of facial expression images of the current user within a certain time period.

[0048] The "certain time period" cannot be too short or too long. If the time is set too short, it will have a potential impact on the accuracy of emotion recognition, and if the time is set too long, it may lead to negative emotions that cannot be timely adjusted and intervened. Generally, it is limited within t seconds, 30 < t < 60, and the embodiment is set to 30 seconds.

[0049] Step S2, performing noise reduction filtering preprocessing operation on the acquired physiological signals, and performing preprocessing operation on the acquired facial expression images.

[0050] In the embodiment, the acquired skin electricity, skin temperature, heart rate and blood volume pulse four physiological signals are subjected to low-pass filtering denoising. The skin electricity signal low-pass filtering range is [0, 0.08] Hz, the skin temperature low-pass filtering range is [0, 0.2] Hz, the heart rate low-pass filtering range is [0, 0.4] Hz, and the blood volume pulse low-pass filtering range is [0, 15] Hz. The time domain analysis features of the skin electricity signal include the mean, standard deviation, median, maximum value and ratio thereof, the mean, standard deviation, median, maximum value and ratio thereof after first-order difference of the skin electricity signal, the mean, standard deviation, median, maximum value and ratio thereof after second-order difference of the skin electricity signal, the frequency domain analysis features of the skin electricity signal include the spectral power in the frequency band [0, 0.08] Hz, the time domain analysis features of the skin temperature signal include the mean, standard deviation, median, maximum value and ratio thereof, the frequency domain analysis features of the skin temperature signal include the spectral power in the frequency band [0, 0.2] Hz, the time domain analysis features of the heart rate signal include the mean, standard deviation, heart rate variability HRV mean, heart rate variability HRV standard deviation, the frequency domain analysis features of the heart rate signal include the power spectral density in the frequency band [0, 0.4] Hz, the time domain analysis features of the blood volume pulse signal include the mean, standard deviation, median, maximum value and ratio thereof, the frequency domain analysis features of the blood volume pulse signal include the power spectral density in the frequency band [0, 0.15] Hz, and the acquired facial expression image is converted to a resolution of [224x224] and the image depth is converted from 24 bits to 8 bits.

[0051] Step S3, inputting the preprocessed physiological signal data and facial expression data into a hybrid multi-layer neural network for feature extraction, emotion recognition classification and outputting emotion classification judgment results.

[0052] The structure of the hybrid multi-layer neural network used in the embodiment is as follows: Figure 3As shown, wherein the LSTM (Long Short-Term Memory) neural network is applied to process the physiological signal data changing over time, the time-frequency analysis features of the four physiological signals are extracted through the LSTM neural network, at the same time, the VGG-16 convolutional neural network is used to process the preprocessed facial expression image layer by layer, and the high-dimensional features related to the emotion are extracted, then the MCB (Multimodal Compact Bilinear Pooling) is used to fuse the LSTM output from the physiological signal and the VGG-16 output from the facial expression image, so as to obtain the joint feature vector representing two different modalities, then the fused features are given different feature weights through the attention mechanism module to enhance the attention to the key features, finally the output features are mapped to the probability distribution of different emotion categories through a fully connected layer of the LSTM neural network and a Softmax activation function, and finally the emotion category with the highest probability is taken as the recognition and classification result. The emotion classification result is set as positive emotion (happy), neutral emotion (calm) and negative emotion (angry, anxious and sad).

[0053] Step S4, judge whether the emotion classification result in step S3 is negative emotion, if "yes", proceed to step S5, if "no", proceed to step S6.

[0054] Step S5, under the precondition that the emotion classification result in step S4 is "yes", judge whether the deep pressure inflatable vest is in the inflated state, if "yes", proceed to step S8, if "no", proceed to step S7.

[0055] Step S6, under the precondition that the emotion classification result in step S4 is "no", judge whether the deep pressure inflatable vest is in the inflated state, if "yes", proceed to step S9, if "no", return to step S1 to repeat the whole method process and continue to acquire and classify the emotional state;

[0056] Step S7, under the precondition that the emotion classification result in step S4 is "yes" and the inflation state of the deep pressure inflatable vest in step S5 is "no", the deep pressure inflatable vest performs the inflation action, and returns to step S1 to repeat the whole method process and continue to acquire and classify the emotional state until the deep pressure inflatable vest is enabled to deflate when the emotion reaches the normal index.

[0057] If the emotion classification result is one of negative emotions (angry, anxious and sad) and the deep pressure inflatable vest is in the non-inflated state, the deep pressure inflatable vest will produce the inflation action.

[0058] The structure of the deep pressure inflatable vest used in the embodiment is as followsFigure 4 As shown, a pump release valve device is used for the inflation and deflation operations.

[0059] Step S8: Under the premise that the emotion classification judgment result in step S4 is "yes" and the inflation status of the deep pressure inflatable vest in step S5 is "yes", continue to maintain the inflation status of the deep pressure inflatable vest, and return to step S1 to repeat the entire method process, continue to acquire and classify the emotion status, until the emotion reaches the normal index, then enable the deep pressure inflatable vest to deflate.

[0060] This embodiment takes into account that if the user's emotion classification judgment result is negative (anger, anxiety, sadness) in two consecutive time periods, that is, the deep pressure inflatable vest has been triggered to inflate in the previous time, the deep pressure inflatable vest will remain inflated in the next time.

[0061] In step S9, under the premise that the emotion classification judgment result in step S4 is "no" and the inflation status of the deep pressure inflatable vest in step S6 is "yes", the deep pressure inflatable vest will deflate and return to step S1 to repeat the entire method process to continue to acquire and classify the emotion state.

[0062] In this embodiment, if the user's emotion classification judgment results are negative emotions (anger, anxiety, sadness) and positive emotions (happiness) or neutral emotions (calm) in a continuous period of time, that is, the deep pressure inflatable vest has been triggered to inflate in the previous time, then the emotion regulation state will be deactivated in the next time, and the deep pressure inflatable vest will perform an air deflation action.

[0063] like Figure 2 As shown, this embodiment also provides a system for emotion recognition and regulation of autistic children based on wearable devices, which includes an acquisition module, a data analysis module, a wireless communication module, and an regulation module. The data analysis module is connected to the wireless communication module, the acquisition module is connected to both the data analysis module and the wireless communication module, the regulation module is connected to the wireless communication module, and the wireless communication module is connected to a terminal.

[0064] The acquisition module is used to acquire data reflecting the user's emotional state in real time over a certain period of time.

[0065] The data analysis module is used to analyze various types of data collected by the acquisition module to determine the emotion category.

[0066] The wireless communication module includes one or more of a network controller, a Bluetooth transceiver, a WiFi transceiver, and a mobile network transceiver, and is used to transmit the data collected by the acquisition module to the data analysis module, and to transmit the action commands generated by the data analysis results to the adjustment module.

[0067] Among them, one or more of the network controller, Bluetooth transceiver, WiFi transceiver and mobile network transceiver are determined by the use of the device, the high-definition camera and the health monitoring smart bracelet of the embodiment adopt the 5G networking module and the Bluetooth 5.0 module of the device, and the deep pressure inflatable vest adopts the Bluetooth 5.0 module to drive the instruction action.

[0068] The WiFi transceiver realizes wireless networking in a short distance, and the embodiment adopts a 1000M WiFi wireless transceiver.

[0069] The adjusting module is configured to provide timely emotional adjustment measures when the data analysis module analyzes that the autistic child produces negative emotions.

[0070] The terminal is configured to receive the data analysis result from the wearable device and the action state of the adjusting module, and provide an intuitive user interface to enable the user to understand the received information.

[0071] Specifically, the specific process of each module in the system of the embodiment and the implementation of the functions of the modules can be referred to the related description in the method embodiment, which will not be described here.

[0072] The embodiment also provides a computer readable storage medium, which stores computer instructions, and the computer instructions are used to make a computer execute the method for autistic child emotion recognition and adjustment based on a wearable device.

[0073] Scenario description:

[0074] For example, a child with autism has recently been emotionally unstable and often exhibits negative emotions such as irritability and restlessness. Doctors recommend using our wearable device to monitor his emotional state so that negative emotions can be adjusted in a timely manner.

[0075] Feature collection:

[0076] Facial expression collection: use the high-definition camera module of the wearable device. Capture the user's micro-expression changes, for example, when the user feels angry, anxious or sad, his face may show slight changes in expression. These data are analyzed by VGG-16 neural network to assist in determining the emotional state.

[0077] Physiological signal data collection: use the health monitoring smart bracelet to continuously monitor the user's physiological indicators to collect data about skin electrical signals, skin temperature signals, heart rate signals, and blood volume pulse signals in real time. These data can reflect the user's physiological and emotional reactions in advance.

[0078] Emotion recognition and adjustment:

[0079] All collected data will be transmitted to a data analysis module, which includes data processing unit, command and dispatch unit, etc. These modules work together to comprehensively analyze the data to determine whether the user is in a potentially negative emotional state.

[0080] If the system determines that the user is in a negative emotional state, the adjustment module will be triggered. This module uses a deep pressure inflatable vest to make the user feel hugged by applying pressure, i.e. deep pressure therapy (DPT), so as to calm the negative emotions.

[0081] Although the specific embodiments of the present application are described above, those skilled in the art should understand that these are only illustrative, and the protection scope of the present application is defined by the appended claims. Those skilled in the art can make various changes or modifications to these embodiments without departing from the principles and essence of the present application, and such changes and modifications fall within the protection scope of the present application.

Claims

1. A method for autism children emotion recognition and regulation based on wearable device, characterized in that, It comprises the following steps: Step S1, real-time acquisition of physiological signals of the current user in a certain time period, including skin electricity signal, skin temperature signal, heart rate signal, blood volume pulse signal, while real-time acquisition of facial expression images of the current user in the corresponding time period; Step S2, the acquired physiological signals are subjected to noise reduction filtering pretreatment operation, and the acquired facial expression images are subjected to pretreatment operation; Step S3, the pretreated physiological signal data and facial expression image data are input into a hybrid multi-layer neural network for feature extraction, emotion recognition classification, and output of emotion classification judgment result; Step S4, whether the emotion classification judgment result in step S3 is a negative emotion, if the emotion classification is a negative emotion, step S5 is performed, if the emotion classification is not a negative emotion, step S6 is performed; Step S5, whether the deep pressure inflatable vest is in an inflated state, if the deep pressure inflatable vest is in an inflated state, step S8 is performed, if the deep pressure inflatable vest is not in an inflated state, step S7 is performed; Step S6, whether the deep pressure inflatable vest is in an inflated state, if the deep pressure inflatable vest is in an inflated state, step S9 is performed, if the deep pressure inflatable vest is not in an inflated state, the whole method process is repeated and executed, and the emotion state is continuously acquired and classified, returning to step S1; Step S7, the deep pressure inflatable vest is inflated, and the whole method process is repeated and executed, and the emotion state is continuously acquired and classified, until the emotion reaches a normal index, enabling the deep pressure inflatable vest to deflate; Step S8, the inflated state of the deep pressure inflatable vest is maintained, and the whole method process is repeated and executed, and the emotion state is continuously acquired and classified, until the emotion reaches a normal index, enabling the deep pressure inflatable vest to deflate; Step S9, the deep pressure inflatable vest is deflated, and the whole method process is repeated and executed, and the emotion state is continuously acquired and classified.

2. The method of autism children emotion recognition and regulation based on wearable device according to claim 1, characterized in that, In step S2, the skin electricity, skin temperature, heart rate and blood volume pulse signals are subjected to low-pass filtering and noise reduction; wherein: The time domain analysis features of the skin electricity signal include the mean, standard deviation, median, maximum and ratio thereof, the mean, standard deviation, median, maximum and ratio of the first-order differential of the skin electricity signal, the mean, standard deviation, median, maximum and ratio of the second-order differential of the skin electricity signal, and the frequency domain analysis feature of the skin electricity signal includes the spectral power; The time domain analysis features of the skin temperature signal include the mean, standard deviation, median, maximum and ratio thereof, and the frequency domain analysis feature of the skin temperature signal includes the spectral power; The time domain analysis features of the heart rate signal include the mean, standard deviation, heart rate variability HRV mean and heart rate variability HRV standard deviation, and the frequency domain analysis feature of the heart rate signal includes the power spectral density; The time domain analysis features of the blood volume pulse signal include the mean, standard deviation, median, maximum and ratio thereof, and the frequency domain analysis feature of the blood volume pulse signal includes the power spectral density.

3. A system for autism children emotion recognition and regulation based on wearable device, for implementing a method for autism children emotion recognition and regulation based on wearable device according to any one of claims 1-2, characterized in that, The device comprises an acquisition module, a data analysis module, a wireless communication module, and an adjustment module, the data analysis module is connected with the wireless communication module, the acquisition module is connected with the data analysis module and the wireless communication module respectively, the adjustment module is connected with the wireless communication module, and the wireless communication module is connected with a terminal; The acquisition module is used for acquiring data reflecting the emotional state of a user in a certain time period in real time; The data analysis module is used for pre-processing and analyzing the data collected by the acquisition module, realizing the judgment and output of the emotional category; The wireless communication module comprises one or more of a network controller, a Bluetooth transceiver, a WiFi transceiver, and a mobile network transceiver, and is used for transmitting the collected data of the acquisition module to the data analysis module and transmitting the action instruction generated by the data analysis result to the adjustment module; The adjustment module is used for providing timely emotional adjustment measures when the data analysis module analyzes that the autistic child produces negative emotions. The terminal is used for receiving the data analysis result from the wearable device and the action state of the adjustment module, and providing an intuitive user interface to enable the user to understand the received information.

4. The system for autism children emotion recognition and regulation based on wearable device according to claim 3, characterized in that, The acquisition module comprises a facial image acquisition module and a physiological signal acquisition module connected with the data analysis module respectively; The facial image acquisition module is used for acquiring facial expression image data of a user; The physiological signal acquisition module is used for acquiring physiological signal data of the user.

5. The system for autism children emotion recognition and regulation based on wearable device according to claim 4, characterized in that, The facial image acquisition module comprises a high-definition camera connected with the data analysis module; The high-definition camera is used for acquiring facial expression image data of a user in real time.

6. The system for autism children emotion recognition and regulation based on wearable device according to claim 4, characterized in that, The physiological signal acquisition module comprises a health monitoring smart bracelet connected with the data analysis module; The health monitoring smart bracelet is used for monitoring and acquiring four physiological signals of the user, including skin electricity, skin temperature, heart rate, and blood volume pulse.

7. The system for autism children emotion recognition and regulation based on wearable device according to claim 3, characterized in that, The data analysis module comprises a data processing unit, a hybrid multi-layer neural network, and a command and dispatch unit; The data processing unit is used for acquiring time-frequency domain analysis features corresponding to the four physiological signals of skin electricity, skin temperature, heart rate, and blood volume pulse, and simultaneously converting the resolution and depth of the acquired facial expression image; The hybrid multi-layer neural network is used for performing feature extraction and emotion recognition operations on the pre-processed data to obtain an emotional classification judgment result; The command and dispatch unit is used for providing command, dispatch, and coordination data or instructions for the connected wearable devices and terminal.

8. The system for autism children emotion recognition and regulation based on wearable device according to claim 3, characterized in that, The wireless communication module comprises a wireless Bluetooth transceiver, a WiFi transceiver, and a mobile network transceiver, and is used for data communication and transmission.

9. The system for autism children emotion recognition and regulation based on wearable device according to claim 3, characterized in that, The adjustment module comprises a deep pressure inflatable vest; the deep pressure inflatable vest is a wearable vest based on deep pressure therapy (DPT) to relieve negative emotions such as anxiety, and can receive the inflation and deflation instructions of the data analysis module in the system to execute or terminate the process of emotional adjustment of the user.

10. The system for emotion recognition and regulation for autistic children based on wearable devices as claimed in claim 7 wherein, The mixed multi-layer neural network is used for feature extraction of the time-frequency analysis features of the four physiological signals after data preprocessing, VGG-16 is used for feature extraction of the facial expression image after preprocessing, then convolution operation is used for transformation of the two feature vectors to obtain the fusion features, the attention mechanism is used for analysis and capture of the relationship between the features, and finally the emotion classification judgment result is output through the full connection layer.

Images