A head therapy instrument control method and system based on a multi-element stimulation mode

Abstract

The application discloses a kind of head physiotherapy instrument control method and system based on multivariate stimulation mode, it is related to head physiotherapy technical field, including, according to the geometric characteristics of facial region, depth expression feature, speech emotion score and skin electric emotion score, through fusion algorithm calculation comprehensive emotion state value, according to emotion state value judges emotion category, according to emotion category selects physiotherapy instrument mode;Physiotherapy instrument uses, real-time acquisition skin conductance value and through photoelectric volume pulse wave sensor acquisition heart rate variability data, through skin conductance value and heart rate variability data, dynamically adjusts physiotherapy instrument mode.The application is accurate to assess the emotion state of user by comprehensive acquisition facial image, speech signal and skin conductance data, combines depth learning and machine learning algorithm, solves the problem, such as low emotion evaluation precision, fixed treatment mode in prior art.

Description

Technical Field

[0001] This invention relates to the field of head physiotherapy technology, and in particular to a control method and system for a head physiotherapy device based on multiple stimulation methods. Background Technology

[0002] With the increasing demand for health management and emotion regulation, the development of emotion monitoring and head therapy devices based on biosignals has gradually become an important research direction. In recent years, methods for monitoring emotional states based on facial expressions, speech signals, and physiological feedback data (such as skin conductance values ​​and heart rate variability) have been widely applied. By acquiring facial images through cameras and speech signals through microphones, combined with advanced image processing technology and speech signal analysis methods, emotional information can be effectively extracted from facial expressions and speech. Furthermore, skin conductance sensors can provide emotion-related physiological signals, such as the electrical conductance response (EDA), whose changes can reflect an individual's emotional states such as tension, anxiety, and relaxation. Utilizing these signals, combined with appropriate control strategies, more precise and personalized emotion regulation and therapy plans can be provided to individuals.

[0003] However, existing technologies for fusing and processing multiple biological signals still have some limitations. First, many current emotion monitoring methods rely primarily on single physiological feedback or behavioral features (such as facial expressions or voice characteristics), resulting in low accuracy in emotion assessment, especially in complex emotional states where a single signal cannot comprehensively reflect emotional changes. Second, existing head therapies are relatively simple in dynamically adjusting treatment modes, lacking real-time monitoring and feedback mechanisms. Most head therapies simply treat according to initially set rules, failing to fully consider the user's immediate emotional changes. Therefore, how to dynamically adjust treatment modes in real-time monitoring by combining multiple stimulation methods (such as sound, vision, and touch) is a problem that urgently needs to be solved in existing technologies. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a head therapy device control system based on multiple stimulation methods, which solves the problems of inaccurate emotion assessment and lack of dynamic adjustment of treatment mode in the prior art.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides a computer device comprising: acquiring a user's facial image, voice signal, and skin conductance value;

[0008] Using MTCNN based on convolutional neural networks, facial contours are detected from facial images and geometric features of facial regions are calculated. Deep expression features are extracted from facial regions using a deep learning model. Speech features are extracted from speech signals using Mel frequency cepstral coefficients. Speech emotion scores are generated using support vector machines. The average value and change of skin conductance are calculated through a sliding window to generate skin conductance emotion scores.

[0009] Based on the geometric features of the facial region, deep expression features, voice emotion score, and skin conductance emotion score, a comprehensive emotion state value is calculated through a fusion algorithm. The emotion category is determined based on the emotion state value, and the physiotherapy mode is selected based on the emotion category.

[0010] When the physiotherapy device is in use, it collects skin conductivity values ​​in real time and heart rate variability data through a photoplethysmography (PPG) sensor. Based on the skin conductivity values ​​and heart rate variability data, the physiotherapy device mode is dynamically adjusted.

[0011] As a preferred embodiment of the head therapy device control method based on multiple stimulation modes described in this invention, the method employs MTCNN based on convolutional neural networks to detect facial contours from facial images and calculate the geometric features of the facial region. A deep learning model is used to extract deep expression features from the facial region, and Mel-frequency cepstral coefficients are used to extract speech features from the speech signal. The specific steps are as follows:

[0012] Using MTCNN based on convolutional neural networks, the three sub-networks of MTCNN detect facial contours from facial images through candidate box generation, exact regression, and facial key point localization, and extract the coordinates of facial key points. By calculating the relative positions and geometric relationships between facial key points, the geometric features of the face are obtained.

[0013] VGG-Face uses convolutional and fully connected layers of a deep convolutional neural network to perform feature mapping on facial images. Each convolutional operation uses the ReLU activation function to extract multi-scale facial expression information and generate deep expression features.

[0014] By processing the speech signal in frames, windowing and short-time Fourier transform are performed on each frame to convert the speech signal from the time domain to the frequency domain. The Mel filter bank is then used to filter the spectrum to obtain the Mel spectrum.

[0015] Logarithmic Mel spectrum is converted into Mel frequency cepstral coefficients by logarithmic operations and discrete cosine transform to obtain speech features.

[0016] As a preferred embodiment of the head therapy device control method based on multiple stimulation modes described in this invention, the steps of generating a voice emotion score using a support vector machine and calculating the average value and change of skin conductance values ​​through a sliding window to generate a skin conductance emotion score are as follows:

[0017] The Z-score normalization method is used to standardize the speech features. The normalized speech features are input into a support vector machine model, and the speech emotion score is output.

[0018] The average and variation of skin conductance values ​​are calculated using a sliding window, and a skin conductance-emotional score is generated using the sigmoid function. The expression is as follows:

[0019] ;

[0020] in, For skin electrophoresis mood score, This represents the average skin conductivity value. This represents the change in skin conductivity.

[0021] As a preferred embodiment of the head therapy device control method based on multiple stimulation modes described in this invention, the specific steps for calculating the comprehensive emotional state value through a fusion algorithm are as follows:

[0022] Based on geometric features, deep expression features, speech emotion scores, and electrodermal emotion scores of the facial region, a cross-attention mechanism is used to map geometric features and deep expression features to a common feature space, obtaining a non-linear relationship between the geometric features and deep expression features. Furthermore, a sigmoid function is used to adjust the dynamic coupling relationship between speech emotion scores and electrodermal emotion scores to calculate a comprehensive emotion state value, expressed as follows:

[0023] ;

[0024] in, The overall emotional state value, Geometric features For the interaction matrix, For deep facial expression features, To score the emotion of the voice, For skin electrophoresis mood score, For adjustment coefficients, Here is the numerical stability constant. Variance representing geometric characteristics, The variance of deep facial expression features.

[0025] In a preferred embodiment of the head therapy device control method based on multiple stimulation modes described in this invention, the specific steps for determining the emotion category based on the emotion state value are as follows:

[0026] Set a low arousal threshold based on the user's historical emotional data. and high wake-up threshold ;

[0027] When the overall emotional state value Less than the low wake-up threshold At that time, the emotion category is calm;

[0028] When the overall emotional state value Located at low wake-up threshold and high wake-up threshold In between, the emotional category is mild tension;

[0029] When the overall emotional state value Greater than the high wake-up threshold At that time, the emotion category is high tension.

[0030] As a preferred embodiment of the head therapy device control method based on multiple stimulation modes described in this invention, the specific steps for selecting the therapy device mode according to the emotion category are as follows:

[0031] When the mood category is calm, choose low-frequency pulse stimulation, heat therapy, and vibration massage;

[0032] When the emotional category is mild tension, choose medium-frequency pulse stimulation, electrical stimulation, and vibration massage;

[0033] When the emotional category is high tension, choose high-frequency pulse stimulation, sound waves, and cold compresses.

[0034] As a preferred embodiment of the head therapy device control method based on multiple stimulation modes described in this invention, wherein: when the therapy device is in use, skin conductivity values ​​are collected in real time, and heart rate variability data is collected through a photoplethysmography (PPG) sensor. Based on historical emotional data, threshold values ​​for skin conductivity and heart rate variability are defined, and the stimulation intensity is dynamically adjusted based on the skin conductivity values, expressed as:

[0035] ;

[0036] in, The adjusted stimulus intensity, For maximum stimulus intensity, To regulate the sensitivity of skin conductivity to stimulation intensity, To monitor skin conductivity in real time, The threshold value for skin conductivity;

[0037] The stimulation frequency is dynamically adjusted based on heart rate variability data, expressed as:

[0038] ;

[0039] in, The adjusted stimulation frequency, The maximum stimulation frequency, The sensitivity of heart rate variability to stimulation frequency regulation. For real-time heart rate variability data, This is the threshold for heart rate variability.

[0040] Secondly, this invention provides a head physiotherapy device control system based on multiple stimulation methods, comprising: a data acquisition module for acquiring the user's facial image, speech signal, and skin conductance value; a feature extraction module for using MTCNN based on a convolutional neural network to detect facial contour coordinates from the facial image, perform normalization processing, calculate the geometric features of the facial region, extract deep expression features from the facial region using a deep learning model, extract speech features from the speech signal using Mel-frequency cepstral coefficients, generate a speech emotion score using a support vector machine, and calculate the average value and variation of skin conductance value through a sliding window to generate a skin conductance emotion score; an emotion state calculation and classification module for calculating a comprehensive emotion state value based on the geometric features of the facial region, deep expression features, speech emotion score, and skin conductance emotion score using a fusion algorithm, determining the emotion category based on the emotion state value, and selecting the physiotherapy device mode based on the emotion category; and a treatment and adjustment module for dynamically adjusting the physiotherapy device mode based on the skin conductance value and heart rate variability data collected in real time and through a photoplethysmography (PPG) sensor during use.

[0041] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the head physiotherapy device control system based on multiple stimulation modes as described in the first aspect of the present invention.

[0042] The beneficial effects of this invention are as follows: By comprehensively collecting facial images, voice signals, and skin conductance data, and combining deep learning and machine learning algorithms, it accurately assesses the user's emotional state, solving the problems of low accuracy in emotion assessment and fixed treatment modes in existing technologies. Through dynamically adjusting the treatment mode, this invention effectively improves the personalization and real-time nature of the treatment process, helping users receive appropriate treatment in different emotional states. This innovative solution not only brings more intelligence to the control method of head therapy devices but also improves the scientific nature and precision of treatment effects. Attached Figure Description

[0043] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a flowchart of the head therapy device control based on multiple stimulation methods in Example 1.

[0045] Figure 2 This is a schematic diagram of the head therapy device control system based on multiple stimulation methods in Example 1. Detailed Implementation

[0046] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0047] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0048] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0049] Example 1, referring to Figure 1 and Figure 2 This is the first embodiment of the present invention, which provides a head physiotherapy device control system based on multiple stimulation methods, including the following steps:

[0050] The data acquisition module collects the user's facial images, voice signals, and skin conductance values;

[0051] Furthermore, facial images of the user are captured via a camera, and facial regions are extracted from the facial images using OpenCV's Haar Cascade classifier;

[0052] It should be noted that the acquired images are input into the Haar Cascade classifier. By analyzing the features of different regions in the image, the facial region is identified and located. Through feature extraction and classification operations, the Haar Cascade classifier can accurately detect the position of the face in the image and outline the facial region.

[0053] Collects the user's voice signal through a microphone;

[0054] It should be noted that when the user's voice signal is collected in real time through the microphone, the microphone converts the sound of the surrounding environment into electrical signals and performs digital processing. The collected voice signal is converted into continuous time series data and noise is removed.

[0055] Skin conductivity values ​​are collected using a skin conductance sensor;

[0056] It should be noted that when collecting skin conductivity values ​​using a wireline sensor, the sensor contacts the user's skin through electrodes, generating an electric field on the skin surface using a tiny current. As the current passes through the skin, the skin's conductivity value changes with the user's emotional state and physiological responses. The wireline sensor monitors these changes in real time and converts the skin conductivity value into an electrical signal. The collected electrical signal is then amplified, filtered, and digitized.

[0057] The feature extraction module uses MTCNN based on convolutional neural networks to detect facial contour coordinates from facial images and perform normalization processing, calculate the geometric features of the facial region, extract deep expression features from the facial region using a deep learning model, extract speech features from the speech signal using Mel frequency cepstral coefficients, generate speech emotion scores using support vector machines, and calculate the average value and change of skin conductance values ​​through a sliding window to generate skin conductance emotion scores.

[0058] Furthermore, using MTCNN based on convolutional neural networks, the three sub-networks of MTCNN detect facial contours from facial images through candidate box generation, exact regression, and facial key point localization, and extract the coordinates of facial key points. By calculating the relative positions and geometric relationships between facial key points, the geometric features of the face are obtained.

[0059] It should be noted that MTCNN detects facial contours from facial images and extracts the coordinates of facial key points, including the eyes, nose, and mouth. It processes the image through multiple convolutional operations to gradually locate the facial region and its key feature points. The extracted facial key point coordinates are used to calculate the relative positions and geometric relationships between key points to obtain the geometric features of the face. These features describe the shape, angles, and expression changes of the face. The three sub-networks of MTCNN are P-Net, R-Net, and O-Net, and the output includes the coordinate information of the left and right eyes, the tip of the nose, and the left and right corners of the mouth.

[0060] VGG-Face uses convolutional and fully connected layers of a deep convolutional neural network to perform feature mapping on facial images. Each convolutional operation uses the ReLU activation function to extract multi-scale facial expression information and generate deep expression features.

[0061] It should be noted that the VGG-Face model uses a deep convolutional neural network to perform feature mapping on facial images and extract deep expression features. The VGG-Face network extracts high-dimensional features that can effectively represent changes in facial expressions through multiple layers of convolution, pooling and fully connected layers, thus generating deep expression features.

[0062] By processing the audio signal in frames, performing a short-time Fourier transform on each frame of the audio signal, the audio signal is converted from the time domain to the frequency domain. The Mel filter bank is then used to filter the spectrum to obtain the Mel spectrum.

[0063] It should be noted that by processing audio signals in frames, the continuous audio signal is divided into several small frames, each typically 20 to 40 milliseconds in length, with some overlap between frames. For each frame of audio signal, a short-time Fourier transform is applied to convert it from the time domain to the frequency domain, obtaining the spectrum of that frame. The short-time Fourier transform uses a window function to perform local processing on the audio signal, enabling it to capture the time and frequency variations of the audio signal.

[0064] Next, the spectrum is filtered using a Mel filter bank. The Mel filter bank is designed based on the auditory characteristics of the human ear, decomposing the spectrum into multiple Mel-scale frequency bands to simulate the perceptual characteristics of the human ear, enhancing sensitivity to low-frequency components and reducing sensitivity to high-frequency components. The filtered spectrum is the Mel spectrum.

[0065] The logarithmic Mel spectrum is converted into Mel frequency cepstral coefficients by logarithmic operations and discrete cosine transform to obtain audio features;

[0066] It should be noted that in the audio signal processing, the Mel spectrum is first logarithmically transformed into a logarithmic Mel spectrum. Then, the logarithmic Mel spectrum is processed by discrete cosine transform to obtain Mel frequency cepstral coefficients (MFCC). Mel frequency cepstral coefficients can effectively characterize the short-time characteristics of audio signals and are widely used in speech recognition and emotion analysis. The Mel spectrum simulates the auditory characteristics of the human ear and can effectively reflect the frequency distribution of audio signals. Usually, the first 13 Mel spectrum coefficients are selected as audio features.

[0067] The Z-score normalization method is used to standardize the speech features. The normalized speech features are input into a support vector machine model, and the speech emotion score is output.

[0068] It should be noted that the mean and standard deviation of each audio feature are calculated, and these values ​​are used to standardize each feature. The standardized audio features are then used as input to a support vector machine (SVM) model for training and prediction. The SVM constructs an optimal hyperplane to map the audio features to a high-dimensional space for better classification or regression. Ultimately, the SVM model outputs a speech emotion score, representing the emotional features contained in the audio data, further providing a quantitative basis for emotion analysis.

[0069] The average and variation of skin conductance values ​​are calculated using a sliding window, and a skin conductance-emotional score is generated using the sigmoid function. The expression is as follows:

[0070] ;

[0071] in, For skin electrophoresis mood score, This represents the average skin conductivity value. This refers to the change in skin conductivity.

[0072] It should be noted that the real-time acquired skin conductivity values ​​are processed using a sliding window technique. The sliding window divides the skin conductivity value sequence into small segments and calculates the average value and change within each segment.

[0073] The emotion state calculation and classification module calculates a comprehensive emotion state value based on the geometric features of the facial region, deep expression features, voice emotion score, and skin conductance emotion score through a fusion algorithm. It then determines the emotion category based on the emotion state value and selects the physiotherapy mode based on the emotion category.

[0074] Furthermore, based on the geometric features of the facial region, deep expression features, speech emotion scores, and electrodermal emotion scores, a cross-attention mechanism is used to map the geometric features and deep expression features to a common feature space, thereby obtaining the nonlinear relationship between the geometric features and deep expression features.

[0075] It should be noted that, through the cross-attention mechanism, the geometric features and deep expression features of the facial region are mapped to a common feature space. In this feature space, the cross-attention mechanism can capture the non-linear relationship between the geometric features and the deep expression features, thereby effectively fusing the two types of features. This process further improves the accuracy of facial emotion analysis by calculating the interdependence and weights between different features.

[0076] Furthermore, the dynamic coupling relationship between speech emotion score and electrodermal emotion score is adjusted using the Sigmoid function to calculate the comprehensive emotion state value, expressed as follows:

[0077] ;

[0078] in, The overall emotional state value, Geometric features For the interaction matrix, For deep facial expression features, To score the emotion of the voice, For skin electrophoresis mood score, For adjustment coefficients, Here is the numerical stability constant. Variance representing geometric characteristics, The variance of deep facial expression features;

[0079] It should be noted that the Sigmoid function is introduced to adjust the speech emotion score. and skin electrophoresis mood score The dynamic coupling relationship between them causes the emotion score to vary within a certain range. It is a moderating coefficient that controls the skin conductance mood score. Voice emotion score The degree of influence; combining the interaction between geometric features and deep facial expression features with dynamically adjusted speech and electrodermal emotion scores, through... The activation function further strengthens the non-linear relationship between features; the calculated emotional state value Normalized, using variance and After standardization, the numerical stability constant is obtained. This is used to ensure that no division by zero error occurs during the calculation process;

[0080] Set a low arousal threshold based on the user's historical emotional data. and high wake-up threshold ;

[0081] It should be noted that the mean and standard deviation of the emotional state values ​​are calculated, and the dynamic classification threshold is calculated based on the mean and standard deviation of the emotional state values. The expression is as follows:

[0082] ;

[0083] in, To achieve a low wake-up threshold, To achieve a high wake-up threshold, The mean of emotional state values. The mean and standard deviation of the emotional state values;

[0084] When the overall emotional state value Less than the low wake-up threshold At that time, the emotion category is calm;

[0085] When the overall emotional state value Located at low wake-up threshold and high wake-up threshold In between, the emotional category is mild tension;

[0086] When the overall emotional state value Greater than the high wake-up threshold At that time, the emotion category is high tension;

[0087] When the mood category is calm, choose low-frequency pulse stimulation, heat therapy, and vibration massage;

[0088] It should be noted that when the emotional category is calm, low-frequency, low-intensity pulse stimulation should be selected. Low-frequency stimulation can gently activate the nervous system, helping people maintain a relaxed and calm state, reducing physical tension. Combined with a moderate temperature (such as 38-40℃), it heats the treatment area, promotes blood circulation, further relaxes muscles, and helps the body restore calm. The gentle vibration frequency and intensity promote slight blood flow, avoiding over-activation of the nervous system, which is suitable for maintaining a calm state.

[0089] When the emotional category is mild tension, choose medium-frequency pulse stimulation, electrical stimulation, and vibration massage;

[0090] It should be noted that when the emotional category is mild tension, choose medium-frequency pulse stimulation. Medium frequency can more effectively stimulate the nervous system and muscles, helping to relieve mild tension without over-activating the body. Moderate electrical stimulation can penetrate deep into the muscle layer, relieving fatigue and tension, and helping the body gradually relax. Choose vibration massage with medium frequency and intensity, which can promote blood circulation and effectively soothe muscles, eliminating mild stress and tension.

[0091] When the emotional category is high tension, choose high-frequency pulse stimulation, sound waves, and cold compresses.

[0092] It should be noted that when the emotional category is high tension, choosing high-frequency, high-intensity pulse stimulation can quickly activate nerves and muscles, stimulate the body to produce more energy responses, and help relieve severe tension or anxiety. Using low-frequency sound waves to deliver deep stimulation can help relax deep muscles, improve blood circulation, and relieve physical tension and anxiety. Applying cold compresses to muscles and joints can help relieve muscle inflammation or excessive tension and reduce the body's stress response.

[0093] The treatment and adjustment module collects skin conductivity values ​​in real time and heart rate variability data through a photoplethysmography sensor during use. Based on the skin conductivity values ​​and heart rate variability data, the initial treatment mode is dynamically adjusted.

[0094] Furthermore, during use, the physiotherapy device collects skin conductivity values ​​in real time and heart rate variability data via a photoplethysmography (PPG) sensor. Based on historical emotional data, it defines thresholds for skin conductivity and heart rate variability, and dynamically adjusts the stimulation intensity using skin conductivity values. The expression is:

[0095] ;

[0096] in, The adjusted stimulus intensity, For maximum stimulus intensity, To regulate the sensitivity of skin conductivity to stimulation intensity, To monitor skin conductivity values ​​in real time, The threshold value for skin conductivity;

[0097] The stimulation frequency is dynamically adjusted based on heart rate variability data, expressed as:

[0098] ;

[0099] in, The adjusted stimulation frequency, The maximum stimulation frequency, The sensitivity of heart rate variability to stimulation frequency regulation. For real-time heart rate variability data, This is the threshold for heart rate variability.

[0100] It should be noted that in the initial treatment mode, the user's skin conductance value is first monitored in real time by a skin conductance sensor. When the skin conductance value exceeds the preset skin conductance value threshold, it indicates that the user's emotions are in a high arousal state. At this time, the stimulation intensity should be appropriately increased to help the user reduce the emotional arousal level. Conversely, when the skin conductance value is lower than the skin conductance value threshold, it indicates that the user's emotions are in a low arousal state. At this time, the stimulation intensity should be appropriately reduced to avoid overstimulation. Through this dynamic adjustment mechanism, the treatment effect can be optimized according to the user's real-time emotional state.

[0101] Furthermore, heart rate variability (HRV) data can also be used to dynamically adjust treatment patterns. HRV reflects the regularity of the heartbeat, and lower HRV is usually associated with higher emotional arousal levels. Therefore, when HRV data is below a preset threshold, it indicates that the user is in a high arousal state. In this case, the stimulation frequency should be appropriately increased to help the user reduce their emotional arousal level. Conversely, when HRV data is above the threshold, it indicates that the user is in a low arousal state. In this case, the stimulation frequency should be appropriately reduced to avoid overstimulation. Through this dynamic adjustment mechanism, the treatment effect can be optimized based on the user's real-time emotional state.

[0102] This embodiment also provides a computer device, including: acquiring a user's facial image, speech signal, and skin conductance values; using MTCNN based on a convolutional neural network to detect facial contours from the facial image and calculate the geometric features of the facial region; using a deep learning model to extract deep expression features from the facial region; using Mel-frequency cepstral coefficients to extract speech features from the speech signal; using a support vector machine to generate a speech emotion score; calculating the average value and variation of skin conductance values ​​through a sliding window to generate a skin conductance emotion score; calculating a comprehensive emotion state value through a fusion algorithm based on the geometric features of the facial region, deep expression features, speech emotion score, and skin conductance emotion score; determining the emotion category based on the emotion state value; and selecting a physiotherapy mode based on the emotion category; when the physiotherapy device is in use, acquiring skin conductance values ​​in real time and acquiring heart rate variability data through a photoplethysmography (PPG) sensor; and dynamically adjusting the physiotherapy mode based on the skin conductance values ​​and heart rate variability data.

[0103] This embodiment also provides a computer device applicable to the control system of a head physiotherapy device based on multiple stimulation methods, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the head physiotherapy device control system based on multiple stimulation methods as proposed in the above embodiment.

[0104] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0105] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the head therapy device control method based on multiple stimulation modes as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0106] In summary, this invention, by comprehensively acquiring facial images, voice signals, and skin conductance data, and combining deep learning and machine learning algorithms, accurately assesses the user's emotional state, solving the problems of low accuracy in emotion assessment and fixed treatment modes in existing technologies. By dynamically adjusting the treatment mode, this invention effectively improves the personalization and real-time nature of the treatment process, helping users receive appropriate treatment in different emotional states. This innovative solution not only brings more intelligence to the control method of head therapy devices but also improves the scientific nature and precision of treatment effects.

[0107] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A head physiotherapy device control system based on multiple stimulation methods, characterized in that: It includes a data acquisition module, a feature extraction module, an emotion state calculation and classification module, and a treatment and regulation module. The data acquisition module is used to collect the user's facial images, voice signals, and skin conductance values; The feature extraction module is used to detect facial contour coordinates from facial images using MTCNN based on convolutional neural networks, perform normalization processing, calculate the geometric features of the facial region, extract deep expression features from the facial region using a deep learning model, extract speech features from the speech signal using Mel frequency cepstral coefficients, generate speech emotion scores using support vector machines, and calculate the average value and change of skin conductance values ​​through a sliding window to generate skin conductance emotion scores. The emotion state calculation and classification module is used to calculate a comprehensive emotion state value based on the geometric features of the facial region, deep expression features, voice emotion score, and skin conductance emotion score through a fusion algorithm. The module then determines the emotion category based on the emotion state value and selects the physiotherapy mode based on the emotion category. The treatment and adjustment module is used to collect skin conductivity values ​​in real time and heart rate variability data through a photoplethysmography (PPG) sensor during the use of the physiotherapy device. Based on the skin conductivity and heart rate variability data, the physiotherapy device mode is dynamically adjusted. The specific steps are as follows. Based on historical emotion data, thresholds for skin conductance and heart rate variability are defined. Stimulus intensity is dynamically adjusted using skin conductance values, expressed as follows: ； in, The adjusted stimulus intensity, For maximum stimulus intensity, To regulate the sensitivity of skin conductivity to stimulation intensity, To monitor skin conductivity values ​​in real time, The threshold value for skin conductivity; The stimulation frequency is dynamically adjusted based on heart rate variability data, expressed as: ； in, The adjusted stimulation frequency, The maximum stimulation frequency, The sensitivity of heart rate variability to stimulation frequency regulation. For real-time heart rate variability data, This is the threshold for heart rate variability.

2. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it performs the following steps. include, Collect user's facial images, voice signals, and skin conductivity values; Using MTCNN based on convolutional neural networks, facial contours are detected from facial images and geometric features of facial regions are calculated. Deep expression features are extracted from facial regions using a deep learning model. Speech features are extracted from speech signals using Mel frequency cepstral coefficients. Speech emotion scores are generated using support vector machines. The average value and change of skin conductance are calculated through a sliding window to generate skin conductance emotion scores. Based on the geometric features of the facial region, deep expression features, voice emotion score, and skin conductance emotion score, a comprehensive emotion state value is calculated through a fusion algorithm. The emotion category is determined based on the emotion state value, and the physiotherapy mode is selected based on the emotion category. When the physiotherapy device is in use, it collects skin conductivity values ​​in real time and heart rate variability data through a photoplethysmography (PPG) sensor. Based on the skin conductivity and heart rate variability data, the physiotherapy mode is dynamically adjusted. The specific steps are as follows. Based on historical emotion data, thresholds for skin conductance and heart rate variability are defined. Stimulus intensity is dynamically adjusted using skin conductance values, expressed as follows: ； in, The adjusted stimulus intensity, For maximum stimulus intensity, To regulate the sensitivity of skin conductivity to stimulation intensity, To monitor skin conductivity values ​​in real time, The threshold value for skin conductivity; The stimulation frequency is dynamically adjusted based on heart rate variability data, expressed as: ； in, The adjusted stimulation frequency, The maximum stimulation frequency, The sensitivity of heart rate variability to stimulation frequency regulation. For real-time heart rate variability data, This is the threshold for heart rate variability.

3. The computer device as described in claim 2, characterized in that: The method uses MTCNN, a convolutional neural network-based algorithm, to detect facial contours from facial images and calculate the geometric features of the facial regions. It then uses a deep learning model to extract deep expression features from the facial regions and Mel-frequency cepstral coefficients to extract speech features from the speech signal. The specific steps are as follows: Using MTCNN based on convolutional neural networks, the three sub-networks of MTCNN detect facial contours from facial images through candidate box generation, exact regression, and facial key point localization, and extract the coordinates of facial key points. By calculating the relative positions and geometric relationships between facial key points, the geometric features of the face are obtained. VGG-Face uses convolutional and fully connected layers of a deep convolutional neural network to perform feature mapping on facial images. Each convolutional operation uses the ReLU activation function to extract multi-scale facial expression information and generate deep expression features. By processing the speech signal in frames, windowing and short-time Fourier transform are performed on each frame to convert the speech signal from the time domain to the frequency domain. The Mel filter bank is then used to filter the spectrum to obtain the Mel spectrum. Logarithmic Mel spectrum is converted into Mel frequency cepstral coefficients by logarithmic operations and discrete cosine transform to obtain speech features.

4. The computer device as described in claim 3, characterized in that: The process of generating speech emotion scores using support vector machines, and calculating the average and variation of skin conductance values ​​through a sliding window to generate skin conductance emotion scores, involves the following specific steps: The Z-score normalization method is used to standardize the speech features. The normalized speech features are input into a support vector machine model, and the speech emotion score is output. The average and variation of skin conductance values ​​are calculated using a sliding window, and a skin conductance-emotional score is generated using the sigmoid function. The expression is as follows: ； in, For skin electrophoresis mood score, This represents the average skin conductivity value. This represents the change in skin conductivity.

5. The computer device as described in claim 4, characterized in that: The specific steps for calculating the comprehensive emotional state value using a fusion algorithm are as follows: Based on geometric features, deep expression features, speech emotion scores, and electrodermal emotion scores of the facial region, a cross-attention mechanism is used to map geometric features and deep expression features to a common feature space, obtaining a non-linear relationship between the geometric features and deep expression features. Furthermore, a sigmoid function is used to adjust the dynamic coupling relationship between speech emotion scores and electrodermal emotion scores to calculate a comprehensive emotion state value, expressed as follows: ； in, The overall emotional state value, Geometric features For the interaction matrix, For deep facial expression features, To score the emotion of the voice, For skin electrophoresis mood score, For adjustment coefficients, Here is the numerical stability constant. Variance representing geometric characteristics, The variance of deep facial expression features.

6. The computer device as described in claim 5, characterized in that: The specific steps for determining the emotion category based on the emotion state value are as follows: Set a low arousal threshold based on the user's historical emotional data. and high wake-up threshold ; When the overall emotional state value Less than the low wake-up threshold At that time, the emotion category is calm; When the overall emotional state value Located at low wake-up threshold and high wake-up threshold In between, the emotional category is mild tension; When the overall emotional state value Greater than the high wake-up threshold At that time, the emotion category is high tension.

7. The computer device as described in claim 6, characterized in that: The specific steps for selecting the therapy device mode based on the emotion category are as follows: When the mood category is calm, choose low-frequency pulse stimulation, heat therapy, and vibration massage; When the emotional category is mild tension, choose medium-frequency pulse stimulation, electrical stimulation, and vibration massage; When the emotional category is high tension, choose high-frequency pulse stimulation, sound waves, and cold compresses.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it performs the following steps. include, Collect user's facial images, voice signals, and skin conductivity values; Using MTCNN based on convolutional neural networks, facial contours are detected from facial images and geometric features of facial regions are calculated. Deep expression features are extracted from facial regions using a deep learning model. Speech features are extracted from speech signals using Mel frequency cepstral coefficients. Speech emotion scores are generated using support vector machines. The average value and change of skin conductance are calculated through a sliding window to generate skin conductance emotion scores. Based on the geometric features of the facial region, deep expression features, voice emotion score, and skin conductance emotion score, a comprehensive emotion state value is calculated through a fusion algorithm. The emotion category is determined based on the emotion state value, and the physiotherapy mode is selected based on the emotion category. When the physiotherapy device is in use, it collects skin conductivity values ​​in real time and heart rate variability data through a photoplethysmography (PPG) sensor. Based on the skin conductivity and heart rate variability data, the physiotherapy mode is dynamically adjusted. The specific steps are as follows. Based on historical emotion data, thresholds for skin conductance and heart rate variability are defined. Stimulus intensity is dynamically adjusted using skin conductance values, expressed as follows: ； in, The adjusted stimulus intensity, For maximum stimulus intensity, To regulate the sensitivity of skin conductivity to stimulation intensity, To monitor skin conductivity values ​​in real time, The threshold value for skin conductivity; The stimulation frequency is dynamically adjusted based on heart rate variability data, expressed as: ； in, The adjusted stimulation frequency, The maximum stimulation frequency, The sensitivity of heart rate variability to stimulation frequency regulation. For real-time heart rate variability data, This is the threshold for heart rate variability.

Images