Pressure detection model training method and device, electronic equipment, and storage medium

Abstract

The application relates to the fields of artificial intelligence and medical health, and discloses a training method and device of a stress detection model, an electronic device and a storage medium, which comprises the following steps: constructing a stress detection model, wherein the stress detection model comprises a feature extraction layer, a feature fusion layer and a result prediction layer which are connected in sequence; the feature extraction layer comprises an action unit feature recognition network, an emotion feature recognition network, a performance feature recognition network and a latent feature recognition network; constructing a training set, each data sample in the training set is a facial image of an identified object, and the data sample contains annotations of facial action unit features, facial emotion features, facial performance features and facial image coding features; inputting the training set into the stress detection model for training to obtain the stress detection model. The application greatly reduces the stress detection cost, improves the detection result efficiency, improves the operability, facilitates user use, and has high practicability.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and healthcare technology, specifically to a training method, apparatus, electronic device, and storage medium for a stress detection model. Background Technology

[0002] Stress is a natural human response that prompts us to cope with life's challenges and threats, often occurring when we feel we cannot manage or control them. Moderate stress can be beneficial, increasing alertness and coping with the challenges of changing environments; for example, it can improve athletic performance, motivation, and responsiveness. However, excessive stress is detrimental to physical and mental health, increasing the risk of stroke, heart attack, ulcers, and mental illnesses such as depression, and can also worsen existing conditions.

[0003] Existing pressure detection methods typically rely on specialized equipment, such as heart rate monitors and skin conductance sensors. However, these specialized devices are expensive to obtain and are not readily available. Summary of the Invention

[0004] To address the aforementioned issues, this application provides a training method, apparatus, electronic device, and storage medium for a stress detection model, thereby overcoming or partially overcoming the shortcomings of the prior art.

[0005] Firstly, this application provides a method for training a stress detection model, including:

[0006] A stress detection model is constructed, comprising a feature extraction layer, a feature fusion layer, and a result prediction layer connected in sequence; wherein, the feature extraction layer includes: an action unit feature recognition network, an emotion feature recognition network, an expression feature recognition network, and a latent feature recognition network;

[0007] Construct a training set, wherein each data sample in the training set is a facial image of the object to be identified, and the data sample contains annotations of facial action unit features, facial emotion features, facial expression features, and facial image coding features.

[0008] The training set is input into the stress detection model for training to obtain the stress detection model.

[0009] Secondly, this application also provides a training device for a stress detection model, the device comprising:

[0010] The first building unit is used to build a stress detection model, which includes a feature extraction layer, a feature fusion layer, and a result prediction layer connected in sequence; wherein, the feature extraction layer includes: an action unit feature recognition network, an emotion feature recognition network, an expression feature recognition network, and a latent feature recognition network;

[0011] The second construction unit is used to construct a training set, wherein each data sample in the training set is a facial image of the object to be identified, and the data sample contains annotations of facial action unit features, facial emotion features, facial expression features, and facial image coding features.

[0012] The training unit is used to input the training set into the stress detection model for training, so as to obtain a trained stress detection model.

[0013] Thirdly, embodiments of this application also provide an electronic device, including: a processor; and a memory arranged to store computer-executable instructions, which, when executed, cause the processor to perform a training method for any of the stress detection models described above.

[0014] Fourthly, embodiments of this application also provide a computer-readable storage medium storing one or more programs, which, when executed by an electronic device including multiple applications, cause the electronic device to perform the training method for any of the aforementioned stress detection models.

[0015] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects:

[0016] This application constructs a stress detection model based on multi-dimensional features. The model includes a feature extraction layer, a feature fusion layer, and a result prediction layer connected sequentially. The feature extraction layer serves as the input layer, and the result prediction layer as the output layer. The feature extraction layer comprises an action unit feature recognition network, an emotion feature recognition network, a performance feature recognition network, and a latent feature recognition network. A training set is constructed, containing a large number of data samples, each a facial image of the target object, with annotations for facial action unit features, facial emotion features, facial performance features, and facial image encoding features. The training set is input into the stress detection model for training. The data samples first enter the feature extraction layer, extracting features in four dimensions, then proceed to the feature fusion layer for feature fusion, and finally, the fused features enter the result prediction layer for stress detection. This application constructs and trains a stress detection model that integrates multi-dimensional facial features. Integrating multi-dimensional facial features provides more comprehensive and accurate stress detection results. The complementary nature of features across different dimensions allows for more effective capture of subtle facial changes, resulting in high accuracy. Compared to existing stress detection methods that rely on specialized equipment, facial image analysis requires no additional hardware support, only a common camera. This significantly reduces detection costs, improves efficiency, enhances operability, and facilitates user operation. Furthermore, it is highly practical and applicable to a wide range of scenarios, such as monitoring fatigued driving, health monitoring, and psychological assessment. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 A flowchart illustrating a training method for a stress detection model according to an embodiment of this application is shown.

[0019] Figure 2 A schematic diagram of the structure of a pressure detection model according to an embodiment of this application is shown;

[0020] Figure 3 A schematic diagram of the structure of a training device for a stress detection model according to an embodiment of this application is shown;

[0021] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0024] In many aspects of life, it is necessary to monitor people's stress levels, such as the condition of truck drivers and bus drivers to determine if they are under excessive stress. In the process of mental health diagnosis and treatment, stress testing can also be used to assess a patient's mental health or mental state.

[0025] In existing technologies, pressure detection methods typically require devices such as heart rate monitors and skin conductance sensors, which suffer from poor convenience and operability.

[0026] In response, this application proposes a training method for a stress detection model. The trained stress detection model can conveniently, quickly, and accurately detect stress on the target object. Figure 1 This diagram illustrates a flowchart of the training method for a stress detection model according to an embodiment of this application. Figure 1 As can be seen, this application includes steps S110 to S130:

[0027] Step S110: Construct a stress detection model, which includes a feature extraction layer, a feature fusion layer, and a result prediction layer connected in sequence; wherein, the feature extraction layer includes: an action unit feature recognition network, an emotion feature recognition network, an expression feature recognition network, and a latent feature recognition network.

[0028] This application constructs a pressure detection model. Figure 2 This paper shows a schematic diagram of the structure of a pressure detection model according to an embodiment of the present application. Figure 2 As can be seen, the stress detection model 200 includes a feature extraction layer 210, a feature fusion layer 220, and a result prediction layer 230. The feature extraction layer 210 serves as the input layer, and the result prediction layer 230 serves as the output layer. The feature extraction layer 210 includes four sub-layers: an action unit feature recognition network 211, an emotion feature recognition network 212, an expression feature recognition network 213, and a latent feature recognition network 214. These four sub-layers are independent of each other.

[0029] Human stress can manifest in many ways, such as body movements, facial expressions, emotional changes, and behavioral performance. To comprehensively extract features from all aspects of stress performance, this application constructs a feature extraction layer 210 for multi-dimensional feature extraction. The feature extraction layer 210 is mainly responsible for extracting multi-dimensional facial features related to stress detection from the facial images of the recognition object. It mainly includes four independent and isolated sub-layers, namely, an action unit feature recognition network 211, an emotion feature recognition network 212, a performance feature recognition network 213, and a latent feature recognition network 214. The sub-layers can be constructed using any deep learning model. Preferably, in some embodiments of this application, the action unit feature recognition network, the emotion feature recognition network, the performance feature recognition network, and the latent feature recognition network are constructed based on convolutional neural networks (CNNs).

[0030] The Action Unit Feature Recognition Network 211 is primarily designed for facial action unit features. The Facial Action Coding System (FACS) divides facial action features into several independent yet interconnected Action Units (AUs) based on the anatomical characteristics of the human face. These AUs are used to describe facial movements, such as cheek lifting, eyebrow lowering, eyelid lifting, corner of the mouth downturned, and corner of the mouth upturned. The Action Unit Feature Recognition Network 211 effectively extracts these features. Its input is a human facial image, and its output is any one of the labels (cheek lifting, eyebrow lowering, eyelid lifting, corner of the mouth downturned, corner of the mouth upturned, etc.) or the confidence value (probability value) of the label. This application does not limit the action unit feature labels; they can be added or removed according to the business scenario.

[0031] Negative emotions often stem from stress; therefore, facial emotional features can reflect the psychological stress a person is under. For example, happiness, sadness, anger, and fear are facial emotional features. The emotion feature recognition network 212 is primarily used to extract facial emotional features. Its input is a human facial image. The emotion feature recognition network 212 can perform emotion recognition on the facial image, and its output is any one of the labels such as happy, sad, angry, and fear, or the confidence value of the label. This application does not limit the emotional feature labels; they can be added or removed according to the business scenario.

[0032] When people are under stress, their faces often show micro-expressions and changes in complexion, such as dark circles under the eyes, frown lines, nasolabial folds, and pale skin. The facial feature recognition network 213 is mainly used for extracting facial features. Its input is a human facial image, and its output is any one of the labels such as dark circles under the eyes, frown lines, nasolabial folds, and pale skin, or the confidence value of the label; this application does not limit the facial feature labels and they can be added or removed according to the business scenario.

[0033] Facial latent features (also known as facial image coding features) can be understood as features that describe the content of an image when it is encoded. They can be understood as a kind of high-level features. In this application, they are automatically learned and extracted by the latent feature recognition network 214, which can capture more abstract and complex image information.

[0034] In some embodiments, the feature fusion layer is built based on an autoencoder. This application primarily employs a convolutional autoencoder, which is mainly suitable for image data processing. An autoencoder learns the latent features of the data by compressing the input data into a smaller representation (encoder) and then reconstructing it back into the original data. After the four features mentioned above pass through the feature fusion layer, a richer and more tightly integrated fused feature vector is obtained.

[0035] In some embodiments, the result prediction layer is built on a deep neural network (DNN). Deep neural networks provide powerful solutions to complex problems, possessing capabilities such as automatic feature learning, processing large-scale data, and transfer learning. In some embodiments of this application, feature extraction utilizes a convolutional neural network (CNN) and result prediction employs deep learning network (DNN) technology. The combination of the two can automatically extract valuable features from large-scale data, reducing manual intervention and improving the accuracy and efficiency of detection.

[0036] Step S120: Construct a training set, wherein each data sample in the training set is a facial image of the object to be identified, and the data sample includes annotations of facial action unit features, facial emotion features, facial expression features, and facial image coding features.

[0037] A training set is constructed, comprising multiple data samples; in principle, the more data samples, the better. The data samples are facial images of the objects to be identified. In real-world scenarios, the objects are typically natural persons, and the data samples are human facial images. For each data sample, i.e., a single facial image, multi-dimensional features are labeled, including: facial action unit features, facial emotion features, facial expression features, and facial image encoding features. This application does not limit the labeling method, but recommends using semi-automatic labeling methods, such as template-based labeling: utilizing standardized templates or rules to assist labeling, suitable for data with a certain structure and regularity; or machine learning-assisted labeling: using pre-trained models to perform initial labeling of the data, followed by human review and modification, improving labeling efficiency and accuracy. Semi-automatic labeling can ensure both efficiency and accuracy.

[0038] For facial action unit features, the Facial Action Coding System (FACS) divides the human face into several facial action units (AUs) that are both independent and interconnected, based on the anatomical characteristics of the face. These facial action units (AUs) are used to describe facial movements, such as cheek lifting, eyebrow lowering, eyelid lifting, corners of the mouth turning down, and corners of the mouth turning up.

[0039] For facial emotion features, such as happiness, sadness, anger, and fear, annotations can be made based on the facial expressions of the person in the image.

[0040] For facial features such as dark circles, frown lines, nasolabial folds, and pale skin, annotations can be made based on the facial features of the person in the facial image.

[0041] Facial action unit features, facial emotion features, and facial expression features are the three explicit features related to stress detection; while facial image coding features, which mainly reflect the features obtained when encoding facial images, are implicit features and can therefore also be called facial image coding features. They can be automatically labeled using existing image basis large models.

[0042] By performing the above processing on each face image in the training set, a massive amount of training samples can be obtained.

[0043] Step S130: Input the training set into the stress detection model for training to obtain the stress detection model.

[0044] Finally, the training set is input into the stress detection model for training, and the trained stress detection model is obtained.

[0045] The training process will be described below using a single data sample as an example.

[0046] In some embodiments of this application, a combination of supervised and unsupervised training is recommended during the training process. Specifically, the feature extraction layer is trained in a supervised manner; the feature fusion layer is trained in an unsupervised manner; and the result prediction layer is trained in a supervised manner. Both supervised and unsupervised training are machine learning methods. Supervised training refers to the model learning using labeled input data during training. The key to this approach is that each training sample has a corresponding target output (label). The model adjusts its parameters by comparing its prediction results with the actual labels, gradually bringing the predictions closer to the true values. Unsupervised training refers to the model learning on unlabeled training data. Unlike supervised learning, unsupervised learning does not have a clear target output; the model's goal is to discover potential structures or patterns in the data. This application combines supervised and unsupervised learning to fully leverage the advantages of both, thereby improving the model's performance and generalization ability.

[0047] When a data sample is input into the stress detection model, the labeled data sample first enters the feature extraction layer 210, and then enters the action unit feature recognition network 211, emotion feature recognition network 212, performance feature recognition network 213, and latent feature recognition network 214 respectively.

[0048] For the action unit feature recognition network 211, the action unit feature recognition network extracts features from the data sample to obtain at least one action unit feature. Specifically, this includes: the action unit feature recognition network extracts features from the data sample to obtain the probability value of each action unit appearing in the data sample, the action unit including: cheek lifting, eyebrow lowering, eyelid lifting, corner of mouth downward, and corner of mouth upward; the probability values ​​of each action unit appearing in the data sample are used to construct a feature vector as the action unit feature.

[0049] The input to the action unit feature recognition network 211 is a facial image from the data sample, and the output is multiple recognized action units. Typically, multiple action units can be recognized simultaneously from a single facial image. Specifically, the output is the recognition probability value corresponding to each facial action unit. These recognition probability values ​​for each facial action unit are used to construct a feature vector, which is then used as the facial action unit feature for subsequent stress detection.

[0050] For the emotion feature recognition network 212, the emotion feature recognition network performs emotion recognition on the data samples to obtain facial emotion features. Specifically, the emotion feature recognition network performs emotion recognition on the data samples to obtain the probability values ​​of each emotion label to which the data samples belong. The emotion labels include: happiness, sadness, anger, and fear. The probability values ​​of each emotion label to which the data samples belong are used to construct a feature vector as the facial emotion features.

[0051] Emotion recognition networks can identify emotion tags from a user's facial images. These emotion tags include, but are not limited to, happiness, sadness, anger, and fear. Specifically, the emotion recognition network can identify the probability value (i.e., confidence score) corresponding to each emotion tag, construct a feature vector from the probability value of each emotion tag, and use this feature vector as facial emotion features for subsequent stress detection.

[0052] For the facial feature recognition network 213, the network is mainly used to extract features from the data samples to obtain at least one facial feature. Specifically, the network performs facial feature recognition on the data samples to obtain the probability value of each facial feature appearing in the data samples. These facial features include: dark circles under the eyes, frown lines, nasolabial folds, and pale skin. The probability values ​​of each facial feature appearing in the data samples are used as a feature vector, which serves as the facial feature. The input to the facial feature recognition network is a facial image from the data samples, and the output is the probability value of one or more facial features identified. Facial features include, but are not limited to, dark circles under the eyes, frown lines, nasolabial folds, and pale skin. The probability values ​​of each facial feature are used as a feature vector, which is then used as the facial feature.

[0053] The latent feature recognition network 214 is primarily used for feature extraction to obtain at least one latent facial feature. Latent facial features typically refer to details or features in a facial image that are not easily observed directly. These features may include subtle changes in expression, minute differences in skin tone, lighting effects, and facial landmarks that may be occluded or blurred. The input to the latent feature recognition network 214 is a facial image, and its output is a facial image encoding vector, which represents the latent facial features and is used for subsequent stress detection.

[0054] After feature extraction, at least one action unit feature, facial emotion feature, at least one facial expression feature, and at least one facial latent feature are obtained. These features are then fed into the feature fusion layer 220, where they are sequentially fused, reduced in dimensionality, and increased in dimensionality to obtain a feature fusion vector.

[0055] The input and output of the feature fusion layer 230 are vectors of the same dimension. It is recommended that the feature fusion layer 230 be trained in an unsupervised manner. Specifically, the feature fusion layer 230 first reduces the high-dimensional sparse features obtained by the initial fusion to low-dimensional compact features, and then increases the dimensionality of the low-dimensional compact features to the same high-dimensional sparse features.

[0056] More specifically, the initial fusion can be understood as simply "placing" these feature vectors together. For example, suppose the first vector is vector (a, b, c) and the second vector is vector (d, e). When concatenating, the second vector is placed after the last element of the first vector according to the original order of the elements in the second vector. This is denoted as vector z, and vector z is (a, b, c, d, e).

[0057] Furthermore, assume that the first and second vectors are identical matrices, and that both the first and second vectors are 2x2 two-dimensional matrices, represented as follows:

[0058]

[0059] The concatenation method is specified as vertical concatenation. During concatenation, the elements of the second vector are "placed" after the last row of the first vector, following the original order of the elements. This is denoted as vector z, which is a 2x4 matrix. Specifically:

[0060]

[0061] If the first semantic vector and the second semantic vector are irregular matrices, then the missing elements can be set to empty or zero.

[0062] After initial fusion, a high-dimensional sparse feature vector is obtained, denoted as high-dimensional sparse feature. A high-dimensional sparse feature vector refers to a feature vector in a high-dimensional space where the dimension of the feature vector is very large, but most of the feature values ​​are zero or close to zero.

[0063] Then, dimensionality reduction is performed on the high-dimensional sparse features. Dimensionality reduction can reduce computational complexity, eliminate noise, improve model interpretability, alleviate overfitting, and reveal data structure. Overall, dimensionality reduction helps improve the overall performance and efficiency of the model, while also enhancing the feasibility of data analysis. After dimensionality reduction, the resulting feature vectors are denoted as low-dimensional compact features.

[0064] Then, a second dimensionality-up process is performed. This process enhances feature representation, introduces interactive features, and increases the model's complexity. The resulting feature vector is the feature fusion vector. Through dimensionality reduction and dimensionality increase, not only are noise and useless information removed from the original fusion vector, but interactive features are also introduced, enriching the expression of the fusion vector and making the prediction results more accurate.

[0065] The dimensionality reduction and dimensionality increase processing of the above feature vectors can be performed using any of the existing technologies, and this application does not limit this to any particular method. This application recommends using an autoencoder.

[0066] Finally, the feature fusion vector is input to the result prediction layer 240, which outputs a stress level label. In some embodiments, the stress level label is set to four categories: no stress, slight stress, moderate stress, and high stress. The stress level label can be set according to business needs, and this application does not limit it.

[0067] During training, backpropagation is performed based on the loss function of each training round to adjust the relevant parameters of the stress detection model 200 until the training requirements are met, such as reaching the required number of training rounds or the loss function value being less than a preset threshold.

[0068] After training, a well-trained stress detection model is obtained. In application, its data flow is the same as in the training process. The input is a user's facial image, which is fed into the stress detection model and enters the action unit feature recognition network, emotion feature recognition network, performance feature recognition network, and latent feature recognition network for feature extraction. This yields facial action unit features, facial emotion features, facial performance features, and facial latent features. These features enter the feature fusion layer to obtain a feature fusion vector, which then enters the result prediction layer to obtain the final recognition result, i.e., the stress label.

[0069] A well-trained stress detection model can be applied to detect stress in various scenarios, such as assessing students' mental health, online medical consultations, and detecting fatigued driving. Stress detection results can be used as a standalone indicator or combined with other indicators, making it highly practical.

[0070] Depend on Figure 1As shown in the method, this application constructs a stress detection model based on multi-dimensional features. The model includes a feature extraction layer, a feature fusion layer, and a result prediction layer connected sequentially. The feature extraction layer serves as the input layer, and the result prediction layer as the output layer. The feature extraction layer includes an action unit feature recognition network, an emotion feature recognition network, a performance feature recognition network, and a latent feature recognition network. A training set is constructed, containing a large number of data samples, each a facial image of the target object, with annotations for facial action unit features, facial emotion features, facial performance features, and facial image encoding features. The training set is input into the stress detection model for training. The data samples first enter the feature extraction layer, extracting features in four dimensions, then proceed to the feature fusion layer for feature fusion, and finally, the fused features enter the result prediction layer for stress detection. This application constructs and trains a stress detection model that integrates multi-dimensional facial features. Compared with existing stress detection methods that rely on special equipment, facial image analysis does not require additional hardware support, only common cameras are needed. This greatly reduces detection costs, and the detection results are accurate and efficient. It also improves operability and makes it convenient for users. Furthermore, it is highly practical and has a wide range of applications, such as monitoring fatigued driving, health monitoring, and psychological assessment.

[0071] Figure 3 A schematic diagram of a training apparatus for a stress detection model according to an embodiment of this application is shown. Figure 3 It can be seen that the training device 300 for the stress detection model includes:

[0072] The first construction unit 310 is used to construct a stress detection model, which includes a feature extraction layer, a feature fusion layer, and a result prediction layer connected in sequence; wherein, the feature extraction layer includes: an action unit feature recognition network, an emotion feature recognition network, an expression feature recognition network, and a latent feature recognition network;

[0073] The second construction unit 320 is used to construct a training set, wherein each data sample in the training set is a facial image of the object to be identified, and the data sample contains annotations of facial action unit features, facial emotion features, facial expression features, and facial image coding features.

[0074] The training unit 330 is used to input the training set into the stress detection model for training, so as to obtain a trained stress detection model.

[0075] In some embodiments, in the above-described apparatus, the action unit feature recognition network, the emotion feature recognition network, the performance feature recognition network, and the latent feature recognition network are constructed based on convolutional neural networks; the feature fusion layer is constructed based on an autoencoder; and the result prediction layer is constructed based on a deep neural network.

[0076] In some embodiments, in the above apparatus, the training unit 330 is configured to train each network of the feature extraction layer in a supervised manner during the training process; train the feature fusion layer in an unsupervised manner; and train the result prediction layer in a supervised manner.

[0077] In some embodiments, in the above-described apparatus, the training unit 330 is configured to input a data sample into the feature extraction layer, wherein the action unit feature recognition network extracts features from the data sample to obtain at least one action unit feature, the emotion feature recognition network performs emotion recognition on the data sample to obtain facial emotion features, the performance feature recognition network extracts features from the data sample to obtain at least one facial performance feature, and the latent feature recognition network extracts features to obtain at least one latent facial feature; the feature fusion layer sequentially fuses, reduces the dimensionality of, and increases the dimensionality of the at least one action unit feature, the facial emotion feature, the at least one facial performance feature, and the at least one latent facial feature to obtain a feature fusion vector; and the result prediction layer identifies the feature fusion vector to obtain a stress level label, wherein the stress level label is any one of no stress, slight stress, moderate stress, and high stress.

[0078] In some embodiments, in the above-described apparatus, the training unit 330 is used to: 1) perform feature extraction on the data sample by the action unit feature recognition network to obtain the probability value of each action unit appearing in the data sample, the action units including: cheek lifting, eyebrow lowering, eyelid lifting, corner of mouth downward, and corner of mouth upward; 2) construct a feature vector from the probability values ​​of each action unit appearing in the data sample as the action unit feature; and 3) perform emotion recognition on the data sample by the emotion feature recognition network to obtain the probability value of each emotion tag to which the data sample belongs, the emotion tags including: happy, sad, angry, and fear; and 4) construct a feature vector from the probability values ​​of each emotion tag to which the data sample belongs as the facial emotion feature.

[0079] In some embodiments, in the above-described apparatus, the training unit 330 is used for the facial expression recognition network to perform facial expression recognition on the data sample, and to obtain the probability value of each facial expression appearing in the data sample, the facial expressions including: dark circles, frown lines, nasolabial folds, and pale skin; the probability value of each facial expression appearing in the data sample is used as a feature vector as the facial expression feature.

[0080] In some embodiments, in the above-described apparatus, the training unit 330 is used to perform preliminary splicing of the at least one action unit feature, the facial emotion feature, the at least one facial expression feature, and the at least one facial latent feature by the feature fusion layer to form a high-dimensional sparse feature; to perform low-dimensional processing on the high-dimensional sparse feature to obtain a low-dimensional compact feature; and to perform high-dimensional processing on the low-dimensional compact feature to obtain the feature fusion vector.

[0081] It is understood that the training device for the stress detection model described above can implement each step of the training method for the stress detection model provided in the foregoing embodiments. The relevant explanations of the training method for the stress detection model are applicable to the training device for the stress detection model, and will not be repeated here.

[0082] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 4 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0083] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0084] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0085] The processor reads the corresponding computer program from non-volatile memory into main memory and then runs it, forming a training device for the stress detection model at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0086] A stress detection model is constructed, comprising a feature extraction layer, a feature fusion layer, and a result prediction layer connected in sequence; wherein, the feature extraction layer includes: an action unit feature recognition network, an emotion feature recognition network, an expression feature recognition network, and a latent feature recognition network;

[0087] Construct a training set, wherein each data sample in the training set is a facial image of the object to be identified, and the data sample contains annotations of facial action unit features, facial emotion features, facial expression features, and facial image coding features.

[0088] The training set is input into the stress detection model for training to obtain the stress detection model.

[0089] The above is as stated in this application. Figure 3The method executed by the training device for the pressure detection model disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0090] The electronic device can also perform Figure 3 The method for implementing the training device of the pressure detection model is described, and the training device for the pressure detection model is implemented in... Figure 3 The functions of the embodiments shown are not described again in this application.

[0091] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by an electronic device including multiple applications, enable the electronic device to perform... Figure 3 The method executed by the training device of the pressure detection model in the illustrated embodiment is specifically used to perform:

[0092] A stress detection model is constructed, comprising a feature extraction layer, a feature fusion layer, and a result prediction layer connected in sequence; wherein, the feature extraction layer includes: an action unit feature recognition network, an emotion feature recognition network, an expression feature recognition network, and a latent feature recognition network;

[0093] Construct a training set, wherein each data sample in the training set is a facial image of the object to be identified, and the data sample contains annotations of facial action unit features, facial emotion features, facial expression features, and facial image coding features.

[0094] The training set is input into the stress detection model for training to obtain the stress detection model.

[0095] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0096] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0097] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0098] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0099] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0100] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0101] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0102] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0103] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0104] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A training method for a stress detection model, characterized in that, include: A stress detection model is constructed, comprising a feature extraction layer, a feature fusion layer, and a result prediction layer connected in sequence. The feature extraction layer includes an action unit feature recognition network, an emotion feature recognition network, a performance feature recognition network, and a latent feature recognition network. The action unit feature recognition network extracts features from data samples to obtain at least one action unit feature; the emotion feature recognition network identifies emotions from the data samples to obtain facial emotion features; the performance feature recognition network extracts features from the data samples to obtain at least one facial performance feature; and the latent feature recognition network extracts features to obtain at least one latent facial feature. The data samples are facial images of the target object. Latent facial features refer to details or features that are not easily observed directly in a facial image, including subtle changes in expression, minor differences in skin tone, lighting effects, and occluded or blurred facial features. The feature fusion layer sequentially fuses, reduces, and increases the dimensionality of the at least one action unit feature, the facial emotion feature, the at least one facial expression feature, and the at least one facial latent feature to obtain a feature fusion vector; The result prediction layer identifies the feature fusion vector to obtain a stress level label, which is any one of no stress, slight stress, moderate stress, and high stress. Construct a training set, wherein each data sample in the training set is a facial image of the object to be identified, and the data sample contains annotations of facial action unit features, facial emotion features, facial expression features, and facial latent features. The training set is input into the stress detection model for training to obtain the stress detection model.

2. The method according to claim 1, characterized in that, The action unit feature recognition network, the emotion feature recognition network, the performance feature recognition network, and the latent feature recognition network are all constructed based on convolutional neural networks. The feature fusion layer is built based on an autoencoder; The result prediction layer is built on a deep neural network.

3. The method according to claim 1, characterized in that, The step of inputting the training set into the stress detection model for training includes: During the training process, each network of the feature extraction layer is trained in a supervised manner; The feature fusion layer is trained in an unsupervised manner; The result prediction layer is trained in a supervised manner.

4. The method according to claim 1, characterized in that, The action unit feature recognition network extracts features from the data samples to obtain at least one action unit feature, including: The action unit feature recognition network extracts features from the data samples to obtain the probability value of each action unit appearing in the data samples. The action units include: cheek lifting, eyebrow lowering, eyelid lifting, corner of mouth downward, and corner of mouth upward. The probability values ​​of each action unit appearing in the data sample are used to construct a feature vector, which serves as the feature of the action unit. The emotion recognition network performs emotion recognition on the data samples to obtain facial emotion features, including: The emotion recognition network performs emotion recognition on the data samples to obtain the probability value of each emotion tag to which the data sample belongs. The emotion tags include: happiness, sadness, anger, and fear. The probability values ​​of each emotion label to which the data sample belongs are used to construct a feature vector, which serves as the facial emotion feature.

5. The method according to claim 1, characterized in that, The facial feature recognition network extracts features from the data samples to obtain at least one facial feature, including: The facial feature recognition network performs facial feature recognition on the data samples to obtain the probability value of each facial feature appearing in the data samples. The facial features include: dark circles, frown lines, nasolabial folds, and pale skin. The probability values ​​of each facial expression appearing in the data samples are used to construct a feature vector, which serves as the facial expression feature.

6. The method according to claim 1, characterized in that, The feature fusion layer sequentially performs preliminary fusion, dimensionality reduction, and dimensionality increase processing on the at least one action unit feature, the facial emotion feature, the at least one facial expression feature, and the at least one facial latent feature to obtain a feature fusion vector, including: The feature fusion layer initially splices together the at least one action unit feature, the facial emotion feature, the at least one facial expression feature, and the at least one facial latent feature to form a high-dimensional sparse feature. The high-dimensional sparse features are processed to reduce their dimensionality, resulting in low-dimensional compact features. The low-dimensional compact features are processed to obtain the feature fusion vector by high-dimensionalization.

7. A training device for a pressure detection model, characterized in that, The device includes: The first building unit is used to construct a stress detection model, which includes a feature extraction layer, a feature fusion layer, and a result prediction layer connected in sequence. The feature extraction layer includes an action unit feature recognition network, an emotion feature recognition network, a performance feature recognition network, and a latent feature recognition network. The action unit feature recognition network extracts features from data samples to obtain at least one action unit feature; the emotion feature recognition network identifies emotions from the data samples to obtain facial emotion features; the performance feature recognition network extracts features from the data samples to obtain at least one facial performance feature; and the latent feature recognition network extracts features to obtain at least one latent facial feature. The data samples are facial images of the object being identified. The latent facial features refer to details or features that are not easily observed directly in a facial image, including subtle changes in expression, minor differences in skin tone, lighting effects, and occluded or blurred facial features. The feature fusion layer sequentially fuses, reduces, and increases the dimensionality of the at least one action unit feature, the facial emotion feature, the at least one facial expression feature, and the at least one facial latent feature to obtain a feature fusion vector; The result prediction layer identifies the feature fusion vector to obtain a stress level label, which is any one of no stress, slight stress, moderate stress, and high stress. The second construction unit is used to construct a training set, wherein each data sample in the training set is a facial image of the object to be identified, and the data sample contains annotations of facial action unit features, facial emotion features, facial expression features, and facial latent features. The training unit is used to input the training set into the stress detection model for training, so as to obtain a trained stress detection model.

8. An electronic device, comprising: processor; as well as A memory configured to store computer-executable instructions, characterized in that, when executed, the executable instructions cause the processor to perform the training method of the stress detection model according to any one of claims 1 to 6.

9. A computer-readable storage medium storing one or more programs, characterized in that, When the one or more programs are executed by an electronic device including multiple applications, the electronic device performs the training method of the stress detection model according to any one of claims 1 to 6.

Images