Training method and device of pressure detection model, electronic equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence and medical health, and discloses a pressure detection model training method and device, electronic equipment and a storage medium, and the method comprises the steps: constructing a pressure detection model which comprises a feature extraction layer, a feature fusion layer and a result prediction layer which are connected in sequence; wherein the feature extraction layer comprises an action unit feature recognition network, an emotion feature recognition network, an expression feature recognition network and a recessive feature recognition network; a training set is constructed, each data sample in the training set is a face image of the recognition object, and the data samples comprise marks of face action unit features, face emotion features, face expression features and face image coding features; and inputting the training set into a pressure detection model for training to obtain the pressure detection model. The method greatly reduces the pressure detection cost, is high in detection result efficiency, improves the operability, and is convenient for a user to use. And the practicability is high.

Description

Technical Field

[0001] The present application relates to the fields of artificial intelligence and medical health technology, and specifically to a training method, device, electronic device and storage medium for a stress detection model. Background Art

[0002] Stress is a natural human response that prompts us to cope with challenges and threats in life, and usually occurs in situations that we feel we cannot manage or control. Small amounts of psychological stress can be beneficial, increasing a person's alertness to cope with the challenges of various environmental changes. For example, it can improve athletic performance, motivation, and response to the environment. However, excessive psychological stress is harmful to a person's physical and mental health. For example, it increases the risk of stroke, heart attack, ulcers, and mental illness (such as depression), and can also worsen existing diseases.

[0003] Existing stress detection methods usually rely on special equipment, such as heart rate monitors, galvanic skin response sensors, etc. However, the acquisition of these special devices requires additional costs and is less convenient. Summary of the invention

[0004] In response to the above problems, the embodiments of the present application provide a method, device, electronic device and storage medium for training a pressure detection model to overcome or partially overcome the shortcomings of the prior art.

[0005] In a first aspect, the present application provides a method for training a stress detection model, comprising:

[0006] Constructing a stress detection model, the stress detection model includes a feature extraction layer, a feature fusion layer, and a result prediction layer connected in sequence; wherein the feature extraction layer includes: an action unit feature recognition network, an emotion feature recognition network, an expression feature recognition network, and a latent feature recognition network;

[0007] Constructing a training set, wherein each data sample in the training set is a facial image of an identification object, and the data sample includes annotations of facial action unit features, facial emotion features, facial expression features, and facial image coding features;

[0008] The training set is input into the pressure detection model for training to obtain the pressure detection model.

[0009] In a second aspect, the present application also provides a training device for a pressure detection model, the device comprising:

[0010] A first construction unit is used to construct a stress detection model, wherein the stress detection model includes a feature extraction layer, a feature fusion layer, and a result prediction layer connected in sequence; wherein the feature extraction layer includes: an action unit feature recognition network, an emotion feature recognition network, an expression feature recognition network, and a latent feature recognition network;

[0011] A second construction unit is used to construct a training set, wherein each data sample in the training set is a facial image of an identification object, and the data sample includes annotations of facial action unit features, facial emotion features, facial expression features, and facial image coding features;

[0012] The training unit is used to input the training set into the pressure detection model for training to obtain a trained pressure detection model.

[0013] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor; and a memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor executes any of the above-mentioned training methods for the pressure detection model.

[0014] In a fourth aspect, an embodiment of the present application also provides a computer-readable storage medium, which stores one or more programs. When the one or more programs are executed by an electronic device including multiple applications, the electronic device executes any of the above-mentioned training methods for the pressure detection model.

[0015] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects:

[0016] The present application constructs a stress detection model, which uses multi-dimensional features as the recognition basis. The stress detection model includes a feature extraction layer, a feature fusion layer and a result prediction layer which are connected in sequence. The feature extraction layer serves as the input layer, and the result prediction layer serves as the output layer, wherein the feature extraction layer includes: an action unit feature recognition network, an emotion feature recognition network, an expression feature recognition network, and an implicit feature recognition network; a training set is constructed, wherein the training set includes a large number of data samples, each of which is a facial image of an identification object, and the facial action unit features, facial emotion features, facial expression features, and facial image coding features thereof are annotated; the training set is input into the stress detection model for training, wherein the data samples first enter the feature extraction layer to extract features of four dimensions, and then reach the feature fusion layer to fuse the features, and finally the fused features enter the result prediction layer to detect stress. This application constructs and trains a model that integrates multi-dimensional facial features for stress detection. The integration of multi-dimensional facial features can provide more comprehensive and accurate stress detection results. Features of different dimensions complement each other and can more effectively capture subtle facial changes. The detection results are highly accurate. Compared with the stress detection method that relies on special equipment in the prior art, facial image analysis does not require additional hardware support and only requires a common camera, which greatly reduces the detection cost, improves the efficiency of the detection results, improves operability, and facilitates user use. It is also highly practical and has a wide range of usage scenarios, such as monitoring fatigue driving, health monitoring, psychological assessment, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 A schematic diagram showing a flow chart of a method for training a pressure detection model according to an embodiment of the present application is shown;

[0019] Figure 2 A schematic diagram of the structure of a pressure detection model according to an embodiment of the present application is shown;

[0020] Figure 3 A schematic diagram of the structure of a training device for a pressure detection model according to an embodiment of the present application is shown;

[0021] Figure 4 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0023] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0024] In many scenarios of life, people's stress needs to be monitored, such as the status of truck drivers and bus drivers to see if they are under too much stress. In the process of mental health diagnosis and treatment, stress testing can also be used to judge the patient's mental health or mental state.

[0025] In the prior art, stress detection methods usually require heart rate monitors, galvanic skin response sensors and other equipment to implement, which has the problems of poor convenience and operability.

[0026] In this regard, the present application proposes a method for training a pressure detection model, and the trained pressure detection model can be used to conveniently, quickly and accurately perform pressure detection on a detection object. Figure 1 A schematic diagram of a training process of a pressure detection model according to an embodiment of the present application is shown. Figure 1 It can be seen that the present application includes steps S110 to S130:

[0027] Step S110, constructing a stress detection model, the stress detection model includes a feature extraction layer, a feature fusion layer and a result prediction layer connected in sequence; wherein the feature extraction layer includes: an action unit feature recognition network, an emotion feature recognition network, an expression feature recognition network, and a latent feature recognition network.

[0028] This application constructs a pressure detection model. Figure 2 A schematic diagram of the structure of a pressure detection model according to an embodiment of the present application is shown. Figure 2 It can be seen that the stress detection model 200 includes: a feature extraction layer 210, a feature fusion layer 220 and a result prediction layer 230, the feature extraction layer 210 is used as an input layer, and the result prediction layer 230 is used as an output layer, wherein the feature extraction layer 210 includes four sub-layers, namely, an action unit feature recognition network 211, an emotion feature recognition network 212, an expression feature recognition network 213, and an implicit feature recognition network 214. The four sub-layers are independent of each other.

[0029] Human stress can be manifested in many aspects, such as human body movements, facial expressions, emotional changes, and behavioral manifestations, etc. In order to comprehensively extract the characteristics of various aspects of stress expression, the present application constructs a feature extraction layer 210 for multi-dimensional feature extraction. The feature extraction layer 210 is mainly responsible for extracting multi-dimensional facial features related to detecting stress from the facial image of the identified object. It mainly includes four independent and isolated sub-layers, namely, an action unit feature recognition network 211, an emotion feature recognition network 212, an expression feature recognition network 213, and an implicit feature recognition network 214. The sub-layers can be constructed using any deep learning model. Preferably, in some embodiments of the present application, the action unit feature recognition network, the emotion feature recognition network, the expression feature recognition network, and the implicit feature recognition network are constructed based on a convolutional neural network CNN.

[0030] The action unit feature recognition network 211 is mainly set up for facial action unit features. The facial coding system FACS (Facial Action Coding System) divides facial action features into a number of facial action units AU (Action Units) that are both independent and interconnected according to the anatomical characteristics of the human face. These facial action units AU are used to describe facial actions of the human face, such as cheek lifting, eyebrow lowering, eyelid lifting, mouth corners down, mouth corners up, etc. The action unit feature recognition network 211 can extract these features very well. Its input is a facial image of a person, and its output is any one of the labels such as cheek lifting, eyebrow lowering, eyelid lifting, mouth corners down, mouth corners up, etc., or the confidence value (probability value) of the label. This application does not limit the action unit feature labels, which can be increased or decreased according to the business scenario.

[0031] People's bad moods often come from stress, so people's facial emotional features can also reflect the psychological stress that people bear, such as happiness, sadness, anger, and fear. The emotional feature recognition network 212 is mainly used to extract facial emotional features. Its input is a facial image of a person. The emotional feature recognition network 212 can perform emotional recognition on the facial image, and its output is any one of the labels such as happiness, sadness, anger, fear, etc., or the confidence value of the label. This application does not limit the emotional feature labels, which can be increased or decreased according to the business scenario.

[0032] When people are under stress, their faces often show some micro-expressions and changes in complexion, such as dark circles, wrinkles, nasolabial folds, and pale skin. The expression feature recognition network 213 is mainly used to extract facial expression features. Its input is a facial image of a person, and its output is any one of the labels such as dark circles, wrinkles, nasolabial folds, and pale skin, or the confidence value of the label; this application does not limit the expression feature labels, and they can be increased or decreased according to the business scenario.

[0033] Facial latent features (also called facial image coding features) can be understood as features that describe the content of an image when the image is encoded, and can be understood as a high-level feature. In this application, they are automatically learned and extracted by the latent feature recognition network 214, and can capture more abstract and complex image information.

[0034] In some embodiments, the feature fusion layer is constructed based on an auto-encoder. In this application, a convolutional auto-encoder is mainly used, which is mainly suitable for processing image data. The auto-encoder learns the implicit features of the data by compressing the input data into a smaller representation (encoding) and reconstructing it back to the original data. After the four features of the above features pass through the feature fusion layer, a fused feature vector with richer and tighter features is obtained.

[0035] In some embodiments, the result prediction layer is constructed based on a deep neural network DNN, which provides a powerful solution to complex problems and has the capabilities of automatic feature learning, large-scale data processing, transfer learning, etc. In some embodiments of the present application, feature extraction uses convolutional neural network CNN and result prediction uses deep learning network DNN technology. The combination of the two can automatically extract valuable features from large-scale data, reduce manual intervention, and improve the accuracy and efficiency of detection.

[0036] Step S120, constructing a training set, wherein each data sample in the training set is a facial image of the recognition object, and the data sample includes annotations of facial action unit features, facial emotion features, facial expression features, and facial image coding features.

[0037] Construct a training set, including multiple data samples in the training set. In principle, the more data samples, the better. The data sample is the facial image of the identification object. In actual scenarios, the identification object is usually a natural person, and the data sample is a facial image of a person. Multi-dimensional features are annotated for a data sample, i.e., a facial image, including: facial action unit features, facial emotion features, facial expression features, and facial image encoding features. This application does not limit the annotation method. This application recommends the use of semi-automatic annotation methods, such as template-based annotation: using standardized templates or rules to assist in annotation, which is suitable for data with certain structures and regularities; another example is machine learning assisted: using a pre-trained model to preliminarily annotate the data, and then reviewing and modifying it by humans to improve the efficiency and accuracy of annotation. Semi-automatic annotation can not only ensure the efficiency of annotation, but also ensure the accuracy of annotation.

[0038] For facial action unit features, the Facial Action Coding System (FACS) divides the face into several independent yet interrelated facial action units (AUs) based on the anatomical characteristics of the human face, and uses these facial action units (AUs) to describe facial movements, such as cheek lifting, eyebrow lowering, eyelid lifting, mouth corners downward, mouth corners upward, etc.

[0039] For facial emotion features, such as happiness, sadness, anger, fear, etc., annotations can be made based on the facial expressions of the person in the facial image.

[0040] For facial features, such as dark circles, wrinkles, nasolabial folds, pale skin, etc., annotations can be made based on the facial condition of the person in the facial image.

[0041] Facial action unit features, facial emotion features, and facial expression features are three dimensions of explicit features related to stress detection; while facial image coding features, which are mainly reflected in the features obtained when encoding facial images of human faces, are implicit features, and can also be called facial image coding features. They can be automatically labeled using the image base model in the prior art.

[0042] By performing the above processing on each face image in the training set, a massive amount of training samples can be obtained.

[0043] Step S130: input the training set into the pressure detection model for training to obtain the pressure detection model.

[0044] Finally, the training set is input into the stress detection model for training to obtain a trained stress detection model.

[0045] The following describes the training process using a data sample as an example.

[0046] During the training process, in some embodiments of the present application, it is recommended to use a combination of supervised and unsupervised methods for training. Specifically, each network of the feature extraction layer is trained in a supervised manner; the feature fusion layer is trained in an unsupervised manner; and the result prediction layer is trained in a supervised manner. Among them, both supervised and unsupervised training methods are a machine learning method, in which supervision refers to the model using labeled input data for learning during the training process. The key to this method is that each training sample has a corresponding target output (label), and the model adjusts its own parameters by comparing the difference between its predicted results and the actual labels, so that the prediction gradually approaches the true value. Unsupervised means that the model learns on unlabeled training data. Unlike supervised learning, unsupervised learning does not have a clear target output, and the goal of the model is to discover potential structures or patterns from the data. The present application combines supervised learning and unsupervised learning to give full play to the advantages of both, thereby improving the performance and generalization ability of the model.

[0047] A data sample is input into the stress detection model. The labeled data sample first enters the feature extraction layer 210, and specifically enters the action unit feature recognition network 211, the emotion feature recognition network 212, the expression feature recognition network 213, and the implicit feature recognition network 214.

[0048] For the action unit feature recognition network 211, the action unit feature recognition network extracts features from the data sample to obtain at least one action unit feature. Specifically, it includes: the action unit feature recognition network extracts features from the data sample to obtain the probability value of each action unit appearing in the data sample, and the action unit includes: cheek lifting, eyebrow lowering, eyelid lifting, mouth corner down, and mouth corner up; the probability value of each action unit appearing in the data sample is constructed as a feature vector as the action unit feature.

[0049] That is, the input of the action unit feature recognition network 211 is the facial image in the data sample, and the output is the multiple action units recognized. Usually, multiple action units can be recognized simultaneously from a facial image. The specific output is the recognition probability value corresponding to each facial action unit, and the recognition probability values ​​corresponding to each facial action unit are constructed into a feature vector, which is used as the facial action unit feature for subsequent pressure detection.

[0050] For the emotion feature recognition network 212, the emotion feature recognition network performs emotion recognition on the data sample to obtain facial emotion features. Specifically, the emotion feature recognition network performs emotion recognition on the data sample to obtain probability values ​​of each emotion label to which the data sample belongs, and the emotion labels include: happiness, sadness, anger, and fear; the probability values ​​of each emotion label to which the data sample belongs are constructed into a feature vector as the facial emotion feature.

[0051] The emotion feature recognition network can identify emotion labels from the user's facial pictures, including but not limited to happiness, sadness, anger, fear, etc. The emotion feature recognition network can specifically identify the probability value (i.e., confidence) corresponding to each emotion label, construct the probability value corresponding to each emotion label into a feature vector, and use the feature vector as a facial emotion feature for subsequent stress detection.

[0052] For the expression feature recognition network 213, the expression feature recognition network is mainly used to extract features from the data sample to obtain at least one facial expression feature. Specifically, the expression feature recognition network performs facial expression recognition on the data sample to obtain the probability value of each facial expression appearing in the data sample, and the facial expression includes: dark circles, Sichuan-shaped wrinkles, nasolabial folds, and pale skin; the probability value of each facial expression appearing in the data sample is built as a feature vector as the facial expression feature. The input of the expression feature recognition network is the facial image in the data sample, and the output is the probability value of one or more recognized facial expressions, and the facial expressions include but are not limited to dark circles, Sichuan-shaped wrinkles, nasolabial folds, and pale skin, etc. The probability value of each facial expression appearing is built as a feature vector, and the feature vector is used as the facial expression feature.

[0053] The hidden feature recognition network 214 is mainly used to extract features and obtain at least one hidden facial feature. Hidden facial features usually refer to details or features that are not easily observed directly in facial images. These features may include subtle changes in expression, slight differences in skin color, light and shadow effects, and some facial landmarks that may be obscured or blurred. The input of the hidden feature recognition network 214 is a facial image of a human face, and the output of the hidden feature recognition network 214 is a facial image encoding vector, which is a facial hidden feature used for subsequent pressure detection.

[0054] After feature extraction, at least one action unit feature, facial emotion feature, at least one facial expression feature, and at least one facial latent feature are obtained. These features enter the feature fusion layer 220, and the feature fusion layer performs preliminary fusion, dimensionality reduction, and dimensionality increase processing on these features in turn to obtain a feature fusion vector.

[0055] The input and output of the feature fusion layer 230 are vectors of the same dimension. It is recommended that the feature fusion layer 230 perform model training using an unsupervised method. Specifically, the feature fusion layer 230 first reduces the high-dimensional sparse features obtained by the preliminary fusion to low-dimensional compact features, and then upgrades the low-dimensional compact features to the same high-dimensional sparse features.

[0056] More specifically, preliminary fusion can be understood as simply "placing" these feature vectors together. For example, assuming that the first vector is vector (a, b, c) and the second vector is vector (d, e), when concatenating, according to the original order of the elements in the second vector, it can be "placed" after the last element of the first vector, recorded as vector z, and vector z is (a, b, c, d, e).

[0057] Assume that the first vector and the second vector are isomorphic matrices, and assume that the first vector and the second vector are both 2*2 two-dimensional matrices, which are expressed as:

[0058]

[0059] Specify vertical splicing. When splicing, the elements in the second vector are placed after the last row of the first vector in their original order. This is recorded as vector z. Vector z is a 2*4 matrix, specifically:

[0060]

[0061] If the first semantic vector and the second semantic vector are heterogeneous matrices, the missing elements can be set to empty or zero.

[0062] After the initial fusion, a high-dimensional sparse feature vector is obtained, which is recorded as a high-dimensional sparse feature. A high-dimensional sparse feature vector refers to a situation in which the dimension of the feature vector is very large in a high-dimensional space, but most of the eigenvalues ​​are zero or close to zero.

[0063] Then, the high-dimensional sparse features are subjected to dimensionality reduction. Dimensionality reduction can reduce computational complexity, eliminate noise, improve the interpretability of the model, alleviate overfitting, reveal data structure, etc. In general, dimensionality reduction helps improve the overall performance and efficiency of the model, and also improves the feasibility of data analysis. After dimensionality reduction, the obtained feature vector is recorded as a low-dimensional compact feature.

[0064] Then, the dimensionality is increased again. The dimensionality increase can enhance the feature expression ability, introduce interactive features, and increase the complexity of the model. After the dimensionality increase, the feature vector obtained is the feature fusion vector. Through the dimensionality reduction and dimensionality increase processing, not only the noise and useless information of the original fusion vector are removed, but also the interactive features are introduced and the expression of the fusion vector is enriched, making the prediction result more accurate.

[0065] The dimensionality reduction and dimensionality increase processing of the above feature vectors can be performed by any of the existing technologies, and this application does not limit this. This application recommends using an autoencoder for this.

[0066] Finally, the feature fusion vector is input to the result prediction layer 240, and the result prediction layer 240 outputs the stress level label. In some embodiments, the stress level label is set to four categories: no stress, slight stress, medium stress, and high stress. The stress level label can be set according to business needs, and this application does not limit this.

[0067] During the training process, back propagation is performed based on the loss function of each training round, and the relevant parameters of the pressure detection model 200 are adjusted until the training requirements are met, such as reaching the number of training times or the loss function value is less than a preset threshold.

[0068] After training, a trained stress detection model is obtained. During the application process, the data flow is the same as the training process. The input is a user's facial picture. The facial picture is input into the stress detection model and enters the action unit feature recognition network, the emotion feature recognition network, the expression feature recognition network, and the latent feature recognition network for feature extraction to obtain facial action unit features, facial emotion features, facial expression features, and facial latent features. These features enter the feature fusion layer to obtain the feature fusion vector, and the feature fusion vector enters the result prediction layer to obtain the final recognition result, i.e., the stress label.

[0069] The trained stress detection model can be applied to stress detection of people in various scenarios, such as student mental health assessment, online diagnosis and treatment, fatigue driving detection, etc. The stress detection result can be used as a single indicator or combined with other indicators, which is highly practical.

[0070] Depend on Figure 1It can be seen from the method shown that the present application constructs a stress detection model, which is based on multi-dimensional features for identification. The stress detection model includes a feature extraction layer, a feature fusion layer and a result prediction layer connected in sequence, the feature extraction layer serves as an input layer, and the result prediction layer serves as an output layer, wherein the feature extraction layer includes: an action unit feature recognition network, an emotion feature recognition network, an expression feature recognition network, and an implicit feature recognition network; a training set is constructed, wherein the training set includes a large number of data samples, each data sample is a facial image of an identification object, and the facial action unit features, facial emotion features, facial expression features, and facial image coding features therein are annotated; the training set is input into the stress detection model for training, the data samples first enter the feature extraction layer, extract features of four dimensions in total, and then reach the feature fusion layer for feature fusion, and finally the fused features enter the result prediction layer for stress detection. This application constructs and trains a model that integrates multi-dimensional facial features for stress detection. Compared with the stress detection method in the prior art that relies on special equipment, facial image analysis does not require additional hardware support and only requires a common camera, which greatly reduces the detection cost, and the detection results are accurate and efficient; it improves operability and is convenient for users to use; and it is highly practical and has a wide range of usage scenarios, such as monitoring fatigue driving, health monitoring, psychological assessment, etc.

[0071] Figure 3 A schematic diagram of the structure of a training device for a pressure detection model according to an embodiment of the present application is shown. Figure 3 It can be seen that the training device 300 of the pressure detection model includes:

[0072] A first construction unit 310 is used to construct a stress detection model, wherein the stress detection model includes a feature extraction layer, a feature fusion layer, and a result prediction layer connected in sequence; wherein the feature extraction layer includes: an action unit feature recognition network, an emotion feature recognition network, an expression feature recognition network, and a latent feature recognition network;

[0073] A second construction unit 320 is used to construct a training set, wherein each data sample in the training set is a facial image of an identification object, and the data sample includes annotations of facial action unit features, facial emotion features, facial expression features, and facial image coding features;

[0074] The training unit 330 is used to input the training set into the pressure detection model for training to obtain a trained pressure detection model.

[0075] In some embodiments, in the above-mentioned device, the action unit feature recognition network, the emotion feature recognition network, the expression feature recognition network, and the latent feature recognition network are constructed based on convolutional neural networks; the feature fusion layer is constructed based on autoencoders; and the result prediction layer is constructed based on deep neural networks.

[0076] In some embodiments, in the above-mentioned device, the training unit 330 is used to train the networks of the feature extraction layer in a supervised manner during the training process; to train the feature fusion layer in an unsupervised manner; and to train the result prediction layer in a supervised manner.

[0077] In some embodiments, in the above-mentioned device, the training unit 330 is used to input a data sample into the feature extraction layer, the action unit feature recognition network performs feature extraction on the data sample to obtain at least one action unit feature, the emotion feature recognition network performs emotion recognition on the data sample to obtain facial emotion features, the expression feature recognition network performs feature extraction on the data sample to obtain at least one facial expression feature, and the latent feature recognition network performs feature extraction to obtain at least one facial latent feature; the feature fusion layer sequentially fuses, reduces dimension, and increases dimension on the at least one action unit feature, the facial emotion feature, the at least one facial expression feature, and the at least one facial latent feature to obtain a feature fusion vector; the result prediction layer identifies the feature fusion vector to obtain a stress level label, and the stress level label is any one of no stress, slight stress, moderate stress, and high stress.

[0078] In some embodiments, in the above-mentioned device, the training unit 330 is used for the action unit feature recognition network to perform feature extraction on the data sample to obtain the probability value of each action unit appearing in the data sample, and the action unit includes: cheek lifting, eyebrow drooping, eyelid lifting, downward corner of the mouth, and upward corner of the mouth; the probability value of each action unit appearing in the data sample is constructed into a feature vector as the action unit feature; and the emotion feature recognition network is used to perform emotion recognition on the data sample to obtain the probability value of each emotion label belonging to the data sample, and the emotion labels include: happiness, sadness, anger, and fear; the probability value of each emotion label belonging to the data sample is constructed into a feature vector as the facial emotion feature.

[0079] In some embodiments, in the above-mentioned device, the training unit 330 is used for the expression feature recognition network to perform facial expression recognition on the data sample to obtain the probability value of each facial expression appearing in the data sample, and the facial expression includes: dark circles, wrinkles, nasolabial folds, and pale skin; the probability value of each facial expression appearing in the data sample is constructed as a feature vector as the facial expression feature.

[0080] In some embodiments, in the above-mentioned device, the training unit 330 is used for the feature fusion layer to preliminarily splice the at least one action unit feature, the facial emotion feature, the at least one facial expression feature, and the at least one facial latent feature to form a high-dimensional sparse feature; the high-dimensional sparse feature is processed in a low-dimensional manner to obtain a low-dimensional compact feature; the low-dimensional compact feature is processed in a high-dimensional manner to obtain the feature fusion vector.

[0081] It can be understood that the training device for the above-mentioned pressure detection model can implement the various steps of the training method for the pressure detection model provided in the aforementioned embodiments. The relevant explanations on the training method for the pressure detection model are applicable to the training device for the pressure detection model and will not be repeated here.

[0082] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 4 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0083] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0084] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0085] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a training device for the pressure detection model at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0086] Constructing a stress detection model, the stress detection model includes a feature extraction layer, a feature fusion layer, and a result prediction layer connected in sequence; wherein the feature extraction layer includes: an action unit feature recognition network, an emotion feature recognition network, an expression feature recognition network, and a latent feature recognition network;

[0087] Constructing a training set, wherein each data sample in the training set is a facial image of an identification object, and the data sample includes annotations of facial action unit features, facial emotion features, facial expression features, and facial image coding features;

[0088] The training set is input into the pressure detection model for training to obtain the pressure detection model.

[0089] The above application Figure 3The method performed by the training device of the pressure detection model disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0090] The electronic device may also perform Figure 3 The method for executing the training device of the pressure detection model in the present invention is implemented, and the training device of the pressure detection model is implemented in Figure 3 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.

[0091] The present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 3 The method performed by the training device of the pressure detection model in the illustrated embodiment is specifically used to perform:

[0092] Constructing a stress detection model, the stress detection model includes a feature extraction layer, a feature fusion layer, and a result prediction layer connected in sequence; wherein the feature extraction layer includes: an action unit feature recognition network, an emotion feature recognition network, an expression feature recognition network, and a latent feature recognition network;

[0093] Constructing a training set, wherein each data sample in the training set is a facial image of an identification object, and the data sample includes annotations of facial action unit features, facial emotion features, facial expression features, and facial image coding features;

[0094] The training set is input into the pressure detection model for training to obtain the pressure detection model.

[0095] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0096] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0097] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0098] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0099] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0100] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0101] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0102] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0103] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0104] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A training method for a pressure detection model, characterized in that: include: Constructing a stress detection model, the stress detection model includes a feature extraction layer, a feature fusion layer, and a result prediction layer connected in sequence; wherein the feature extraction layer includes: an action unit feature recognition network, an emotion feature recognition network, an expression feature recognition network, and a latent feature recognition network; Constructing a training set, wherein each data sample in the training set is a facial image of an identification object, and the data sample includes annotations of facial action unit features, facial emotion features, facial expression features, and facial image coding features; The training set is input into the pressure detection model for training to obtain the pressure detection model.

2. The method according to claim 1, characterized in that The action unit feature recognition network, the emotion feature recognition network, the performance feature recognition network, and the latent feature recognition network are constructed based on convolutional neural networks; The feature fusion layer is constructed based on the autoencoder; The result prediction layer is constructed based on a deep neural network.

3. The method according to claim 1, characterized in that The step of inputting the training set into the pressure detection model for training includes: During the training process, each network of the feature extraction layer is trained in a supervised manner; The feature fusion layer is trained in an unsupervised manner; The result prediction layer is trained in a supervised manner.

4. The method according to claim 3, characterized in that The step of inputting the training set into the pressure detection model for training includes: For a data sample, the data sample is input into the feature extraction layer, the action unit feature recognition network performs feature extraction on the data sample to obtain at least one action unit feature, the emotion feature recognition network performs emotion recognition on the data sample to obtain facial emotion features, the expression feature recognition network performs feature extraction on the data sample to obtain at least one facial expression feature, and the implicit feature recognition network performs feature extraction on the data sample to obtain at least one facial implicit feature; The feature fusion layer sequentially fuses, reduces, and increases the dimension of the at least one action unit feature, the facial emotion feature, the at least one facial expression feature, and the at least one facial latent feature to obtain a feature fusion vector; The result prediction layer identifies the feature fusion vector to obtain a stress level label, where the stress level label is any one of no stress, slight stress, medium stress, and high stress.

5. The method according to claim 4, characterized in that The action unit feature recognition network extracts features from the data sample to obtain at least one action unit feature, including: The action unit feature recognition network extracts features from the data sample to obtain a probability value of each action unit appearing in the data sample, wherein the action units include: cheek lifting, eyebrow lowering, eyelid lifting, mouth corner downward, and mouth corner upward; Constructing the probability value of each action unit appearing in the data sample into a feature vector as the action unit feature; The emotion feature recognition network performs emotion recognition on the data sample to obtain facial emotion features, including: The emotion feature recognition network performs emotion recognition on the data sample to obtain probability values ​​of each emotion label to which the data sample belongs, wherein the emotion labels include: happiness, sadness, anger, and fear; The probability values ​​of each emotion label to which the data sample belongs are constructed into a feature vector as the facial emotion feature.

6. The method according to claim 4, characterized in that The facial expression feature recognition network extracts features from the data sample to obtain at least one facial expression feature, including: The facial expression recognition network performs facial expression recognition on the data sample to obtain a probability value of each facial expression appearing in the data sample, wherein the facial expressions include: dark circles, wrinkles, nasolabial folds, and pale skin; The probability value of each facial expression appearing in the data sample is constructed as a feature vector as the facial expression feature.

7. The method according to claim 4, characterized in that The feature fusion layer sequentially performs preliminary fusion, dimension reduction, and dimension increase processing on the at least one action unit feature, the facial emotion feature, the at least one facial expression feature, and the at least one facial latent feature to obtain a feature fusion vector, including: The feature fusion layer preliminarily splices the at least one action unit feature, the facial emotion feature, the at least one facial expression feature, and the at least one facial latent feature to form a high-dimensional sparse feature; The high-dimensional sparse features are processed into low-dimensional features to obtain low-dimensional dense features; The low-dimensional compact features are processed into high dimensions to obtain the feature fusion vector.

8. A training device for a pressure detection model, characterized in that: The device comprises: A first construction unit is used to construct a stress detection model, wherein the stress detection model includes a feature extraction layer, a feature fusion layer, and a result prediction layer connected in sequence; wherein the feature extraction layer includes: an action unit feature recognition network, an emotion feature recognition network, an expression feature recognition network, and a latent feature recognition network; A second construction unit is used to construct a training set, wherein each data sample in the training set is a facial image of an identification object, and the data sample includes annotations of facial action unit features, facial emotion features, facial expression features, and facial image coding features; The training unit is used to input the training set into the pressure detection model for training to obtain a trained pressure detection model.

9. An electronic device, comprising: processor; as well as A memory arranged to store computer executable instructions, characterized in that when the executable instructions are executed, the processor executes the training method of the pressure detection model described in any one of claims 1 to 7.

10. A computer-readable storage medium storing one or more programs, characterized in that: When the one or more programs are executed by an electronic device including a plurality of application programs, the electronic device executes the training method of the pressure detection model described in any one of claims 1 to 7.

Images