Facial pain recognition method and device, computer device, and storage medium

Abstract

The present application relates to the technical field of image recognition, and discloses a facial pain recognition method, device, equipment and medium. The method comprises the following steps: obtaining a facial pain image sample; inputting the facial pain image sample into a facial pain image labeling module to calculate a first loss value; inputting the facial pain image sample into a pain clue mining module to calculate a second loss value; inputting the facial pain image sample into a facial pain recognition module for training to calculate a third loss value; calculating a total loss value according to the first loss value, the second loss value and the third loss value; when the total loss value meets a preset training end condition, a trained facial pain recognition module is determined; wherein the trained facial pain recognition module is used for facial pain recognition of a target facial pain image. In the present application, the facial pain image labeling module and the pain clue mining module are used to assist the training of the facial pain recognition module, so as to improve the generalization of the facial pain recognition module.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, and in particular to a facial pain recognition method and device, a computer device and a storage medium. BACKGROUND

[0002] With the continuous development of science and technology, facial pain recognition technology as a comprehensive image recognition technology has been widely used in many technical fields.

[0003] However, due to the training images used by the facial pain recognition model during training, there are factors such as poor image quality, coarse annotation granularity, and difficulty in extracting clues, which makes the facial pain recognition model obtained by training have poor generalization.

[0004] For example, taking coarse annotation granularity as an example, the annotation method mainly based on discrete labels cannot reflect the real and subtle facial expression changes, and is prone to the problem of mismatch between annotation information and expression details.

[0005] Again, due to the lack of obvious facial pain features, it is often difficult to give a clear definition from the outline or color, making it difficult for traditional image classification algorithms to accurately model it.

[0006] Therefore, there is an urgent need for a facial pain recognition method to improve the generalization of the facial pain recognition model. SUMMARY

[0007] The embodiments of the present application provide a facial pain recognition method, device, computer device and storage medium to solve the problem of poor generalization of the facial pain recognition model.

[0008] A facial pain recognition method, the method comprising:

[0009] obtaining a facial pain image sample;

[0010] inputting the facial pain image sample into a facial pain image annotation module to calculate a first loss value;

[0011] inputting the facial pain image sample into a pain clue mining module to calculate a second loss value;

[0012] inputting the facial pain image sample into a facial pain recognition module for training to calculate a third loss value;

[0013] According to the first loss value, the second loss value and the third loss value, a total loss value is calculated;

[0014] when the total loss value meets a preset training end condition, a trained facial pain recognition module is determined;

[0015] The trained facial pain recognition module is used for facial pain recognition on a target facial pain image.

[0016] The method can further include:

[0017] obtaining a low-quality facial pain image;

[0018] inputting the low-quality facial pain image into a facial pain image enhancement module to obtain the facial pain image sample.

[0019] The method can further include:

[0020] The total loss value is calculated based on the first loss value, the second loss value and the third loss value.

[0021] The total loss value is calculated based on the first loss value, the second loss value, the third loss value and the fourth loss value.

[0022] The method can further include:

[0023] inputting the facial pain image sample into a facial pain image annotation module to obtain a predicted image annotation vector;

[0024] The activation loss value, the region regression loss value, the identity decoupling loss value and the annotation mask loss value are calculated based on the predicted image annotation vector.

[0025] The first loss value is calculated based on the activation loss value, the region regression loss value, the identity decoupling loss value and the annotation mask loss value.

[0026] The method can further include:

[0027] inputting the facial pain image sample into a pain clue mining module to obtain a clue mining vector and a judgment probability vector;

[0028] The generator loss value and the discriminator loss value are calculated based on the clue mining vector and the judgment probability vector.

[0029] The second loss value is calculated based on the generator loss value and the discriminator loss value.

[0030] The method can further comprise: optionally, inputting the facial pain image sample into a facial pain recognition module for training, calculating a third loss value, and comprising:

[0031] inputting the facial pain image sample into the facial pain recognition module for training, and obtaining a predicted pain category vector;

[0032] inputting the predicted pain category vector and a real pain category vector of the facial pain image sample into a third loss function, and obtaining the third loss value.

[0033] The third loss function can be as follows:

[0034]

[0035] wherein, the predicted pain category vector, the real pain category vector, the weight matrix, the bias vector.

[0036] A facial pain recognition device can comprise:

[0037] a facial pain image sample acquisition unit configured to acquire a facial pain image sample;

[0038] a first loss calculation unit configured to input the facial pain image sample into a facial pain image labeling module, and calculate a first loss value;

[0039] a second loss calculation unit configured to input the facial pain image sample into a pain clue mining module, and calculate a second loss value;

[0040] a third loss calculation unit configured to input the facial pain image sample into a facial pain recognition module for training, and calculate a third loss value;

[0041] a total loss calculation unit configured to calculate a total loss value according to the first loss value, the second loss value and the third loss value;

[0042] a total loss value judgment unit configured to determine a trained facial pain recognition module when the total loss value meets a preset training end condition.

[0043] The trained facial pain recognition module is configured to perform facial pain recognition on a target facial pain image.

[0044] A computer device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements a face pain recognition method as described above when executing the computer program.

[0045] A computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement a face pain recognition method as described above.

[0046] The face pain recognition method, device, computer device and storage medium described above, by inputting the target image sample into the face pain image labeling module, the pain clue mining module and the face pain recognition module respectively, to calculate the corresponding first loss value, the second loss value and the third loss value respectively, and then based on the first loss value, the second loss value and the third loss value, the total loss value is calculated, when the total loss value meets the preset training end condition, the trained face pain recognition module is obtained, and the trained face pain recognition module is used for face pain recognition of the target face pain image. It can be seen that in the present application, the face pain image labeling module and the pain clue mining module are used to assist the training of the face pain recognition module, from the perspective of multi-task fusion, the performance of the face pain recognition module is improved, and the purpose of improving the generalization of the face pain recognition module is achieved. BRIEF DESCRIPTION OF DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained according to these drawings without creative labor for those skilled in the art.

[0048] Figure 1 is a flowchart of a face pain recognition method disclosed by embodiment one of the present application;

[0049] Figure 2 is a partial flowchart of a face pain recognition method disclosed by embodiment one of the present application;

[0050] Figure 3 is a partial flowchart of a face pain recognition method disclosed by embodiment one of the present application;

[0051] Figure 4 is a partial flowchart of a face pain recognition method disclosed by embodiment one of the present application;

[0052] Figure 5 is a partial flowchart of a face pain recognition method disclosed by embodiment one of the present application;

[0053] Figure 6 is a structural schematic diagram of a facial pain recognition device disclosed by embodiment two of the present application;

[0054] Figure 7 is a structural schematic diagram of a computer device disclosed by embodiment three of the present application. DETAILED DESCRIPTION

[0055] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0056] It should be understood that when used in the specification and the appended claims of the present application, the term “comprising” indicates the presence of the described features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0057] It should also be understood that the term “and / or” used in the specification and the appended claims of the present application means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0058] As used in the specification and the appended claims of the present application, the term “if” can be interpreted as “when” or “upon” or “in response to a determination” or “in response to detecting” depending on the context. Similarly, the phrase “if determined” or “if detected [the described condition or event]” can be interpreted as meaning “upon determining” or “in response to determining” or “upon detecting [the described condition or event]” or “in response to detecting [the described condition or event]” depending on the context.

[0059] In addition, in the description of the present application and the appended claims, the terms “first”, “second”, “third”, etc. are only used for differentiation in description, and cannot be understood as indicating or implying relative importance.

[0060] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0061] This invention discloses a facial pain recognition method, apparatus, computer device, and storage medium. The method involves inputting target image samples into a facial pain image annotation module, a pain cue mining module, and a facial pain recognition module to calculate corresponding first, second, and third loss values. Then, based on these first, second, and third loss values, a total loss value is calculated. When the total loss value meets a preset training termination condition, a trained facial pain recognition module is obtained. This trained module is used to perform facial pain recognition on target facial pain images. Therefore, this embodiment uses the facial pain image annotation module and the pain cue mining module to assist in the training of the facial pain recognition module. From a multi-task fusion perspective, this improves the performance of the facial pain recognition module and enhances its generalization ability.

[0062] Example 1

[0063] like Figure 1 The diagram shown is a flowchart of a facial pain recognition method disclosed in Embodiment 1 of the present invention. This method is applicable to electronic devices capable of facial pain recognition from facial images, such as mobile phones, tablets, all-in-one machines, and servers. In this embodiment, the technical solution assists the training of the facial pain recognition module through a facial pain image annotation module and a pain cue mining module. From a multi-task fusion perspective, this improves the performance of the facial pain recognition module, thereby enhancing its generalization ability by enabling facial pain recognition of target facial pain images.

[0064] Specifically, the method in this embodiment may include the following steps:

[0065] S101: Obtain facial pain image samples.

[0066] In a specific implementation, the facial pain image sample in the embodiment can be obtained by an image acquisition device with an image shooting function, which can include a mobile phone, a tablet computer, a camera, a camera, and the like. The facial image shot by the image acquisition device at least includes complete facial information of the person being shot, and a facial image with better image quality is selected as the facial pain image sample. Accordingly, the facial pain image sample can be obtained.

[0067] S102: input the facial pain image sample into a facial pain image labeling module to calculate a first loss value.

[0068] In a specific implementation, the facial pain image labeling module in the embodiment includes a One-Shot region alignment module and a Dual-Head collaborative training module.

[0069] The region alignment module is composed of a 3D vertex estimator component, a manual calibration component, two reversible transformation components, and an ellipse fitting component; and the collaborative training module is composed of a base model, a Dual-Head multi-task module, and an identity decoupling module.

[0070] Accordingly, the facial pain image sample is input into the facial pain image labeling module to obtain a predicted region labeling vector, and then a loss function is calculated based on the predicted region labeling vector and a real region labeling vector of the facial pain image sample to calculate the first loss value.

[0071] S103: input the facial pain image sample into a pain clue mining module to calculate a second loss value.

[0072] In a specific implementation, the pain clue mining module in the embodiment is a conditional GAN (CGAN), which includes a query part and a reference part. The query part is composed of a latent clue encoder CCE and an auxiliary clue classifier ACC, and the reference part is composed of a clue transfer generator CTG and a clue adversarial discriminator CAD. The query part is used to generate a difference feature map based on an input image, and the reference part is used to generate a corresponding weak texture image based on the feature difference map. Based on this, the facial pain image sample is input into the pain clue mining module to generate a predicted pain clue vector of the facial pain image sample, and then a loss is calculated according to the predicted pain clue vector and a real pain clue vector of the facial pain image sample to calculate the second loss value.

[0073] S104: input the facial pain image sample into a facial pain recognition module for training to calculate a third loss value.

[0074] ​In a specific implementation, the facial pain recognition module in the embodiment mainly consists of a deep residual network. After inputting the facial pain image sample into the facial pain recognition module, a predicted pain category vector of the facial pain image sample is generated, and then loss calculation is performed according to the predicted pain category vector and a real pain category vector of the facial pain image sample, to obtain a third loss value.

[0075] S105: According to the first loss value, the second loss value and the third loss value, a total loss value is calculated.

[0076] In a specific implementation, the first loss value, the second loss value and the third loss value can be input into a total loss calculation formula in the embodiment to calculate the total loss value.

[0077] The total loss value calculation formula can be as follows:

[0078]

[0079] wherein, the first loss value is L1, the second loss value is L2, the third loss value is L3, and the total loss value is L.

[0080] S106: When the total loss value meets a preset training end condition, a trained facial pain recognition module is determined.

[0081] The trained facial pain recognition module is used for facial pain recognition of a target facial pain image.

[0082] In a specific implementation, the training end condition in the embodiment can be that the total loss value no longer decreases, and the trained facial pain recognition module is determined. After obtaining the trained facial pain recognition module, a target facial pain image to be subjected to facial pain recognition is input into the facial pain recognition module, and the facial pain category of the target facial pain image can be obtained.

[0083] To sum up, the application discloses a facial pain recognition method, target image samples are respectively input into a facial pain image labeling module, a pain clue mining module and a facial pain recognition module to respectively calculate corresponding first loss values, second loss values and third loss values, then a total loss value is calculated based on the first loss values, the second loss values and the third loss values, when the total loss value meets a preset training end condition, a trained facial pain recognition module is obtained, and the trained facial pain recognition module is used for facial pain recognition of a target facial pain image. It can be seen that in the embodiment, the facial pain image labeling module and the pain clue mining module are used to assist the training of the facial pain recognition module, the performance of the facial pain recognition module is improved from the perspective of multi-task fusion, and the purpose of improving the generalization of the facial pain recognition module can be achieved.

[0084] In a specific implementation based on Figure 1 , step S101 can be implemented by the following steps, as shown in Figure 1 :

[0085] S201: Obtain a low-quality facial pain image.

[0086] In a specific implementation, when the facial pain image is obtained in the embodiment, it can be affected by an image acquisition device or an image acquisition environment, so that the collected facial pain image is a low-quality facial pain image.

[0087] For example, when a relatively poor image acquisition device is used, it is difficult to collect a high-quality facial pain image, and only a low-quality facial pain image can be obtained.

[0088] For another example, the image acquisition environment is dark, so that the collected facial pain image is blurred.

[0089] S202: Input the low-quality facial pain image into a facial pain image enhancement module to obtain a facial pain image sample.

[0090] In a specific implementation, the facial pain image enhancement module in the embodiment includes a data enhancement module and a model training module. The data enhancement module focuses on three degradation modes of low illumination, blurring and low resolution, and introduces combined degradation to improve the diversity of the degradation modes. The model training module is implemented through a deep multi-scale residual pyramid structure, a normalization flow structure and a reference mechanism. Based on this, after the facial pain image sample is input into the facial pain image enhancement module, the facial pain image sample is generated.

[0091] To sum up, in the embodiment, the low-quality facial pain image is subjected to image enhancement by the facial pain image enhancement module, thereby avoiding the problem that the facial pain recognition model has poor training effect due to the low-illumination, blur, defocus and other image problems of the facial pain image sample. In addition, the facial pain image enhancement module in the embodiment can also improve the pain recognition effect of the facial pain recognition model on the target facial pain image.

[0092] Based on Figure 1 In a specific implementation, when the low-quality facial pain image is input into the facial pain image enhancement module to obtain the facial pain image sample, and the corresponding fourth loss value is calculated based on the facial pain image sample, step S105 can be implemented in the following manner:

[0093] The total loss value is calculated according to the first loss value, the second loss value, the third loss value and the fourth loss value.

[0094] In a specific implementation, the first loss value, the second loss value and the third loss value can be input into the total loss calculation formula to calculate the total loss value in the embodiment.

[0095] The total loss calculation formula can be as follows:

[0096]

[0097] wherein, the first loss value is L1, the second loss value is L2, the third loss value is L3, the fourth loss value is L4, and the total loss value is L.

[0098] In addition, the fourth loss value can be obtained in the following manner:

[0099] The low-quality facial pain image is input into the facial pain image enhancement module for image enhancement to obtain a predicted enhanced image vector (i.e., the image vector of the facial pain image sample), forward likelihood loss calculation and backward reconstruction loss calculation are performed based on the predicted enhanced image vector to obtain a forward likelihood loss value and a backward reconstruction loss value, and the forward likelihood loss value and the backward reconstruction loss value are input into a fourth loss calculation formula to obtain the fourth loss value.

[0100] The fourth loss calculation formula can be as follows:

[0101]

[0102] wherein, the forward likelihood loss value is Lf, denotes a backward reconstruction loss value, is a hyperparameter, denotes a fourth loss value.

[0103] It can be seen that in the embodiment, the total loss value is calculated according to the first loss value, the second loss value, the third loss value and the fourth loss value, so as to realize the training of the facial pain image labeling module, the pain clue mining module and the facial pain image enhancement module to assist the training of the facial pain recognition module, and the generalization of the facial pain recognition module can be further improved.

[0104] Based on Figure 1 In a specific implementation of the embodiment, step S102 can be implemented by the following steps, as shown in FIG. 10: Figure 3

[0105] S301: inputting the facial pain image sample into the facial pain image labeling module to obtain a predicted image labeling vector.

[0106] In a specific implementation, the facial pain image labeling module in the embodiment can be a trained facial pain image labeling module, or the facial pain image labeling module in the embodiment can also be an untrained facial pain image labeling module. The facial pain image sample is input into the facial pain image labeling module to obtain a predicted image labeling result.

[0107] S302: calculating an activation loss value, a region regression loss value, an identity decoupling loss value and a labeling mask loss value according to the predicted image labeling vector, respectively.

[0108] In a specific implementation, the activation loss value can be calculated based on the predicted image labeling vector and a real image labeling vector of the facial pain image sample; the region regression loss value can be calculated based on the predicted image labeling vector and the real image labeling vector of the facial pain image sample; the identity decoupling loss value can be calculated based on the predicted image labeling vector and the real image labeling vector of the facial pain image sample; and the labeling mask loss value can be calculated based on the predicted image labeling vector and the real image labeling vector of the facial pain image sample. The real image labeling vector of the facial pain image sample is an image labeling vector obtained by artificial labeling in advance.

[0109] S303: calculating a first loss value based on the activation loss value, the region regression loss value, the identity decoupling loss value and the labeling mask loss value.

[0110] The activation loss value, the region regression loss value, the identity decoupling loss value and the labeling mask loss value are input into a first loss calculation formula to calculate the first loss value.

[0111] ​The first loss calculation formula can be as follows:

[0112]

[0113] wherein, indicates an activation loss value, indicates a region regression loss value, indicates an identity decoupling loss value, indicates a label mask loss value, indicates a first loss value.

[0114] Based on Figure 1 In a specific implementation of the method, step S103 can be implemented by the following steps, as shown in Figure 4

[0115] S401: inputting the facial pain image sample into the pain clue mining module to obtain a clue mining vector and a determination probability vector.

[0116] In a specific implementation, the pain clue mining module in the embodiment can be a trained pain clue mining module, or the pain clue mining module in the embodiment can also be an untrained pain clue mining module. The facial pain image sample is input into the pain clue mining module, and the pain clue mining module extracts a difference feature map through a latent clue encoder and an auxiliary clue classifier on the facial pain image sample, and then generates a weak texture image according to the difference feature map through a clue transfer generator, that is, a predicted clue mining vector is obtained, and then the weak texture image is determined through an adversarial discriminator to obtain a determination probability vector of whether the weak texture image is a required pain clue.

[0117] S402: based on the clue mining vector and the determination probability vector, a generator loss value and a discriminator loss value are respectively calculated.

[0118] In a specific implementation, the generator loss value can be calculated based on the clue mining vector in the embodiment, and the discriminator loss value can be calculated based on the determination probability vector.

[0119] S403: based on the generator loss value and the discriminator loss value, a second loss value is calculated.

[0120] The generator loss value and the discriminator loss value are input into a second loss calculation formula to calculate the second loss value.

[0121] The second loss calculation formula can be as follows:

[0122]

[0123] wherein,​ denotes a generator loss value, denotes a discriminator loss value, denotes a second loss value.

[0124] based on Figure 1 In a specific implementation, step S104 can be implemented by the following steps, as shown in Figure 5

[0125] S501: input the facial pain image sample into the facial pain recognition module for training to obtain a predicted pain category vector;

[0126] In a specific implementation, the facial pain recognition module in this embodiment is mainly composed of a deep residual network. After the facial pain image sample is input into the facial pain recognition module, the facial pain recognition module extracts the latent features of the facial pain image sample, and then generates a predicted pain category vector of the facial pain image sample based on the extracted latent features.

[0127] The predicted pain category vector corresponds to a pain category label. The pain category label can be six basic expressions defined in the Facial Action Coding System (FACS) established by Ekman and Friesen: happiness (Happy), anger (Angry), surprise (Surprise), fear (Fear), disgust (Disgust), and sadness (Sad). That is, happiness, anger, surprise, fear, disgust, and sadness are respectively a pain category label.

[0128] For example, after the facial image is input into the facial pain recognition module, a predicted pain category vector of the facial image is obtained. The pain category label corresponding to the predicted pain category vector is anger. It can be determined that the pain category of the facial pain image is anger.

[0129] S502: input the predicted pain category vector and a real pain category vector of the facial pain image sample into a third loss function to calculate a third loss value.

[0130] The real pain category vector of the facial pain image sample is a pain category vector obtained by artificial labeling in advance.

[0131] Specifically, the third loss function can be as follows:

[0132]

[0133] wherein, denotes a real pain category vector, denotes a weight matrix, ​a deviation vector, a third loss value, a predicted pain category vector.

[0134] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0135] Embodiment two

[0136] As Figure 6 shown, it is a structure schematic diagram of a facial pain recognition device disclosed in embodiment two of the present application, which is suitable for electronic devices capable of facial pain recognition of facial images, such as mobile phones, tablet computers, all-in-one machines, servers and other devices. The specific implementation is as follows:

[0137] In a specific implementation, the facial pain recognition device in the embodiment can include the following units:

[0138] The facial pain image sample acquisition unit 601 is configured to acquire facial pain image samples.

[0139] The first loss calculation unit 602 is configured to input the facial pain image samples into the facial pain image labeling module and calculate a first loss value.

[0140] The second loss calculation unit 603 is configured to input the facial pain image samples into the pain clue mining module and calculate a second loss value.

[0141] The third loss calculation unit 604 is configured to input the facial pain image samples into the facial pain recognition module for training and calculate a third loss value.

[0142] The total loss calculation unit 605 is configured to calculate a total loss value according to the first loss value, the second loss value and the third loss value.

[0143] The total loss value judgment unit 606 is configured to determine a trained facial pain recognition module when the total loss value meets a preset training end condition.

[0144] The trained facial pain recognition module is configured to perform facial pain recognition on a target facial pain image.

[0145] In an implementation manner, the facial pain image sample acquisition unit 601 can be configured to:

[0146] acquire low-quality facial pain images;

[0147] input the low-quality facial pain images into a facial pain image enhancement module to obtain facial pain image samples.

[0148] In an implementation manner, the facial pain image sample is used to calculate a corresponding fourth loss value; and the total loss calculation unit 605 can be specifically configured to:

[0149] The total loss value is calculated according to the first loss value, the second loss value, the third loss value and the fourth loss value.

[0150] In an implementation manner, the first loss calculation unit 602 can be specifically configured to:

[0151] The facial pain image sample is input into the facial pain image labeling module to obtain a predicted image labeling vector;

[0152] The activation loss value, the region regression loss value, the identity decoupling loss value and the labeling mask loss value are respectively calculated according to the predicted image labeling vector;

[0153] The first loss value is calculated based on the activation loss value, the region regression loss value, the identity decoupling loss value and the labeling mask loss value.

[0154] In an implementation manner, the second loss calculation unit 603 can be specifically configured to:

[0155] The facial pain image sample is input into the pain clue mining module to obtain a clue mining vector and a judgment probability vector;

[0156] The generator loss value and the discriminator loss value are respectively calculated based on the clue mining vector and the judgment probability vector;

[0157] The second loss value is calculated based on the generator loss value and the discriminator loss value.

[0158] In an implementation manner, the third loss calculation unit 604 can be specifically configured to:

[0159] The facial pain image sample is input into the facial pain recognition module for training to obtain a predicted pain category vector;

[0160] The predicted pain category vector and a real pain category vector of the facial pain image sample are input into a third loss function to calculate the third loss value.

[0161] In an implementation manner, the third loss function is as follows:

[0162]

[0163] wherein, represents the predicted pain category vector, represents the real pain category vector, represents a weight matrix and, represents a bias vector.

[0164] For specific limitations of the facial pain recognition device, reference can be made to the relevant limitations of the facial pain recognition method in the foregoing, which will not be repeated here. Each module in the facial pain recognition device described above can be implemented in whole or in part by software, hardware, and combinations thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each of the above modules.

[0165] Embodiment Three

[0166] Embodiment three of the present application discloses a computer device, which can be a server, and its internal structure diagram can be as shown in Figure 7 The computer device includes a processor, a memory, a network interface, and a database connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with external terminals through network connection. The computer program is executed by the processor to implement a facial pain recognition method.

[0167] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements the following steps when executing the computer program:

[0168] Obtaining a facial pain image sample;

[0169] Inputting the facial pain image sample into a facial pain image labeling module to calculate a first loss value;

[0170] Inputting the facial pain image sample into a pain clue mining module to calculate a second loss value;

[0171] Inputting the facial pain image sample into a facial pain recognition module for training to calculate a third loss value;

[0172] According to the first loss value, the second loss value, and the third loss value, a total loss value is calculated;

[0173] When the total loss value meets a preset training end condition, a trained facial pain recognition module is determined;

[0174] The trained facial pain recognition module is used for facial pain recognition of a target facial pain image.

[0175] Embodiment Four

[0176] Embodiment Four of the present application discloses a computer readable storage medium, when the instructions in the computer readable storage medium are executed by the processor in the computer device, the computer device can execute each step of any embodiment of the face pain recognition method disclosed by the present application. The computer readable storage medium can be non-volatile or volatile.

[0177] In one embodiment, a computer readable storage medium is provided, and a computer program is stored on the computer readable storage medium. The computer program is executed by the processor to implement the following steps:

[0178] Obtaining a face pain image sample;

[0179] Inputting the face pain image sample into a face pain image labeling module to calculate a first loss value;

[0180] Inputting the face pain image sample into a pain clue mining module to calculate a second loss value;

[0181] Inputting the face pain image sample into a face pain recognition module for training to calculate a third loss value;

[0182] According to the first loss value, the second loss value and the third loss value, a total loss value is calculated;

[0183] When the total loss value meets a preset training end condition, a trained face pain recognition module is determined;

[0184] The trained face pain recognition module is used for face pain recognition of a target face pain image.

[0185] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments of each method. Any reference to memory, storage, database or other medium used in each embodiment provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0186] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0187] The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, but not to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A facial pain recognition method, characterized by, The method comprises: obtaining a facial pain image sample; inputting the facial pain image sample into a facial pain image labeling module to calculate a first loss value; inputting the facial pain image sample into a pain clue mining module to calculate a second loss value; inputting the facial pain image sample into a facial pain recognition module for training to calculate a third loss value; calculating a total loss value according to the first loss value, the second loss value and the third loss value; when the total loss value meets a preset training end condition, determining a trained facial pain recognition module; wherein the trained facial pain recognition module is used for facial pain recognition of a target facial pain image.

2. The facial pain recognition method of claim 1, wherein, The obtaining of the facial pain image sample comprises: obtaining a low-quality facial pain image; inputting the low-quality facial pain image into a facial pain image enhancement module to obtain the facial pain image sample.

3. The facial pain recognition method of claim 2, wherein, The facial pain image sample is used to calculate a corresponding fourth loss value; wherein the calculating of the total loss value according to the first loss value, the second loss value and the third loss value comprises: calculating the total loss value according to the first loss value, the second loss value, the third loss value and the fourth loss value.

4. The facial pain recognition method of claim 1, wherein, The inputting of the facial pain image sample into the facial pain image labeling module to calculate the first loss value comprises inputting the facial pain image sample into a facial pain image labeling module to obtain a predicted image labeling vector; calculating an activation loss value, a region regression loss value, an identity decoupling loss value and a labeling mask loss value according to the predicted image labeling vector, respectively; calculating the first loss value based on the activation loss value, the region regression loss value, the identity decoupling loss value and the labeling mask loss value.

5. The facial pain recognition method of claim 1, wherein, The inputting of the facial pain image sample into the pain clue mining module to calculate the second loss value comprises: inputting the facial pain image sample into a pain clue mining module to obtain a clue mining vector and a judgment probability vector; calculating a generator loss value and a discriminator loss value based on the clue mining vector and the judgment probability vector, respectively; calculating the second loss value based on the generator loss value and the discriminator loss value.

6. The facial pain recognition method of claim 1, wherein, The inputting of the facial pain image sample into the facial pain recognition module for training to calculate the third loss value comprises: inputting the facial pain image sample into a facial pain recognition module for training to obtain a predicted pain category vector; inputting the predicted pain category vector and a real pain category vector of the facial pain image sample into a third loss function to obtain the third loss value.

7. The facial pain recognition method of claim 6, wherein, The third loss function is as follows: wherein, denotes a predicted pain category vector, denotes a true pain category vector, denotes a weight matrix, denotes a bias vector.

8. A facial pain recognition apparatus, characterized by, comprises: a facial pain image sample acquisition unit configured to obtain a facial pain image sample; a first loss calculation unit configured to input the facial pain image sample into a facial pain image labeling module to calculate a first loss value; a second loss calculation unit configured to input the facial pain image sample into a pain clue mining module to calculate a second loss value; a third loss calculation unit configured to input the facial pain image sample into a facial pain recognition module for training and calculate a third loss value; a total loss calculation unit configured to calculate a total loss value according to the first loss value, the second loss value and the third loss value; a total loss value judgment unit configured to determine the trained facial pain recognition module when the total loss value meets a preset training end condition. The trained facial pain recognition module is configured to perform facial pain recognition on a target facial pain image.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the facial pain recognition method in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the facial pain recognition method in any one of claims 1 to 7.

Images