A method for identifying postoperative pain in children based on an explainable prototype

Abstract

The application discloses a postoperative children's pain recognition method based on an interpretable prototype, aiming to improve the interpretability of postoperative children's pain detection by comparing the similarity of facial feature positions. First, facial expression frames and sparse optical flow frames of each frame of picture are extracted from postoperative children's pain videos. Then, the feature vector of the facial feature position is extracted by using the ResNet convolution network, and the randomly initialized prototype vector is optimized by minimizing the loss function. Then, the similarity of the feature vector and the prototype vector is calculated, and the pain level recognition result of each frame of picture is obtained through the full connection network. Finally, the facial feature positions corresponding to the feature vector and the facial feature positions corresponding to the prototype vector are visually compared. The application improves the interpretability of the postoperative children's pain recognition process by showing the similarity of the facial feature positions on the facial expression and the sparse optical flow, which helps medical professionals better understand the automatic pain recognition result.

Description

Technical Field

[0001] The present invention relates to the technical field of video processing, and in particular to a method for identifying pain in children after surgery based on an interpretable prototype. Background Art

[0002] The negative impact of postoperative pain in children is widespread. If not promptly identified and addressed, it can develop into chronic pain and long-term pain-related disorders. Therefore, identifying pain in children after surgery is a widespread area of ​​concern for medical experts. With the development of artificial intelligence algorithms, many automated pain recognition methods based on facial expressions have emerged. However, these methods still have some significant drawbacks.

[0003] First, traditional artificial intelligence algorithms directly identify pain results through neural networks such as ResNet convolutional networks and fully connected networks. However, the black-box neural networks lack transparency, which makes pain recognition lack sufficient explainability. Even if a pain prediction result can be given, it cannot provide a basis for judgment. This makes it difficult for medical professionals to fully understand and trust the judgment of the neural network. Secondly, the automatic pain recognition method based only on static facial expressions ignores the temporal information of the previous and next frames and is not sensitive enough to subtle changes in facial expressions, which may be an important indicator of pain severity. Finally, within the same pain level, different children will show different facial expressions. Existing technologies have not effectively integrated this individual difference into the pain recognition process, which directly affects the accuracy and reliability of pain recognition. Summary of the Invention

[0004] The purpose of the present invention is to solve the above-mentioned defects in the prior art and provide a method for identifying postoperative pain in children based on an explainable prototype.

[0005] The purpose of the present invention can be achieved by taking the following technical solutions:

[0006] A method for identifying postoperative pain in children based on an explainable prototype, the method comprising the following steps:

[0007] S1, face detection step, frame extraction is performed on all collected videos of postoperative pain in children, face detection is performed on each frame to obtain facial expression frames, and corresponding sparse optical flow frames are calculated between two consecutive facial expression frames;

[0008] S2, feature vector extraction step, input the facial expression frame and the corresponding sparse optical flow frame into the trained ResNet convolutional network to obtain a facial feature map, and then extract the feature vectors of the facial features from the facial feature map;

[0009] S3, prototype vector optimization step, randomly initialize the prototype vector, first use the feature vectors of all facial features to optimize the prototype vector by minimizing the loss function, then traverse all feature vectors to find the feature vector with the highest similarity to the prototype vector as the final prototype vector;

[0010] S4, a pain recognition step, wherein steps S1 and S2 are executed on a video of a child experiencing postoperative pain to be recognized, to obtain a feature vector for each frame of the video, and similarity is calculated between the feature vector of each frame and the prototype vector. The similarity is then input into a trained fully connected network to obtain a pain level recognition result for each frame;

[0011] S5. Visualization step, showing the positions of facial features corresponding to the feature vector of each frame image and the positions of facial features corresponding to the prototype vector.

[0012] Furthermore, the process of step S1 is as follows:

[0013] S101, extracting frames from all collected videos of postoperative pain in children at time intervals, and obtaining multiple frames of images from each video through frame extraction;

[0014] S102, extract the face area from each frame of the picture using face detection to obtain the facial expression frame Where m and n represent the number of pixels in the frame's length and width, respectively. The channel represents RGB color, which includes three color channels. i = 1, 2, 3, ..., N, represents the i-th frame among all N facial expression frames. Face detection uses the method proposed by Bulat et al. in their IEEE conference paper "How far are we from solving the 2D & 3D face alignment problem?" This method identifies the coordinates of 68 facial landmarks from a 2D image and extracts the facial region.

[0015] S103, between two consecutive facial expression frames, respectively, by the formula:

[0016] I x u+I y v+I t =0

[0017] The optical flow intensity u along the x direction and the optical flow intensity v along the y direction of each pixel in the facial expression frame are calculated, where I x Represents the brightness gradient of each pixel along the x direction, I y Represents the brightness gradient of each pixel along the y direction, I tIt represents the brightness gradient of each pixel between two adjacent frames. The calculation method adopts Lucas Kanade sparse optical flow algorithm, that is, a sliding window is established on the facial expression frame, and the brightness gradient of all pixels in the sliding window along the x direction, along the y direction, and between two adjacent frames is calculated, and then the brightness gradient is calculated by formula I. x u+I y v+I t = 0 to construct the equation group, and use the least squares method to solve the optical flow intensity u along the x direction and the optical flow intensity v along the y direction for each pixel in the sliding window. Figure 3 As shown, the direction of the arrow on the sparse optical flow frame indicates the direction of the optical flow, and the length of the arrow indicates the size of the optical flow.

[0018] The optical flow intensity u along the x direction and the optical flow intensity v along the y direction of all pixels in the i-th facial expression frame are merged in the last dimension to obtain a sparse optical flow frame The sparse optical flow frames record the subtle dynamic changes in facial expressions, allowing the ResNet convolutional network to learn the temporal information of the previous and next frames.

[0019] Furthermore, the process of step S2 is as follows:

[0020] S201, facial expression frame and the corresponding sparse optical flow frames Merge in the last dimension to get the merged frame X in the i-th facial expression frame i ∈R m×n×5 Taking the merged frames as input, the ResNet convolutional network not only learns static facial expressions, but also learns subtle dynamic changes in facial expressions through optical flow information;

[0021] S202, put X i Input into the trained ResNet convolutional network to obtain the facial feature map M∈R m′×n′×p , where m' and n' represent the number of pixels in the length and width of the facial feature map respectively, and p represents the channel dimension of the facial feature map. The structure of the ResNet convolutional network is as follows:

[0022] The connections from the input layer to the output layer are:

[0023] 1 2D convolutional layer, 1 BN layer, 1 ReLU layer, 1 2D maximum pooling layer, 8 residual blocks, 1 2D average pooling layer; in the structure of each residual block, the connections are: 1 2D convolutional layer, 1 BN layer, 1 ReLU layer, 1 2D convolutional layer, 1 BN layer, 1 ReLU layer;

[0024] S203. Extract all facial key points from X through face detection, then obtain K facial feature locations from all facial key points, and divide them into K facial key point sets. Based on the commonly used facial coding action units for pain recognition proposed by Mascia et al. in the 1981 paper "Clinical pain measures and personality variables in chronic pain patients," the present invention extracts the four facial features most relevant to pain expression: eyebrows, eyes, nose, and mouth.

[0025] S204. Divide X into m′×n′ regions according to the size of M. The size of each region is Because each area of ​​M is from the corresponding area of ​​X, through the ResNet convolutional network, respectively, according to and Therefore, X can be divided into m′×n′ regions according to the size of M.

[0026] S205, traverse the K sets of facial key points, record the area with the most key points in the k-th set of facial key points in the m′×n′ areas, and m′×n′×p Extract the channel vector of the region and obtain the feature vector V of the kth facial features position k ∈R p , where k = 1, 2, 3, ..., K, represents the kth position of K facial features, and p represents the channel dimension of the facial feature map.

[0027] Furthermore, the process of step S3 is as follows:

[0028] S301, random initialization prototype vector Where p represents the channel dimension of the facial feature map, express The cth category in the pain scale, k = 1, 2, 3, ..., K, represents the kth location of K facial features, and q = 1, 2, 3, ..., Q, represents the qth prototype vector of Q facial features. Multiple prototype vectors are constructed for each facial feature location. The purpose is to learn multiple representative pain expressions through multiple prototype vectors. For example, in the extreme pain scale, the representative pain expression of the eyes may be closed or open.

[0029] S302, through each feature vector V k and each prototype vector Calculate distance:

[0030]

[0031] ‖‖2 represents the norm of 2, and we continue to calculate V k and Similarity:

[0032]

[0033] where the constant ε<10 -5 , allowing in the operation When it is 0;

[0034] S303, calculated Similarity, and then input into the trained fully connected network, we get Prediction score of pain level The structure of the fully connected network is as follows:

[0035] The connections from the input layer to the output layer are:

[0036] neurons, 1 fully connected layer, neurons;

[0037] Through a simple fully connected layer, the weighted coefficient of similarity to pain level is obtained, which can intuitively reflect the importance of similarity to pain recognition;

[0038] S304. In order to achieve the pain level classification task, a cross entropy loss function is constructed:

[0039]

[0040] in is the i-th facial expression frame A category vector in which The index position corresponding to the pain level category is 1, and all other positions are 0;

[0041] At the same time, in the same pain level, in order to increase the eigenvector V k and prototype vector Similarity, construct clustering loss function:

[0042]

[0043] where argmax(Y i ) represents the true pain level category;

[0044] At the same time, in order to reduce the eigenvector V in different pain levels k and prototype vector Similarity, construct separation loss function:

[0045]

[0046] The total loss function is constructed as follows:

[0047]

[0048] Wherein λ1 is the weighting coefficient of the clustering loss function, and λ2 is the weighting coefficient of the separation loss function;

[0049] By minimizing the total loss function, not only the pain level classification task is realized, but also the prototype vector is optimized; the prototype vector is constantly close to the feature vector corresponding to the facial feature position in the process of minimizing the total loss function, and is far away from the feature vector not corresponding to the facial feature position.

[0050] S305, traverse to find all feature vectors V k Neutralize the prototype vector The feature vector with the highest similarity between the prototype vector is taken as the final prototype vector. The purpose is to replace the prototype vector with the feature vector obtained from the real existing sample, which is conducive to displaying the facial feature position corresponding to the prototype vector in the visualization step.

[0051] Further, the step S4 process is as follows:

[0052] S401, first execute step S1 to obtain the facial expression frame and sparse optical flow frame of each frame of picture for a postoperative child pain video to be identified, and then execute step S2 to obtain the feature vector of each frame of picture;

[0053] S402, according to steps S302 and S303, the similarity of the feature vector and the prototype vector is calculated, and then the obtained similarity is input into the trained full connection network to obtain The predicted score ρ of the pain level category;

[0054] S403, the pain level with the highest predicted score is taken as the pain level recognition result of each frame of picture. The prediction of pain recognition is based on the similarity of the feature vector and the prototype vector, which is conducive to displaying and comparing the facial feature position corresponding to the feature vector and the facial feature position corresponding to the prototype vector respectively in the visualization step.

[0055] Further, the step S5 process is as follows:

[0056] S501, display the facial feature position corresponding to the feature vector of each frame of picture;

[0057] S502, traverse to find the prototype vector with the highest similarity to the feature vector, and display the facial feature position corresponding to the prototype vector.

[0058] Another object of the present application is to disclose a postoperative child pain recognition device based on interpretable prototype, for executing the postoperative child pain recognition method described above, comprising the following modules:

[0059] The face detection module extracts frames from all collected postoperative child pain videos, detects faces in each frame to obtain facial expression frames, and calculates corresponding sparse optical flow frames between consecutive two facial expression frames;

[0060] The feature vector extraction module inputs the facial expression frames and the corresponding sparse optical flow frames into the trained ResNet convolutional network to obtain a facial feature map, and then extracts the feature vector of the facial feature position from the facial feature map;

[0061] The prototype vector optimization module randomly initializes the prototype vector, first optimizes the prototype vector by minimizing the loss function using the feature vectors of all facial feature positions, and then traverses to find the feature vector with the highest similarity to the prototype vector as the final prototype vector;

[0062] The pain recognition module calls and executes the face detection module and the feature vector extraction module to obtain the feature vector of each frame of the video, calculates the similarity between the feature vector of each frame and the prototype vector, and then inputs the similarity into the trained fully connected network to obtain the pain level recognition result of each frame.

[0063] The visualization module displays the facial feature position corresponding to the feature vector of each frame and the facial feature position corresponding to the prototype vector.

[0064] The present application has the following advantages and effects compared with the prior art:

[0065] 1. The postoperative child pain recognition method based on the interpretable prototype is proposed, the representative facial feature position is obtained through the construction of the prototype vector, and the similarity between the representative facial feature position and the facial feature position of the real sample is displayed through visualization, so that medical professionals can understand the similarity, therefore, the present application has interpretability, and medical professionals can intuitively understand the decision-making process of postoperative child pain recognition, and more trust the results of pain recognition;

[0066] 2. The present application can provide more accurate and comprehensive pain assessment by analyzing facial expression frames and sparse optical flow frames to identify pain, so that the ResNet convolutional network not only learns static facial expressions, but also learns the subtle dynamic changes of facial expressions such as blinking and frowning through optical flow information, which are key indicators of pain expression, therefore, compared with the method relying only on static images, the present application can provide more accurate and comprehensive pain assessment;

[0067] 3. The present application constructs multiple prototype vectors on each facial feature position, and learns multiple representative pain expression methods through multiple prototype vectors, therefore, the present application considers the individual differences of children in pain expression, thereby improving the accuracy and generalization ability of pain recognition. DETAILED DESCRIPTION

[0068] The accompanying drawings, which are included to provide a further understanding of the present application, constitute a part of this application and illustrate the illustrative embodiments of the present application and their description serve to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0069] Figure 1 is a flow chart of the postoperative child pain recognition method of the present application;

[0070] Figure 2 is a schematic diagram of facial expression frames and sparse optical flow frames in the postoperative child pain recognition method of the present application;

[0071] Figure 3 is a visualization step schematic diagram of the postoperative child pain recognition method of the present application. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical scheme and advantages of the embodiments of the present application more clear, the technical scheme in the embodiments of the present application will be described clearly and completely below with reference to the drawings of the embodiments of the present application, obviously, the described embodiments are part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0073] EMBODIMENT

[0074] This embodiment specifically discloses a method for identifying pain in children after surgery based on an explainable prototype, comprising the following steps:

[0075] S1, face detection step, frame extraction is performed on all collected videos of postoperative pain in children, face detection is performed on each frame to obtain facial expression frames, and corresponding sparse optical flow frames are calculated between two consecutive facial expression frames;

[0076] The specific implementation process of step S1 is as follows:

[0077] S101, extracting frames every 5 frames from 700 videos of postoperative pain in children with a duration of 10 seconds and 30 frames per second, and obtaining 60 RGB images from each video through frame extraction;

[0078] S102, extract the face area from each frame of the picture using face detection to obtain a facial expression frame with a length of 224 pixels and a width of 224 pixels The channel is RGB color including 3 color channels, i = 1, 2, 3, ..., 42000, representing the i-th one in 42000 facial expression frames;

[0079] S103, using the Lucas Kanade sparse optical flow algorithm on two consecutive facial expression frames, that is, establishing a sliding window on the facial expression frame, first calculating the brightness gradient of all pixels in the sliding window along the x direction, the brightness gradient along the y direction, and the brightness gradient between two adjacent frames, and then using formula I x u+I y v+I t =0 to construct a set of equations, and use the least squares method to solve the optical flow intensity u along the x direction and the optical flow intensity v along the y direction for each pixel in the sliding window.

[0080] The optical flow intensity u along the x direction and the optical flow intensity v along the y direction of all pixels in the i-th facial expression frame are merged in the last dimension to obtain a sparse optical flow frame

[0081] S2, feature vector extraction step, input the facial expression frame and the corresponding sparse optical flow frame into the trained ResNet convolutional network to obtain a facial feature map, and then extract the feature vectors of the facial features from the facial feature map;

[0082] The specific implementation process of step S2 is as follows:

[0083] S201, facial expression frame and the corresponding sparse optical flow frames Merge in the last dimension to get the merged frame X∈R in the i-th facial expression frame224×224×5 ;

[0084] S202. Input X into the trained ResNet convolutional network to obtain a facial feature map M∈R with a length of 7 pixels, a width of 7 pixels, and a channel dimension of 128. 7×7×128 , the ResNet convolutional network adopts ResNet18, and its structure is as follows:

[0085] The connections from the input layer to the output layer are:

[0086] 1 2D convolutional layer, 1 BN layer, 1 ReLU layer, 1 2D maximum pooling layer, 8 residual blocks, 1 2D average pooling layer; in the structure of each residual block, the connections are: 1 2D convolutional layer, 1 BN layer, 1 ReLU layer, 1 2D convolutional layer, 1 BN layer, 1 ReLU layer;

[0087] S203, extracting 68 facial key points from X through face detection, and then obtaining the positions of the four facial features most relevant to pain expression, including eyebrows, eyes, nose, and mouth, from all facial key points, and dividing them into four facial key point sets;

[0088] S204, divide X into 7×7 regions according to the size of M, and the size of each region is 32×32;

[0089] S205, traverse the four facial key point sets, record the area with the most key points in the 7×7 area of ​​the k-th facial key point set, and 7×7×128 Extract the channel vector of the region and obtain the feature vector V of the kth facial features position k ∈R 128 , where k = 1, 2, 3, 4, representing the kth position of the facial features of the four people.

[0090] S3, prototype vector optimization step, randomly initialize the prototype vector, first use the feature vectors of all facial features to optimize the prototype vector by minimizing the loss function, then traverse all feature vectors to find the feature vector with the highest similarity to the prototype vector as the final prototype vector;

[0091] The specific implementation process of step S3 is as follows:

[0092] S301, randomly initialize 4×4×3 prototype vectors Where p represents the channel dimension of the facial feature map, c = 1, 2, 3, 4, represents the cth category of the four pain levels, k = 1, 2, 3, 4, represents the kth location of the four facial features, and q = 1, 2, 3, represents the qth location of the three prototype vectors in each facial feature position;

[0093] S302, through each feature vector V k and each prototype vector Calculate distance:

[0094]

[0095] ‖‖2 represents the norm of 2, and we continue to calculate V k and Similarity:

[0096]

[0097] S303. Calculate 4×4 similarities and input them into the trained fully connected network to obtain the predicted scores ρ∈R for the four pain levels. 4 , where the structure of the fully connected network is as follows:

[0098] The connections from the input layer to the output layer are:

[0099] 4×4 neurons, 1 fully connected layer, 4 neurons;

[0100] S304. In order to achieve the pain level classification task, a cross entropy loss function is constructed:

[0101]

[0102] where Y i ∈R 4 is the i-th facial expression frame The category vector of The pain level category is 2, then Y i ={0,1,0,0};

[0103] At the same time, in the same pain level, in order to increase the eigenvector V k and prototype vector Similarity, construct clustering loss function:

[0104]

[0105] where argmax(Y i ) represents the real pain level category, assuming Y i ={0,1,0,0}, then argmax(Y i )=2;

[0106] At the same time, in order to reduce the eigenvector V in different pain levels k and prototype vector Similarity, construct separation loss function:

[0107]

[0108] Construct the total loss function:

[0109]

[0110] By minimizing the total loss function, not only the pain level classification task is achieved, but also the prototype vector is optimized;

[0111] S305, traverse and search for all eigenvectors V k Neutralize prototype vector The feature vector with the highest similarity between them is taken as the final prototype vector.

[0112] S4, a pain recognition step, wherein steps S1 and S2 are executed on a video of a child experiencing postoperative pain to be recognized, to obtain a feature vector for each frame of the video, and similarity is calculated between the feature vector of each frame and the prototype vector. The similarity is then input into a trained fully connected network to obtain a pain level recognition result for each frame;

[0113] The specific implementation process of step S4 is as follows:

[0114] S401: Take a video of a child suffering from postoperative pain and first perform step S1 to obtain the following: Figure 2 The facial expression frame and the sparse optical flow frame of each frame of the picture are shown, and then step S2 is performed to obtain the feature vector of each frame of the picture;

[0115] S402: Calculate similarity between the feature vector and the prototype vector according to steps S302 and S303, and then input the obtained similarity into the trained fully connected network to obtain prediction scores ρ = {10.25, 24.82, 4.66, 3.09} for the four pain levels;

[0116] S403 , the prediction score of the second type of pain level is the highest, so the predicted category of the pain level of the frame image to be identified is 2.

[0117] S5. Visualization step, showing the positions of facial features corresponding to the feature vector of each frame image and the positions of facial features corresponding to the prototype vector.

[0118] The specific implementation process of step S5 is as follows:

[0119] S501, display the facial features corresponding to the feature vector of the frame image to be identified, such as Figure 3 As shown;

[0120] S502, traverse and search for the prototype vector with the highest similarity to the feature vector, such as Figure 3As shown, the position of the facial features corresponding to the prototype vector is displayed. Because the position of the facial features is very similar to that of the frame image to be identified, and the known sample pain level label of the facial features corresponding to the prototype vector is 2, the predicted category of the pain level of the frame image to be identified is 2.

[0121] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A method for identifying pain in children after surgery based on an interpretable prototype, characterized by: The method for identifying pain in children after surgery comprises the following steps: S1, face detection step, frame extraction is performed on all collected videos of postoperative pain in children, face detection is performed on each frame to obtain facial expression frames, and corresponding sparse optical flow frames are calculated between two consecutive facial expression frames; S2, feature vector extraction step, input the facial expression frame and the corresponding sparse optical flow frame into the trained ResNet convolutional network to obtain a facial feature map, and then extract the feature vectors of the facial features from the facial feature map; S3, prototype vector optimization step, randomly initialize the prototype vector, first use the feature vectors of all facial features to optimize the prototype vector by minimizing the loss function, then traverse all feature vectors to find the feature vector with the highest similarity to the prototype vector as the final prototype vector; S4, a pain recognition step, wherein steps S1 and S2 are executed on a video of a child experiencing postoperative pain to be recognized, to obtain a feature vector for each frame of the video, and similarity is calculated between the feature vector of each frame and the prototype vector. The similarity is then input into a trained fully connected network to obtain a pain level recognition result for each frame; S5. Visualization step, showing the positions of facial features corresponding to the feature vector of each frame image and the positions of facial features corresponding to the prototype vector.

2. A method for identifying postoperative pain in children based on an explainable prototype according to claim 1, characterized in that: The process of step S1 is as follows: S101, extracting frames from all collected videos of postoperative pain in children at time intervals, and obtaining multiple frames of images from each video through frame extraction; S102, extract the face area from each frame of the picture using face detection to obtain the facial expression frame Where m and n represent the number of pixels in the length and width of the frame respectively, the channel is RGB color including 3 color channels, i = 1, 2, 3, ..., N, represents the i-th of all N facial expression frames; S103, between two consecutive facial expression frames, respectively, by the formula: I x u+I y v+I t =0 The optical flow intensity u along the x direction and the optical flow intensity v along the y direction of each pixel in the facial expression frame are calculated, where I x Represents the brightness gradient of each pixel along the x direction, I y Represents the brightness gradient of each pixel along the y direction, I t It represents the brightness gradient of each pixel between two adjacent frames; The optical flow intensity u along the x direction and the optical flow intensity v along the y direction of all pixels in the i-th facial expression frame are merged in the last dimension to obtain a sparse optical flow frame 3. A method for identifying postoperative pain in children based on an explainable prototype according to claim 2, characterized in that: The process of step S2 is as follows: S201, facial expression frame and the corresponding sparse optical flow frames Merge in the last dimension to get the merged frame X∈R in the i-th facial expression frame m×n×5 ; S202, input X into the trained ResNet convolutional network to obtain the facial feature map M∈R m′×n′×p , where m' and n' represent the number of pixels in the length and width of the facial feature map respectively, and p represents the channel dimension of the facial feature map. The structure of the ResNet convolutional network is as follows: The connections from the input layer to the output layer are: 1 2D convolutional layer, 1 BN layer, 1 ReLU layer, 1 2D maximum pooling layer, 8 residual blocks, 1 2D average pooling layer; in the structure of each residual block, the connections are: 1 2D convolutional layer, 1 BN layer, 1 ReLU layer, 1 2D convolutional layer, 1 BN layer, 1 ReLU layer; S203, extracting all facial key points from X through face detection, and then obtaining K facial feature positions from all facial key points, and dividing them into K facial key point sets; S204. Divide X into m′×n′ regions according to the size of M. The size of each region is S205, traverse the K sets of facial key points, record the area with the most key points in the k-th set of facial key points in the m′×n′ areas, and m′×n′×p Extract the channel vector of the region and obtain the feature vector V of the kth facial features position k ∈R p , where k = 1, 2, 3, ..., K, represents the kth position of K facial features, and p represents the channel dimension of the facial feature map.

4. A method for identifying postoperative pain in children based on an explainable prototype according to claim 3, characterized in that: The process of step S3 is as follows: S301, random initialization prototype vector Where p represents the channel dimension of the facial feature map, express The cth category in the pain level, k = 1, 2, 3, ..., K, represents the kth position of the facial features of K people, q = 1, 2, 3, ..., Q, represents the qth prototype vector of the Q facial features of each person; S302, through each feature vector V k and each prototype vector Calculate distance: || ||2 represents the norm of 2, and we continue to calculate V k and Similarity: where the constant ε<10 -5 , allowing in the operation When it is 0; S303, calculated Similarity, and then input into the trained fully connected network, we get Prediction score of pain level The structure of the fully connected network is as follows: The connections from the input layer to the output layer are: neurons, 1 fully connected layer, neurons; S304. In order to achieve the pain level classification task, a cross entropy loss function is constructed: in is the i-th facial expression frame A category vector in which The index position corresponding to the pain level category is 1, and all other positions are 0; At the same time, in the same pain level, in order to increase the eigenvector V k and prototype vector Similarity, construct clustering loss function: where argmax(Y i ) represents the true pain level category; At the same time, in order to reduce the eigenvector V in different pain levels k and prototype vector Similarity, construct separation loss function: Construct the total loss function: Where λ1 is the weighted coefficient of the clustering loss function, and λ2 is the weighted coefficient of the separation loss function; By minimizing the total loss function, not only the pain level classification task is achieved, but also the prototype vector is optimized; S305, traverse and search for all eigenvectors V k Neutralize prototype vector The feature vector with the highest similarity between them is taken as the final prototype vector.

5. The method for identifying postoperative pain in children based on an explainable prototype according to claim 4, characterized in that: The process of step S4 is as follows: S401, for a video of a child suffering from postoperative pain to be identified, first perform step S1 to obtain a facial expression frame and a sparse optical flow frame for each frame, and then perform step S2 to obtain a feature vector for each frame; S402: According to steps S302 and S303, the feature vector and the prototype vector are similarly calculated, and the obtained similarity is input into the trained fully connected network to obtain The predicted score of the pain class ρ; S403: The pain level with the highest prediction score is used as the pain level recognition result of each frame of the image.

6. The method for identifying postoperative pain in children based on an explainable prototype according to claim 5, characterized in that: The process of step S5 is as follows: S501, displaying the positions of facial features corresponding to the feature vectors of each frame of the image; S502 , traverse and search for the prototype vector with the highest similarity to the feature vector, and display the positions of the facial features corresponding to the prototype vector.

Images