A method for assessing pain after anesthesia based on patients' facial expressions

Abstract

The present invention relates to the field of image analysis technology, and specifically to a method for assessing post-anesthesia pain in combination with a patient's facial expression. The optical flow method is used to analyze the pixel movement of facial images, obtain optical flow vectors and information points, and quantitatively capture changes in facial expressions. In view of the differences in the effectiveness of information points, facial feature points are screened to reduce data redundancy. Taking into account the differences in the facial structures of patients, the pulling characteristics of the optical flow vectors are analyzed to determine the pulling direction of the image block. Painful expressions, such as closed eyes and eyebrows, can be characterized by the pulling direction. By analyzing the chaos and differences in the distribution of pulling directions, a facial pain index is determined, providing an accurate reference for pain assessment. Finally, based on a data set of facial images of multiple patients and pain index screening, a neural network is trained to establish a mapping relationship between facial expression and pain degree. This method not only improves the accuracy of pain assessment, but also significantly improves the evaluation efficiency, thereby realizing automated pain assessment of facial images of patients to be tested.

Description

Technical Field

[0001] The present invention relates to the technical field of image analysis, and in particular to a method for assessing pain after anesthesia based on a patient's facial expression. Background Art

[0002] In the medical field, pain assessment after anesthesia is crucial, directly impacting the patient's recovery and medical outcomes. Because patients may struggle to accurately express their pain due to medication effects, confusion, or language barriers, facial expression-based pain assessment methods are emerging to improve accuracy and efficiency.

[0003] Facial expressions are an important way for humans to express emotions, and pain, as a strong emotional experience, often leaves obvious traces on the face. Therefore, by analyzing the patient's facial muscle movements and expression changes, the degree of pain can be indirectly assessed. However, when using facial recognition technology for pain recognition in practical applications, facial feature points are often used for recognition. However, due to differences in facial structure and muscle and skin distribution among different people, and some feature points have nothing to do with pain recognition, traditional facial pain recognition using feature points may require the simultaneous processing of a large amount of non-pain redundant data, which reduces efficiency and affects the final accuracy. Summary of the Invention

[0004] In order to solve the technical problem that facial structures and muscle-skin distribution vary among different people, and some feature points are irrelevant to pain recognition, the traditional use of feature points for facial pain recognition may require processing a large amount of non-pain redundant data, which reduces efficiency and affects the final accuracy. The purpose of the present invention is to provide a method for post-anesthesia pain assessment combined with the patient's facial expression. The technical solution adopted is as follows:

[0005] Obtain two temporally adjacent facial image frames after the patient's anesthesia surgery, and use the temporally previous facial image frame as the current image;

[0006] Based on the optical flow method, the motion changes of pixels in the two frames of facial images are analyzed to obtain optical flow vectors and the pixels corresponding to the optical flow vectors in the current image are used as information points; the current image is divided into blocks to obtain image blocks; the difference between the distribution characteristics of the information points in the local neighborhood and in the image blocks is analyzed, and facial feature points are selected from the information points;

[0007] Determine the pull eigenvalue corresponding to each facial feature point based on the positional relationship and length difference between the optical flow vectors; determine the pull direction of each image block based on the pull eigenvalue corresponding to the facial feature point in each image block; analyze the distribution disorder and distribution difference of the pull directions of all image blocks in the current image to determine the facial pain index of the current image;

[0008] A facial pain index screening dataset corresponding to facial images of multiple patients is used for training a neural network to obtain a trained neural network; and post-anesthesia pain assessment is performed on the facial images of the tested patients based on the trained neural network.

[0009] Furthermore, the method for acquiring facial feature points includes:

[0010] In the current image, for any information point, take the information point as the center of the circle and the length of the optical flow vector corresponding to the information point as the radius to obtain the local neighborhood corresponding to the information point;

[0011] In the local neighborhood corresponding to the information point, the density of all information points in the local neighborhood is calculated as the first density;

[0012] In the image block to which the information point belongs, calculating the density of all information points in the image block as a second density;

[0013] multiplying the normalized value of the difference between the first density and the second density corresponding to the information point by the first density, and normalizing the obtained product to obtain an information factor corresponding to the information point;

[0014] Among all the information points, the information points whose information factors are greater than the preset information threshold are taken as facial feature points.

[0015] Furthermore, the method for obtaining the pulling characteristic value includes:

[0016] For an optical flow vector corresponding to any facial feature point, determine the stretched optical flow of the optical flow vector according to the positional relationship between the optical flow vector and other optical flow vectors;

[0017] Calculate the angle between the optical flow vector and each corresponding stretched optical flow as the angle deviation value, the difference between the length of the optical flow vector and the length of each corresponding stretched optical flow as the length deviation factor, and the Euclidean distance between the starting point of the optical flow vector and the starting point of each corresponding stretched optical flow as the distance factor;

[0018] Arrange the stretched optical flow of the optical flow vector in ascending order according to the distance factor to obtain a sorted sequence, and under the sorted sequence, calculate the Pearson correlation coefficient between the angle deviation values ​​of all stretched optical flows and the distance factor as a first correlation factor, and calculate the Pearson correlation coefficient between the length deviation factors of all stretched optical flows and the distance factor as a second correlation factor;

[0019] The normalized value of the sum of the first correlation factor and the second correlation factor corresponding to all stretched optical flows of the optical flow vector is used as the stretched feature value of the facial feature point corresponding to the optical flow vector.

[0020] Furthermore, the method for obtaining the stretched optical flow includes:

[0021] The optical flow vector corresponding to any facial feature point is selected as the optical flow to be measured, and a perpendicular line is drawn through the starting point of the optical flow to be measured to the straight line where the optical flow to be measured is located, and the perpendicular line is used as the transverse line;

[0022] The optical flow vector intersecting the transverse line is used as the stretched optical flow of the optical flow to be measured.

[0023] Furthermore, the method for obtaining the pulling direction includes:

[0024] In each image block, the direction of the optical flow vector corresponding to the maximum pulling eigenvalue is used as the pulling direction of each image block.

[0025] Furthermore, the method for obtaining the facial pain index includes:

[0026] In the current image, obtaining facial symmetry lines on the left and right sides of the face and determining a facial region where each image block is located, wherein the facial region is divided into a left region and a right region;

[0027] Construct a coordinate system with the facial symmetry line in the current image as the vertical axis and the perpendicular facial symmetry line as the horizontal axis;

[0028] For any image block, the straight line in the pulling direction is used as the moving line, the intersection of the moving line and the edge line of the facial area is used as the first intersection point, and the intersection of the moving line and the facial symmetry line is used as the second intersection point;

[0029] The image block with the pulling direction pointing to the first intersection is regarded as the first area, and the image block with the pulling direction pointing to the second intersection is regarded as the second area;

[0030] For all first regions in the current image, a ratio of the information entropy of the ordinate of the first intersection to the information entropy of the ordinate of the second intersection is used as a diffusion index;

[0031] For all second regions in the current image, a ratio of the information entropy of the ordinate of the first intersection to the information entropy of the ordinate of the second intersection is used as an aggregation index;

[0032] The sum of the ordinates of the first intersection point and the second intersection point in the second area is used as a first sum, the sum of the ordinates of the first intersection point and the second intersection point in the first area is used as a second sum, and the difference between the first sum and the second sum is normalized as the height difference;

[0033] The product of the diffusion index, aggregation index and height difference value corresponding to the current image is normalized and used as the facial pain index corresponding to the current image.

[0034] Furthermore, the method for obtaining the facial symmetry line includes:

[0035] Take any point on the upper boundary and the lower boundary of the current image and draw a line connecting them, and use the straight line where the line is located as the dividing line;

[0036] For any dividing line, the number of information points in the current image that are symmetrical about the dividing line is used as the symmetry index of the dividing line;

[0037] The segmentation line with the largest symmetry index is taken as the facial symmetry line of the current image.

[0038] Furthermore, the method for obtaining the data set includes:

[0039] Among the facial pain indices corresponding to all patients' facial images, facial images with a value greater than or equal to the pain threshold are selected as target images;

[0040] The optical flow vectors of facial feature points in the target image are used as pulling vectors, and the pulling vectors corresponding to all target images are used as data sets.

[0041] Furthermore, the post-anesthesia pain assessment based on the trained neural network on the facial image of the patient to be tested includes:

[0042] The optical flow vectors of all facial feature points in the facial image to be tested are obtained and used as the input of the trained neural network, and the pain level of the facial image to be tested is output.

[0043] Furthermore, the method for obtaining the image block includes:

[0044] The current image is segmented using the superpixel segmentation method to obtain several superpixel blocks, each of which is an image block.

[0045] The present invention has the following beneficial effects:

[0046] To accurately assess postoperative pain in patients undergoing anesthesia, the optical flow method was first used to analyze the motion changes of pixels in facial images. Optical flow vectors were obtained and used to identify information points, which objectively and quantitatively capture subtle changes in facial expression. Given that not all information points are effective for characterizing postoperative pain, the differences between the distribution characteristics of information points in their local neighborhood and within the image block were analyzed, and facial feature points were selected from the information points to reduce data redundancy. Due to differences in facial structure and muscle-skin distribution among patients, when a patient experiences pain, facial muscles may shift in orientation, resulting in muscle tension. Therefore, the tension characteristics between optical flow vectors were analyzed to determine the direction of tension for each image block. Facial expressions of pain often exhibit features such as closed and gathered eyes and eyebrows, which can be characterized by tension direction. Therefore, the distribution of tension direction across image blocks was analyzed to determine a facial pain index, which can provide a more accurate reference for assessing postoperative pain in patients undergoing anesthesia. Finally, by screening the facial pain index data set corresponding to the facial images of multiple patients and training the neural network, a mapping relationship between facial expressions and pain levels can be established, thereby realizing the subsequent automated pain assessment of the facial images of the patients to be tested, thereby improving the assessment accuracy and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0048] Figure 1 A flowchart of a method for assessing post-anesthesia pain in combination with a patient's facial expression provided by one embodiment of the present invention;

[0049] Figure 2 A schematic diagram of a stretched optical flow provided by one embodiment of the present invention;

[0050] Figure 3 A schematic diagram of the distribution of optical flow vectors when a face is subjected to a force, provided by one embodiment of the present invention;

[0051] Figure 4 A schematic diagram of a first intersection and a second intersection provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0052] To further illustrate the technical means and efficacy employed by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and efficacy of a method for assessing post-anesthesia pain based on a patient's facial expression, as proposed by the present invention. In the following description, references to different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0053] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0054] The following describes in detail a specific scheme of a method for assessing post-anesthesia pain in combination with a patient's facial expression provided by the present invention with reference to the accompanying drawings.

[0055] See also Figure 1 , which shows a method flow chart of a method for assessing post-anesthesia pain in combination with a patient's facial expression, provided by one embodiment of the present invention. The method comprises the following steps:

[0056] Step S1: Acquire two temporally adjacent facial image frames after the patient's anesthesia surgery, and use the temporally previous facial image frame as the current image.

[0057] For patients who are unconscious after anesthesia, or who are unable to directly express pain, such as children and the elderly, doctors often assess their pain status based on facial expressions. With the rapid development of computer vision and artificial intelligence technologies, pain assessment methods based on facial expression analysis are gaining attention to improve assessment efficiency. These methods allow for objective post-anesthesia pain assessments and provide doctors with more accurate management references.

[0058] First, it is necessary to obtain the facial image of the patient after anesthesia. In view of the differences in facial structure and muscle and skin distribution among different people, in order to more accurately identify the pain state represented by facial expressions, in an embodiment of the present invention, two frames of facial images that are adjacent in time sequence after the patient's anesthesia are obtained, which helps to capture subtle changes in the face more objectively and quantitatively in the subsequent process.

[0059] Specifically, a high-definition camera or a medical-specific camera device is used to record continuous video during the patient's post-anesthesia recovery period. During the shooting process, the position, angle, and focal length of the camera must be fixed. Use professional video processing software to extract images from the recorded video at a set time interval or frame rate. Since the embodiment of the present invention mainly focuses on the patient's facial state, in order to avoid interference from other background areas, the extracted images can be operated based on a trained semantic segmentation network to obtain a facial image that only contains the patient's face. Then, in the extracted facial image sequence, any two frames of facial images that are adjacent in time sequence are selected, for example, the nth frame and the n+1th frame are selected. In the two adjacent facial image frames, the previous frame (the nth frame) is used as the current image.

[0060] It should be noted that the time interval when extracting images can be set to 1 second. The specific time interval can be adjusted according to the implementation scenario and is not limited here. The training process of the semantic segmentation network is briefly described as follows: (1) Images containing human faces are used as training data. The pixels in the facial area are marked as 1 and the other pixels are marked as 0 to obtain label data. (2) The semantic segmentation network adopts an encoding-decoding structure, and the training data and label data are normalized and input into the network. The semantic segmentation encoder is used to extract the features of the input data and obtain a feature map. The semantic segmentation decoder performs sampling transformation on the feature map and outputs the semantic segmentation result. (3) The network is trained using the cross entropy loss function.

[0061] In the embodiments of the present invention, the acquisition and collection of patient facial data are authorized by relevant users, and the process does not violate relevant laws and regulations and does not violate public order and good customs.

[0062] Step S2: Analyze the motion changes of pixels in the two facial images based on the optical flow method to obtain optical flow vectors and use the pixels corresponding to the optical flow vectors in the current image as information points; divide the current image into blocks to obtain image blocks; analyze the differences between the distribution characteristics of the information points in the local neighborhood and in the image blocks, and filter facial feature points among the information points.

[0063] Pain is often accompanied by tension in facial muscles. Therefore, by analyzing the movement changes of pixels in facial images using the optical flow method, optical flow vectors and information points are obtained, which can capture the movement of facial muscles, which is crucial for recognizing painful expressions. Since not all information points are effective in characterizing the movement of facial muscles, in order to improve and reduce computational redundancy, facial feature points can be further screened out from all information points. For feature points that can characterize facial muscle movement, they will undergo relatively obvious displacement under expressions such as frowning. Therefore, in an embodiment of the present invention, the difference in the distribution characteristics of information points in the local neighborhood and in the image block is analyzed, and information points with relatively obvious displacement are screened out as facial feature points.

[0064] First, two frames of facial images are used as input to the optical flow method, which outputs the optical flow vector in the current image, reflecting the motion information of the pixels in the current image. The pixels with optical flow vectors are used as information points, which represent the location where the movement occurs. These movements are usually closely related to changes in facial expressions.

[0065] It should be noted that the process of obtaining the optical flow vector based on the optical flow method is a well-known technology, and the specific process will not be described here in detail.

[0066] The current image can then be divided into blocks to more carefully analyze the changes in facial expressions within a small range, so that more representative facial feature points that can more significantly reflect facial muscle movements can be screened out in the subsequent process.

[0067] Preferably, in one embodiment of the present invention, the method for obtaining an image block includes:

[0068] Superpixel segmentation can divide an image into regions with similar attributes (superpixel blocks), reducing image complexity while preserving key image features such as edges, texture, and shape. Therefore, in this embodiment of the present invention, a superpixel segmentation method is used to segment the current image to obtain several superpixel blocks, each of which is an image block.

[0069] It should be noted that in order to ensure that the area of ​​the image block is not too small, in this embodiment of the present invention, the number of superpixel blocks is set to 16. The specific number can be adjusted according to the implementation scenario and is not limited here. The superpixel segmentation method is a well-known technology and the specific process will not be repeated here.

[0070] Superpixel segmentation divides the current image into image blocks of several texture structure pixels, which can be regarded as dividing the patient's face into different areas. At this time, the distribution of information points in the image block can represent the direction changes of facial muscles within a range. In order to further identify more representative facial feature points, the difference between the distribution characteristics of information points in a smaller local neighborhood and in the image block can be analyzed to screen facial feature points.

[0071] Preferably, in one embodiment of the present invention, the method for acquiring facial feature points includes:

[0072] In the current image, for any information point, the local neighborhood corresponding to the information point is obtained with the information point as the center of the circle and the length of the optical flow vector corresponding to the information point as the radius. By setting the local neighborhood, the relationship between the information point and its surrounding pixels can be more accurately evaluated, as well as whether the information point can represent a significant facial feature.

[0073] In the local neighborhood corresponding to the information point, the density of all information points in the local neighborhood is calculated as the first density. Specifically, the ratio of the number of all information points in the local neighborhood to the area of ​​the local neighborhood can be used as the first density. The larger the first density is, the denser the distribution of information points is in the local neighborhood corresponding to the information point, that is, the denser the distribution of pixels with motion changes is, then the more representative the information point is in representing muscle changes when the face is in pain.

[0074] Then, in the image block to which the information point belongs, the density of all information points in the image block is calculated as the second density. Similarly, the ratio of the number of all information points in the image block to the area of ​​the image block is taken as the second density. The larger the second density, the denser the distribution of the information points in the image block to which the information point belongs.

[0075] Then, by comparing the local neighborhood density (first density) and the image block density (second density) corresponding to the information point, it is possible to evaluate whether the information point has significant characteristics in its local neighborhood. If the first density is higher than the second density, then the information point can be regarded as a facial feature point with more significant motion characteristics.

[0076] Therefore, the normalized value of the difference between the first and second densities corresponding to the information point is calculated. The larger the normalized value, the greater the degree to which the local neighborhood density of the information point exceeds the density of the image block. This value is then multiplied by the first density, and the resulting product is normalized to obtain the information factor corresponding to the information point. In this case, the larger the information factor, the more significant the local facial motion characteristics of the information point, and the better it can express the movement changes of facial muscles. Normalization is a technical means well known to those skilled in the art. The normalization function can be linear normalization or standard normalization, etc. The specific normalization method is not limited here.

[0077] Finally, among all the information points, the information points whose information factors are greater than the preset information threshold are taken as facial feature points.

[0078] It should be noted that, in the embodiment of the present invention, the preset information threshold is 0.65, and the specific value can be adjusted according to the implementation scenario and is not limited here.

[0079] Step S3: Determine the pulling feature value corresponding to each facial feature point based on the positional relationship and length difference between the optical flow vectors; determine the pulling direction of each image block based on the pulling feature value corresponding to the facial feature point in each image block; analyze the distribution disorder and distribution difference of the pulling directions of all image blocks in the current image to determine the facial pain index of the current image.

[0080] In facial expressions of pain, muscle movement and deformation are prominent features, and facial muscles pull on each other. Optical flow vectors can capture the motion information of pixels in an image, which is crucial for analyzing the dynamic changes in facial expressions. Therefore, we analyze the positional relationships and length differences between optical flow vectors to determine the pull eigenvalue corresponding to each facial feature point, reflecting the degree of pull at each facial feature point. Then, within each image block, the pull direction of that block can be determined based on the pull eigenvalue of each facial feature point. This helps analyze the deformation and muscle movement of different facial regions, further supporting the recognition of painful expressions. Finally, given that facial expressions of pain tend to close and cluster around the eyes and eyebrows, while areas like the mouth and nostrils appear open, we can comprehensively assess the pain level of the entire face by analyzing the distribution of the pull directions and differences across all image blocks in the current image. This distribution analysis can capture both subtle changes and overall trends in facial pain, providing a reliable basis for determining facial pain indicators.

[0081] First, based on the positional relationship and length difference characteristics between the optical flow vectors, the pulling feature value corresponding to each facial feature point is determined. Preferably, in one embodiment of the present invention, the method for obtaining the pulling feature value includes:

[0082] Choose the optical flow vector corresponding to any facial feature point as the optical flow to be measured, draw a perpendicular line through the starting point of the optical flow to be measured, use the perpendicular line as the transverse line, and use the optical flow vector that intersects the transverse line as the stretched optical flow of the optical flow to be measured. By analyzing the stretched optical flow, we can understand the pulling of the facial muscles by the optical flow to be measured in the direction of the transverse line, which is important for analyzing the deformation of facial features. Figure 2 , which shows a schematic diagram of stretched optical flow in one embodiment of the present invention, wherein the yellow arrow represents the optical flow to be measured, the yellow straight line represents the transverse line of the optical flow to be measured, and the blue arrow is the stretched optical flow of the optical flow to be measured.

[0083] In facial expression analysis, when a force acts on the face, such as a muscle contraction, the area near the point of force application is usually subjected to a greater pull, while the area farther from the point of force application experiences a gradually decreasing degree of pull. This principle is applied to the analysis of optical flow vectors. That is, for a certain optical flow vector, the further away from the optical flow vector the stretched optical flow is from the optical flow vector, the greater the deviation angle and length difference between the two will tend to be due to the decreasing effect of the degree of pull. This deviation and difference provide important information about the degree of facial pull. Figure 3 , which shows a schematic diagram of the distribution of optical flow vectors when the face is subjected to force in one embodiment of the present invention.

[0084] Therefore, for any optical flow vector corresponding to a facial feature point, the angle between the optical flow vector and each corresponding stretched optical flow can be calculated as the angle deviation value. The difference between the length of the optical flow vector and the length of each corresponding stretched optical flow is used as the length deviation factor. The Euclidean distance between the starting point of the optical flow vector and the starting point of each corresponding stretched optical flow is used as the distance factor.

[0085] Then, the stretched optical flow of the optical flow vector is arranged in ascending order according to the distance factor to obtain a sorted sequence. Under the sorted sequence, the Pearson correlation coefficient between the angular deviation values ​​of all stretched optical flows and the distance factor is calculated as the first correlation factor. Based on the above analysis, it can be seen that as the distance factor increases, the angular deviation value will also increase under the action of traction. Therefore, if the first correlation factor is closer to 1, it means that there is a positive correlation between the distance factor and the angular deviation value, then the more it conforms to the change trend under the traction effect, the greater the degree of traction.

[0086] Similarly, the Pearson correlation coefficient between the length deviation factor and the distance factor of all stretched optical flows is calculated as the second correlation factor. If the second correlation factor is closer to 1, it means that there is a positive correlation between the distance factor and the length deviation factor, which also indicates a greater degree of stretching.

[0087] Finally, the sum of the first and second correlation factors corresponding to all stretched optical flows of the optical flow vector is normalized to obtain the stretch feature value of the facial feature point corresponding to the optical flow vector. A larger stretch feature value indicates more pronounced muscle stretch at the facial feature point and a greater degree of stretch. Normalization is a well-known technique for those skilled in the art, and the normalization function can be linear normalization or standard normalization. The specific normalization method is not limited here.

[0088] At this point, the stretch feature value corresponding to each facial feature point in the current image can be obtained. The stretch feature value reflects the degree of stretching of the facial feature point in a specific direction. Therefore, by analyzing the stretch feature value of the facial feature point in the image block, the stretching direction of the image block can be determined, which helps to more accurately understand the changes in facial expressions.

[0089] Preferably, in one embodiment of the present invention, the method for obtaining the pulling direction includes:

[0090] Based on the above analysis, it can be seen that when the stretch feature value of a facial feature point is larger, it means that the muscle stretch at the facial feature point is more obvious and the degree of stretch is greater. Therefore, in each image block, the optical flow vector corresponding to the maximum stretch feature often represents the most obvious deformation trend in the image block. Therefore, the direction of the optical flow vector corresponding to the maximum stretch feature value is used as the stretch direction of each image block.

[0091] When a patient experiences pain, they exhibit a range of facial expressions, resulting in various deformations of facial organs. For example, eyebrows may droop or furrow, eyes may close or squint, appearing to be gathered together, nostrils may widen, and the corners of the mouth may droop, appearing to be widened. Consequently, the pulling directions of different image blocks may differ. Therefore, in this embodiment of the present invention, by analyzing the distribution disorder and distribution differences of the pulling directions of image blocks in the current image, the degree of facial pain in the current image can be quantified to obtain a facial pain index.

[0092] Preferably, in one embodiment of the present invention, the method for obtaining the facial pain index includes:

[0093] Since the patient's head may slightly twist during the facial image acquisition process, the position of each image block in the current image can be determined first. Since the left and right sides of the patient's face are generally symmetrical, the facial symmetry lines on the left and right sides of the face can be obtained in the current image to determine the facial area where each image block is located. The facial area can be simply divided into a left area and a right area.

[0094] The method for obtaining a facial symmetry line includes: randomly selecting a point on each of the upper and lower boundaries of the current image to draw a line connecting the points, and using the line along the connecting line as a dividing line; for each dividing line, using the number of information points in the current image that are symmetrical about the dividing line as a symmetry index for the dividing line; the larger the symmetry index corresponding to a particular dividing line, the better the symmetry of the left and right faces of the current image when the dividing line is used as the symmetry line, and thus using the dividing line with the largest symmetry index as the facial symmetry line of the current image. It should be noted that the boundary in this embodiment of the present invention refers to the border of the image.

[0095] Then, a coordinate system is constructed with the facial symmetry line as the vertical axis and the perpendicular facial symmetry line as the horizontal axis.

[0096] For any image block, the straight line in the pulling direction is taken as the moving line, the intersection of the moving line and the edge line of the facial area is taken as the first intersection point, and the intersection of the moving line and the facial symmetry line is taken as the second intersection point. Figure 4 , which shows a schematic diagram of the first and second intersection points in one embodiment of the present invention. The arrow indicates the pulling direction of the image block, the line on which the arrow points represents the movement line, the red dot indicates the first intersection point, and the green dot indicates the second intersection point. It should be noted that edge lines can be obtained using the Canny operator, which is a well-known technique and the specific process is not described here.

[0097] The image block with the pulling direction pointing to the first intersection is regarded as the first area. Since the first intersection is the intersection of the moving line and the edge line of the facial area, if the pulling direction points to the first intersection, then the pulling direction of the image block can be regarded as spreading toward the outside of the face; the image block with the pulling direction pointing to the second intersection is regarded as the second area. Since the second intersection is the intersection of the moving line and the facial symmetry line, if the pulling direction points to the second intersection, then the pulling direction of the image block can be regarded as converging toward the inside of the face.

[0098] When expressing painful emotions, facial muscles often undergo more obvious longitudinal deformation. In comparison, the lateral deformation of the face may not be as significant as the longitudinal deformation. Therefore, for all first areas in the current image, the first area is an image block that diffuses outward from the face. Therefore, the position disorder of the first intersection is higher than that of the second intersection. Information entropy is highly sensitive to slight changes in data. In the evaluation of facial muscle deformation, even if the deformation is small, it can be more accurately represented. Therefore, the ratio of the information entropy of the ordinate of the first intersection to the information entropy of the ordinate of the second intersection is used as a diffusion index. At this time, the larger the diffusion index, the greater the degree of outward diffusion of the first area, which indicates that the deformation of the facial muscles is greater, and the higher the degree of pain.

[0099] Similarly, for all the second areas in the current image, the second area is an image block where the face is concentrated inward, so the position disorder degree of the first intersection is higher than that of the second intersection. Therefore, the ratio of the information entropy of the vertical coordinate of the first intersection to the information entropy of the vertical coordinate of the second intersection is used as the aggregation index. The larger the aggregation index, the greater the degree of inward aggregation of the second area, which indicates that the deformation of the facial muscles is greater, and the higher the degree of pain.

[0100] Based on the above analysis, it can be seen that when the face makes an expression of pain, the eyebrows, eyes, and nose often show inward-gathering muscle deformation, while the mouth and nostrils often show outward-diffusion muscle deformation, so the vertical coordinate of the second area should be larger than the vertical coordinate of the first area, so the sum of the vertical coordinates of the first intersection point and the second intersection point in the second area is calculated as the first sum; the sum of the vertical coordinates of the first intersection point and the second intersection point in the first area is calculated as the second sum; the difference between the first sum and the second sum is normalized as the height difference. If the height difference is positive, it means that the muscle direction in the facial expression at this time is more consistent with the characteristics of expressions such as pain and pain. Since the difference between the first sum and the second sum may be positive or negative, the normalization here can be used. function.

[0101] Finally, the product of the diffusion index, aggregation index, and height difference corresponding to the current image is normalized to obtain the corresponding facial pain index. A larger facial pain index indicates that the facial muscles in the current image are more consistent with a painful expression. Normalization is a well-known technique for those skilled in the art, and the normalization function can be linear normalization or standard normalization. The specific normalization method is not limited here.

[0102] Step S4: screening a dataset of facial pain indicators corresponding to facial images of multiple patients for training a neural network to obtain a trained neural network; and performing post-anesthesia pain assessment on the facial images of the patient to be tested based on the trained neural network.

[0103] In order to improve the efficiency of post-operative pain assessment for patients after anesthesia, in an embodiment of the present invention, facial images of multiple patients can be analyzed based on the aforementioned steps to obtain facial pain indicators, and then the data set can be screened for training the neural network to obtain a trained neural network. In the subsequent process, post-operative pain assessment can be directly performed on the facial images of the patients to be tested based on the trained neural network.

[0104] Preferably, in one embodiment of the present invention, the method for obtaining a data set includes:

[0105] Among the facial pain indicators corresponding to all patients' facial images, facial images with a value greater than or equal to the pain threshold are used as target images. At this time, the target images can express the pain state more accurately and representatively, thereby helping to improve the learning efficiency and accuracy of the neural network.

[0106] Optical flow vectors can represent changes in facial muscles, so the optical flow vectors of facial feature points in the target image are used as pulling vectors, and the pulling vectors corresponding to all target images are used as a data set. At this time, the data set can more realistically and effectively reflect the patient's pain state, providing rich and accurate information for neural network training.

[0107] It should be noted that the pain threshold here is set to 0.5. The specific value can be adjusted according to the implementation scenario and is not limited here.

[0108] After obtaining the data set, the neural network can be trained: the pain index is graded, for example, [0.5, 0.7) is level one pain, [0.7, 0.9) is level two pain, and [0.9, 1] is level three pain. At this time, each traction vector in the data set corresponds to a pain level. The network structure uses Encoder-FC, and the loss function is the cross entropy function; the input is the data set, and the output is the pain level. During the neural network training process, training is performed using the gradient descent method until the loss function converges and the training is completed.

[0109] It should be noted that the training process of a neural network is a well-known technology and is only briefly described here without going into details about the specific process.

[0110] After obtaining the trained neural network, post-anesthesia pain assessment can be performed on the facial image of the patient to be tested based on the trained neural network. Preferably, in one embodiment of the present invention, the process includes:

[0111] Based on the above steps, the optical flow vectors of all facial feature points in the facial image to be tested are obtained and used as the input of the trained neural network, thereby outputting the pain level of the facial image to be tested.

[0112] In summary, to accurately assess postoperative pain in patients undergoing anesthesia, we first used optical flow to analyze the motion changes of pixels in facial images. This method then obtained optical flow vectors and information points, which objectively and quantitatively captured subtle changes in facial expression. Given that not all information points are effective for characterizing postoperative pain, we analyzed the differences between the distribution characteristics of information points in their local neighborhood and within the image block, filtering facial feature points from these information points to reduce data redundancy. While facial structure and muscle-skin distribution vary among patients, pain can cause changes in facial muscle orientation, resulting in muscle tension. Therefore, we analyzed the tension characteristics between optical flow vectors to determine the direction of tension for each image block. Facial expressions of pain often exhibit features such as closed and gathered eyes and eyebrows, which can be characterized by tension direction. Therefore, we analyzed the distribution of tension direction disturbances and differences across image blocks to determine a facial pain index, which can provide a more accurate reference for assessing postoperative pain in patients. Finally, by screening the facial pain index data set corresponding to the facial images of multiple patients and training the neural network, a mapping relationship between facial expressions and pain levels can be established, thereby realizing the subsequent automated pain assessment of the facial images of the patients to be tested, thereby improving the assessment accuracy and efficiency.

[0113] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0114] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. A method for assessing pain after anesthesia combined with the patient's facial expression, characterized in that: The method comprises: Obtain two temporally adjacent facial image frames after the patient's anesthesia surgery, and use the temporally previous facial image frame as the current image; Based on the optical flow method, the motion changes of pixels in the two frames of facial images are analyzed to obtain optical flow vectors and the pixels corresponding to the optical flow vectors in the current image are used as information points; the current image is divided into blocks to obtain image blocks; the difference between the distribution characteristics of the information points in the local neighborhood and in the image blocks is analyzed, and facial feature points are selected from the information points; Determine the pull eigenvalue corresponding to each facial feature point based on the positional relationship and length difference between the optical flow vectors; determine the pull direction of each image block based on the pull eigenvalue corresponding to the facial feature point in each image block; analyze the distribution disorder and distribution difference of the pull directions of all image blocks in the current image to determine the facial pain index of the current image; A facial pain index data set corresponding to facial images of multiple patients is selected for use in training a neural network to obtain a trained neural network; and post-anesthesia pain assessment is performed on the facial images of the patient to be tested based on the trained neural network; The method for acquiring facial feature points includes: in a current image, for any information point, taking the information point as the center of a circle and the length of an optical flow vector corresponding to the information point as the radius, obtaining a local neighborhood corresponding to the information point; in the local neighborhood corresponding to the information point, calculating the density of all information points in the local neighborhood as a first density; in an image block to which the information point belongs, calculating the density of all information points in the image block as a second density; multiplying a normalized value of the difference between the first density and the second density corresponding to the information point by the first density, and normalizing the obtained product to obtain an information factor corresponding to the information point; and selecting, among all information points, an information point whose information factor is greater than a preset information threshold as a facial feature point; The method for obtaining a stretched feature value includes: for an optical flow vector corresponding to any facial feature point, determining the stretched optical flow of the optical flow vector according to the positional relationship between the optical flow vector and other optical flow vectors; calculating the angle between the optical flow vector and each corresponding stretched optical flow as an angle deviation value, using the difference between the length of the optical flow vector and the length of each corresponding stretched optical flow as a length deviation factor, and using the Euclidean distance between the starting point of the optical flow vector and the starting point of each corresponding stretched optical flow as a distance factor; arranging the stretched optical flows of the optical flow vector in ascending order according to the distance factor to obtain a sorted sequence, and under the sorted sequence, calculating the Pearson correlation coefficient between the angle deviation values ​​of all stretched optical flows and the distance factor as a first correlation factor, and calculating the Pearson correlation coefficient between the length deviation factors and the distance factors of all stretched optical flows as a second correlation factor; and using the normalized sum of the first correlation factor and the second correlation factor corresponding to all stretched optical flows of the optical flow vector as the stretched feature value of the facial feature point corresponding to the optical flow vector; The method for obtaining the stretched optical flow includes: selecting an optical flow vector corresponding to any facial feature point as the optical flow to be measured, drawing a perpendicular line to the straight line where the optical flow to be measured is located through the starting point of the optical flow to be measured, and using the perpendicular line as the transverse displacement line; and using the optical flow vector intersecting with the transverse displacement line as the stretched optical flow of the optical flow to be measured.

2. The method for assessing post-anesthesia pain based on the patient's facial expression according to claim 1, characterized in that: Methods for obtaining the pulling direction include: In each image block, the direction of the optical flow vector corresponding to the maximum pulling eigenvalue is used as the pulling direction of each image block.

3. The method for assessing post-anesthesia pain based on the patient's facial expression according to claim 1, wherein: Methods for obtaining facial pain indicators include: In the current image, obtain the facial symmetry lines on the left and right sides of the face and determine the facial region where each image block is located, where the facial region is divided into a left region and a right region; Construct a coordinate system with the facial symmetry line in the current image as the vertical axis and the perpendicular facial symmetry line as the horizontal axis; For any image block, the straight line in the pulling direction is used as the moving line, the intersection of the moving line and the edge line of the facial area is used as the first intersection point, and the intersection of the moving line and the facial symmetry line is used as the second intersection point; The image block with the pulling direction pointing to the first intersection is regarded as the first area, and the image block with the pulling direction pointing to the second intersection is regarded as the second area; For all first regions in the current image, a ratio of the information entropy of the ordinate of the first intersection to the information entropy of the ordinate of the second intersection is used as a diffusion index; For all second regions in the current image, a ratio of the information entropy of the ordinate of the first intersection to the information entropy of the ordinate of the second intersection is used as an aggregation index; The sum of the ordinates of the first intersection point and the second intersection point in the second area is used as a first sum, the sum of the ordinates of the first intersection point and the second intersection point in the first area is used as a second sum, and the difference between the first sum and the second sum is normalized as the height difference; The product of the diffusion index, aggregation index and height difference value corresponding to the current image is normalized and used as the facial pain index corresponding to the current image.

4. The method for assessing post-anesthesia pain by combining the patient's facial expression according to claim 3, characterized in that: Methods for obtaining facial symmetry lines include: Take any point on the upper and lower boundaries of the current image and connect them, and use the straight line as the dividing line; For any dividing line, the number of information points in the current image that are symmetrical about the dividing line is used as the symmetry index of the dividing line; The segmentation line with the largest symmetry index is taken as the facial symmetry line of the current image.

5. The method for assessing pain after anesthesia based on the patient's facial expression according to claim 1, wherein: Methods for obtaining datasets include: Among the facial pain indices corresponding to all patients' facial images, facial images with a value greater than or equal to the pain threshold are selected as target images; The optical flow vectors of facial feature points in the target image are used as pulling vectors, and the pulling vectors corresponding to all target images are used as data sets.

6. The method for assessing post-anesthesia pain by combining the patient's facial expression according to claim 1, characterized in that: Post-anesthesia pain assessment is performed on the facial images of the patient under test based on the trained neural network, including: The optical flow vectors of all facial feature points in the facial image to be tested are obtained and used as the input of the trained neural network, and the pain level of the facial image to be tested is output.

7. The method for assessing post-anesthesia pain in combination with the patient's facial expression according to claim 1, characterized in that: The image block acquisition method includes: The current image is segmented using the superpixel segmentation method to obtain several superpixel blocks, each of which is an image block.

Images