A method and system for visual assessment of post-anesthesia pain expression

Abstract

The application discloses a post-anesthesia pain expression visual assessment method and system, and relates to the technical field of image processing, which comprises the following steps: acquiring facial image data and time period correlation data of a user after anesthesia in a preset time period, and performing time sequence alignment to obtain time sequence facial image data and time sequence correlation data; obtaining expression active area feature data based on a facial key point data set and expression active area data; identifying non-pain interference state data in combination with a time coincidence proportion value and a region response difference value, and performing region time filtering and weighted correction to obtain credible expression feature data; and integrating the credible expression feature data through time continuity to obtain pain expression visual assessment data, and improving the stability of pain expression candidate change screening.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a visual assessment method and system for post-anesthesia pain expression. Background Technology

[0002] Existing technologies typically involve collecting facial images or videos of patients after surgery and then using face detection, facial landmark localization, facial expression action unit recognition, or machine learning classification models to perform pain-related analysis on changes in facial movements.

[0003] However, in scenarios where individuals are under continuous observation after anesthesia, facial images may contain not only pain-related expressions but also non-pain-related facial changes caused by natural eye closure, shivering, speaking in response, occlusion, or manipulation. Current visual assessment methods primarily focus on extracting facial expression features from the facial images themselves, lacking analysis of the correspondence between external events and facial region changes during image acquisition. This easily leads to the inclusion of non-pain-related facial changes in the pain expression analysis process, resulting in insufficient stability of the screening results for pain expression candidates. Therefore, there is a problem of poor assessment stability due to interference from non-painful facial movements. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a visual assessment method and system for assessing pain expressions after anesthesia.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows:

[0006] In a first aspect, the present invention discloses a visual assessment method for post-anesthesia pain expressions, comprising the following steps: acquiring facial image data and time-related data of a user during a preset time period after anesthesia, and performing time-series alignment processing to obtain time-series facial image data and time-series related data; performing imaging validity detection and facial key point localization processing based on the time-series facial image data to obtain image validity state data, facial key point dataset, and expression active region data; performing continuous time window change feature analysis based on the facial key point dataset and expression active region data to obtain expression active region feature data; and performing time-series coupling analysis based on the time-series related data and expression active region feature data to obtain the time overlap ratio between time-series related events and changes in expression active regions. The system identifies non-painful interference state data based on temporal overlap ratio and regional response difference values, and obtains regional temporal interference mapping data based on the non-painful interference state data. It then performs regional temporal filtering on the facial expression active region feature data based on the regional temporal interference mapping data to obtain filtered facial expression feature data. Finally, it obtains feature credibility data based on effective image state data, facial keypoint datasets, and non-painful interference state data, and performs weighted correction on the filtered facial expression feature data to obtain credible facial expression feature data. The credible facial expression feature data undergoes temporal continuity integration processing to obtain visual evaluation data for pain expressions. This visual evaluation data is used to characterize changes in pain expression candidates after filtering for non-painful interference sources.

[0007] Secondly, this invention discloses a visual assessment system for post-anesthesia pain expression, comprising the following modules: a data acquisition module, used to acquire facial image data and time-related data of the user during a preset time period after anesthesia, and perform time-series alignment processing to obtain time-series facial image data and time-series related data; an image detection module, used to perform imaging validity detection and facial key point localization processing based on the time-series facial image data to obtain image validity state data, facial key point dataset, and expression active area data; a feature analysis module, used to perform continuous time window change feature analysis based on the facial key point dataset and expression active area data to obtain expression active area feature data; and a time-series coupling analysis module, used to perform time-series coupling analysis based on the time-series related data and expression active area feature data to obtain the time overlap ratio between time-series related events and changes in expression active areas. The system comprises the following modules: a time overlap ratio and a region response difference value; an interference identification module, used to identify non-painful interference state data based on the time overlap ratio and region response difference value, and to obtain region temporal interference mapping data based on the non-painful interference state data; a feature filtering module, used to perform region temporal filtering on the facial expression active region feature data based on the region temporal interference mapping data, to obtain filtered facial expression feature data; a credibility analysis module, used to obtain feature credibility data based on image valid state data, facial key point dataset, and non-painful interference state data, and to perform weighted correction on the filtered facial expression feature data based on the feature credibility data, to obtain credible facial expression feature data; and a data determination module, used to perform temporal continuity integration processing on the credible facial expression feature data, to obtain pain expression visual evaluation data, which is used to characterize the changes in pain expression candidates after filtering by non-painful interference sources.

[0008] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0009] 1. This invention performs temporal coupling analysis based on temporal correlation data and facial expression active area feature data to obtain the time overlap ratio and regional response difference value, thereby identifying non-pain interference state data. This enables the differentiation between pain expression candidate changes and non-pain source interference changes, thereby improving the stability of pain expression candidate change screening and effectively solving the problem of insufficient stability of pain expression candidate change screening caused by non-pain source interference in the prior art.

[0010] 2. This invention performs time-series alignment processing on facial image data and time-series associated data to obtain time-series facial image data and time-series associated data, thereby establishing a time correspondence between facial image frames and time-series associated events. This enables subsequent identification of non-painful interference state data to have a clear time basis, avoiding the problem of unclear interference sources caused by judging based solely on isolated image frames.

[0011] 3. By obtaining regional temporal interference mapping data based on non-pain interference state data, and performing regional temporal filtering processing on facial expression active region feature data based on regional temporal interference mapping data, it is possible to filter the time period and facial expression active region affected by interference, thereby achieving targeted elimination or downweighting of non-pain source interference features and reducing the risk of effective facial expression features being mistakenly deleted as a whole. Attached Figure Description

[0012] The disclosure of this invention is illustrated with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. In the drawings, the same reference numerals are used to refer to the same parts. Wherein:

[0013] Figure 1 This is an overall flowchart of the present invention;

[0014] Figure 2 This is a flowchart of the non-pain interference state data recognition process of the present invention;

[0015] Figure 3 This is a flowchart of the regional time filtering and weighted correction process of the present invention;

[0016] Figure 4 This is a system architecture diagram of the present invention. Detailed Implementation

[0017] It is readily understood that, based on the technical solution of this invention, those skilled in the art can propose various interchangeable structural methods and implementations without altering the essential spirit of the invention. Therefore, the following detailed embodiments and accompanying drawings are merely illustrative examples of the technical solution of this invention and should not be considered as the entirety of the invention or as limitations or restrictions on the technical solution of this invention.

[0018] In existing technologies, visual assessment of pain expressions typically involves acquiring facial images of the user and combining them with face detection, facial landmark localization, expression region feature extraction, or classification models to identify and evaluate changes in facial movements. However, after anesthesia, when the user is within a preset time period, facial images may show changes not originating from pain, such as natural eye closure, shivering, responding to speech, or occlusion due to manipulation. Existing methods tend to directly incorporate such changes into the pain expression analysis process, leading to insufficient stability in the selection of candidate pain expression changes.

[0019] To address the aforementioned issues, this invention acquires facial image data and time-related data of users during a preset time period after anesthesia, and performs temporal alignment processing to obtain temporal facial image data and temporal related data. Based on the temporal facial image data, it obtains effective image state data, facial key point datasets, and facial expression active region data. It then obtains facial expression active region feature data and performs temporal coupling analysis in conjunction with the temporal related data to obtain time overlap ratios and regional response difference values. Based on these values, it identifies non-pain interference state data, obtains regional temporal interference mapping data, and performs regional temporal filtering and weighted correction on the facial expression active region feature data to obtain reliable facial expression feature data. Finally, it integrates temporal continuity to obtain visual evaluation data of pain expressions. This method can filter non-pain source interference from both temporal and regional dimensions, improving the stability and reliability of screening candidate changes in pain expressions.

[0020] like Figure 1 As shown, Figure 1 The overall flowchart of this invention includes the following steps: acquiring facial image data and time-related data of a user during a preset time period after anesthesia, and performing time-series alignment processing to obtain time-series facial image data and time-series related data; performing imaging validity detection and facial key point localization processing based on the time-series facial image data to obtain image validity state data, facial key point dataset, and expression active region data; performing continuous time window change feature analysis based on the facial key point dataset and expression active region data to obtain expression active region feature data; and performing time-series coupling analysis based on the time-series related data and expression active region feature data to obtain the time overlap ratio and regional response difference value between time-series related events and changes in expression active regions. The process involves identifying non-painful interference state data based on temporal overlap ratios and regional response differences, and obtaining regional temporal interference mapping data based on this data. Regional temporal filtering is then applied to the facial expression active region feature data based on the regional temporal interference mapping data to obtain filtered facial expression feature data. Feature credibility data is obtained based on effective image state data, facial keypoint datasets, and non-painful interference state data. This credibility data is then used to weight and correct the filtered facial expression feature data to obtain credible facial expression feature data. Finally, temporal continuity integration processing is performed on the credible facial expression feature data to obtain visual evaluation data for pain expressions. This visual evaluation data is used to characterize changes in candidate pain expressions after filtering for non-painful interference sources.

[0021] Further, temporal facial image data and temporal associated data are obtained. Specifically, the method is as follows: Facial image frames are acquired, and image time identifiers, frame numbers, and acquisition status identifiers are written to each frame to obtain facial image data, which includes facial image frames, image time identifiers, frame numbers, and acquisition status identifiers; Time-related events within a preset time period are acquired, and event time identifiers, event type identifiers, and event durations are written to each event to obtain time-related data, which consists of time-related events, event time identifiers, event type identifiers, and event durations; Image time windows are formed based on the image time identifiers according to a preset time window length; event time windows corresponding to time-related events are determined based on the event time identifiers; the image time windows and event time windows are matched to obtain time window matching data; based on the time window matching data, events falling within the same preset time window are... A correspondence is established between facial image frames within a time window and time-related events. An event matching identifier is written to the user's facial image frame to obtain time-series facial image data. The time-series facial image data includes user facial image frames sorted by image time identifier, frame number, acquisition status identifier, and event matching identifier. Based on the time window matching data, time-related events corresponding to the same preset time window are sorted by event time identifier to obtain time-series related data. The time-series related data includes preset time window identifier, time-series related events, event time identifier, event type identifier, event duration, and event arrangement data. When there are no time-series related events within the same preset time window, a no-event identifier is written to the corresponding user facial image frame. When there are multiple time-series related events within the same preset time window, event arrangement data is obtained according to the order of event time identifiers, and the event arrangement data is written to the time-series related data.

[0022] In this embodiment, the image time identifier, frame sequence number, acquisition status identifier, event time identifier, event type identifier, and event duration are all basic fields written into the data record, used to represent the time, order, type, and duration of the image or event.

[0023] Time window matching data refers to the correspondence between image time windows and event time windows. It is obtained by: forming image time windows based on image time identifiers and preset time window lengths; forming event time windows based on event time identifiers and event durations; and establishing a correspondence between corresponding facial image frames and time-related events when the two overlap.

[0024] Event sorting data refers to data obtained by arranging multiple related events within the same preset time window according to the chronological order of their event time identifiers. The acquisition method is as follows: read the event time identifiers of related events within the same preset time window and sort them from earliest to latest.

[0025] The image time window number corresponding to the facial image frames within the preset time period can be obtained using a conventional time window division formula:

[0026] ;

[0027] In the formula, Indicates the first The image time window position value corresponding to the frame of the facial image. , This represents the total number of facial image frames. Indicates the first Image time markers for facial image frames. Indicates the start time of the preset time period. Indicates the preset time window length.

[0028] By writing time fields for facial image frames and time-related events respectively, and matching them using image time windows and event time windows, a unified temporal basis is formed for facial image data and time-related data. The absence of event identifiers avoids misinterpreting event-free periods as having external correlations, and the event arrangement data can handle multiple concurrent events within the same preset time window, reducing errors caused by relying solely on image frames for judgment.

[0029] Furthermore, effective image state data, facial key point datasets, and expression active region data are obtained. Specifically, the following methods are employed: Effective image state data is obtained by analyzing temporal facial image data, including the proportion of visible faces, the proportion of occluded faces, image clarity, and effective image identifiers; Effectiveness screening is performed based on the effective image state data to obtain image frames that meet preset validity conditions, and these are then summarized as effective facial image data; Facial key point localization and scale normalization are performed on the effective facial image data to obtain a facial key point dataset; The facial key point dataset is matched with key point combinations of preset expression active regions to obtain expression active region data, which includes region identifiers, region key point combinations, and region location ranges.

[0030] In this embodiment, the image valid state data is used to indicate whether an image frame is suitable for subsequent expression analysis, and it includes the proportion of visible face, the proportion of occluded face, image clarity, and image valid identifier.

[0031] The visible facial proportion represents the ratio of the visible facial area to the total facial area; the occluded facial proportion represents the ratio of the occluded facial area to the total facial area, i.e., the proportion of unoccluded pixels to the total number of pixels; image sharpness is used to indicate whether an image is blurry and can be obtained using the Laplacian variance method.

[0032] The facial keypoint dataset is a set of keypoint coordinates obtained by scaling the initial facial keypoint data. The expression activity region data is obtained by matching the facial keypoint dataset with keypoints from a predefined expression activity region; it is used to distinguish expression analysis areas such as the eye area, eyebrow area, lip area, and jaw area. The pixels involved in the sharpness calculation are grayscale pixels located within the visible facial area of ​​the valid facial image data.

[0033] Image sharpness was obtained using the Laplacian variance method, specifically as follows:

[0034] ;

[0035] In the formula, This represents the image sharpness of the i-th frame. This represents the number of pixels in the i-th frame that are involved in the sharpness calculation. This represents the response value of the p-th pixel in the i-th frame of the image after processing by the Laplacian operator. This represents the mean Laplacian response of all pixels involved in the calculation in the i-th frame of the image.

[0036] The specific method for determining the validity of an image is as follows:

[0037] ;

[0038] In the formula, This represents the valid image identifier for the i-th frame. This represents the proportion of the face visible in the i-th frame image. Indicates the visible proportion threshold. The proportion of facial occlusion in the i-th frame image Indicates the occlusion ratio threshold. Image clarity of the i-th frame. This indicates the resolution threshold.

[0039] The validity of an image is determined by valid status data. If the result of the valid image identifier is 1, it is considered valid; otherwise, it is invalid.

[0040] The facial key point dataset is obtained by performing scale normalization. The specific method is as follows:

[0041] ;

[0042] ;

[0043] In the formula, and Let x and y represent the normalized x and y coordinates of the k-th facial key point in the i-th frame image, respectively. and Let x and y represent the x and y coordinates of the k-th facial keypoint in the i-th frame before normalization, respectively. and Let x and y represent the x and y coordinates of the facial center keypoint in the i-th frame image, respectively. This represents the facial scale value in the i-th frame image.

[0044] The normalized facial key points are statistically analyzed to obtain a facial key point dataset, where the facial scale value can be obtained from the maximum distance between facial contour key points or the width of the facial region.

[0045] By filtering temporal facial image data using effective image state data, the impact of invisible faces, severe occlusion, or blurred images on subsequent feature extraction can be reduced. By locating facial key points and normalizing scale, coordinate deviations caused by acquisition distance, differences in facial size, and changes in image position can be reduced. By combining key points in areas with active facial expressions for region matching, the key points of the entire face can be transformed into data with regional identifiers and regional location ranges.

[0046] Furthermore, the feature data of the active facial expression region is obtained. Specifically, the method is as follows: Based on the active facial expression region data, the corresponding regional keypoint sequence is extracted from the facial keypoint dataset; inter-frame distance changes are calculated based on the regional keypoint sequence to obtain regional morphological change data; inter-frame displacement amplitude and direction are calculated on the regional keypoint sequence to obtain regional motion change data; periodic statistics are performed on the regional motion change data based on a preset continuous time window to obtain regional motion rhythm data; the duration of regional change data is obtained based on the number of consecutive frames where the regional morphological change data exceeds a preset change threshold; the regional morphological change data, regional motion change data, regional motion rhythm data, and regional change duration data are associated according to regional identifiers and time identifiers to obtain the active facial expression region feature data. The active facial expression region feature data includes regional identifiers, time identifiers, regional morphological change data, regional motion change data, regional motion rhythm data, and regional change duration data.

[0047] In this embodiment, the region keypoint sequence refers to the sequence of keypoint coordinates in consecutive image frames for the same area with active facial expressions. Region morphological change data is used to represent the distance changes between keypoints within the same region. Region motion change data is used to represent the inter-frame displacement amplitude and direction of the overall position of the same region. Region motion rhythm data is used to represent whether the region motion changes are repetitive within a continuous time window. Region change duration data is used to represent the duration of a region morphological change after exceeding a preset change threshold.

[0048] The formula for calculating the distance between preset key points within the area is:

[0049] ;

[0050] In the formula, This represents the distance value of the keypoint in the r-th region of active facial expression in the i-th frame of the image. and This represents the normalized coordinates of the first preset keypoint within the r-th active facial expression region in the i-th frame image. and This represents the normalized coordinates of the second preset keypoint within the r-th active facial expression region in the i-th frame image.

[0051] Regional morphological change data can be obtained from the changes in the distance between key points in adjacent frames:

[0052] ;

[0053] In the formula, This represents the region morphological change data of the r-th expression-active area in the i-th frame image. This represents the distance value of the keypoint in the r-th region of active facial expression in the i-th frame of the image. This represents the distance value between key points in the same active facial expression area in the previous frame image. This indicates the prevention of extremely small positive numbers with a denominator of 0.

[0054] Regional motion change data can be obtained through the inter-frame displacement of the region's center point:

[0055] ;

[0056] ;

[0057] In the formula, This represents the inter-frame displacement magnitude of the r-th facial expression active region in the i-th frame image. This represents the inter-frame displacement direction of the r-th facial expression active region in the i-th frame image. and Let x and y represent the x and y coordinates of the center point of the r-th facial expression active region in the i-th frame image, respectively. and These represent the x and y coordinates of the center point of the same active facial expression area in the previous frame image, respectively.

[0058] The duration of regional changes can be obtained by counting the number of consecutive frames exceeding a preset change threshold:

[0059] ;

[0060] In the formula, This represents the duration of changes in the r-th region of active facial expression in the i-th frame of the image. This represents the number of consecutive frames before and after the i-th frame that satisfy the preset change threshold for regional morphological change data. This indicates the time interval between adjacent image frames.

[0061] By using regional morphological change data to reflect the distance changes of key points within the active facial expression region, regional motion change data to reflect the overall displacement of the region, regional motion rhythm data to reflect repetitive motion characteristics within a continuous time window, and regional change duration data to reflect the duration of change, the feature data of the active facial expression region simultaneously possesses regional and temporal dimensions. This step provides foundational data for subsequently determining the temporal overlap ratio, regional response difference value, and candidate interference type identification, avoiding reliance solely on a single frame of static facial expression for judgment and improving the stability of non-pain interference identification.

[0062] Furthermore, the identification of non-painful interference state data is specifically carried out as follows: Time-series related events, event start and end times, event type identifiers, and event durations are extracted from time-series related data to obtain a time-series related event sequence; region identifiers, time identifiers, and region change features are extracted from facial expression active region feature data to obtain a region change sequence; based on the comparison of region change features with preset region change thresholds, the region change time intervals of facial expression active regions within preset time windows are selected; the overlap time between the event time intervals corresponding to time-series related events and the region change time intervals is statistically analyzed, and the ratio is calculated with the corresponding total duration to obtain the time overlap ratio; the mean regional change amplitude within a preset control time window before the occurrence of the time-series related event and the mean regional change amplitude within a preset response time window after the occurrence of the time-series related event are calculated respectively, and the difference between the two is normalized to obtain the region response difference value; based on the event type identifier, region identifier, and region change features, preset interference type rules are matched to obtain candidate interference type identifiers; The interference matching value is obtained by weighted fusion of the time overlap ratio and the regional response difference value. The candidate interference type identifier, event type identifier, region identifier, time identifier, time overlap ratio, regional response difference value and interference matching value are associated to form non-pain interference candidate data. The interference matching value in the non-pain interference candidate data is compared with the preset interference matching threshold. The non-pain interference candidate data with the interference matching value above the threshold is determined as the target non-pain interference candidate data. Otherwise, it is not determined as the target non-pain interference candidate data. The interference type code is obtained based on the candidate interference type identifier in the target non-pain interference candidate data. The interference start and end time is obtained based on the time identifier in the target non-pain interference candidate data. The interference affected area is obtained based on the region identifier in the target non-pain interference candidate data. The interference intensity is obtained based on the interference matching value in the target non-pain interference candidate data. The interference type code, interference start and end time, interference affected area and interference intensity are associated to obtain the non-pain interference state data.

[0063] In this embodiment, the regional change time interval refers to the time range within which the regional change characteristics of the active facial expression area exceed a preset regional change threshold. The time overlap ratio is the ratio of the intersection duration to the union duration between the event time interval and the regional change time interval. The regional response difference value is a normalized value obtained by normalizing the difference in regional change amplitude before and after the occurrence of the time-related event. The interference matching value is a value obtained by weighted fusion of the time overlap ratio and the regional response difference value, used to determine whether the non-pain interference candidate data can be identified as non-pain interference state data.

[0064] The regional response difference value is obtained using the normalized difference calculation formula:

[0065] ;

[0066] In the formula, This represents the difference in regional response between the r-th active facial expression region and the event occurring in the j-th time period. This represents the average change in the r-th active facial expression area within a preset response time window after the occurrence of the associated event in the j-th time period. This represents the average change in the r-th active facial expression area within a preset comparison time window before the occurrence of the associated event in the j-th time period. This indicates the prevention of extremely small positive numbers with a denominator of 0.

[0067] The interference matching value is calculated as follows:

[0068] ;

[0069] ;

[0070] In the formula, This represents the interference matching value between the associated event in the j-th time period and the active facial expression region in the r-th time period. This indicates the proportion of time overlap. Indicates the regional response difference value. The weighting of the influence of the time overlap ratio. This indicates the weight of the impact of regional response differences.

[0071] The method for obtaining the influence weights of time overlap ratio and regional response difference is as follows: collect historical facial image data and historical time period related data, manually label which regional changes belong to non-pain interference, set multiple sets of candidate influence weights, make the sum of the two weights equal to 1, calculate the interference matching value in the historical samples respectively, compare it with the manual labeling results, and select the set of candidate influence weights with the highest recognition accuracy as the influence weight.

[0072] The interference matching threshold is obtained by calculating the interference matching value using the same historical sample, setting multiple candidate interference matching thresholds, judging samples with interference matching values ​​above the candidate thresholds as non-pain interference candidate data, comparing them with the manually labeled results, and selecting the candidate threshold with the lowest combined false positive rate and false negative rate as the preset interference matching threshold.

[0073] The time overlap ratio reflects the degree of synchronization between time-related events and regional changes. The regional response difference reflects whether the magnitude of regional changes has changed significantly before and after the event. The two are fused by interference matching value, thereby avoiding making a single judgment based solely on event records or image changes. This can identify changes from non-pain sources from facial expression active area feature data and improve the stability of screening candidate changes for pain expressions.

[0074] The interference intensity is obtained based on the interference matching value in the target non-pain interference candidate data. Specifically, the interference matching value corresponding to the target candidate interference type identifier, region identifier, and time identifier in the target non-pain interference candidate data is read, and this interference matching value is used as the interference intensity. The interference intensity is a dimensionless value between 0 and 1; the larger the interference matching value, the greater the interference intensity, indicating a stronger non-pain interference effect under the corresponding time identifier and region identifier.

[0075] Furthermore, candidate interference type identifiers are obtained. Specifically, the following methods are used: extracting regional morphological change data, regional motion change data, regional motion rhythm data, and regional change duration data corresponding to the regional identifier from the regional change features; when the regional identifier corresponds to the eye region, obtaining the closure amplitude of the regional morphological change data corresponding to the eye region; if the closure amplitude is above a preset closure amplitude threshold, and the regional morphological change data corresponding to the eyebrow region, lip region, and jaw region are all below a preset morphological change threshold, the candidate interference type identifier is determined as a natural eye-closing interference identifier; when the regional motion change data corresponding to a preset number of expressive active regions are all above a preset motion change threshold, and the corresponding regional motion rhythm data are all above a preset rhythm threshold, the candidate interference type identifier is determined as a shivering interference identifier; when both regional identifiers corresponding to the lip region and the jaw region exist simultaneously in the regional change sequence, the regional morphological change data corresponding to the lip region and the jaw region are obtained respectively. The opening and closing change value is calculated, and the opening and closing duration is obtained from the corresponding regional change duration data of the lip region and the jaw region. If the opening and closing change value is above the preset opening and closing change threshold and the opening and closing duration is above the preset opening and closing duration threshold, the candidate interference type identifier is determined as the response speech interference identifier. When the facial occlusion ratio in the effective image state data is above the preset occlusion threshold and the event type identifier corresponds to the operation event, the candidate interference type identifier is determined as the operation interference identifier. When there are any two or more candidate interference type identifiers among the natural eye closing interference identifier, shivering interference identifier, response speech interference identifier, or operation interference identifier, all two or more candidate interference type identifiers are used as candidate interference type identifiers, and they are associated with the corresponding region identifier and time identifier respectively and written into the non-pain interference candidate data. When there are no natural eye closing interference identifier, shivering interference identifier, response speech interference identifier, or operation interference identifier, the candidate interference type identifier is determined as the no-interference identifier.

[0076] In this embodiment, as Figure 2 As shown, Figure 2This is a flowchart illustrating the process of identifying non-painful interference state data according to the present invention. The flowchart shows the process of identifying non-painful interference state data. Time-series correlation data is used to obtain time-series related event sequences, facial expression active area feature data is used to obtain area change sequences, and area change features are compared with thresholds to obtain area change time intervals. Event time intervals and area change time intervals are used to calculate the time overlap ratio, and the area change amplitude before and after the event is used to calculate the area response difference value. Candidate interference type identifiers are obtained based on preset interference type rules, and non-painful interference candidate data is generated by combining interference matching values. When the interference matching value reaches the interference matching threshold, non-painful interference state data is obtained.

[0077] Candidate interference type identifiers are used to indicate the possible interference types corresponding to non-painful interference candidate data. The natural eye-closing interference identifier indicates a situation where there are significant changes in eye closure while other expressive areas show little change in morphology. The shivering interference identifier indicates a situation where multiple expressive areas simultaneously exhibit rhythmic movement changes. The response-speaking interference identifier indicates a situation where the lips and jaw area show continuous opening and closing changes. The manipulation interference identifier indicates a situation where the facial occlusion ratio is high and the event type identifier corresponds to a manipulation event.

[0078] By differentiating candidate interference types through variations in the active areas of different facial expressions, non-pain-related changes such as natural eye closure, shivering, responding speech, and operational interference can be identified separately. This step does not directly output pain assessment results but provides a typological basis for the candidate data of non-pain interference. This allows subsequent interference matching value comparison, region-time interference mapping, and region-time filtering to process different interference types, thereby reducing the impact of non-pain-related changes on the visual assessment data of pain expressions.

[0079] Further, filtered facial expression feature data is obtained through the following method: Interference type code, interference start and end time, interference affected area, and interference intensity are extracted from the non-pain interference state data; interference time period data is obtained based on the interference start and end time, and interference area data is obtained based on the interference affected area; the interference time period data and interference area data are associated according to the interference type code to obtain regional time interference mapping data, which consists of interference type code, interference time period data, interference area data, and interference intensity; the regional time interference mapping data is matched with the area identifier and time identifier in the facial expression active area feature data to obtain interference matching results; when the interference matching result indicates that the facial expression active area feature data falls within the interference time period and belongs to the interference affected area, a filtering level identifier is written to the facial expression active area feature data based on the interference intensity; facial expression active area feature data whose filtering level identifier reaches the preset rejection level are removed, and facial expression active area feature data whose filtering level identifier does not reach the preset rejection level are downweighted to obtain filtered facial expression feature data.

[0080] In this embodiment, the regional time interference mapping data is data obtained by associating interference type encoding, interference time period data, interference region data, and interference intensity, and is used to indicate which facial expression active regions are affected by a certain non-pain interference within a specific time period.

[0081] The filter level indicator is used to represent the level of filtering after the facial expression activity region feature data is affected by non-painful interference. The filter level indicator can be obtained by comparing the interference intensity with a preset filter level threshold.

[0082] The specific filtration level designation is as follows:

[0083] ;

[0084] In the formula, This represents the filter level label corresponding to the feature data of the r-th expression active region in the i-th frame image. This represents the interference intensity of the associated event in the j-th time period on the r-th facial expression active area. This indicates the preset rejection level threshold; This indicates the preset threshold for the reduction level. The greater the interference intensity, the higher the filtering level indicator.

[0085] The weighted processing yields the filtered feature vector of the active facial expression region:

[0086] ;

[0087] ;

[0088] In the formula, This represents the weighting coefficient of the r-th expression-active region in the i-th frame of the image. This represents the feature vector of the active facial expression region before filtering. This represents the feature vector of the active facial expression region after filtering.

[0089] The greater the interference intensity, the smaller the weighting coefficient, and the lower the degree of preservation of the feature vector of the active facial expression region after filtering.

[0090] The preset rejection level threshold and preset deweighting level threshold can be determined through historical samples. Specifically, the interference intensity in historical samples is compared with the rejection, deweighting, and retention results of manual annotation, multiple candidate thresholds are set, and the threshold group that makes the classification result have the highest consistency rate with the manual annotation result is selected.

[0091] Furthermore, credible facial expression feature data is obtained through the following methods: image credibility is obtained based on the proportion of visible faces, the proportion of occluded faces, and image clarity in the effective image state data; keypoint stability is obtained based on the positional fluctuation amplitude of the same keypoint in the facial keypoint dataset within consecutive frames; interference residual value is obtained based on the interference intensity and interference start and end time in the non-pain interference state data; image credibility, keypoint stability, and interference residual value are weighted and fused to obtain fused credibility; feature credibility data is obtained based on image credibility, keypoint stability, interference residual value, and fused credibility; feature credibility data is matched with filtered facial expression feature data according to region identifier and time identifier to obtain credibility matching results; and the filtered facial expression feature data is weighted and corrected based on the credibility matching results to obtain credible facial expression feature data.

[0092] In this embodiment, as Figure 3 As shown, Figure 3 This is a flowchart of the regional time filtering and weighted correction process of the present invention. The flowchart illustrates the regional time filtering and weighted correction process. Non-pain interference state data is used to extract interference type code, interference start and end time, interference affected area and interference intensity parameters, and generate regional time interference mapping data. After the regional time interference mapping data is matched with facial expression active area feature data, interference matching results are obtained. Features that fall within the interference time period and belong to the interference affected area are written into the filtering level identifier, and are removed or downweighted according to the removal level. The filtered facial expression feature data is then matched with feature credibility data and weighted correction is performed to obtain credible facial expression feature data.

[0093] Image confidence is a numerical value derived from the proportion of visible faces, the proportion of occluded faces, and image sharpness, used to represent the usability of the image frame itself. Keypoint stability is a numerical value derived from the positional fluctuation of the same keypoint within consecutive frames, used to represent the stability of facial keypoint localization. Disturbance residue is a numerical value derived from the interference intensity and the start and end times of the interference, used to represent the degree to which filtered facial expression feature data is still affected by non-painful interference at the corresponding time marker. Fusion confidence is a weighted fusion of image confidence, keypoint stability, and interference residue.

[0094] Image credibility is calculated using the following method:

[0095] ;

[0096] In the formula, This represents the image credibility of the i-th frame. This represents the proportion of the face visible in the i-th frame image. This represents the proportion of facial occlusion in the i-th frame image. This represents the image sharpness of the i-th frame.

[0097] The stability value of a key point can be obtained from the position fluctuation amplitude:

[0098] ;

[0099] In the formula, This represents the keypoint stability value of the r-th expression-active region in the i-th frame image. This represents the positional fluctuation range of key points within the r-th active facial expression region in the i-th frame image across consecutive frames.

[0100] The method for obtaining the position fluctuation amplitude is as follows:

[0101] ;

[0102] In the formula, Indicates the first The frame image corresponds to the preset continuous time window of the first frame. The fluctuation range of the position of the active facial expression area Indicates the first The number of image frames contained within a preset continuous time window corresponding to a frame image. Indicates the first The number of key points contained within each active facial expression area Indicates the number of consecutive time windows within a preset time window. The first frame of the image The first expression active area Normalized x-coordinates of key points Indicates the number of consecutive time windows within a preset time window. The first frame of the image The first expression active area Normalized ordinates of key points.

[0103] The residual value of the interference can be obtained by the exponential decay calculation formula:

[0104] ;

[0105] In the formula, This represents the residual interference value of the r-th facial expression active region in the i-th frame image. This represents the interference intensity of the associated event in the j-th time period on the r-th facial expression active area. This represents the preset attenuation coefficient. This represents the time distance between the timestamp of the i-th frame and the start and end times of the interference associated with the j-th time period.

[0106] The preset attenuation coefficient is obtained from historical samples. Specifically, multiple sets of historical facial image data and corresponding historical time period data are collected. The residual impact level after the end of non-painful interference in each set of historical samples is manually labeled. For each set of historical samples, the regional change amplitude at different time intervals after the end of non-painful interference is statistically analyzed. The ratio of the regional change amplitude at each time interval to the regional change amplitude at the end of interference is calculated to obtain the historical residual ratio. Multiple candidate attenuation coefficients are set. Each candidate attenuation coefficient is substituted into the exponential attenuation model. The error between the model residual ratio and the historical residual ratio is calculated. The candidate attenuation coefficient with the smallest error is selected as the preset attenuation coefficient.

[0107] The fusion credibility is obtained through a custom weighted fusion formula:

[0108] ;

[0109] ;

[0110] In the formula, This represents the fusion confidence of the r-th expression-active region in the i-th frame image. Indicates the credibility of the image. Indicates the stable value at key points. Indicates the residual value of interference. The weights representing the influence of image credibility This indicates the influence weight of the key point stability value. This indicates the impact weight corresponding to the residual value of the interference.

[0111] The weighted adjustment is as follows:

[0112] ;

[0113] In the formula, This represents the weighted and corrected reliable facial expression feature vector. Indicates the credibility of the fusion. This represents the feature vector of the active facial expression region after filtering.

[0114] The method for obtaining the influence weights and preset decay coefficients is as follows: based on historical samples, multiple sets of candidate influence weights and candidate decay coefficients are set, the fusion credibility is calculated, and the weighted and corrected facial expression features are compared with the manually labeled available facial expression features, and the set of candidate values ​​with the highest consistency rate is selected.

[0115] By introducing image confidence, keypoint stability, and interference residue after regional temporal filtering, the filtered facial expression feature data can be weighted and corrected. Image confidence is used to avoid the excessive influence of low-quality image frames, keypoint stability is used to suppress errors caused by unstable keypoint localization, and interference residue is used to reduce the persistent influence of non-painful interference on features near the interference boundary.

[0116] Furthermore, visual assessment data of pain expressions is obtained through the following methods: The credible expression feature data is segmented according to a preset assessment time window to obtain segmented credible expression feature data; the continuity of change of the segmented credible expression feature data between adjacent assessment time windows is calculated to obtain the expression change continuity; the number of continuous frames of the segmented credible expression feature data within the same assessment time window is calculated to obtain the expression change duration; the expression change continuity and expression change duration are compared with preset continuity thresholds and preset duration thresholds, respectively, to obtain candidate expression segment data; the candidate expression segment data is then subjected to temporal smoothing to obtain visual assessment data of pain expressions. This visual assessment data of pain expressions includes candidate expression segment data, expression change continuity, expression change duration, and temporal smoothing results.

[0117] In this embodiment, segmented credible facial expression feature data refers to data obtained by segmenting credible facial expression feature data according to a preset evaluation time window. Facial expression change continuity indicates whether the changes in credible facial expression features are smooth between adjacent evaluation time windows. Facial expression change persistence indicates the degree to which credible facial expression features persist within the same evaluation time window. Temporal smoothing results are obtained by smoothing the candidate facial expression segment data.

[0118] The continuity of facial expression changes is as follows:

[0119] ;

[0120] In the formula, Indicates the first The continuity of facial expression changes within an assessment time window. Indicates the first The average feature vector of segmented credible facial expression feature data within each evaluation time window. This represents the average feature vector of the segmented credible facial expression feature data within the previous evaluation time window. This represents the Euclidean distance. The closer the average eigenvectors are between adjacent evaluation time windows, the greater the continuity of facial expression changes.

[0121] The duration of facial expression changes can be calculated using a proportional formula:

[0122] ;

[0123] In the formula, Indicates the first The duration of facial expression changes within an assessment time window. Indicates the first The number of frames that meet the preset candidate conditions within each evaluation time window. Indicates the first The total number of frames within an evaluation time window. The more frames that meet the preset candidate conditions, the greater the duration of the facial expression change.

[0124] Perform time smoothing to obtain the time smoothing result, specifically:

[0125] ;

[0126] In the formula, Indicates the first The time smoothing results of each evaluation time window Indicates the first The average feature vector of segmented credible facial expression feature data within each evaluation time window. This indicates the time smoothing result of the previous evaluation time window. This represents the smoothing coefficient.

[0127] The larger the smoothing coefficient, the greater the impact of the segmented reliable facial feature data of the current evaluation time window on the time smoothing result; the smaller the smoothing coefficient, the greater the impact of the time smoothing result of the previous evaluation time window on the current result.

[0128] The smoothing coefficient is obtained by setting multiple sets of candidate smoothing coefficients using historical reliable facial expression feature data, comparing the time-smoothed result with manually annotated continuous facial expression segments, and selecting the candidate smoothing coefficient that can reduce short-term jumps and does not significantly delay continuous facial expression changes.

[0129] By segmenting credible facial expression feature data through a preset evaluation time window, the direct impact of single-frame fluctuations on the visual evaluation data of pain expressions can be avoided. Judging the smoothness of changes between adjacent time windows by the continuity of facial expression changes, and judging the persistence of changes within the same time window by the duration of facial expression changes, can reduce misjudgments caused by transient noise or instantaneous movements. Temporal smoothing processing makes the visual evaluation data of pain expressions more stable over time. Overall, this step further integrates the features, after filtering out interference from non-pain sources and weighting corrections, into stable candidate change data for pain expressions.

[0130] like Figure 4 As shown, Figure 4 The system architecture diagram of this invention is shown below. An embodiment of this application provides a visual assessment system for post-anesthesia pain expression, comprising the following modules: a data acquisition module, used to acquire facial image data and time-related data of a user during a preset time period after anesthesia, and perform temporal alignment processing to obtain temporal facial image data and temporal related data; an image detection module, used to perform imaging validity detection and facial key point localization processing based on the temporal facial image data to obtain image validity state data, facial key point dataset, and expression active region data; a feature analysis module, used to perform continuous time window change feature analysis based on the facial key point dataset and expression active region data to obtain expression active region feature data; and a temporal coupling analysis module, used to perform temporal coupling analysis based on the temporal related data and expression active region feature data to obtain the time between time-related events and changes in expression active regions. The system comprises the following modules: a temporal overlap ratio value and a regional response difference value; an interference identification module, used to identify non-painful interference state data based on the temporal overlap ratio value and the regional response difference value, and to obtain regional temporal interference mapping data based on the non-painful interference state data; a feature filtering module, used to perform regional temporal filtering on the facial expression active region feature data based on the regional temporal interference mapping data, to obtain filtered facial expression feature data; a credibility analysis module, used to obtain feature credibility data based on the effective image state data, facial key point dataset, and non-painful interference state data, and to perform weighted correction on the filtered facial expression feature data based on the feature credibility data, to obtain credible facial expression feature data; and a data determination module, used to perform temporal continuity integration processing on the credible facial expression feature data, to obtain pain expression visual evaluation data, which is used to characterize the changes in pain expression candidates after filtering by non-painful interference sources.

[0131] The technical scope of this invention is not limited to the content described above. Those skilled in the art can make various modifications and variations to the above embodiments without departing from the technical concept of this invention, and all such modifications and variations should fall within the protection scope of this invention.

Claims

1. A visual assessment method for post-anesthesia pain expression, characterized in that: Includes the following steps: After anesthesia, facial image data and time-related data of the user during a preset time period are acquired and time-series aligned to obtain time-series facial image data and time-series related data. Based on temporal facial image data, imaging validity detection and facial key point localization are performed to obtain image validity state data, facial key point dataset, and expression active region data. Based on the facial key point dataset and the data of active facial expression regions, continuous time window change feature analysis was performed to obtain the feature data of active facial expression regions. Based on temporal correlation data and facial expression activity area feature data, a temporal coupling analysis was performed to obtain the temporal overlap ratio and regional response difference between time-related events and changes in facial expression activity areas. Non-painful interference state data are identified based on the time overlap ratio and regional response difference, and regional time interference mapping data are obtained based on the non-painful interference state data. Based on the regional temporal interference mapping data, regional temporal filtering processing is performed on the facial expression active region feature data to obtain filtered facial expression feature data. Feature credibility data is obtained based on effective image state data, facial key point dataset, and non-pain interference state data. The filtered expression feature data is then weighted and corrected based on the feature credibility data to obtain credible expression feature data. The reliable facial expression feature data is integrated over time to obtain visual evaluation data of pain expression. This data is used to characterize the changes in candidate pain expressions after filtering out interference from non-pain sources.

2. The visual assessment method for post-anesthesia pain expression according to claim 1, characterized in that: The specific method for obtaining temporal facial image data and temporal correlation data is as follows: The system acquires user facial image frames and writes image time identifier, frame number, and acquisition status identifier into each user facial image frame to obtain facial image data, which includes facial image frames, image time identifier, frame number, and acquisition status identifier. The system acquires time-related events that a user is within a preset time period, and writes an event time identifier, an event type identifier, and an event duration into each time-related event to obtain time-related data. The time-related data consists of time-related events, event time identifiers, event type identifiers, and event durations. Image time windows are formed based on image time markers according to a preset time window length. Event time windows corresponding to time period-related events are determined based on event time markers. Image time windows and event time windows are matched to obtain time window matching data. Based on time window matching data, a correspondence is established between facial image frames falling into the same preset time window and time period-related events, and an event matching identifier is written for the user's facial image frame to obtain time-series facial image data. The time-series facial image data includes user facial image frames sorted according to image time identifiers, frame number, acquisition status identifier, and event matching identifier. Based on time window matching data, time-related events corresponding to the same preset time window are sorted according to the event time identifier to obtain time-series related data. The time-series related data includes preset time window identifier, time-series related events, event time identifier, event type identifier, event duration, and event arrangement data. When there are no time-related events within the same preset time window, a no-event flag is written to the corresponding user facial image frame. When there are multiple time-related events within the same preset time window, the event arrangement data is obtained according to the order of the event time flags, and the event arrangement data is written to the time-related data.

3. The visual assessment method for post-anesthesia pain expression according to claim 1, characterized in that: The specific method for obtaining the effective state data of the image, the facial key point dataset, and the expression active region data is as follows: Based on the analysis of time-series facial image data, effective image state data is obtained, which includes the proportion of visible face, the proportion of occluded face, image clarity, and effective image identifiers. Based on the valid state data of the images, the validity is screened to obtain image frames that meet the preset valid conditions, and then summarized into valid facial image data. Facial key points are located from valid facial image data and scale normalization is performed to obtain a facial key point dataset. The facial key point dataset is matched with the key point combination of the preset expression active region to obtain expression active region data, which includes region identifier, region key point combination and region location range.

4. The method for visually assessing post-anesthesia pain expression according to claim 1, characterized in that: The specific method for obtaining the facial expression active region feature data is as follows: Extract the corresponding region key point sequence from the facial key point dataset based on the facial expression active region data; Frame distance variation is calculated based on the regional key point sequence to obtain regional morphological change data; The inter-frame displacement amplitude and inter-frame displacement direction of the key point sequence in the region are calculated to obtain the regional motion change data. Based on a preset continuous time window, periodic statistics are performed on the regional motion change data to obtain regional motion rhythm data; The duration of regional changes is obtained based on the number of consecutive frames where the regional morphological change data exceeds a preset change threshold. By associating regional morphological change data, regional motion change data, regional motion rhythm data, and regional change duration data according to regional identifiers and time identifiers, we obtain facial expression active region feature data. The facial expression active region feature data includes regional identifiers, time identifiers, regional morphological change data, regional motion change data, regional motion rhythm data, and regional change duration data.

5. The visual assessment method for post-anesthesia pain expression according to claim 1, characterized in that: The specific method for identifying non-painful interference state data is as follows: Extract time-series related events, event start and end times, event type identifiers, and event durations from time-series related data to obtain time-series related event sequences; Extracting region identifiers, time identifiers, and region change features from facial expression activity region feature data to obtain region change sequences; Based on the comparison between regional change characteristics and preset regional change thresholds, the time interval of regional change in the active expression region within the preset time window is selected. The overlap time between the event time interval corresponding to the related event in the statistical period and the regional change time interval is calculated as a ratio to the corresponding total time to obtain the time overlap ratio. The mean regional change amplitude within a preset control time window before the occurrence of the time-related event and the mean regional change amplitude within a preset response time window after the occurrence of the time-related event are calculated respectively. The difference between the two is then normalized to obtain the regional response difference value. Candidate interference type identifiers are obtained by matching preset interference type rules based on event type identifiers, region identifiers, and region change characteristics. The interference matching value is obtained by weighted fusion processing based on the time overlap ratio and the regional response difference. The candidate interference type identifier, event type identifier, region identifier, time identifier, time overlap ratio, region response difference value, and interference matching value are associated to form non-pain interference candidate data; Based on the comparison between the interference matching value within the non-pain interference candidate data and the preset interference matching threshold, the non-pain interference candidate data corresponding to the interference matching value above the interference matching threshold is determined as the target non-pain interference candidate data; otherwise, it is not determined as the target non-pain interference candidate data. The interference type code is obtained based on the candidate interference type identifier in the target non-pain interference candidate data. The interference start and end time is obtained based on the time identifier in the target non-pain interference candidate data. The interference affected area is obtained based on the area identifier in the target non-pain interference candidate data. The interference intensity is obtained based on the interference matching value in the target non-pain interference candidate data. The interference type code, interference start and end time, interference affected area and interference intensity are associated to obtain the non-pain interference state data.

6. The method for visually assessing post-anesthesia pain expression according to claim 5, characterized in that: The specific method for obtaining the candidate interference type identifier is as follows: Extract regional morphological change data, regional motion change data, regional motion rhythm data, and regional change duration data corresponding to regional identifiers from regional change characteristics; When the region identifier corresponds to the eye region, the closure amplitude of the region morphological change data corresponding to the eye region is obtained. If the closure amplitude is above the preset closure amplitude threshold, and the region morphological change data corresponding to the eyebrow region, lip region, and jaw region are all less than the preset morphological change threshold, the candidate interference type identifier is determined as the natural closed eye interference identifier. When the motion change data of the corresponding regions of a preset number of active facial expression regions are all above the preset motion change threshold, and the corresponding region motion rhythm data are all above the preset rhythm threshold, then the candidate interference type is identified as the shivering interference identifier. When both the region identifier corresponding to the lip region and the region identifier corresponding to the mandibular region exist in the region change sequence, the opening and closing change values ​​in the region morphology change data corresponding to the lip region and the mandibular region are obtained respectively, and the opening and closing duration in the region change duration data corresponding to the lip region and the mandibular region is obtained respectively. If the opening and closing change value is above the preset opening and closing change threshold and the opening and closing duration is above the preset opening and closing duration threshold, then the candidate interference type identifier is determined as the response speech interference identifier. When the proportion of facial occlusion in the valid state data of an image is above the preset occlusion threshold, and the event type identifier corresponds to an operation event, then the candidate interference type identifier is determined as the operation interference identifier. When there are any two or more candidate interference type identifiers among the natural eye-closing interference identifier, shivering interference identifier, response speaking interference identifier, or operation interference identifier, then all two or more candidate interference type identifiers are treated as candidate interference type identifiers, and they are associated with the corresponding area identifier and time identifier respectively and written into the non-pain interference candidate data. When there are no interference indicators for natural eye closure, shivering, response speech, or operation, the candidate interference type indicator is determined as the no-interference indicator.

7. The method for visually assessing post-anesthesia pain expression according to claim 1, characterized in that: The specific method for obtaining the filtered facial expression feature data is as follows: Extract the interference type code, interference start and end time, interference affected area and interference intensity from the non-pain interference state data; The interference time period data is obtained based on the start and end times of the interference, and the interference area data is obtained based on the area affected by the interference. Interference time period data and interference area data are associated according to interference type code to obtain regional time interference mapping data. Regional time interference mapping data consists of interference type code, interference time period data, interference area data and interference intensity. The interference matching results are obtained by matching the regional temporal interference mapping data with the regional identifiers and time identifiers in the facial expression active region feature data. When the interference matching result indicates that the feature data of the active facial expression region falls into the interference time period and belongs to the interference-affected area, a filter level label is written to the feature data of the active facial expression region based on the interference intensity. The facial expression feature data that reaches the preset rejection level is removed, and the facial expression feature data that does not reach the preset rejection level is downweighted to obtain filtered facial expression feature data.

8. The method for visually assessing post-anesthesia pain expression according to claim 1, characterized in that: The specific method for obtaining reliable facial expression feature data is as follows: Image credibility is obtained based on the proportion of visible faces, the proportion of occluded faces, and the image sharpness in the effective state data of the image. The stable value of a key point is obtained by measuring the positional fluctuation of the same key point in consecutive frames in a facial key point dataset. The residual value of interference is obtained based on the interference intensity and interference start and end time in the non-painful interference state data; The image credibility, key point stability value, and interference residual value are weighted and fused to obtain the fusion credibility. Feature credibility data is obtained based on image credibility, key point stability value, interference residual value, and fusion credibility. The feature credibility data and the filtered facial expression feature data are matched according to the region identifier and time identifier to obtain the credibility matching result; The filtered facial expression feature data is weighted and corrected based on the credibility matching results to obtain credible facial expression feature data.

9. The method for visually assessing post-anesthesia pain expression according to claim 1, characterized in that: The specific method for obtaining the visual assessment data of pain expressions is as follows: The reliable facial expression feature data is segmented according to the preset evaluation time window to obtain segmented reliable facial expression feature data. The continuity of expression change is obtained by calculating the continuity of change of segmented reliable facial expression feature data between adjacent evaluation time windows. The duration of facial expression change is obtained by calculating the number of sustained frames of segmented reliable facial expression feature data within the same evaluation time window. The continuity of facial expression changes and the duration of facial expression changes are compared with preset continuity thresholds and preset duration thresholds, respectively, to obtain candidate facial expression segment data; The candidate expression segment data is subjected to time smoothing to obtain pain expression visual evaluation data, which includes candidate expression segment data, expression change continuity, expression change duration, and time smoothing results.

10. A system for visually assessing post-anesthesia pain expression using any one of claims 1-9, characterized in that: Includes the following modules: The data acquisition module is used to acquire facial image data and time-related data of users during a preset time period after anesthesia, and perform time-series alignment processing to obtain time-series facial image data and time-series related data. The image detection module is used to perform imaging validity detection and facial key point localization processing based on temporal facial image data, and to obtain image validity state data, facial key point dataset and expression active area data. The feature analysis module is used to perform continuous time window change feature analysis based on facial key point datasets and facial expression active area data to obtain facial expression active area feature data; The temporal coupling analysis module is used to perform temporal coupling analysis based on temporal correlation data and facial expression active area feature data to obtain the temporal overlap ratio and regional response difference value between time-related events and changes in facial expression active areas. The interference identification module is used to identify non-painful interference state data based on the time overlap ratio and regional response difference, and to obtain regional time interference mapping data based on the non-painful interference state data. The feature filtering module is used to perform regional temporal filtering on the feature data of active facial expressions based on the regional temporal interference mapping data, so as to obtain filtered facial expression feature data. The credibility analysis module is used to obtain feature credibility data based on image valid state data, facial key point dataset, and non-pain interference state data, and to perform weighted correction on filtered expression feature data based on feature credibility data to obtain credible expression feature data. The data determination module is used to perform temporal continuous integration processing on credible facial expression feature data to obtain visual evaluation data of pain expressions. The visual evaluation data of pain expressions is used to characterize the changes in candidate pain expressions after filtering out interference from non-pain sources.

Images