A pediatric pain assessment system

Abstract

The present invention relates to the field of pain assessment technology, specifically a pediatric pain assessment system, including a muscle group rhythm recognition module, a behavioral linkage screening module, a periodic fluctuation extraction module, a behavioral-physiological difference module, and a pain grading judgment module. In the present invention, a linkage offset recognition path is constructed through the angular relationship between facial muscle groups and the start-up timing, the multi-muscle coordination imbalance manifestation is identified, and a regional-level dynamic behavior judgment basis is formed. The response overlap of multiple behavioral characteristics in the same time period is used to establish a behavioral coordination discrimination mechanism, exclude path signals that do not form effective coordination, enhance the effectiveness of behavioral characteristics, introduce a continuous trend structure of peak time in rhythmic changes, construct a periodic response mutation monitoring channel, improve the recognition sensitivity of short-term fluctuation concentrated segments, extract response mismatch segments based on the trigger time misalignment state between behavioral and physiological responses, and locate pain judgment key points in the timing overlap area.

Description

Technical Field

[0001] The present invention relates to the technical field of pain assessment, and in particular to a pediatric pain assessment system. Background Art

[0002] Pain assessment technology is a key branch in medicine and nursing. It mainly studies how to objectively and accurately identify and quantify patients' pain experience. This field covers a variety of methods such as physiological indicator monitoring, behavioral observation scales, self-reporting tools, image recognition and AI-assisted analysis, and is widely used in clinical diagnosis and treatment, chronic disease management, postoperative recovery and emergency treatment.

[0003] Among them, the pediatric pain assessment system is a pain identification and assessment system designed specifically for children. It aims to overcome problems such as children's limited ability to express themselves and difficulty in subjective description. The system helps medical staff accurately judge the degree of pain in children by integrating technologies such as facial expression recognition, crying analysis, physiological parameter monitoring and multidimensional behavioral assessment scales. Its main uses include pain assessment during clinical treatment, pain management during routine operations such as postoperative care and vaccination, as well as continuous monitoring of children with chronic diseases, thereby providing accurate and scientific support for pediatric medical services.

[0004] Existing technologies often make threshold judgments based on single-frame facial images or crying amplitude curves, ignoring the coupling state of path angles between multiple muscles and the initiation sequence of movements. This makes it difficult to identify rhythm misalignment between movements, and no interactive judgment framework is established between behavioral characteristics, resulting in a lack of linkage reference benchmarks for different types of behavioral signals. Paths with failed linkage cannot be effectively identified and included in the evaluation. Physiological curves and behavioral events belong to independent trajectories, and rhythmic mutation reactions cannot be monitored in advance through peak trend evolution, resulting in the inability to fully extract the periodic structure. The misalignment state where behavior precedes or lags behind physiological signals lacks differential inversion judgment within a time period, and the abnormal initiation sequence in pain expression is misjudged or missed. The evaluation results lack a stable cross-mapping basis, making it difficult to achieve hierarchical output in the context of multidimensional and complex data, affecting the adaptation accuracy and response timeliness of pain management measures. Summary of the Invention

[0005] In order to solve the problems in the prior art, such as threshold judgment is often made through single-frame facial images or crying amplitude curves, the coupling state of the path angles between multiple muscles and the action start sequence is ignored, it is difficult to identify the rhythm misalignment phenomenon between actions, no interactive judgment framework is built between behavioral characteristics, resulting in a lack of linkage reference benchmarks for different types of behavioral signals, the paths of failed linkage cannot be effectively identified and included in the evaluation, physiological curves and behavioral events belong to independent trajectories, rhythm mutation reactions cannot be monitored in advance through the evolution of peak trends, resulting in the inability to fully extract the periodic structure, the misalignment state of the behavior preceding or lagging behind the physiological signal lacks differential inversion judgment within a time period, the abnormal start sequence in pain expression is misjudged or missed, the evaluation results lack a stable cross-mapping basis, it is difficult to achieve hierarchical output in the context of multi-dimensional complex data, and the technical problems of affecting the adaptation accuracy and response timeliness of pain management measures, an embodiment of the present invention provides a pediatric pain assessment system. The technical solution is as follows:

[0006] In one aspect, a pediatric pain assessment system is provided, comprising:

[0007] The muscle group rhythm recognition module extracts the starting position and movement direction of the zygomaticus major, frontalis, and depressor anguli oris muscles based on a sequence of collected children's facial images. It calculates the angles and activation intervals between the muscles, identifies movement rhythm misalignments, and selects combinations of step differences to determine the facial linkage angle offset.

[0008] The behavioral linkage screening module obtains the onset and duration of crying frequency, jaw muscle stretching, body vibration, and blinking behavioral responses based on the facial linkage angle offset, analyzes the response overlap between multiple behaviors, eliminates behavioral channels lacking linkage, and obtains invalid behavioral path data;

[0009] The periodic fluctuation extraction module calls the invalid behavior path data, draws facial rhythm, heart rate fluctuation and muscle activity curves, identifies the changing trend of signal peak spacing in adjacent time periods, screens mutation concentration areas, and obtains rhythm mutation response segments;

[0010] The behavior-physiology difference module analyzes the starting time of the crying sound and the heart rate response according to the rhythm mutation response segment, calculates the time difference therebetween, identifies the misalignment interval of the behavior and physiological response, and obtains the behavior-physiology misalignment segment.

[0011] On the other hand, the facial linkage angle offset includes the angle of the muscle group movement trajectory, the action start time label, and the linkage coordination abnormality mark; the invalid behavior path data includes the behavior response time distribution, the channel linkage density index, and the abnormal channel number set; the rhythm mutation response segment includes the signal peak time interval sequence, the periodic fluctuation amplitude data, and the mutation period segment index; the behavior physiological misalignment segment includes the starting response time comparison group, the behavior physiological synchronization offset result, and the cross-type signal misalignment segment label.

[0012] On the other hand, the muscle group rhythm recognition module includes:

[0013] The muscle point localization submodule analyzes the starting areas of the zygomatic major, frontalis, and depressor anguli oris muscles marked in consecutive frames of the collected child facial image sequence. By comparing the spatial position changes of each muscle area in adjacent frames, the coordinate correspondence of the same muscle in the difference frames is determined, and a muscle position data sequence is established.

[0014] The rhythm parameter extraction submodule calculates the change in the movement direction angle of each muscle in consecutive frames based on the relative displacement of the muscle position between each frame in the muscle position data sequence, determines the frame position sequence relationship at which the muscle starts to move, and obtains the muscle movement rhythm parameter by sorting out the change trend of each group of muscles in the image sequence;

[0015] The linkage rhythm screening submodule screens the relationship between the directional angle trends and starting frames of multiple groups of muscles in the muscle movement rhythm parameters, determines whether the directional trends between different muscles are consistent and whether the movement start sequence is coordinated, and counts the combinations that do not have the same direction or movement synchronization to obtain the facial linkage angle offset.

[0016] On the other hand, the behavior linkage screening module includes:

[0017] The crying frequency analysis submodule obtains the corresponding image sequence time period based on the facial linkage angle offset, analyzes the start and end frames of the vocalization signal in the crying frequency within the time period, calculates the frame distance distribution between consecutive vocalization segments, determines whether there is an interval change in the frame distance fluctuation, screens consecutive segments with vocalization clustering characteristics, and obtains the vocalization density distribution;

[0018] The facial stretch determination submodule analyzes the vertical displacement direction of the chin region in consecutive image frames based on the image segments covered by the dense distribution of vocalizations, compares the directional consistency of the displacement trajectory between the starting frame and the ending frame, determines whether a continuous upward or downward movement trend is formed, and selects image segments with continuous unidirectional displacement of the trajectory to obtain the stretch trajectory deviation degree;

[0019] The behavior overlap detection submodule calls the time period corresponding to the stretching trajectory offset, calculates the first and last occurrence frame marks of the blink action in the frame sequence, performs time axis cross-judgment with the body vibration frame mark interval in the same segment, filters the behavior channel data that does not form an overlapping segment, and obtains invalid behavior path data.

[0020] On the other hand, the periodic fluctuation extraction module includes:

[0021] The rhythm curve construction submodule calls the image frame segments covered by the invalid behavior path data, analyzes the facial action frame sequence, heart rate signal density and muscle activity frequency band, calculates the corresponding change trajectory positions of the three types of signals in consecutive frames, draws continuity curves in chronological order, and obtains signal rhythm distribution information;

[0022] The peak distance change identification submodule calculates the frame intervals between consecutive peak points based on the changing position of the curve trajectory in the signal rhythm distribution information, compares the change amplitudes of adjacent intervals in consecutive frame segments, selects the frame segment sequences with key changes in continuity, and obtains the curve interval fluctuation trend;

[0023] The abnormal fluctuation screening submodule determines whether there is inconsistent change direction or delayed response distribution in the three types of curves of face, heart rate and muscle based on the time period marked by the fluctuation trend of the curve interval, analyzes the concentrated time period of synchronization deviation in multiple segments of data, and obtains the rhythm mutation response segment.

[0024] On the other hand, the analysis of the concentrated period of synchronization deviation in multiple segments of data adopts the formula:

[0025] ;

[0026] Calculate the synchronization deviation value , obtain the rhythm mutation response segment, where, Represents the number of time periods, Representative Heart rate signal within a period of time, Represents the mean of the heart rate signal in all time periods, Represents the standard deviation of the heart rate signal in all time periods, Representative Muscle signals over time, Represents the mean value of muscle signals in all time periods, Represents the standard deviation of muscle signals over all time periods, Representative Facial signals over time, represents the mean value of facial signals in all time periods, Represents the standard deviation of facial signals in all time periods, Representative Time stamp for a period of time, represents the mean of the time stamps for all time periods, Represents a non-zero constant used to smooth the square root term to prevent instability caused by zero.

[0027] On the other hand, the behavior-physiology comparison module includes:

[0028] The response time extraction submodule analyzes the starting points of continuous sound waves in the vocalization trajectory based on the frame sequence range of the rhythm mutation response segment, and simultaneously detects the first dense fluctuation segment in the heart rate data. It compares the arrangement order of the first frames of the two segments on the time axis, determines their sequence relationship and establishes the corresponding terms of the frame positions to obtain the starting offset of the vocalization heart rate;

[0029] The time period difference calculation submodule analyzes the continuous fluctuation segments of crying and heart rate corresponding to the image frame axis based on the frame sequence position in the vocalization heart rate starting offset, compares the time span between the start and end frames of the two types of signals, and selects the areas with relatively stable frame segment change directions to obtain the behavior duration comparison value;

[0030] The signal consistency judgment submodule compares the fluctuation frame segments within the behavior duration comparison quantity with the fluctuation direction in the rising segment of the crying amplitude and the heart rate change trajectory to determine whether they present synchronous characteristics within the same time period, screens out intervals with reverse changes or fluctuation faults, and obtains the behavioral and physiological dislocation segments.

[0031] On the other hand, the comparison of the rising section of the crying amplitude and the fluctuation direction in the heart rate change trajectory adopts the formula:

[0032] ;

[0033] Calculating the Volatility Synchrony Index , to determine whether they present synchronization characteristics within the same time, where, Representative The amplitude of the rising segment of the frame crying sound, Representative Heart rate in the frame heart rate change track, Represents the mean value of the amplitude in the entire rising section of crying amplitude, Represents the mean of the heart rate in the entire heart rate change trajectory, Represents the total number of frames in the current comparison frame segment, Represents a non-zero constant used to smooth the square root term to prevent instability caused by zero.

[0034] In another aspect, the system further comprises:

[0035] The pain grading determination module monitors whether there are changes in facial, behavioral, and physiological signals simultaneously within the behavioral and physiological misalignment segments, identifies overlapping areas of the three types of signals, analyzes whether the characteristic positions specified in the pain judgment criteria appear, determines the pain severity grade of the corresponding area, and obtains a multi-source pain grading area.

[0036] The multi-source pain grading area includes a multi-dimensional signal fusion result, a pain grade classification number, and a reference standard mapping index.

[0037] On the other hand, the pain grading determination module includes:

[0038] The signal change detection submodule analyzes the continuous expression contour changes in the facial image, the vocalization fluctuations in the sound trajectory, and the fluctuation starting point of the heart rate curve based on the frame sequence interval covered by the behavioral physiological dislocation segment, and determines whether the signal has a linkage change within the same frame segment to obtain the signal change synchronization amount;

[0039] The overlapping segment screening submodule compares the starting trajectories of facial, behavioral, and physiological signals in the same frame sequence based on the frame segment range involved in the signal change synchronization quantity, screens the frame segments that have intersections and continue to extend in the time sequence direction, and then counts the number of overlapping ranges in the continuous distribution segment to obtain the total number of overlapping distribution segments;

[0040] The feature level matching submodule analyzes whether the corresponding position is located in the key segment set in the pain level reference sequence based on the signal overlap structure and frame segment persistence characteristics in the total amount of overlap distribution fragments, judges the degree of matching between the linkage form presented by the frame segment and the level standard, determines the corresponding pain level grade, and obtains the multi-source pain grade area.

[0041] The beneficial effects brought about by the technical solutions provided by the embodiments of the present invention include at least:

[0042] Through the angular relationship and start-up timing between facial muscle groups, a linkage offset recognition path is constructed to identify the imbalance of multi-muscle coordination and form a basis for regional-level dynamic behavior judgment. The response overlap of multiple behavioral characteristics in the same time period is used to establish a behavioral coordination discrimination mechanism, exclude path signals that do not form effective coordination, enhance the effectiveness of behavioral characteristics, introduce a continuous trend structure of peak time in rhythmic changes, construct a periodic response mutation monitoring channel, and improve the recognition sensitivity of short-term fluctuation concentration segments. Based on the trigger time misalignment state between behavioral and physiological responses, the response mismatch segment is extracted, and the key points of pain judgment are located in the timing overlap area, and the level mapping block is delineated. The overall combination judgment core is based on timing difference, channel coordination, and periodic change, forming a multi-source fusion analysis chain across behavioral and physiological dimensions, which improves the recognition clarity and level differentiation ability of pain expression under non-standard behavioral conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 A schematic diagram of the system of the present invention;

[0045] Figure 2 Schematic diagram of the system framework of the present invention;

[0046] Figure 3 This is a flow chart of the muscle group rhythm recognition module of the present invention;

[0047] Figure 4 This is a flow chart of the behavior linkage screening module of the present invention;

[0048] Figure 5 This is a flow chart of the periodic fluctuation extraction module of the present invention;

[0049] Figure 6 This is a flow chart of the behavior-physiology comparison module of the present invention;

[0050] Figure 7 This is a flow chart of the pain grading determination module of the present invention. DETAILED DESCRIPTION

[0051] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0052] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0053] In the embodiments of the present invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same. The terms "of," "corresponding," and "corresponding" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same.

[0054] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0055] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0056] The embodiment of the present invention provides a pediatric pain assessment system, such as Figure 1 As shown, the system includes:

[0057] The muscle group rhythm recognition module uses a sequence of collected children's facial images to call the starting positions and movement directions of the zygomaticus major, frontalis, and depressor anguli oris muscles in the image sequence. It calculates the movement angle and movement time interval between each pair of muscles, determines whether the movement rhythms of multiple muscles are misaligned or asynchronous, filters out inconsistent movement combinations, and obtains the facial linkage angle offset.

[0058] The behavioral linkage screening module uses facial linkage angle offset to obtain crying frequency, jaw muscle stretch ratio, body vibration changes, and blink counts within a time period. It then calculates the start time and duration of each behavioral response, analyzes the amount of temporal overlap between different behaviors, and filters out behavioral channels that do not participate in linkage as invalid information, generating invalid behavioral path data.

[0059] The periodic fluctuation extraction module uses invalid behavior path data to draw curves of changes in facial movement rhythm, heart rate fluctuation trajectory, and muscle activity duration. It checks the highest point position of each curve in adjacent time periods and records the trend of the time difference between the highest points. It determines which time periods have unstable or sudden signal distributions, selects time periods with sudden fluctuations as abnormal segments, and obtains rhythm mutation response segments.

[0060] The behavioral-physiological misalignment module analyzes the onset time of crying and heart rate responses in the rhythm mutation response segment, calculates the time difference between the two types of data, identifies the time period when the behavioral response is ahead or behind the physiological response, and determines whether there is any inconsistency in the difference type signal in this time period, thereby obtaining the behavioral-physiological misalignment segment;

[0061] The pain grading determination module is based on the behavioral and physiological misalignment segments. It monitors whether there are changes in facial, behavioral and physiological signals at the same time within the time period, identifies the overlapping areas of the three types of signals, checks the number and duration of the overlapping parts of the signals, analyzes whether the characteristic positions in the pain judgment criteria appear, determines the pain level grade of the corresponding area, and obtains the multi-source pain grading area.

[0062] The facial linkage angle offset includes the angle of muscle group movement trajectory, the action start time label, and the linkage coordination abnormality mark. The invalid behavior path data includes the behavior response time distribution, the channel linkage density index, and the abnormal channel number set. The rhythm mutation response segment includes the signal peak time interval sequence, the periodic fluctuation amplitude data, and the mutation period segment index. The behavior-physiology misalignment segment includes the starting response time comparison group, the behavior-physiology synchronization offset results, and the cross-type signal misalignment segment label. The multi-source pain grading area includes the multi-dimensional signal fusion results, the pain level classification number, and the reference standard mapping index.

[0063] The behavioral channel refers to each independent behavioral signal source in the system, which serves as a separate data input path to reflect the child's non-verbal behavioral response within a specific time period, such as the crying channel (crying frequency, intensity, and spectral distribution), the jaw muscle channel (jaw muscle stretching ratio, muscle tension state), the body vibration channel (slight limb movements or trunk tremor frequency), and the blinking channel (the frequency and time interval of blinking).

[0064] like Figure 2 and Figure 3 As shown, the muscle group rhythm recognition module includes:

[0065] The muscle point localization submodule analyzes the starting areas of the zygomatic major, frontalis, and depressor anguli oris muscles marked in consecutive frames of the collected child facial image sequence. By comparing the spatial position changes of each muscle area in adjacent frames, the coordinate correspondence of the same muscle in the difference frames is determined, and a muscle position data sequence is established.

[0066] In the image preprocessing stage, each frame of the image is subjected to regional recognition. By extracting the coordinates of the center point of the target muscle group area marked in the image, a muscle coordinate information list indexed by the frame number is formed. For example, in the 5th frame of the image, the zygomatic major muscle is identified at the 120-pixel horizontal coordinate and 80-pixel vertical coordinate position of the image, which is recorded as (120, 80). In the 6th frame, if the muscle recognition coordinate becomes (125, 85), it means that the muscle has moved to the upper right direction between the two adjacent frames. The process is repeated to traverse the center point changes of all target muscle areas in each frame of the entire image sequence, and the spatial coordinate difference results of each target muscle between consecutive frames are recorded. The data are stored to form structured spatial movement data. Then, the position relationship of the same muscle in different images in adjacent frames is compared. That is, the position change of the muscle in the two frames before and after is compared from front to back according to the frame sequence index. In this way, the full frame image is traversed in sequence. In the continuous comparison process, in order to ensure the tracking continuity of the same muscle, only the situation where the position change amplitude of the marked area is within a reasonable range is recorded. If the position drift between the two frames exceeds three times the diameter of the preset recognition area, the muscle positioning of the frame fails, and the frame data needs to be removed from the continuous sequence. Finally, a coordinate trajectory linked list data set of each muscle across all valid frames in the image sequence is formed, which is the muscle position data sequence.

[0067] The rhythm parameter extraction submodule calculates the change in the movement direction angle of each muscle in consecutive frames based on the relative displacement of muscle positions between each frame in the muscle position data sequence, determines the frame sequence relationship when the muscle starts to move, and obtains the muscle movement rhythm parameters by sorting out the change trend of each muscle group in the image sequence;

[0068] The coordinate points of the same muscle in the previous and next frames are read sequentially. The spatial direction change trend is evaluated by connecting the displacement direction line segments formed by the pair of coordinate points. This operation is performed in any two adjacent frames to obtain a series of directional angle change information. On this basis, the movement direction trend sequence formed by each muscle in the entire image sequence is traversed to extract the time period with more drastic direction changes or stable direction. At the same time, the frame number of each muscle in the image sequence where it first undergoes significant displacement is recorded as the starting frame. The judgment criterion is that the position offset between the two frames exceeds the preset pixel difference of 5 pixels and the movement trend with the same direction continues for at least 3 frames. The frame is considered to be the movement starting point. For example, if the displacement of the zygomatic major muscle first meets this criterion in the 8th frame, the starting frame number is 8. Subsequently, the direction change sequence and starting frame number of the frontalis muscle and the depressor anguli oris muscle are extracted in the same way, and the direction trend data and starting timing corresponding to each muscle are obtained. Finally, the muscle movement rhythm parameters are output for subsequent screening and analysis.

[0069] The linkage rhythm screening submodule screens the relationship between the directional angle trends of multiple groups of muscles and the starting frame in the muscle movement rhythm parameters, determines whether the directional trends of different muscles are consistent and whether the movement start sequence is coordinated, and counts the combinations that do not have the same direction or movement synchronization to obtain the facial linkage angle offset;

[0070] The three target muscles are combined into three groups of muscle pairs in pairs, and the movement direction change trends and starting frame sequence information of the two muscles in each group in the image sequence are compared in turn. First, for each combination, the direction change data of the two muscles are compared frame by frame in the image sequence to determine whether similar direction trends are presented in the same time period. The situation where the direction difference between frames does not exceed 10 degrees is defined as a consistent direction trend. If the trend continues for more than three consecutive frames, it is recorded as having the same direction. At the same time, the starting frame numbers of the two muscles are compared. If the difference between the two numbers does not exceed 2 frames, that is, the starting action occurs in adjacent frames or the time difference is less than or equal to 66 milliseconds, it is considered to be a coordinated start timing. If a group of combinations If the difference in the middle direction trend exceeds 15 degrees, or the difference in the starting frame number is greater than 3 frames, it is judged as an asynchronous linkage combination. All muscle combinations that do not meet the linkage conditions are counted and marked in number. At the same time, the corresponding direction difference range and the difference in the starting frame number are recorded. During the system initialization stage, the threshold of 10 degrees set for this judgment standard is derived from the median result of common direction fluctuations during normal facial muscle movement in children. The direction angle difference range of 0 to 5 degrees is high synchronization, 5 to 10 degrees is medium synchronization, 10 to 15 degrees is low synchronization, and more than 15 degrees is asynchronous. Finally, all muscle combinations that do not meet the direction consistency and starting timing coordination and their analysis results are formed into a structured list and output as the facial linkage angle offset.

[0071] like Figure 2 and Figure 4 As shown, the behavioral linkage screening module includes:

[0072] The crying frequency analysis submodule obtains the corresponding image sequence time period based on the facial linkage angle offset, analyzes the start and end frames of the vocalization signal in the crying frequency within this period, calculates the frame distance distribution between consecutive vocalization segments, determines whether there is interval change in the frame distance fluctuation, screens consecutive segments with vocalization clustering characteristics, and obtains the vocalization density distribution;

[0073] Extract the crying frequency data synchronized with the time period, identify all audio segments with obvious sound wave energy fluctuations, align the audio track with the image frame rate, extract the instantaneous energy value of the audio signal in units of each frame, and compare the energy value of each frame with the background noise baseline energy to identify the frame with energy value significantly higher than the baseline energy as the sound frame. For example, set the baseline to the mean energy value of the first 100 frames of audio in silence plus the standard deviation. If the energy of a frame is at least 20% higher than the value, the frame is recorded as the sound start frame. Continuously detect the start and end frames of each sound segment and record them as a sound The frame distance difference between each sound segment is analyzed, and then the frame distance distribution list between the sound segments is obtained to determine whether there is a dense segment with a frame distance within the set sound aggregation threshold of 50 frames. When the frame distance difference of consecutive sound segments is less than 50 frames and the number of consecutive occurrences reaches more than three times, it is defined as a sound aggregation segment. For example, the frame distances between the three consecutive segments from the 100th frame to the 130th frame, the 160th frame to the 185th frame, and the 200th frame to the 230th frame are 30 frames and 25 frames respectively, which constitute a sound aggregation group as a whole. All audio segments that meet the standard are counted to obtain the sound dense distribution amount.

[0074] The facial stretch measurement submodule analyzes the vertical displacement direction of the chin area in consecutive image frames based on the image segments covered by the dense distribution of vocalizations. It compares the directional consistency of the displacement trajectory between the starting and ending frames to determine whether a continuous upward or downward trend is forming. It then selects image segments with continuous unidirectional displacement to obtain the stretch trajectory deviation degree.

[0075] First, the coordinates of the center position of the chin area in each frame of the image sequence are extracted and arranged in frame order to form a coordinate trajectory list. The vertical coordinate difference between each two adjacent frames is calculated to obtain the frame-by-frame vertical displacement value of the chin center position. Then, by summarizing the positive and negative directions of all displacement values ​​between the starting frame and the ending frame, it is analyzed whether the chin movement shows continuous unidirectionality during this time period. The judgment standard is: if the vertical coordinates in all consecutive frames in the entire segment continue to increase, it is judged as a downward movement trend. Conversely, if the vertical coordinates continue to decrease, it is judged as an upward movement trend. If the direction of more than 20% of the frame segments is reversed, it does not constitute a unidirectional movement. For example, from the 100th frame to the 150th frame, the chin coordinates gradually increase from 90 to 135, and in this section, the coordinates of only 8 frames decrease, accounting for 16%, which is less than the 20% threshold. It can be regarded as a stable downward trend. The 20% direction reversal threshold is set by analyzing the median frequency distribution of coordinate fluctuation values ​​in 50 segments of children's facial actual vocalization images to reduce misjudgment caused by jitter or image blur, and screen out image segments with continuous unidirectional vertical displacement characteristics. The vertical distance between the starting and ending positions in the segment is quantified as the stretching trajectory offset of the segment.

[0076] The behavior overlap detection submodule uses the time period corresponding to the stretch trajectory offset to calculate the first and last occurrences of the blink action in the frame sequence. This is then cross-checked with the body vibration frame interval within the same segment, filtering out the behavior channel data that does not form an overlapping segment to obtain invalid behavior path data.

[0077] Extract the frame sequence information of blinking and body vibration within the time period. First, mark the blink status of each frame in the image sequence, record the blink start frame and blink end frame, and form the start and end frame sequence numbers of each blink segment. For example, frames 102 to 108 identify a complete blink process. In this way, a list of all blink frame segments in the current time period is compiled. Then, the action frame interval output by the body vibration recognition module is extracted to construct a vibration segment frame sequence number set. Then, cross-check the frame segments of the two behavior channels. The method is: for each blink frame segment, the start and end frame interval overlap check is performed with the body vibration frame segment. If there is no time axis overlap with the current blink segment in any vibration segment, that is, the end frame of the blink frame segment is earlier than the start frame of the vibration frame segment, or the start frame of the blink frame segment is later than the end frame of the vibration frame segment, it is considered that there is no behavior overlap. Count all blink and vibration combinations of such non-overlapping behavior segments to form a non-overlapping behavior channel set. The non-overlapping behavior time segments and behavior types are marked as invalid behavior path data.

[0078] like Figure 2 and Figure 5 As shown, the periodic fluctuation extraction module includes:

[0079] The rhythm curve construction submodule calls the image frame segments covered by the invalid behavior path data, analyzes the facial action frame sequence, heart rate signal density and muscle activity frequency band, calculates the corresponding change trajectory positions of the three types of signals in consecutive frames, draws continuity curves in chronological order, and obtains signal rhythm distribution information;

[0080] Three types of signal data within this time range are extracted in sequence, namely facial action frame sequence, heart rate signal density and muscle activity frequency band. For facial action frame sequence, the movement direction and amplitude of facial key points in each frame image within the corresponding time period are first obtained. By counting the vertical and horizontal displacements of the zygomatic major, frontalis and depressor anguli oris muscles in consecutive frames, they are converted into a sequence of facial activity intensity values ​​for each frame. For example, from the 110th to the 130th frame, if the zygomatic major muscle moves upward by 1 pixel in each frame, the action intensity value is 1. For heart rate signal density, the pulse wave signal synchronized with the image frame is collected, and the heart rate signal per second is converted into the number of image frames. Average pulse number, for example, if the video is 30 frames per second, if the heart rate in that second is 90 times per minute, then the average density per frame is 1.5 times, and in this way a sequence of heart rate density change values ​​is generated for all frame segments; for the muscle activity frequency band, the position and energy size of the main frequency band in each frame of the electromyography signal are analyzed, the number of main frequency changes in every 10 frames is counted and normalized into a frequency value, and the three types of values ​​are respectively corresponding to the frame number to construct a continuous curve with the horizontal axis as the frame sequence and the vertical axis as the intensity value. By connecting the numerical changes of the three types of signals point by point, three types of continuous rhythm change curves of face, heart rate and muscle are formed, and the signal rhythm distribution information is drawn and output.

[0081] The peak distance change identification submodule calculates the frame intervals between consecutive peak points based on the changing position of the curve trajectory in the signal rhythm distribution information, compares the change amplitude of adjacent intervals in consecutive frame segments, screens the frame segments with key changes in continuity, and obtains the curve interval fluctuation trend;

[0082] All local maximum positions are identified in each curve and recorded as the peak position sequence of the signal. The frame sequence difference between two adjacent peak points is calculated segment by segment to form a frame distance sequence. Then, the frame distance sequence is compared item by item to obtain the difference between each frame distance segment and the next frame distance segment. For example, if a signal curve has peaks at frames 110, 130, 152, and 177, the frame distances are 20, 22, and 25, respectively, and the adjacent frame distance differences are 2 and 3 frames. Each difference is judged whether it is greater than the frame distance fluctuation threshold. The fluctuation threshold is set to 5 frames based on the signal frequency stability in children's normal physiological rhythm. If three consecutive frame distance differences are greater than this value, the segment is determined to be a key change segment. For example, if the frame distances are 20, 28, 36, and 45, respectively, the adjacent differences are 8, 8, and 9 frames, which meets the conditions. The set of frame segment numbers that meet the change amplitude judgment conditions is output as the curve interval fluctuation trend.

[0083] The abnormal fluctuation screening submodule determines whether there are inconsistent change directions or delayed response distributions in the facial, heart rate, and muscle curves based on the time periods marked by the curve interval fluctuation trends. It analyzes the concentrated periods of synchronization deviation in multiple data segments and obtains the rhythm mutation response segments.

[0084] To analyze the concentrated period of synchronization deviation in multiple data segments, the formula is used:

[0085] ;

[0086] Calculate the synchronization deviation value The synchronization deviation value is a quantitative indicator used to measure the relative coordination and timing consistency deviation of facial signals, heart rate signals and muscle signals in a certain period of time. It is used to analyze the concentrated period of synchronization deviation in multiple segments of data and obtain the rhythm mutation response segment. Represents the number of time periods, Representative Heart rate signal within a period of time, Represents the mean of the heart rate signal in all time periods, Represents the standard deviation of the heart rate signal in all time periods, Representative Muscle signals over time, Represents the mean value of muscle signals in all time periods, Represents the standard deviation of muscle signals over all time periods, Representative Facial signals over time, represents the mean value of facial signals in all time periods, Represents the standard deviation of facial signals in all time periods, Representative Time stamp for a period of time, represents the mean of the time stamps for all time periods, Represents a non-zero constant, used to smooth the square root term to prevent instability caused by zero;

[0087] set up , the data collection frequency is recorded every 10 seconds, and the physiological sensor monitoring results are obtained as follows:

[0088] Heart rate signal (unit: bpm): , , ;

[0089] Muscle signal (unit: μV, using myoelectric RMS): , , ;

[0090] Facial signal (unit: μV): , , ;

[0091] Time stamp (unit: s): , , ;

[0092] Smoothing constant: (Permanent value, used to maintain stability);

[0093] average value:

[0094] ;

[0095] ;

[0096] ;

[0097] ;

[0098] Standard Deviation:

[0099] ;

[0100] ;

[0101] ;

[0102] Calculate item by item:

[0103] Item 1:

[0104] ;

[0105] Item 2:

[0106] ;

[0107] Item 3:

[0108] ;

[0109] Sum and average:

[0110] ;

[0111] The results show that The value of is 0, and the deviation trends of the three types of signals in the three time periods are completely offset under the calculated weights. There is no obvious overall synchronization offset, and the synchronization state is relatively balanced within this window.

[0112] like Figure 2 and Figure 6 As shown, the behavioral and physiological contrast module includes:

[0113] The response time extraction submodule analyzes the starting points of continuous sound waves in the vocalization trajectory based on the frame sequence range of the rhythm mutation response segment, and simultaneously detects the first dense fluctuation segment in the heart rate data. It compares the arrangement order of the first frames of the two segments on the time axis, determines their sequence relationship, and establishes the corresponding terms of the frame positions to obtain the starting offset of the vocalization heart rate.

[0114] According to the frame sequence range of the rhythm mutation response segment, the audio data and heart rate data in the target time period are extracted, and the sound wave envelope analysis is performed on the audio data. The frame position of the first sound wave rising edge in the continuous sound signal is extracted as the sound start frame. For example, if the audio energy rises for three consecutive frames for the first time in the 210th frame and exceeds the silent energy benchmark, the 210th frame is recorded as the starting point of the sound trajectory. At the same time, the fluctuation density detection is performed on the heart rate data in the same segment, and the starting position with the most intensive pulse waveform changes is selected as the first heart rate response frame. Specifically, the heart rate mutation per second is analyzed. The frequency of occurrence of variable values ​​is as follows: if a high-frequency fluctuation segment appears within 5 seconds after the start of a certain frame and the fluctuation amplitude exceeds the average amplitude of the previous cycle by more than 20%, the frame is marked as the starting frame of intensive heart rate fluctuation. For example, if continuous heart rate jump points appear starting from the 218th frame, the 218th frame is regarded as the starting point of the heart rate response. The numbering sequence of the phonation start frame and the heart rate response frame is compared, and the one with the smaller number is recorded as the first response signal. At the same time, the frame sequence positions of the two are correspondingly marked as a group of position matching items, and finally the starting response offset data of the phonation and heart rate signals in all rhythm mutation segments are formed.

[0115] The time period difference calculation submodule analyzes the continuous fluctuation segments of crying and heart rate in the image frame axis based on the frame sequence position in the vocalization heart rate start offset, compares the time span between the start and end frames of the two types of signals, and selects the areas with relatively stable frame segment change directions to obtain the behavior duration comparison value;

[0116] For each pairing, extract the complete fluctuation segments of the crying and heart rate signals in the image frame axis. Record the frame numbers corresponding to the first rising point and the last peak point of each type of signal in the segment to form the start and end time intervals of each signal. Calculate their respective time spans as the behavior duration. For example, the crying fluctuation segment was detected from frames 210 to 245, and the heart rate fluctuation segment was detected from frames 218 to 252, corresponding to durations of 35 and 34 frames, respectively. Next, cross-compare the relative positions between the start and end frames of the two signals. If the start frame interval between the two does not exceed 10 frames and the end frame interval does not exceed 15 frames, and there is no sharp reversal in the signal waveform trend within the frame segment, then the segment is considered to be a relatively stable behavior period. The criterion for a sharp reversal is that the waveform has more than two reverse jumps within 10 frames or the change amplitude exceeds 20%. Count all time periods that meet this stability condition and sort the time span comparison values ​​of the two types of signals in turn. Output the numerical values ​​and time period positions of all behavior duration comparison quantities.

[0117] The signal consistency judgment submodule compares the fluctuation frame segments within the behavior duration comparison quantity, the rising segment of the crying amplitude and the fluctuation direction of the heart rate change trajectory, and determines whether they show synchronization characteristics within the same time period. It screens out intervals with reverse changes or fluctuation faults to obtain behavioral and physiological misalignment segments;

[0118] Compare the rising section of crying amplitude with the fluctuation direction of heart rate change trajectory, and use the formula:

[0119] ;

[0120] Calculating the Volatility Synchrony Index The fluctuation synchronization index is an indicator used to measure whether the changes in crying amplitude and heart rate fluctuate synchronously in the same time period. and To assess whether the two are consistent in their changing trends, that is, whether they are strengthened or weakened at the same time in the same period, and then to determine whether they show synchronous characteristics in the same time. Representative The amplitude of the rising segment of the frame crying sound, Representative Heart rate in the frame heart rate change track, Represents the mean value of the amplitude in the entire rising section of crying amplitude, Represents the mean of the heart rate in the entire heart rate change trajectory, Represents the total number of frames in the current comparison frame segment, Represents a non-zero constant, used to smooth the square root term to prevent instability caused by zero;

[0121] The sampling frequency of the contrast frame segment is set to 100 frames per second, with a total duration of 5 seconds. The sampling device is used to uniformly control the number of frames. The crying signal is collected through a microphone, and the amplitude value is calculated every 10ms using the short-time energy method. The data is normalized after preprocessing and filtering. The 5th, 120th, 240th, 360th, and 480th frames in the comparison time period are selected as representatives. The data are as follows:

[0122] Crying amplitude in frame 5 , heart rate ;

[0123] Crying amplitude at frame 120 , heart rate ;

[0124] Crying amplitude at frame 240 , heart rate ;

[0125] Crying amplitude at frame 360 , heart rate ;

[0126] Crying amplitude at frame 480 , heart rate .

[0127] Mean crying amplitude Calculated by averaging the selected frame amplitudes:

[0128] ;

[0129] Average heart rate Calculate in the same way:

[0130] ;

[0131] Smoothing parameter Used to prevent the denominator from being zero, the value is 0.01, which is set according to the minimum theoretical fluctuation of the denominator after signal standardization. When the standard amplitude change is less than 0.05, Make sure the denominator is non-zero.

[0132] Compute the synchronization term for each frame:

[0133] Frame 5:

[0134] ;

[0135] Frame 120:

[0136] ;

[0137] Frame 240:

[0138] ;

[0139] Frame 360:

[0140] ;

[0141] Frame 480:

[0142] ;

[0143] Substitute the calculated values ​​of 5 frames into the overall average:

[0144] ;

[0145] The results show that the instantaneous fluctuation synchronization index between the crying amplitude and heart rate fluctuations in the comparison frame segment is 68.445. The numerical value represents the degree of match between the current behavioral characteristics and physiological signals. This value is derived from the synthesis of multiple synchronization items to form a quantitative description of the overall behavioral and physiological fluctuation relationship of the current frame segment.

[0146] like Figure 2 and Figure 7 As shown, the pain grading determination module includes:

[0147] The signal change detection submodule analyzes the continuous expression contour changes in facial images, the vocalization fluctuations in the sound trajectory, and the starting point of the heart rate curve based on the frame sequence interval covered by the behavioral and physiological misalignment segment. It determines whether the signal has a linked change within the same frame segment and obtains the signal change synchronization amount.

[0148] The facial image data, sound track and heart rate curve information are extracted in sequence, and the facial contour extraction operation is performed on each frame of the facial image data. By comparing the changes in the coordinates of the contour points of parameters such as eyelid opening and closing, mouth corner position, and eyebrow height in consecutive frames, a sequence of facial expression changes is formed. Energy analysis and frequency band recognition are performed on the sound track in the same segment, and the audio energy value and main frequency structure of each frame are extracted. The frame sequence with three consecutive frames whose energy values ​​are higher than the background benchmark is recorded as the starting point of the voice, and the number of voice frequency band fluctuations is counted with every 5 frames as a window segment. The heart rate curve is analyzed within the frame segment. Perform a jump point extraction operation to record the starting points of continuous rise or fall in the pulse curve to form a fluctuation starting point sequence. Then, extract the change starting point frame numbers of the three types of signals respectively, compare them against the frame sequence axis, and judge whether the three types of signals have significant changes at the same time in the same frame segment. Set the judgment standard as the maximum frame difference of the starting frames of the three signal changes does not exceed 4 frames, that is, the changes start at the same time within a range of about 133 milliseconds, which is identified as a linked change frame segment. According to this standard, all frame segments that meet the conditions in the behavioral and physiological dislocation segment are counted and recorded, and summarized as the signal change synchronization amount.

[0149] The overlapping segment screening submodule compares the starting trajectories of facial, behavioral, and physiological signals in the same frame sequence based on the frame segment range involved in the signal change synchronization quantity, and screens the frame segments that have intersections and continue to extend in the time sequence direction. It then counts the number of overlapping ranges in the continuous distribution segment to obtain the total number of overlapping distribution segments.

[0150] The starting trajectories of changes in facial, behavioral, and physiological signals were sequentially extracted within a time period. The starting frames and continuation trends of the three types of signals were compared on the same frame axis. For each signal, it was determined whether it fluctuated continuously within the frame segment starting from the starting frame, that is, the change value maintained an increasing or decreasing trend for at least five consecutive frames. The time axis was superimposed according to the frame segment span. Frame segments where the starting frames of the three types of signal fluctuations were within the same frame segment and there was an extended trend for at least eight frames thereafter were considered linkage continuation segments. The start and end frames of each signal within the corresponding frame segment were recorded, and a frame segment overlap set was constructed. The intersection judgment was performed on all start and end frame sequences in this set. If the frame segment interval of the three types of signals had an overlapping frame sequence range of at least six frames, it was recorded as a valid overlap distribution segment. The number of all such overlap segments was counted one by one as the total number of overlap distribution segments. The minimum number of intersection frames set in the overlap judgment was six frames, which was based on the median value of the clinical data on the synchronization probability of facial reactions and heart rate changes. The total number of overlap distribution segments was output.

[0151] The feature level matching submodule analyzes the signal overlap structure and frame segment persistence characteristics in the total amount of overlap distribution segments to determine whether their corresponding positions are located in the key segments set in the pain level reference sequence. It then determines the degree of match between the linkage morphology presented by the frame segments and the level standard, determines the corresponding pain level, and obtains the multi-source pain level region.

[0152] The peak intensity of facial movements, the rate of increase in crying amplitude, and the amplitude of heart rate fluctuations in each segment are compared and mapped to the key segment positions set in the pain level reference sequence according to the frame sequence position. In each frame segment, the change pattern of the three types of signals is compared item by item to see whether it matches the characteristic curve type defined in the level standard. For example, if from frame 250 to frame 270, the three types of signals all show a synchronous rise, and the peak intensity of facial movements exceeds 1.5 times the amplitude of the action benchmark, the average energy of crying is higher than twice the silent benchmark, and the heart rate fluctuation has a jump frequency of more than 10% for 8 consecutive frames, then the segment is marked as a high-matching segment and then mapped according to the level area in the reference sequence where the segment is located. If the segment is in the third-level pain segment, it is recorded as the frame segment corresponding to the third-level pain. All frame segments that meet the mapping conditions are confirmed and labeled in turn for matching levels, and the frame sequence positions and level values ​​of all corresponding pain levels are output to form a multi-source pain grading area.

[0153] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. A and B can be singular or plural. Furthermore, the character " / " as used herein generally indicates an "or" relationship between the associated objects, but it may also indicate an "and / or" relationship. For specific understanding, please refer to the context.

[0154] In this disclosure, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, "at least one of a, b, or c" can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0155] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0156] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0157] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0158] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of the device or unit, which can be electrical, mechanical or other forms.

[0159] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0160] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0161] If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical disks.

[0162] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A pediatric pain assessment system, characterized in that: The system comprises: The muscle group rhythm recognition module extracts the starting position and movement direction of the zygomaticus major, frontalis, and depressor anguli oris muscles based on a sequence of collected children's facial images. It calculates the angles and activation intervals between the muscles, identifies movement rhythm misalignments, and selects combinations of step differences to determine the facial linkage angle offset. The behavioral linkage screening module obtains the onset and duration of crying frequency, jaw muscle stretching, body vibration, and blinking behavioral responses based on the facial linkage angle offset, analyzes the response overlap between multiple behaviors, eliminates behavioral channels lacking linkage, and obtains invalid behavioral path data; The periodic fluctuation extraction module calls the invalid behavior path data, draws facial rhythm, heart rate fluctuation and muscle activity curves, identifies the changing trend of signal peak spacing in adjacent time periods, screens mutation concentration areas, and obtains rhythm mutation response segments; The periodic fluctuation extraction module includes: The rhythm curve construction submodule calls the image frame segments covered by the invalid behavior path data, analyzes the facial action frame sequence, heart rate signal density and muscle activity frequency band, calculates the corresponding change trajectory positions of the three types of signals in consecutive frames, draws continuity curves in chronological order, and obtains signal rhythm distribution information; The peak distance change identification submodule calculates the frame intervals between consecutive peak points based on the changing position of the curve trajectory in the signal rhythm distribution information, compares the change amplitudes of adjacent intervals in consecutive frame segments, selects the frame segment sequences with key changes in continuity, and obtains the curve interval fluctuation trend; The abnormal fluctuation screening submodule determines whether there is inconsistent change direction or delayed response distribution in the three types of facial, heart rate and muscle curves based on the time period marked by the fluctuation trend of the curve interval, analyzes the concentrated time period of synchronization deviation in multiple segments of data, and obtains the rhythm mutation response segment; The analysis of the concentrated period of synchronization deviation in multiple segments of data adopts the formula: ； Calculate the synchronization deviation value , obtain the rhythm mutation response segment, where, Represents the number of time periods, Representative Heart rate signal within a period of time, Represents the mean of the heart rate signal in all time periods, Represents the standard deviation of the heart rate signal in all time periods, Representative Muscle signals over time, Represents the mean value of muscle signals in all time periods, Represents the standard deviation of muscle signals over all time periods, Representative Facial signals over time, represents the mean value of facial signals in all time periods, Represents the standard deviation of facial signals in all time periods, Representative Time stamp of a period of time, represents the mean of the time stamps for all time periods, Represents a non-zero constant, used to smooth the square root term to prevent instability caused by zero; The behavior-physiology mismatch module analyzes the start time of the crying sound and the heart rate response according to the rhythm mutation response segment, calculates the time difference therebetween, identifies the misalignment interval between the behavior and the physiological response, and obtains the behavior-physiology misalignment segment; The behavior-physiology comparison module includes: The response time extraction submodule analyzes the starting points of continuous sound waves in the vocalization trajectory based on the frame sequence range of the rhythm mutation response segment, and simultaneously detects the first dense fluctuation segment in the heart rate data. It compares the arrangement order of the first frames of the two segments on the time axis, determines their sequence relationship and establishes the corresponding terms of the frame positions to obtain the starting offset of the vocalization heart rate; The time period difference calculation submodule analyzes the continuous fluctuation segments of crying and heart rate corresponding to the image frame axis based on the frame sequence position in the vocalization heart rate starting offset, compares the time span between the start and end frames of the two types of signals, and selects the areas with relatively stable frame segment change directions to obtain the behavior duration comparison value; The signal consistency judgment submodule compares the fluctuation frame segments within the behavior duration comparison quantity with the fluctuation direction in the crying amplitude rising segment and the heart rate change trajectory to determine whether they show synchronization characteristics within the same time period, and screens out intervals with reverse changes or fluctuation faults to obtain behavioral and physiological dislocation segments; The comparison of the rising section of the crying amplitude and the fluctuation direction in the heart rate change trajectory uses the formula: ； Calculating the Volatility Synchrony Index , to determine whether they present synchronization characteristics within the same time, where, Representative The amplitude of the rising segment of the frame crying sound, Representative Heart rate in the frame heart rate change track, Represents the mean value of the amplitude in the entire rising section of crying amplitude, Represents the mean of the heart rate in the entire heart rate change trajectory, Represents the total number of frames in the current comparison frame segment, Represents a non-zero constant, used to smooth the square root term to prevent instability caused by zero; The pain grading determination module monitors whether there are changes in facial, behavioral, and physiological signals simultaneously within the behavioral and physiological misalignment segments, identifies overlapping areas of the three types of signals, analyzes whether the characteristic positions specified in the pain judgment criteria appear, determines the pain severity grade of the corresponding area, and obtains a multi-source pain grading area. The multi-source pain grading area includes a multi-dimensional signal fusion result, a pain grade classification number, and a reference standard mapping index.

2. The pediatric pain assessment system according to claim 1, wherein: The facial linkage angle offset includes the angle of the muscle group movement trajectory, the action start time label, and the linkage coordination abnormality mark; the invalid behavior path data includes the behavior response time distribution, the channel linkage density index, and the abnormal channel number set; the rhythm mutation response segment includes the signal peak time interval sequence, the periodic fluctuation amplitude data, and the mutation period segment index; the behavior physiological dislocation segment includes the starting response time comparison group, the behavior physiological synchronization offset result, and the cross-type signal dislocation segment label.

3. The pediatric pain assessment system according to claim 1, wherein: The muscle group rhythm recognition module includes: The muscle point localization submodule analyzes the starting areas of the zygomatic major, frontalis, and depressor anguli oris muscles marked in consecutive frames of the collected child facial image sequence. By comparing the spatial position changes of each muscle area in adjacent frames, the coordinate correspondence of the same muscle in the difference frames is determined, and a muscle position data sequence is established. The rhythm parameter extraction submodule calculates the change in the movement direction angle of each muscle in consecutive frames based on the relative displacement of the muscle position between each frame in the muscle position data sequence, determines the frame position sequence relationship at which the muscle starts to move, and obtains the muscle movement rhythm parameter by sorting out the change trend of each group of muscles in the image sequence; The linkage rhythm screening submodule screens the relationship between the directional angle trends and starting frames of multiple groups of muscles in the muscle movement rhythm parameters, determines whether the directional trends between different muscles are consistent and whether the movement start sequence is coordinated, and counts the combinations that do not have the same direction or movement synchronization to obtain the facial linkage angle offset.

4. The pediatric pain assessment system according to claim 1, wherein: The behavior linkage screening module includes: The crying frequency analysis submodule obtains the corresponding image sequence time period based on the facial linkage angle offset, analyzes the start and end frames of the vocalization signal in the crying frequency within the image sequence time period, calculates the frame distance distribution between consecutive vocalization segments, determines whether there is an interval change in the frame distance fluctuation, screens consecutive segments with vocalization clustering characteristics, and obtains the vocalization density distribution amount; The facial stretch determination submodule analyzes the vertical displacement direction of the chin region in consecutive image frames based on the image segments covered by the dense distribution of vocalizations, compares the directional consistency of the displacement trajectory between the starting frame and the ending frame, determines whether a continuous upward or downward movement trend is formed, and selects image segments with continuous unidirectional displacement of the trajectory to obtain the stretch trajectory deviation degree; The behavior overlap detection submodule calls the time period corresponding to the stretching trajectory offset, calculates the first and last occurrence frame marks of the blink action in the frame sequence, performs time axis cross-judgment with the body vibration frame mark interval in the same segment, filters the behavior channel data that does not form an overlapping segment, and obtains invalid behavior path data.

5. The pediatric pain assessment system according to claim 1, wherein: The pain grading determination module includes: The signal change detection submodule analyzes the continuous expression contour changes in the facial image, the vocalization fluctuations in the sound trajectory, and the fluctuation starting point of the heart rate curve based on the frame sequence interval covered by the behavioral physiological dislocation segment, and determines whether the signal has a linkage change within the same frame segment to obtain the signal change synchronization amount; The overlapping segment screening submodule compares the starting trajectories of facial, behavioral, and physiological signals in the same frame sequence based on the frame segment range involved in the signal change synchronization quantity, screens the frame segments that have intersections and continue to extend in the time sequence direction, and then counts the number of overlapping ranges in the continuous distribution segment to obtain the total number of overlapping distribution segments; The feature level matching submodule analyzes whether the corresponding position is located in the key segment set in the pain level reference sequence based on the signal overlap structure and frame segment persistence characteristics in the total amount of overlap distribution fragments, judges the degree of matching between the linkage form presented by the frame segment and the level standard, determines the corresponding pain level grade, and obtains the multi-source pain grade area.

Images