Method and system for identifying, monitoring and early warning abnormal state of patient

Abstract

The invention provides a patient abnormal state identification monitoring early warning method and system, and relates to the technical field of patient state monitoring. The method comprises the following steps: acquiring a facial image sequence of a patient, tracking a texture direction and constructing a variation comparison sequence, sorting linkage differences to generate dynamic clues, extracting non-shielding track mark picture distribution, identifying life fluctuations in a segmented manner, generating a fluctuation list through cross pairing, establishing a mapping chain to mark centralized anomalies, and generating an early warning scheme. According to the method, the recognition precision of facial subtle changes is improved, region credibility is calibrated in combination with a shielding ratio, non-shielding continuous image regions are screened, a physiological signal change sequence is segmented through trend continuity and direction consistency, high-sensitivity fluctuation paragraphs are extracted, and the credible image regions and physiological data are subjected to cross pairing in time and space; and merging intersection signals, identifying continuous high-frequency abnormal segments, and constructing a multi-dimensional mapping chain, thereby realizing high-precision early warning and real-time monitoring of complex state changes.

Description

Technical Field

[0001] This invention relates to the field of patient condition monitoring technology, and in particular to a method and system for identifying, monitoring and warning of abnormal patient conditions. Background Technology

[0002] Patient status monitoring technology involves the continuous detection and information identification of patients' physiological parameters and behavioral manifestations. Its core lies in the real-time or periodic monitoring of patients' physical condition through the perception, collection, analysis, and correlation judgment of multi-source physiological information. This technology systematically integrates vital sign detection, physiological information collection and modeling, medical information interaction, and judgment and early warning mechanisms, and is widely used in scenarios such as telemedicine, hospital ward monitoring, rehabilitation management, and home care. Current development trends in this technology are based on multi-parameter monitoring systems, medical IoT information structures, and intelligent early warning mechanisms, supporting the dynamic analysis and identification of patients' physiological data.

[0003] Among them, the patient abnormal state identification, monitoring, and early warning method refers to a technical solution that continuously compares the real-time physiological parameter changes of patients with historical reference data, and combines event-triggered logic rules to identify potential abnormal vital sign trends or behavioral manifestations. The technical aspects covered by this method include: the collection of basic physiological data such as heart rate, respiratory rate, blood pressure, body temperature, and blood oxygen saturation; dynamic trend judgment using continuous time series verification; abnormal value identification based on set threshold ranges and fluctuation frequency rules; feature comparison using historical data samples to identify sudden nonlinear changes; and early warning prompts via message push or audio-visual signals. This type of method is generally executed through a combination of electrophysiological sensors, non-contact monitoring equipment, and directional analysis logic to achieve high-frequency, high-temporal-resolution abnormality detection and judgment.

[0004] Existing technologies largely focus on unidirectional trend recognition of physiological parameters, lacking the involvement of facial behavioral information in the overall judgment mechanism, resulting in a single perspective in state analysis. While relying on numerical sensor data, it is difficult to isolate abnormal signals caused by external interference and changes in equipment stability, making the actual judgment results susceptible to non-essential factors. The lack of verification and screening standards for facial image validity leads to the inclusion of partially occluded or distorted images in the analysis, reducing judgment accuracy. In the trend linkage judgment among multiple physiological parameters, the lack of cross-indicator temporal pairing capability fails to simultaneously demonstrate the multidimensional coupling characteristics of physiological state changes. Some early warning logics rely on static threshold settings, failing to combine the correspondence between behavioral performance and physiological fluctuations, resulting in a slow response to early abnormal states. These shortcomings may lead to the failure to recognize minor behavioral signals, frequent false alarms, or delayed responses to genuine abnormal states in practical applications, limiting their adaptability in scenarios requiring high continuity and accuracy. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the existing technology and to propose a method and system for identifying, monitoring and warning of abnormal patient conditions.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A method for identifying, monitoring, and issuing early warnings of abnormal patient conditions includes the following steps: S1: Obtain facial image sequences collected by the robot in the ICU ward, track the muscle texture direction of corresponding regions between image frames, construct a comparison sequence according to the difference of displacement increment between frames, and generate a set of facial dynamic clues; S2: Based on the active area in the set of facial dynamic clues, extract the continuous display trajectory in the image frame, calculate the ratio of occluded pixels to displacement, classify the occlusion stability according to time period, extract the unoccluded block, and generate facial image distribution markers. S3: For the time period corresponding to the distribution mark on the facial image, retrieve continuous heart rate, blood pressure, blood oxygen and respiratory rate, extract the sequence of changes in vital signals, filter according to the consistency of trends, and generate a set of vital fluctuation indexes; S4: Align the set of facial dynamic cues with the credible regions in the facial image distribution markers, pair the time points in the set of vital fluctuations with the time periods of facial changes, extract the signal change regions in the intersecting time periods, and generate a list of facial and physiological fluctuations. S5: Based on the time segments in the facial and physiological fluctuation list, establish a mapping between facial image change areas and physiological fluctuation sequences, mark and archive consecutive entries with frequencies higher than adjacent areas, and obtain an abnormal state identification, monitoring and early warning scheme.

[0007] As a further aspect of the present invention, the facial dynamic cue set includes regional change direction features, inter-frame displacement increment difference sequence, and regional linkage ratio classification results; the facial image distribution markers include continuous display trajectory paths, unobstructed block coordinates, and occlusion interference stability labels; the vital fluctuation index set includes trend continuity segmentation results, fluctuation direction consistent segments, and continuous over-reference length change segments; the facial and physiological fluctuation list includes regional intersection alignment positions, signal change pairing time periods, and three data convergence time periods; and the abnormal state identification monitoring and early warning scheme includes image change and physiological fluctuation mapping chains, frequency anomaly marker entries, and concentrated triggering segment identifiers.

[0008] As a further aspect of the present invention, the specific steps of S1 are as follows: S101: Acquire the sequence of patient facial images continuously collected by the robot in the ICU ward, perform point-by-point calibration on the facial regions of adjacent frames, extract the gray-scale gradient direction of local pixel blocks, track the directional change trend in the sequence, and generate continuous texture direction change values. S102: Call the continuous texture direction change value, extract the displacement increment based on the difference between the horizontal and vertical coordinates of the region in adjacent frames, calculate the displacement increment difference between adjacent frames according to the time series, construct the continuous change sequence of the same region, and generate the inter-frame displacement difference sequence. S103: Based on the inter-frame displacement difference sequence, extract the displacement ratio between each frame in each region, establish a regional linkage response matrix, identify the combination of fluctuating regions by the ratio difference, call the corresponding texture direction change sequence for linkage classification and organization, and generate a set of facial dynamic clues.

[0009] As a further aspect of the present invention, the specific steps of S2 are as follows: S201: Based on the active areas in the set of facial dynamic cues, extract the continuous position coordinates of the regions in the image frame sequence, rearrange the center points of the regions in chronological order, establish the trajectory path, and generate a facial region trajectory sequence. S202: Based on the facial region trajectory sequence, extract the number of occluded pixels and the image displacement value of the corresponding region of the frame, calculate the ratio between the two, and process them according to time period to generate an occlusion interference stable interval. S203: Call the occlusion interference stable interval, filter the occlusion fluctuation area, extract continuous unoccluded image blocks, and generate facial image distribution markers.

[0010] As a further aspect of the present invention, the specific steps of S3 are as follows: S301: Within the monitoring period corresponding to the facial image distribution marker, retrieve the recorded continuous time series values ​​of heart rate, blood pressure, blood oxygen and respiratory rate, extract the time axis change data of vital signs, arrange them by item and generate trajectories in chronological order to obtain the vital sign trajectory sequence. S302: Based on the vital sign trajectory sequence, the change trend of each type of vital signal is segmented into continuous time periods, the direction of change of signal value in adjacent time periods is extracted, the consistency of the direction change in each time period is judged and the repeated change direction areas are screened out, and a set of sequence segments with consistent continuous fluctuation direction is established to obtain the continuous trend segment of vital signal. S303: Call the continuous trend segment of the life signal, calculate the duration of each continuous sequence, extract the sequence segments in the life signal whose continuous duration is greater than the average duration of the same type as the target part, uniformly number and organize the target segments of the signal, and generate a life fluctuation index set.

[0011] As a further aspect of the present invention, the formula for calculating the duration of each continuous sequence is as follows: ; in, Represents the duration of the i-th continuous sequence. This represents the end time of the i-th segment. Represents the starting time point of the i-th segment. This represents the number of signal data points contained in the i-th segment. This represents the signal value of the j-th data point in the i-th segment. This represents the average value of the i-th signal segment.

[0012] As a further aspect of the present invention, the specific steps of S4 are as follows: S401: Compare the image region locations in the facial dynamic clue set with the credible image regions in the facial image distribution markers, extract the position numbers of overlapping pixel blocks appearing in each frame image, sort the intersection regions by time period and reconstruct the region segments, and generate the region intersection alignment result. S402: Based on the region intersection alignment result, extract the image time period corresponding to the intersection region, perform parallel matching with the signal change time period in the life fluctuation index set, search for whether the change duration segment in each life signal has an overlapping interval with any intersection time period, extract the intersection part time point, and generate time period intersection matching result; S403: Call the time period intersection matching results, extract the image region number and vital signal change type within the intersection time period, summarize all signal and image change features under the same time period, reorganize the data structure according to the time period number, and generate a list of facial and physiological fluctuations.

[0013] As a further aspect of the present invention, the specific steps of S5 are as follows: S501: Based on the time segments in the facial and physiological fluctuation list, extract the corresponding image regions and physiological fluctuation sequence data, construct the time correspondence between regions and signals, and generate an image physiological mapping chain; S502: Based on the image physiological mapping chain, extract the image change frequency, calculate the average image change frequency between the current frame and the previous frame, filter out continuous entries with frequencies greater than the adjacent reference range, and generate high-frequency continuous segment markers. S503: Invoke the high-frequency continuous segment marker, integrate the marked time period and physiological fluctuation trajectory, combine and archive the image region number and signal type, reorganize the marker sequence according to the segment number, and generate an abnormal state identification, monitoring and early warning scheme.

[0014] As a further aspect of the present invention, the formula for calculating the average image change frequency between the current frame and the previous frame is as follows: ; in, This represents the average frequency of image changes between the current frame and the previous frame. Represents the first in the current frame The grayscale value of each pixel Represents the first in the previous frame The grayscale value of each pixel This represents the total number of pixels contained in a single frame of an image. This represents the pixel number in the current image. The time series label for the current frame.

[0015] The patient abnormal condition identification, monitoring, and early warning system includes: The image acquisition module acquires a sequence of patient facial images continuously recorded by the robot in the ICU ward, tracks the grayscale direction of the cheekbone, glabella, and philtrum regions in consecutive frames, calculates the displacement change of the grayscale centroid of adjacent frames, determines whether the displacement direction and amplitude between regions show a coordinated trend, classifies and summarizes regions with related change characteristics, and generates a set of facial dynamic cues. The image processing module extracts the continuous trajectory of the region in the image sequence based on the active area of ​​the facial dynamic clue set, makes a proportional judgment on the number of occluded pixels and the degree of horizontal displacement in the trajectory, calculates the persistence and amplitude range of the proportional fluctuation, classifies and continuously displays stable facial blocks according to the proportional fluctuation trend, and generates facial image distribution markers. The physiological detection module retrieves continuous values ​​of heart rate, blood pressure, blood oxygen, and respiratory rate within the time period indicated by the facial image distribution markers. It segments the change trajectory of each type of signal in time sequence, performs consistency screening on the fluctuation direction, identifies the part with continuous fluctuation direction and duration exceeding the average segment of signal fluctuation, and generates a vital fluctuation index set. The fluctuation linkage module overlays the concentrated area of ​​facial dynamic clues with the unobstructed area in the facial image distribution mark, extracts the continuous time period corresponding to the image, compares the signal fluctuation trajectory in the life fluctuation index set, determines whether the direction of change of the image area is consistent with the direction of signal fluctuation and whether the synchronization deviation does not exceed the specified time interval, and generates a list of facial and physiological fluctuations. The early warning marking module extracts image change frequency and physiological signal fluctuation frequency data based on the time segments in the facial and physiological fluctuation list, divides the time period into continuous cycles, statistically compares the frequency range with adjacent cycles, and defines the time period with the frequency significantly exceeding the upper limit of the adjacent reference interval as the concentrated fluctuation area, thereby generating an abnormal state identification, monitoring and early warning scheme.

[0016] Compared with the prior art, the advantages and positive effects of the present invention are as follows: In this invention, by improving the accuracy of facial subtle change recognition, combining the occlusion ratio to calibrate the region's credibility, screening unobstructed continuous image regions, segmenting physiological signal change sequences based on trend continuity and directional consistency, extracting highly sensitive fluctuation segments, cross-pairing credible image regions with physiological data in time and space, merging and converging signals, identifying continuous high-frequency abnormal segments, and constructing a multi-dimensional mapping chain, high-precision early warning and real-time monitoring of complex state changes are achieved. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the steps of the present invention; Figure 2 This is a system module diagram of the present invention. Detailed Implementation

[0019] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0020] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0021] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0022] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0023] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0024] Please see Figure 1 This invention provides a method for identifying, monitoring, and issuing early warnings of abnormal patient conditions, comprising: S1: Obtain the sequence of patient facial images continuously collected by the robot in the ICU ward, continuously track the direction of the corresponding regions between image frames, compare the direction of change of the regions, construct a comparison sequence by the difference of the displacement increment between the same region frames, classify and organize the differences in the linkage ratio between regions, and generate a set of facial dynamic clues. S2: Based on the active area in the facial dynamic cues, extract the continuous display trajectory of the area in the image frame sequence, calculate the ratio of the number of pixels occlusion in the trajectory to the degree of image displacement, classify and label the stability of screen occlusion interference according to the distribution of continuous time periods, extract continuously identifiable unoccluded blocks, and generate facial screen distribution markers. S3: During the monitoring period corresponding to the facial image distribution markers, retrieve the recorded continuous values ​​of heart rate, blood pressure, blood oxygen and respiratory rate, extract the change sequence of vital signals, segment the trend continuity of each type of change trajectory according to the time sequence, filter the change parts in the continuous time period with the same fluctuation direction and the continuous length exceeding the average baseline segment, and generate a vital fluctuation index set. S4: Align the intersection of the facial dynamic cue set with the credible image area in the facial image distribution marker, pair the time points in the life fluctuation index set with the facial image change period, merge and aggregate the signal change areas in the time period of the intersection of the three sources, and generate a list of facial and physiological fluctuations. S5: Based on the time segments merged from the facial and physiological fluctuation lists, establish a mapping chain between the corresponding facial image change areas and physiological fluctuation sequences. Mark and archive items with continuous time length and change frequency greater than the standard frequency range of adjacent areas, classify and construct centralized trigger segment markings for state changes, and obtain an abnormal state identification, monitoring and early warning scheme.

[0025] The facial dynamic cue set includes regional change direction features, inter-frame displacement increment difference sequence, and regional linkage ratio classification results. The facial image distribution markers include continuous display trajectory paths, unobstructed block coordinates, and occlusion interference stability labels. The vital fluctuation index set includes trend continuity segmentation results, fluctuation direction consistent segments, and continuous over-reference length change segments. The facial and physiological fluctuation list includes regional intersection alignment positions, signal change pairing time periods, and three data convergence time periods. The abnormal state identification monitoring and early warning scheme includes image change and physiological fluctuation mapping chain, frequency anomaly marker entries, and concentrated triggering segment identifiers.

[0026] The specific steps of S1 are as follows: S101: Acquire the sequence of patient facial images continuously collected by the robot in the ICU ward, perform point-by-point calibration on the facial regions of adjacent frames, extract the gray-scale gradient direction of local pixel blocks, track the directional change trend in the sequence, and generate continuous texture direction change values. A sequence of facial images of patients continuously acquired by a robot in an ICU ward was obtained. During image acquisition, the robot used a built-in visual positioning module to determine the area of ​​the patient's face and used infrared auxiliary lighting to avoid image quality degradation caused by changes in lighting intensity. Five frames were continuously acquired per second. After acquiring two consecutive frames at a certain moment, the grayscale matrix of the images was read and compared point by point to extract the position and grayscale information of all pixels in the facial area of ​​the two frames. In the initial comparison, it was determined whether each pixel was at the same coordinate position in the two frames. If a point shifted by more than 1 pixel in the x or y direction between the two frames, it was marked as a "drift pixel". Using any 5×5 pixel block in the patient's facial area as the basic unit, points were taken above, below, to the left, and to the right of the center point of each pixel block to calculate the trend of grayscale value changes. The grayscale value difference and relative coordinate change were used to determine the grayscale of the area. Gradient direction: If the grayscale change value in a certain direction is 10 and the opposite direction is 2, then the current direction of the pixel block is the positive main change direction. All pixel blocks in the entire image are processed sequentially, and the grayscale change direction is recorded according to the center point of each block. This process is repeated in consecutive image frames to track the direction change trend of each block. When the direction change of a block from the previous frame to the current frame is greater than 10 degrees, it is marked as a direction jump region. For example, if the direction of a block in the previous frame is 30 degrees and in the current frame is 45 degrees, the direction jump is 15 degrees. The direction change sequence of this block will be continuously recorded. A direction change time series corresponding to the pixel block coordinates is established in the entire image sequence, and its cumulative change value is recorded. When a region changes continuously more than a set number of times (such as a jump of more than 10 degrees in 3 consecutive frames), the direction sequence of this region is classified into the region with significant texture change, and is used for collaborative calculation with other regions in the subsequent displacement processing to finally obtain a complete set of continuous texture direction change values.

[0027] S102: Call the continuous texture direction change value, extract the displacement increment based on the difference between the horizontal and vertical coordinates of the region in adjacent frames, calculate the displacement increment difference between adjacent frames according to the time series, construct the continuous change sequence of the same region, and generate the inter-frame displacement difference sequence. Between adjacent frames, select a pixel block in each facial region and calculate the displacement change of its center point in the horizontal and vertical directions. If the center point of the pixel block was (100, 150) in the previous frame and (102, 152) in the current frame, the displacement change is 2 pixels and 2 pixels respectively, with a total displacement of approximately 2.8 pixels in both directions. Use this method to count the displacement increment of all pixel blocks marked as directional change regions in the entire image. If the displacement increment of a certain region in consecutive frames is greater than 2 pixels in multiple frames, then the region is marked as a motion region. Extract the increment changes between each frame in the time series and calculate the difference between each pair of frames. If a pixel block has a displacement of 2.5 pixels between frame 1 and frame 2, and 1.8 pixels between frame 2 and frame 3, then the inter-frame difference is -0.7 pixels. Record the changes sequentially. The displacement difference of a region between consecutive frames is calculated, and a displacement difference sequence for each pixel is generated. Based on these differences, regions with significant changes are screened out, and the start and end times of the change are marked on the time axis. The displacement difference threshold for distinguishing between stationary and changing regions is set to 1.0 pixel. This threshold is derived from the statistical results of image differences in different patients' stationary states during the experiment. In the stationary state, the inter-frame displacement of more than 90% of regions does not exceed 1.0 pixel, so this is set as the distinction boundary. For example, if the difference of a region in consecutive frames is 0.5, 1.2, and 0.8, then only the second frame can be identified as a motion feature frame. Finally, the displacement change trend of each region is organized into region sequence data and classified into significant motion regions and stable regions for subsequent region response analysis.

[0028] S103: Based on the inter-frame displacement difference sequence, extract the displacement ratio between each frame in each region, establish a regional linkage response matrix, identify the combination of fluctuating regions by the ratio difference, call the corresponding texture direction change sequence for linkage classification and organization, and generate a set of facial dynamic clues. Based on the generated inter-frame displacement difference sequence, ratios are extracted and compared for each facial functional region. For example, the displacement of the left eyebrow region between frames 1 and 2 is 2.0 pixels, and between frames 2 and 3 it is 2.6 pixels, resulting in a ratio of 1.3. The right eye region is 2.1 and 2.2 pixels, resulting in a ratio of 1.05. The corner of the mouth region is 1.5 and 2.1 pixels, resulting in a ratio of 1.4. This method is used to calculate the displacement ratio sequence of all facial functional regions between adjacent frames, constructing a region ratio vector at each time point. When the ratio of multiple regions exceeds a set threshold at the same time, it indicates that the time point is the peak period of region response. The ratio threshold is set with reference to the fluctuation range of the displacement ratio between facial regions under normal static conditions. This range is generally within the range... The ratio is set to no more than 1.2, with 1.25 as the starting limit for abnormal fluctuations. For example, if the ratios of the three regions of left eyebrow, right eye, and corner of mouth are 1.3, 1.05, and 1.4 respectively, then the left eyebrow and corner of mouth are identified as linked regions. The corresponding muscle direction change sequence is called for a second comparison. If the direction jump of both is greater than 10 degrees in the current frame and the jump direction is consistent (e.g., both increase counterclockwise), then it is further determined to be a real linked region group. This group is marked as a linked event in the current frame, and its corresponding time, region coordinates, ratio, and direction change value are recorded together. Finally, it is integrated into a set of facial dynamic clues. All clues are sorted by time to form a linear trajectory cluster of the patient's facial expression dynamics, which is used for further behavior modeling and atlas formation.

[0029] The specific steps of S2 are as follows: S201: Based on the active areas in the set of facial dynamic cues, extract the continuous position coordinates of the regions in the image frame sequence, rearrange the center points of the regions in chronological order, establish the trajectory path, and generate a facial region trajectory sequence. Based on the active regions in the facial dynamic cues set, the coordinates of each facial sub-region identified as an active region are determined in the image frame sequence. The center point position of the corresponding region is extracted frame by frame. Data from each frame from frame 1 to frame 50 is read. The image size is set to 640×480. The center coordinates of a certain region in frame 1 are extracted as (200, 150), and in frame 2 as (202, 151). After extracting the center coordinates of the region in all frames, they are sorted according to the image acquisition time sequence to generate a time-evolving region position sequence. The movement trajectory of the region center between adjacent frames is calculated. By recording the difference in the center point position between two consecutive frames as a two-dimensional coordinate change vector, the entire sequence is accumulated and connected to form a visualized path segment. Each The center point coordinates and time label of the region in each frame are defined. For example, the center point of the corner of the mouth region in frame 10 is (210, 158), corresponding to the second. All time points and spatial points are linked through an index array. When building this sequence, the spatial continuity of each trajectory is judged. If the center position of the region shifts by more than 10 pixels within three consecutive frames, it is marked as a trajectory mutation segment. This judgment condition comes from the statistics of the normal micro-expression range of the patient's facial region under the average image frame rate. Under normal expression, the position change in consecutive frames usually does not exceed 5 pixels. Therefore, 10 pixels is set as the mutation judgment threshold. Finally, each facial activity region forms a trajectory sequence with a timestamp throughout the entire time period. Each trajectory point contains center coordinates, frame number and region ID, which together constitute the facial region trajectory sequence.

[0030] S202: Based on the facial region trajectory sequence, extract the number of occluded pixels and the image displacement value of the corresponding region of the frame, calculate the ratio between the two, and process them according to time period to generate an occlusion interference stable interval. Based on the facial region trajectory sequence, the number of occluded pixels at the corresponding region location is read from each frame of the image. The validity of occluded pixels is determined by a set brightness threshold. If the grayscale value of pixels in a certain region is lower than 30 and there are more than 25 contiguous pixels, then the region is determined to be occluded. The total number of occluded pixels in each frame is accumulated to obtain the number of occluded pixels. In addition, the displacement value of the center point of the region is calculated between adjacent frames. For example, if the center point in the previous frame is (100, 120) and the center point in the next frame is (103, 122), then the displacement value is 3.6 pixels. After obtaining the number of occluded pixels and the displacement value, their ratio is calculated. If the number of occluded pixels is 90 and the displacement is 3.6, then the occlusion ratio is 25.0. The occlusion ratio sequence of each region will be statistically analyzed over a continuous time period, and then calculated according to the set... The system is segmented into intervals, for example, with 5 frames as an analysis period. The average occlusion ratio within this period is calculated. If the average occlusion ratio of a certain region within 5 frames is 28.6, it is classified as a high occlusion interference interval. If the average ratio is less than 20.0, it is classified as a stable occlusion interference interval. The reference values ​​for dividing the occlusion interference intervals are set after analyzing and statistically analyzing a large amount of image data. Most unoccluded or slightly occluded areas have occlusion ratios concentrated in the 15-20 interval. An occlusion ratio less than 20.0 is set as the stability judgment threshold, 20.0-30.0 is a mild interference area, and above 30.0 is a severe interference area. In this way, the stability of the occlusion degree of each region's trajectory in different time periods is identified, and finally, multiple stable occlusion interference intervals are defined for subsequent region screening.

[0031] S203: Call the stable interval of occlusion interference, filter the occlusion fluctuation area, extract continuous unoccluded image blocks, and generate facial image distribution markers; The system retrieves identified stable occlusion interference intervals and filters the marked stable intervals within each region's trajectory segment. Regions where the occlusion ratio remains consistently stable throughout the entire trajectory sequence are prioritized. For example, if a region has an occlusion ratio below 20.0 in 15 out of 20 frames, and the other 5 frames show fluctuations not exceeding 22.5, then this region is considered to have low overall occlusion fluctuation and is recorded as a low-fluctuation region. Image patches are then extracted from these regions, and the occlusion pixel ratio of the corresponding pixel patch in each frame is read. If a pixel patch has a total of 225 pixels in a frame and 10 pixels are occluded, then the occlusion ratio is 4.4%. An unobstructed image threshold of 5% was set. Image blocks with an occlusion ratio below this value were considered unobstructed. The threshold was determined based on statistical comparisons of facial image clarity requirements and occlusion intensity experiments. When the occlusion ratio exceeds 5%, it will affect the recognition of texture and dynamic information in the facial region. Therefore, 5% was used as the reference benchmark for unobstructed image. Finally, continuous image blocks that meet the unobstructed condition were selected from all low-fluctuation regions. Their relative position coordinates and time sequence in the original image were marked according to their region location to generate facial image distribution markers, which are used as coordinate and time sequence references for subsequent image enhancement, recognition extraction, and dynamic restoration processes.

[0032] The specific steps for S3 are as follows: S301: Within the monitoring period corresponding to the facial image distribution markers, retrieve the recorded continuous time series values ​​of heart rate, blood pressure, blood oxygen, and respiratory rate, extract the time axis change data of vital signs, arrange them by item and generate trajectories in chronological order to obtain the vital sign trajectory sequence. Within the monitoring period corresponding to the facial image distribution markers, the start and end timestamps contained in the facial image markers are first read. This time period is then precisely located to the vital signs data module recorded in the hospital information. Raw data of four vital signs parameters—heart rate, blood pressure, blood oxygen saturation, and respiratory rate—are extracted every second recorded within this time period. For each vital sign, its corresponding timestamp and numerical data are extracted. For example, heart rate values ​​are recorded as [t0, 82], [t1, 84], [t2, 86]; blood pressure is recorded as [t0, 110 / 70], [t1, 112 / 71], [t2, 115 / 72]; blood oxygen saturation is [t0, 98%], [t1, 97%], [t2, 96%]; and respiratory rate is [t0, 98%], [t1, 97%], [t2, 96%]. [t0, 18], [t1, 19], [t2, 20] are used to structure and sort these data in chronological order. Each type of parameter is arranged by time to form its own time axis sequence. During the sorting process, each time point is used as the reference on the horizontal axis, and various vital signs values ​​are used as variables on the vertical axis to form a set of curve data points. This further generates the trajectory sequence of the parameter. For example, the heart rate trajectory is a curve that slowly rises over time, the blood pressure trajectory can be split into hyperbolas representing systolic and diastolic blood pressure respectively, and the blood oxygen and respiratory rate trajectories are reflected as fluctuations. By displaying all the data on the same time axis, the vital sign trajectory sequence is constructed, laying the foundation for subsequent trend extraction and anomaly monitoring.

[0033] S302: Based on the vital sign trajectory sequence, the changing trend of each type of vital signal is segmented into continuous time periods, the direction of numerical change of signal values ​​in adjacent time periods is extracted, the consistency of the direction change in each time period is judged and the repeated areas of change direction are screened out, and a set of sequence segments with consistent continuous fluctuation direction is established to obtain the continuous trend segment of vital signal. First, all timestamps and value pairs of this type of sequence are read. Each analysis period is defined as 10 seconds. Within each period, the data is sorted by time, and the direction of the difference between the current value and the previous value is compared point by point. For example, if the heart rate rises from 82 to 84 and then to 87, it is marked as "rising"; if it is 84, 82, and 80, it is marked as "falling"; if it is 82, 83, and 82, it is marked as "fluctuating". This method is used to mark the direction of all periods. During processing, the direction values ​​of adjacent periods are compared. If two or more consecutive periods have the same direction without reversal (e.g., three periods are all "rising"), then this segment is considered a consistent trend interval. If a segment shows "rising, falling, and...", then... If the direction of the "rising" trend alternates, the segment is marked as a region of repeated direction and is screened out. The screening criterion is that at least two directions differ in three adjacent time periods. For example, if the heart rate in the first time period is from 82 to 85, the second time period is from 85 to 83, and the third time period is from 83 to 86, then the directions of these three time periods are rising, falling, and rising in sequence. This is marked as inconsistent and repetitive in direction and is removed from subsequent trend analysis. After screening, all segments with consistent continuous directions are retained, and each segment is assigned a unique number for organization to construct a set of continuous trend segments of vital signs. Each set of trend segments includes the start and end times, parameter names, direction types, and the number of continuous data points, which are used for the next step of duration calculation and target segment identification.

[0034] S303: Call the continuous trend segment of the life signal, calculate the duration of each continuous sequence, extract the sequence segments in the life signal whose continuous duration is greater than the average duration of the same type as the target part, uniformly number and organize the signal target segments, and generate a life fluctuation index set; The specific formula for calculating the duration of each continuous sequence is as follows: ; in, Represents the duration of the i-th continuous sequence. This represents the end time of the i-th segment. Represents the starting time point of the i-th segment. This represents the number of signal data points contained in the i-th segment. This represents the signal value of the j-th data point in the i-th segment. This represents the average value of the i-th signal segment; parameter and These represent the start and end times of the i-th segment of the vital sign trend, respectively, in seconds. This data is acquired in real-time by the timestamp acquisition module in the ICU monitoring system, with an accuracy controlled to 1 second. Based on the actual data, the i-th segment of the heart rate increase trend started at 08:15:20 and ended at 08:17:10, corresponding to a duration of 110 seconds, i.e.: ; parameter This represents the number of signal sampling points within the i-th segment. The ICU sampling frequency is once every 5 seconds to record vital signs values; therefore, a total of 22 signal values ​​are collected within 110 seconds. ; parameter This represents the vital sign value at time j within the i-th segment, specifically the heart rate (in bpm). This value is continuously recorded by the patient's ECG monitoring device and imported in real-time via a data interface. The sampled data are: 83, 84, 84, 85, 87, 86, 88, 87, 89, 90, 88, 87, 89, 91, 92, 91, 90, 89, 88, 88, 87, 86. The overall data is calculated by averaging. ; Next, we calculate the average deviation term: ; The mean absolute deviation is: ; Substitute into the formula to calculate the extended continuous duration value: ; The results indicate that the continuous upward trend of heart rate in segment i is based on the actual duration with the weight of data stability offset, and the extended continuous duration is 112.30 seconds. This value is used as a comprehensive indicator of the trend segment for comparison with the average trend segment value. If this value is higher than the average trend extension duration value, the segment is classified as the target segment of life fluctuation and included in the life fluctuation index set.

[0035] The specific steps of S4 are as follows: S401: Compare the image region locations in the facial dynamic clue set with the reliable image regions in the facial image distribution markers, extract the position numbers of overlapping pixel blocks appearing in each frame image, sort the intersection regions by time period and reconstruct the region segments to generate the region intersection alignment result. First, the image frame numbers and corresponding coordinate information of all regions in the dynamic cue set are read. A structured region coordinate table is constructed for each frame. For example, in frame 12, the coordinates of region A are (x=100, y=120, w=30, h=30). The coordinates of image block regions marked as reliable are extracted from the facial image distribution markers. For example, in frame 12, the reliable region is (x=90, y=110, w=50, h=50). In the current frame, coordinate cross-detection is performed on the above two sets of regions to determine whether there are overlapping segments in the horizontal and vertical directions. The start and end coordinates of the overlapping region are calculated and converted into image numbers. If the right boundary of region A is 130, the left boundary of the reliable region is 90, and the right boundary is 140, then the horizontal overlap range is 10. 0130, similarly comparing vertically, the overlapping range is 120140. This overlapping pixel block is defined as number Z_12_01, where 12 represents the frame number and 01 represents the first overlapping block in the frame. This operation is repeated throughout the entire frame, recording the numbers of all image blocks with overlapping regions. At the same time, all numbered regions are arranged in order of frame number with time as the axis, constructing a continuous region combination sequence composed of multiple numbers. For example, if there are consecutive overlapping regions with numbers Z_12_01 to Z_20_01 in frames 12 to 20, they are merged and defined as the region intersection segment with segment ID RJ_001. The final output intersection alignment result is a set of several region segments, each segment recording the corresponding time range, frame number range, and region number sequence.

[0036] S402: Based on the region intersection alignment results, extract the image time period corresponding to the intersection region, perform parallel matching with the signal change time period in the life fluctuation index set, search for whether the change duration segment in each life signal has an overlapping interval with any intersection time period, extract the intersection part time point, and generate the time period intersection matching result. Based on the region intersection alignment results, the start and end times of each region intersection segment RJ are read. For example, RJ_001 corresponds to times from 08:30:00 to 08:30:20. Then, the time range and direction of change of each vital signal trend segment are read from the vital signal fluctuation index set. For example, the heart rate segment of HR-TD-002 is from 08:30:15 to 08:30:45 and the direction of change is upward. The intersection segment and the vital signal segment are compared side by side on the time axis to determine whether there is an intersection segment between the two time intervals. If the start time of the intersection segment is less than the end time of the signal segment, and the end time is greater than the start time of the signal segment, then it is determined that the two segments have time overlap. For example, RJ_001 and H The overlapping portion of the R-TD-002 time interval is from 08:30:15 to 08:30:20, with an overlap duration of 5 seconds. The time points corresponding to each second within this time interval are extracted at the second level as intersection time points T_HR=[08:30:15, 08:30:16, ..., 08:30:20]. Further, matching relationships with other intersection segments are searched among all signal segments. During the processing, all combinations of non-intersecting time points are excluded. A set of matching entries is constructed, consisting of an array of intersection segment numbers, signal numbers, and intersection time points. Finally, the intersection matching results of all time periods are output in a structured manner to identify the synchronous feature distribution of image change areas and changes in vital signals.

[0037] S403: Call the time period intersection matching results, extract the image region number and vital signal change type within the intersection time period, summarize all signal and image change features under the same time period, reorganize the data structure according to the time period number, and generate a list of facial and physiological fluctuations. The system retrieves the time-intersection matching results, reading the intersection time period T, the corresponding image region number, and the vital signal trend number for each matching entry. First, it searches for the image frame corresponding to time period T in the image dataset, then retrieves the sequence of all intersection region numbers within that time period from the frame. For example, the region numbers corresponding to the time period from 08:30:15 to 08:30:20 are Z_15_01 to Z_20_01. Next, it reads the trend attribute of HR-TD-002 in the vital signal fluctuation index set, marking this segment as an "upward" trend. It then combines the region number and signal trend within each time point into a joint record entry. If the region number in the frame at 08:30:15 is Z_15_... 01. If the signal type is heart rate and the trend is upward, then the joint record is generated as [T=08:30:15, Region ID=Z_15_01, Signal=HR, Trend=Upward]. This process is repeated to generate joint data tables for all time points. After all the data in the table is completed, it is summarized and reorganized according to the intersection time period number, such as RJ_001. All signal change types and image region numbers under this number are classified and merged to construct a comprehensive list containing all signal change trends and region numbers for that time period. The final output is a list of facial and physiological fluctuations. Each record is based on a time period and includes a list of all image region numbers, signal type, and corresponding trend information.

[0038] The specific steps of S5 are as follows: S501: Based on the time segments in the facial and physiological fluctuation list, extract the corresponding image regions and physiological fluctuation sequence data, construct the temporal correspondence between regions and signals, and generate an image physiological mapping chain; First, the time period T, corresponding image region number Z, and physiological signal change trend label S are read from each record item in the list. All frame data of each image region with number Z within the time period T are extracted from the image dataset to construct a region image sequence. Simultaneously, physiological signal values ​​corresponding to the time period T are continuously extracted from the signal recording module to construct time series value pairs of vital parameters. For example, from 08:31:00 to 08:31:15, the heart rate is [85, 86, 87, 87, 88], and the respiratory rate is [19, 20, 21, 21, 22]. Index items are created for each second of time, and the region number Z_t in the image frame is mapped to the signal value pair S_t to form a record item grid. The formula is [T_i, Z_i, S_i]. When processing multiple signal types, an independent index linked list is maintained for each type of signal. For example, the heart rate mapping chain is recorded as [T1=08:31:00, Z1=Z_180_01, HR=85], while the respiratory rate mapping chain is recorded as [T1=08:31:00, Z1=Z_180_01, RR=19]. In this way, a one-to-one or many-to-one mapping correspondence structure is established between facial region change frames and signal time trajectories. The frame sequence number of the image region corresponding to each type of signal is integrated into a separate linked storage data group, which finally forms the image physiological mapping chain. This mapping chain can locate the joint association between the image region number and multiple signal values ​​at any time point.

[0039] S502: Based on the image physiological mapping chain, extract the image change frequency, calculate the average image change frequency between the current frame and the previous frame, filter continuous entries with frequencies greater than the adjacent reference range, and generate high-frequency continuous segment markers. The formula for calculating the average image change frequency between the current frame and the previous frame is as follows: ; in, This represents the average frequency of image changes between the current frame and the previous frame. Represents the first in the current frame The grayscale value of each pixel Represents the first in the previous frame The grayscale value of each pixel This represents the total number of pixels contained in a single frame of an image. This represents the pixel number in the current image. The time series label for the current frame; The formula represents the average frequency of image change between the current frame and the previous frame, where, The current frame The Middle The grayscale value of each pixel The previous frame The Middle The grayscale value of each pixel It is the total number of pixels in a single frame of an image.

[0040] To demonstrate the calculation process of the formula, specific real-world data is used. Assume there is a 5x5 pixel image (therefore...). The grayscale value of each pixel is obtained from the monitoring device, as follows: Current frame pixel grayscale value : [10, 20, 30, 40, 50, 10, 20, 30, 40, 50, 10, 20, 30, 40, 50, 10, 20, 30, 40, 50, 10, 20, 30, 40, 50]; Previous frame pixel grayscale value : [5, 15, 25, 35, 45, 5, 15, 25, 35, 45, 5, 15, 25, 35, 45, 5, 15, 25, 35, 45, 5, 15, 25, 35, 45]; Calculate the absolute value of the difference in grayscale value between each pixel in two frames: ; Sum the differences of all pixels and calculate the average: ; ; This result indicates that from the frame to frame The average pixel change frequency of the image is 5. This means that each pixel changes by an average of 5 gray levels, revealing the uniformity of image variation between the two frames. This calculation can be used to detect dynamic changes in images, such as identifying sudden movements in video surveillance or tracking organ movement in medical imaging.

[0041] S503: Call high-frequency continuous segment markers, integrate the marked time period and physiological fluctuation trajectory, combine and archive the image region number and signal type, reorganize the marker sequence according to the segment number, and generate an abnormal state identification, monitoring and early warning scheme. The high-frequency sustained segment markers are invoked, and the corresponding time period, region number, and frame list are extracted from each marked HF segment. Then, physiological signal records for the same time period are consulted in the image physiological mapping chain, extracting all signal value sequences within that segment and identifying the trend type. For example, the time period of HF_001 is 08:31:05 to 08:31:10, the corresponding region number is Z_185_03, and the corresponding heart rate changes within this segment are [86, 88, 89, 91, 90, 92], with a continuous upward trend. Z_185_03 is combined with the upward trend of heart rate into a mapping pair. Then, the trend features of signals such as respiratory rate, blood pressure, and blood oxygen are extracted sequentially for the corresponding time periods, constructing a signal-image combination list, for example... The respiratory rates within this segment are [20, 21, 21, 22, 23, 23], with an upward trend, forming a joint record item [HF_001, Z_185_03, HR=increasing, RR=increasing]. All signal types are grouped under this number according to the same time period. The above joint records are numbered and archived. All data items are rearranged and grouped according to the HF segment number. Each record in the structure includes the segment number, start and end time, image region number, signal name and its trend direction. Finally, an abnormal state identification, monitoring and early warning scheme is generated. This scheme expresses the synchronicity of rapid changes in regional images and violent fluctuations in physiological signals in the form of a table structure, and provides structured input for subsequent threshold early warning judgment and continuous feature tracking.

[0042] Please see Figure 2 This invention also provides a patient abnormal state identification, monitoring, and early warning system, comprising: The image acquisition module acquires a sequence of patient facial images continuously recorded by the robot in the ICU ward, tracks the grayscale direction of the cheekbone, glabella, and philtrum regions in consecutive frames, calculates the displacement change of the grayscale centroid of adjacent frames, determines whether the displacement direction and amplitude between regions show a coordinated trend, classifies and summarizes regions with related change characteristics, and generates a set of facial dynamic cues. The image processing module extracts the continuous trajectory of the region in the image sequence based on the active area of ​​the facial dynamic cues, makes a proportional judgment on the number of occluded pixels and the degree of horizontal displacement in the trajectory, calculates the persistence and amplitude range of the proportional fluctuation, classifies and continuously displays stable facial blocks according to the proportional fluctuation trend, and generates facial image distribution markers. Within the time period indicated by the facial image distribution markers, the physiological detection module retrieves continuous values ​​of heart rate, blood pressure, blood oxygen, and respiratory rate, segments the change trajectory of each type of signal in time sequence, performs consistency screening on the direction of fluctuation, identifies the part with continuous fluctuation direction and duration exceeding the average segment of signal fluctuation, and generates a set of vital fluctuation indexes. The fluctuation linkage module overlays the concentrated area of ​​facial dynamic clues with the unobstructed area in the facial image distribution markers, extracts the continuous time period corresponding to the image, compares the signal fluctuation trajectory in the life fluctuation index, determines whether the direction of change of the image area is consistent with the direction of signal fluctuation and whether the synchronization deviation does not exceed the specified time interval, and generates a list of facial and physiological fluctuations. The early warning marking module extracts image change frequency and physiological signal fluctuation frequency data based on the time segments in the facial and physiological fluctuation list. It divides time periods into continuous cycles, statistically compares the frequency range with adjacent cycles, and defines the time periods with frequencies that significantly exceed the upper limit of adjacent reference intervals as concentrated fluctuation areas, thereby generating an abnormal state identification, monitoring and early warning scheme.

[0043] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for identifying, monitoring, and issuing early warnings of abnormal patient conditions, characterized in that, Includes the following steps: S1: Obtain facial image sequences collected by the robot in the ICU ward, track the muscle texture direction of corresponding regions between image frames, construct a comparison sequence according to the difference of displacement increment between frames, and generate a set of facial dynamic clues; S2: Based on the active area in the set of facial dynamic clues, extract the continuous display trajectory in the image frame, calculate the ratio of occluded pixels to displacement, classify the occlusion stability according to time period, extract the unoccluded block, and generate facial image distribution markers. S3: For the time period corresponding to the distribution mark on the facial image, retrieve continuous heart rate, blood pressure, blood oxygen and respiratory rate, extract the sequence of changes in vital signals, filter according to the consistency of trends, and generate a set of vital fluctuation indexes; S4: Align the set of facial dynamic cues with the credible regions in the facial image distribution markers, pair the time points in the set of vital fluctuations with the time periods of facial changes, extract the signal change regions in the intersecting time periods, and generate a list of facial and physiological fluctuations. S5: Based on the time segments in the facial and physiological fluctuation list, establish a mapping between facial image change areas and physiological fluctuation sequences, mark and archive consecutive entries with frequencies higher than adjacent areas, and obtain an abnormal state identification, monitoring and early warning scheme.

2. The method for identifying, monitoring, and issuing early warnings of abnormal patient states according to claim 1, characterized in that, The facial dynamic cue set includes regional change direction features, inter-frame displacement increment difference sequence, and regional linkage ratio classification results. The facial image distribution markers include continuous display trajectory paths, unobstructed block coordinates, and occlusion interference stability labels. The vital fluctuation index set includes trend continuity segmentation results, fluctuation direction consistent segments, and continuous over-reference length change segments. The facial and physiological fluctuation list includes regional intersection alignment positions, signal change pairing time periods, and three data convergence time periods. The abnormal state recognition monitoring and early warning scheme includes image change and physiological fluctuation mapping chain, frequency anomaly marker entries, and concentrated triggering segment identifiers.

3. The method for identifying, monitoring, and issuing early warnings of abnormal patient states according to claim 1, characterized in that, The specific steps of S1 are as follows: S101: Acquire the sequence of patient facial images continuously collected by the robot in the ICU ward, perform point-by-point calibration on the facial regions of adjacent frames, extract the gray-scale gradient direction of local pixel blocks, track the directional change trend in the sequence, and generate continuous texture direction change values. S102: Call the continuous texture direction change value, extract the displacement increment based on the difference between the horizontal and vertical coordinates of the region in adjacent frames, calculate the displacement increment difference between adjacent frames according to the time series, construct the continuous change sequence of the same region, and generate the inter-frame displacement difference sequence. S103: Based on the inter-frame displacement difference sequence, extract the displacement ratio between each frame in each region, establish a regional linkage response matrix, identify the combination of fluctuating regions by the ratio difference, call the corresponding texture direction change sequence for linkage classification and organization, and generate a set of facial dynamic clues.

4. The method for identifying, monitoring, and issuing early warnings of abnormal patient states according to claim 1, characterized in that, The specific steps of S2 are as follows: S201: Based on the active areas in the set of facial dynamic cues, extract the continuous position coordinates of the regions in the image frame sequence, rearrange the center points of the regions in chronological order, establish the trajectory path, and generate a facial region trajectory sequence. S202: Based on the facial region trajectory sequence, extract the number of occluded pixels and the image displacement value of the corresponding region of the frame, calculate the ratio between the two, and process them according to time period to generate an occlusion interference stable interval. S203: Call the occlusion interference stable interval, filter the occlusion fluctuation area, extract continuous unoccluded image blocks, and generate facial image distribution markers.

5. The method for identifying, monitoring, and issuing early warnings of abnormal patient states according to claim 1, characterized in that, The specific steps for S3 are as follows: S301: Within the monitoring period corresponding to the facial image distribution marker, retrieve the recorded continuous time series values ​​of heart rate, blood pressure, blood oxygen and respiratory rate, extract the time axis change data of vital signs, arrange them by item and generate trajectories in chronological order to obtain the vital sign trajectory sequence. S302: Based on the vital sign trajectory sequence, the change trend of each type of vital signal is segmented into continuous time periods, the direction of change of signal value in adjacent time periods is extracted, the consistency of the direction change in each time period is judged and the repeated change direction areas are screened out, and a set of sequence segments with consistent continuous fluctuation direction is established to obtain the continuous trend segment of vital signal. S303: Call the continuous trend segment of the life signal, calculate the duration of each continuous sequence, extract the sequence segments in the life signal whose continuous duration is greater than the average duration of the same type as the target part, uniformly number and organize the target segments of the signal, and generate a life fluctuation index set.

6. The method for identifying, monitoring, and issuing early warnings of abnormal patient states according to claim 5, characterized in that, The formula for calculating the duration of each continuous sequence segment is as follows: ； in, Represents the duration of the i-th continuous sequence. This represents the end time of the i-th segment. Represents the starting time point of the i-th segment. This represents the number of signal data points contained in the i-th segment. This represents the signal value of the j-th data point in the i-th segment. This represents the average value of the i-th signal segment.

7. The method for identifying, monitoring, and issuing early warnings of abnormal patient states according to claim 1, characterized in that, The specific steps of S4 are as follows: S401: Compare the image region locations in the facial dynamic clue set with the credible image regions in the facial image distribution markers, extract the position numbers of overlapping pixel blocks appearing in each frame image, sort the intersection regions by time period and reconstruct the region segments, and generate the region intersection alignment result. S402: Based on the region intersection alignment result, extract the image time period corresponding to the intersection region, perform parallel matching with the signal change time period in the life fluctuation index set, search for whether the change duration segment in each life signal has an overlapping interval with any intersection time period, extract the intersection part time point, and generate time period intersection matching result; S403: Call the time period intersection matching results, extract the image region number and vital signal change type within the intersection time period, summarize all signal and image change features under the same time period, reorganize the data structure according to the time period number, and generate a list of facial and physiological fluctuations.

8. The method for identifying, monitoring, and issuing early warnings of abnormal patient states according to claim 1, characterized in that, The specific steps of S5 are as follows: S501: Based on the time segments in the facial and physiological fluctuation list, extract the corresponding image regions and physiological fluctuation sequence data, construct the time correspondence between regions and signals, and generate an image physiological mapping chain; S502: Based on the image physiological mapping chain, extract the image change frequency, calculate the average image change frequency between the current frame and the previous frame, filter out continuous entries with frequencies greater than the adjacent reference range, and generate high-frequency continuous segment markers. S503: Invoke the high-frequency continuous segment marker, integrate the marked time period and physiological fluctuation trajectory, combine and archive the image region number and signal type, reorganize the marker sequence according to the segment number, and generate an abnormal state identification, monitoring and early warning scheme.

9. The method for identifying, monitoring, and issuing early warnings of abnormal patient states according to claim 8, characterized in that, The formula for calculating the average image change frequency between the current frame and the previous frame is as follows: ； in, This represents the average frequency of image changes between the current frame and the previous frame. Represents the first in the current frame The grayscale value of each pixel Represents the first in the previous frame The grayscale value of each pixel This represents the total number of pixels contained in a single frame of an image. This represents the pixel number in the current image. The time series label for the current frame.

10. A patient abnormal state identification, monitoring, and early warning system, characterized in that, The patient abnormal state identification, monitoring, and early warning method according to any one of claims 1-9, wherein the system comprises: The image acquisition module acquires a sequence of patient facial images continuously recorded by the robot in the ICU ward, tracks the grayscale direction of the cheekbone, glabella, and philtrum regions in consecutive frames, calculates the displacement change of the grayscale centroid of adjacent frames, determines whether the displacement direction and amplitude between regions show a coordinated trend, classifies and summarizes regions with related change characteristics, and generates a set of facial dynamic cues. The image processing module extracts the continuous trajectory of the region in the image sequence based on the active area of ​​the facial dynamic clue set, makes a proportional judgment on the number of occluded pixels and the degree of horizontal displacement in the trajectory, calculates the persistence and amplitude range of the proportional fluctuation, classifies and continuously displays stable facial blocks according to the proportional fluctuation trend, and generates facial image distribution markers. The physiological detection module retrieves continuous values ​​of heart rate, blood pressure, blood oxygen, and respiratory rate within the time period indicated by the facial image distribution markers. It segments the change trajectory of each type of signal in time sequence, performs consistency screening on the fluctuation direction, identifies the part with continuous fluctuation direction and duration exceeding the average segment of signal fluctuation, and generates a vital fluctuation index set. The fluctuation linkage module overlays the concentrated area of ​​facial dynamic clues with the unobstructed area in the facial image distribution mark, extracts the continuous time period corresponding to the image, compares the signal fluctuation trajectory in the life fluctuation index set, determines whether the direction of change of the image area is consistent with the direction of signal fluctuation and whether the synchronization deviation does not exceed the specified time interval, and generates a list of facial and physiological fluctuations. The early warning marking module extracts image change frequency and physiological signal fluctuation frequency data based on the time segments in the facial and physiological fluctuation list, divides the time period into continuous cycles, statistically compares the frequency range with adjacent cycles, and defines the time period with the frequency significantly exceeding the upper limit of the adjacent reference interval as the concentrated fluctuation area, thereby generating an abnormal state identification, monitoring and early warning scheme.

Images