Non-contact ar monitoring device and method based on facial time-series images

Abstract

The application relates to the technical field of image recognition, and particularly discloses a non-contact AR monitoring device and method based on facial time sequence images, which comprises an AR glasses body, a camera, a sensor, a processor, a memory, a communication interface and an AR display unit. The camera collects a time sequence image sequence of a patient's facial area, the sensor collects a micro-motion signal or an illumination auxiliary signal, the processor screens effective image frames, extracts facial color time sequence change features or facial micro-motion time sequence change features, generates real-time vital sign data, and when the identity recognition confidence does not satisfy a preset condition, fuses gait information, iris information or wristband radio frequency signals to obtain an identity mark, and compares the real-time vital sign data with illness information to generate a risk state. Through facial time sequence images, identity fusion and field association comparison, the prompt information in the AR visual field is corresponded with patient signs and illness records.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a non-contact AR monitoring device and method based on facial time-series images. Background Technology

[0002] Image recognition technology primarily focuses on processing two-dimensional or time-series images acquired by image acquisition devices, including target detection, region localization, feature extraction, identity matching, and result display. Image data is typically acquired through the combined efforts of cameras, sensors, terminal processors, and memory. Local programs or servers then retrieve information such as image frames, feature points, textures, colors, contours, motion changes, and matching thresholds for comparison, classification, association, and output. Traditional non-contact AR monitoring devices and methods based on facial time-series images involve continuously acquiring sequences of patient facial images using AR glasses cameras. Processing is performed in the following order: facial region localization, image frame filtering, time-series change extraction, patient identity recognition, medical record retrieval, and display of information within the field of view. This can be combined with information such as facial color changes, subtle movements, examination and test records, medical history records, and wristband radio frequency signals obtained from cameras or sensors. The glasses or a connected server then perform data association, risk status assessment, and visual prompts.

[0003] Traditional AR medical assistive devices use cameras to capture images of patients' faces as the entry point. Based on the face matching results, they retrieve medical records and display text or image information within the glasses' field of vision. The processing chain is biased towards identity verification and static record presentation. There is a lack of continuous correlation between the temporal changes of facial images and the patient's current vital signs. The comparison basis between medical record fields and examination and test fields is scattered. In scenarios where the face is occluded or damaged, the correspondence between the patient's identity and medical records is easily interrupted. On-site prompts are difficult to generate corresponding outputs as the patient's state changes. Summary of the Invention

[0004] The purpose of this invention is to provide a non-contact AR monitoring device and method based on facial temporal images, which is used in AR medical monitoring scenarios to collaboratively process the acquisition of temporal images of the patient's facial area, generation of vital signs, confirmation of patient identity, retrieval of medical information, assessment of risk status, and AR field of vision prompts, so that the AR glasses can form a continuous monitoring basis for the correspondence between the patient's current vital signs and medical records.

[0005] To achieve the above objectives, this invention provides a non-contact AR monitoring device based on temporal facial images, comprising an AR glasses body, a camera, a sensor, a processor, a memory, a communication interface, and an AR display unit mounted on the AR glasses body. The camera outputs a temporal image sequence of the patient's facial region to the processor; the sensor outputs facial micro-motion signals or illumination change auxiliary signals to the processor; and the memory stores a monitoring program for the processor to access. The processor runs the monitoring program to filter valid image frames according to facial region localization results and image frame quality conditions, extracts temporal color change features or facial micro-motion change features from the valid image frames, and generates real-time vital sign data. The processor is also used to obtain a patient identification identifier by fusing gait information, iris information, or wristband radio frequency signals when the confidence level of the patient's facial image does not meet preset identification conditions, and to retrieve the patient's condition information via the communication interface based on the patient identification identifier. The processor further compares and correlates real-time vital signs data with patient condition information according to vital signs, examination and test, and medical history fields. When the correlation and comparison results meet preset risk conditions, a risk status is generated, and the AR display unit overlays and outputs prompt information corresponding to the risk status within the AR glasses' field of view.

[0006] As a progressive limiting scheme, the processor determines candidate image frames corresponding to the patient's facial region in the temporal image sequence based on the facial region localization results, and compares the candidate image frames with image frame quality conditions, including facial region integrity, brightness stability, and adjacent image frame displacement. When a candidate image frame simultaneously meets the facial region integrity, brightness stability, and adjacent image frame displacement conditions, the processor considers the candidate image frame as a valid image frame; when a candidate image frame does not simultaneously meet these conditions, the processor discards the candidate image frame.

[0007] Furthermore, the processor performs a temporal continuity check on the valid image frames. This check includes determining whether the acquisition interval between adjacent valid image frames meets a preset time interval condition, and whether the number of consecutive valid image frames meets a preset quantity condition. When both the acquisition interval and the number of consecutive valid image frames meet the conditions, the processor arranges the corresponding valid image frames into a vital signs analysis window. When either the acquisition interval or the number of consecutive valid image frames does not meet the conditions, the processor re-selects valid image frames from the candidate image frames.

[0008] Furthermore, the processor determines the region of interest (ROI) on the patient's face within the vital signs analysis window and extracts the color channel change sequence and pixel displacement change sequence within the ROI region according to the acquisition time sequence. The processor corrects the color channel change sequence for illumination change auxiliary signals to form facial color temporal change features, and subtracts the overall facial displacement from the pixel displacement change sequence to form facial micro-motion temporal change features. Based on this, the processor extracts periodic components from both the facial color temporal change features and the facial micro-motion temporal change features, using the peak interval, fluctuation amplitude, and continuity of these periodic components as the basis for vital sign generation. When the periodic components corresponding to the facial color temporal change features and the facial micro-motion temporal change features meet the consistency condition, the processor generates real-time vital sign data including heart rate, respiratory rate, or blood oxygen saturation; when the periodic components do not meet the consistency condition, the processor marks the corresponding vital sign generation result as pending confirmation.

[0009] As a further refinement of identity verification, when the processor performs identity recognition on the patient's facial image, it first matches the patient's facial image with pre-stored patient facial features and generates an identity recognition confidence score. When the identity recognition confidence score meets preset recognition conditions, the processor uses the matched patient identity as the patient's identity identifier; when the identity recognition confidence score does not meet the preset recognition conditions, the processor reads gait information, iris information, or wristband radio frequency signals according to preset priorities, and performs consistency verification between the read information and the facial recognition result before generating a patient identity identifier.

[0010] As a further limitation on the association of patient information, when the processor retrieves patient information via the communication interface, it uses the patient's identity identifier as an index to read medical records, examination and test records, and medical history records. It then maps the diagnosis field in the medical records, the numerical field in the examination and test records, and the past status field in the medical history records to vital sign fields, examination and test fields, and medical history fields, respectively. When a vital sign field has the same name as a field in real-time vital sign data, the processor updates the corresponding field value according to the acquisition time; when a vital sign field does not have the same name as a field in real-time vital sign data, the processor retains the corresponding field value as the basis for association comparison.

[0011] As a further limitation of risk assessment, when performing correlation comparisons, the processor writes the current vital sign value from the real-time vital sign data into the corresponding vital sign field, and sequentially compares the current vital sign value with the similar test results recorded in the examination and testing field and the previous status recorded in the medical history field. When the difference between the current vital sign value and the similar test results meets a preset risk condition, and the previous status does not record an allowed deviation status, the processor generates a risk status; when the difference between the current vital sign value and the similar test results does not meet the preset risk condition, the processor uses the current vital sign value as the benchmark value for the next correlation comparison.

[0012] As a further definition of display and collaboration, when the AR display unit outputs the prompt information, the processor matches the risk status with a preset display layer, generates an overlay display instruction including a vital signs display layer, a medical summary display layer, and an operation prompt display layer, and sends the overlay display instruction to the AR display unit. When the communication interface receives remote annotation information, the processor spatially associates the remote annotation information with the patient's facial area or body positioning area within the current field of view, and controls the AR display unit to display the corresponding annotation graphic within the AR glasses' field of view.

[0013] This invention also provides a non-contact AR monitoring method based on facial temporal images, executed by the aforementioned device. The method includes: continuously acquiring a sequence of temporal images of the patient's facial region, and filtering valid image frames according to facial region positioning results and image frame quality conditions; extracting facial color temporal change features or facial micro-motion temporal change features from the valid image frames, and generating real-time vital sign data based on at least one temporal change feature; performing identity recognition on the patient's facial images, and retrieving the patient's medical condition information when the identity recognition confidence level meets preset recognition conditions, and fusing gait information, iris information, or wristband radio frequency signals to obtain the patient's identity identifier and retrieve the patient's medical condition information when the identity recognition confidence level does not meet preset recognition conditions; and comparing the real-time vital sign data with the patient's medical condition information according to vital sign fields, examination and test fields, and medical history fields, and generating a risk status and medical condition summary when the comparison result meets preset risk conditions, and overlaying and outputting prompt information corresponding to the risk status within the AR glasses' field of view.

[0014] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: This invention provides a processor with a temporal image sequence of the patient's facial region, facial micro-motion signals, and illumination change auxiliary signals via a camera and sensors. The processor generates valid image frames based on the facial region positioning results and image frame quality conditions, ensuring that real-time vital sign data originates from filtered temporal facial changes. When the confidence level for identity recognition does not meet preset recognition conditions, gait information, iris information, or wristband radio frequency signals are fused to obtain the patient's identity identifier, ensuring that the patient's condition information is matched with the monitored individual. The correlation and comparison of vital sign fields, examination and test fields, and medical history fields form a risk status, ensuring that the prompts output by the AR display unit correspond to the patient's current status. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of the overall collaboration of a non-contact AR monitoring device based on facial temporal images; Figure 2 A schematic diagram illustrating the formation of an effective image frame selection and vital sign analysis window; Figure 3 A schematic diagram illustrating the extraction of facial temporal change features and the generation of real-time vital sign data; Figure 4 A diagram illustrating the mapping between patient identification and patient condition information fields; Figure 5 This is a diagram illustrating the overlay display of AR field-of-view prompts and remote annotation information. Detailed Implementation

[0017] The technical solutions described in this invention will be further explained below with reference to the accompanying drawings and embodiments. In the scope of protection text, the AR glasses body, camera, sensor, processor, memory, communication interface, and AR display unit are all understood as entities or program execution carriers within the same non-contact AR monitoring device; the temporal image sequence of the patient's facial region refers to the set of patient facial image frames continuously acquired by the camera according to the acquisition time; a valid image frame refers to a data frame that enters subsequent vital sign analysis after being filtered by facial region positioning results and image frame quality conditions; real-time vital sign data refers to data objects formed by temporal changes in facial color and facial micro-movements, which can be subsequently associated and compared; the patient identification identifier is an object identifier used by the processor to index patient condition information; patient condition information includes medical records, examination and test records, and medical history records; risk status is the output status formed by the processor after performing field association comparisons between real-time vital sign data and patient condition information; remote annotation information is an annotation object received via the communication interface and superimposed onto the AR display unit. The above terms retain the same meaning in the following embodiments and accompanying text.

[0018] Example 1 Please see Figures 1 to 5 This embodiment provides a non-contact AR monitoring device based on facial time-series images. In this embodiment, the non-contact AR monitoring device includes an AR glasses body, a camera, a sensor, a processor, a memory, a communication interface, and an AR display unit mounted on the AR glasses body. The camera's acquisition direction corresponds to the patient's facial area in the wearer's field of vision. The sensor provides the processor with facial micro-motion signals or illumination change auxiliary signals. After obtaining input data from the camera and sensor, the processor calls the monitoring program in the memory. The communication interface is used to call up the patient's condition information and receive remote annotation information. The AR display unit is used to overlay prompt information and annotation graphics within the AR glasses' field of vision. Figure 1 This is used to demonstrate the data input, processing, communication calls, display output, and remote annotation feedback relationships between the aforementioned physical carriers.

[0019] When a camera captures a patient's facial region, the time-series image sequence output by the camera includes at least image frames arranged according to acquisition time, the acquisition time corresponding to each image frame, and the order information of the image frames in the acquisition sequence. After receiving the time-series image sequence, the processor first performs facial region localization in each image frame and generates facial region localization results. The facial region localization results are used to represent the positional boundaries, contour state, and local areas that can be used for subsequent analysis of the patient's facial region in the image frame. The processor does not directly generate real-time vital sign data from a single frame image; instead, it first compares the facial region localization results with the image frame quality conditions, ensuring that the data frames entering subsequent analysis have continuously readable facial regions and comparable temporal relationships.

[0020] like Figure 2 As shown, the processor determines candidate image frames in the temporal image sequence based on the facial region localization results, and compares these candidate image frames with facial region integrity conditions, brightness stability conditions, and adjacent image frame displacement conditions. The facial region integrity condition determines whether the patient's facial region within the candidate image frame contains the main facial region required for subsequent analysis; the brightness stability condition determines whether there are any illumination abrupt changes between the candidate image frame and adjacent candidate image frames that affect color change readings; the adjacent image frame displacement condition determines whether the facial region displacement between adjacent candidate image frames is within the range that the processor can perform temporal alignment. The specific boundaries of these conditions are determined by the comparable configuration stored in memory. When a candidate image frame simultaneously meets the facial region integrity condition, brightness stability condition, and adjacent image frame displacement condition, the processor writes the candidate image frame into the valid image frame set; when a candidate image frame does not simultaneously meet the above conditions, the processor marks the candidate image frame as rejected and retains the rejection reason for subsequent re-screening or log verification.

[0021] After forming a set of valid image frames, the processor performs a temporal continuity check on the valid image frames according to the acquisition time sequence. The temporal continuity check includes determining whether the acquisition interval between adjacent valid image frames meets a preset time interval condition, and whether the number of consecutive valid image frames meets a preset quantity condition. The acquisition interval is derived from the acquisition time output by the camera, and the number of consecutive valid image frames is calculated by the processor within the current filtering period. When both the acquisition interval and the number of consecutive valid image frames meet the conditions, the processor forms a vital sign analysis window from the corresponding valid image frames; if either the acquisition interval or the number of consecutive valid image frames does not meet the conditions, the processor returns to the candidate image frame filtering stage and reselects candidate image frames from the temporal image sequence. The input to this re-filtering process remains the temporal image sequence output by the camera and the recorded facial region localization results; the output is an updated set of valid image frames or a pending confirmation status.

[0022] The processor identifies the region of interest (ROI) on the patient's face within the vital signs analysis window. The ROI is a further defined analysis area based on the facial region localization results, used to extract temporal variations in facial color and facial micro-motion characteristics. The processor reads the color channel variation sequence and pixel displacement variation sequence within the ROI in chronological order of acquisition. The color channel variation sequence is formed by the temporal changes of the corresponding color channels within the ROI, while the pixel displacement variation sequence is formed by the changes in pixel positions within the ROI over time. The processor performs illumination change auxiliary signal correction on the color channel variation sequence, using the illumination change auxiliary signal output from the sensor as a reference input. Non-physiological color changes caused by ambient light variations are subtracted to obtain the temporal variation features of facial color. The processor performs overall facial displacement subtraction on the pixel displacement variation sequence, targeting the common displacement between adjacent valid image frames caused by overall head movement or wearer's viewing angle movement. After subtraction, the temporal variation features of facial micro-motion are obtained.

[0023] like Figure 3 As shown, the processor extracts periodic components from the temporal changes in facial color and facial micro-motions. The input for periodic component extraction is the temporal changes in facial color and facial micro-motions within the vital signs analysis window. The processor identifies periodic fluctuations according to the acquisition time sequence and extracts the peak interval, fluctuation amplitude, and continuity of the periodic components. The peak interval is used to establish the time basis for heart rate or respiratory rate; the fluctuation amplitude is used to determine whether the signal within the current window exhibits readable changes; and the continuity is used to determine whether the periodic component spans the vital signs analysis window. The processor performs a consistency condition judgment between the periodic components corresponding to the temporal changes in facial color and facial micro-motions. The consistency condition is constituted by the correspondence between the two in terms of peak interval, fluctuation trend, and continuity. When the corresponding periodic components meet the consistency condition, the processor generates real-time vital signs data including heart rate, respiratory rate, or blood oxygen saturation. When the periodic components do not meet the consistency condition, the processor does not output the generated results of the current window as confirmed vital signs but marks it as pending confirmation, providing this pending confirmation status to subsequent display or re-filtering stages.

[0024] When the processor performs facial recognition on a patient's face, it uses the patient's facial image captured by a camera as input. It matches the patient's facial image with pre-stored patient facial features accessible via a memory or communication interface, generating an identity recognition confidence score. The identity recognition confidence score indicates the degree of matching between the patient's facial image and the pre-stored patient facial features. The processor first determines whether the identity recognition confidence score meets preset recognition conditions. If the identity recognition confidence score meets the preset recognition conditions, the processor uses the matched patient identity as the patient's identity identifier. If the identity recognition confidence score does not meet the preset recognition conditions, the processor reads at least one of gait information, iris information, or wristband radio frequency signals according to preset priorities, and performs a consistency check between the read information and the facial recognition result. Gait information provides a basis for identifying the patient's movement pattern, iris information provides a basis for identifying the patient's eye biometric features, and wristband radio frequency signals provide a basis for identifying the patient wearing a wristband. The processor generates a patient identity identifier only when the read information meets the consistency check conditions with the facial recognition result; if the consistency check conditions are not met, the processor marks the identity confirmation result as pending confirmation to avoid incorrect indexing of patient medical information. Figure 4 This is used to display the binding relationship between a patient's facial image, gait information, iris information, wristband radio frequency signal, and patient identification.

[0025] When the processor accesses patient information via the communication interface, it reads medical records, examination and test records, and medical history records using the patient's identity identifier as an index. Medical records provide at least a diagnostic field, examination and test records provide at least a numerical field comparable to real-time vital signs data, and medical history records provide at least a past status field. The processor maps the diagnostic fields in medical records, the numerical fields in examination and test records, and the past status fields in medical history records to vital sign fields, examination and test fields, and medical history fields, respectively. Vital sign fields are used to represent the current vital sign value from real-time vital signs data, examination and test fields are used to represent similar test results from existing examination and test records, and medical history fields are used to indicate whether there are any records in the past status records that allow for deviations from the current status. When a vital sign field has the same name as a field in real-time vital signs data, the processor updates the corresponding field value according to the acquisition time; when a vital sign field does not have the same name as a field in real-time vital signs data, the processor retains the corresponding field value as the basis for comparison and does not forcibly generate a new field name.

[0026] When performing correlation comparisons, the processor writes the current vital sign value from the real-time vital sign data into the corresponding vital sign field, and then sequentially compares the current vital sign value with the similar test results recorded in the examination and testing field and the previous status recorded in the medical history field. The first stage of the sequential comparison is to determine the difference between the current vital sign value and the similar test results, and the second stage is to determine whether the medical history field records an allowable deviation status. If the difference between the current vital sign value and the similar test results meets the preset risk conditions, and no allowable deviation status is recorded in the previous status, the processor generates a risk status; if the difference between the current vital sign value and the similar test results does not meet the preset risk conditions, the processor uses the current vital sign value as the benchmark value for the next correlation comparison. The recording method of the above-mentioned preset risk conditions, allowable deviation status, and benchmark value is jointly determined by the comparison rules stored in the memory and the field status in the patient's condition information. Through this field mapping and sequential comparison, the processor can establish a traceable data chain between the vital sign values ​​from the current acquisition cycle and the existing examination and testing fields and medical history fields.

[0027] When the AR display unit outputs a prompt message, the processor matches the risk status with the preset display level and generates an overlay display instruction. The preset display levels include a vital signs display layer, a medical condition summary display layer, and an operation prompt display layer. The vital signs display layer displays the current vital signs value or the status pending confirmation; the medical condition summary display layer displays the medical condition summary formed by field correlation comparisons; and the operation prompt display layer displays the prompt content corresponding to the risk status. The overlay display instruction is sent by the processor to the AR display unit, which then displays the prompt message within the AR glasses' field of view according to the overlay display instruction. There is a one-to-one correspondence between the prompt message and the risk status. When no risk status is generated, the AR display unit does not output the prompt message corresponding to the risk status; it can only display the status pending confirmation or ordinary field status.

[0028] like Figure 5 As shown, when the communication interface receives remote annotation information, the remote annotation information includes at least the annotation graphic type, annotation location source, and annotation reception time. The processor spatially associates the remote annotation information with the patient's facial region or body positioning region within the current field of view. The input for spatial association is the positioning result of the patient's facial region or body positioning region within the current field of view and the annotation location source in the remote annotation information; the output is the annotation overlay position for use by the AR display unit. The processor writes the annotation overlay position into the overlay display instruction and controls the AR display unit to display the corresponding annotation graphic within the AR glasses' field of view. The remote annotation information and the risk status prompt information can be located at different display levels. The processor distinguishes the display areas of the vital signs display layer, the disease summary display layer, the operation prompt display layer, and the annotation graphic according to preset display levels, ensuring that the prompt information and the remote annotation information remain identifiable within the same field of view.

[0029] For device operation verification, verification can be performed using the acquisition log, filtering log, identity verification log, field mapping log, risk status log, and display output log recorded in the memory. The acquisition log records the time-series image sequence output by the camera and the acquisition time; the filtering log records whether the candidate image frames meet the conditions for facial region integrity, brightness stability, and displacement of adjacent image frames; the identity verification log records the confidence level of identity recognition, preset recognition conditions, and the source of the patient's identity identifier; the field mapping log records the source of the vital signs field, examination and test field, and medical history field; the risk status log records the comparison results of the current vital signs value with similar test results and whether the past status records allowable deviation status; the display output log records whether the AR display unit receives the overlay display command and prompt information.

[0030] Example 2 Please see Figures 1 to 5 This embodiment provides a non-contact AR monitoring method based on facial temporal images. The difference between this embodiment and Embodiment 1 is that Embodiment 1 focuses on the composition and collaborative relationships of the non-contact AR monitoring device, while this embodiment converts the processing procedures within the device into a sequence of steps executed by the AR glasses. S1 to S4 represent the main flow corresponding to the independently protected items of the method. The same camera, sensors, processor, memory, communication interface, and AR display unit can be configured as in Embodiment 1. This embodiment focuses on explaining the input sources, processing rules, intermediate states, and output results of each step.

[0031] S1 continuously acquires a sequence of temporal images of the patient's facial region and filters valid image frames based on facial region localization results and image frame quality conditions. After the camera is aligned with the patient's facial region within the AR glasses' field of view, it continuously outputs image frames according to the acquisition time. The processor generates facial region localization results based on the patient's facial region in the image frames. The facial region localization results are used to define the position and analyzable area of ​​the patient's facial region in the image frames. The processor compares the candidate image frames corresponding to the facial region localization results with the image frame quality conditions, including facial region integrity, brightness stability, and adjacent image frame displacement. Candidate image frames that simultaneously meet the above conditions are included in the valid image frame set; candidate image frames that do not simultaneously meet the above conditions are discarded, and the reason for rejection is recorded. The processor then performs temporal continuity verification on the valid image frames according to the acquisition time sequence. When both the acquisition interval and the number of consecutive valid image frames meet the conditions, a vital signs analysis window is formed; if any condition is not met, candidate image frames are re-filtered. The output of S1 is either the vital signs analysis window or a pending confirmation state, which is received by S2.

[0032] S2 extracts temporal variation features of facial color and facial micro-motion from valid image frames, and generates real-time vital sign data based on at least one temporal variation feature. The processor determines the region of interest (ROI) on the patient's face within the vital sign analysis window, and extracts the color channel variation sequence and pixel displacement variation sequence within the ROI according to the acquisition time sequence. The processor corrects the color channel variation sequence using an illumination change auxiliary signal provided by the sensor to form the temporal variation features of facial color; the processor subtracts the overall facial displacement from the pixel displacement variation sequence to form the temporal variation features of facial micro-motion. The processor extracts periodic components from the temporal variation features of facial color and facial micro-motion, and uses the peak interval, fluctuation amplitude, and continuity of the periodic components as the basis for vital sign generation. When the periodic components corresponding to the two types of temporal variation features meet the consistency condition, the processor generates real-time vital sign data; when the periodic components do not meet the consistency condition, the processor outputs a pending confirmation status. The output of S2 includes real-time vital sign data or a pending confirmation status, and is received by the correlation comparison link or AR display unit in S4.

[0033] S3 performs patient facial image identification. When the identification confidence level meets the preset identification conditions, the patient's medical information is retrieved. When the identification confidence level does not meet the preset identification conditions, gait information, iris information, or wristband radio frequency signal are fused to obtain the patient's identity identifier and the patient's medical information is retrieved. The processor first matches the patient's facial image with pre-stored patient facial features to generate an identification confidence level. When the identification confidence level meets the preset identification conditions, the processor directly uses the matched patient identity as the patient identity identifier and retrieves the patient's medical information via the communication interface. When the identification confidence level does not meet the preset identification conditions, the processor reads at least one of gait information, iris information, or wristband radio frequency signal according to a preset priority and performs consistency verification with the facial recognition result. If the verification is successful, the processor generates a patient identity identifier and uses the patient identity identifier as an index to retrieve medical records, examination and test records, and medical history records; if the verification is unsuccessful, the processor outputs an identity pending confirmation status to prevent subsequent risk statuses from being generated using an uncertain identity as an index. The output of S3 is patient medical information, patient identity identifier, or identity pending confirmation status, which is received by S4.

[0034] S4 compares real-time vital sign data with patient condition information according to vital sign fields, examination and test fields, and medical history fields. When the comparison result meets preset risk conditions, a risk status and condition summary are generated, and the corresponding prompt information is overlaid and output within the AR glasses' field of view. The processor reads the patient's condition information using the patient's identity identifier as an index, mapping the diagnosis fields in the medical record, the numerical fields in the examination and test records, and the past status fields in the medical history record to vital sign fields, examination and test fields, and medical history fields. The processor writes the current vital sign value from the real-time vital sign data into the corresponding vital sign field and sequentially compares the current vital sign value with the similar test results recorded in the examination and test fields and the past status recorded in the medical history field. When the difference between the current vital sign value and the similar test results meets the preset risk conditions and no allowed deviation status is recorded in the past status, the processor generates a risk status and condition summary; when the difference between the current vital sign value and the similar test results does not meet the preset risk conditions, the processor uses the current vital sign value as the benchmark value for the next comparison. The processor matches the risk status with the preset display levels and generates an overlay display command. The AR display unit then displays the vital signs display layer, the medical condition summary display layer, and the operation prompt display layer according to the overlay display command. When the communication interface receives remote annotation information, the processor spatially associates the remote annotation information with the patient's facial area or body positioning area within the current field of view and controls the AR display unit to display the corresponding annotation graphics.

[0035] The method in this embodiment can be verified through the device's operating status. The time-series image sequence output by the camera, the illumination change auxiliary signal and facial micro-motion signal output by the sensor, the valid image frames recorded by the processor, the vital signs analysis window, the time-series change features of facial color, the time-series change features of facial micro-motion, real-time vital signs data, patient identification, patient condition information, risk status, overlay display instructions, and remote annotation information can all be used as traceable data or status objects. The verification objects in S1 are the set of valid image frames and the vital signs analysis window; the verification objects in S2 are the real-time vital signs data or the status awaiting confirmation; the verification objects in S3 are the patient identification or the status awaiting confirmation; and the verification objects in S4 are the risk status, condition summary, prompt information, and annotated graphics. Each verification object is generated by the preceding processing step and received by the following processing step, forming a closed loop from patient facial area acquisition to AR field-of-view output.

[0036] The specific structures, processes, field states, processing order, judgment conditions, parameter sources, module collaboration, component cooperation, and carrier acceptance relationships in the above embodiments are used to explain the possible implementations of the present invention and should not limit the present invention to the specific embodiments listed. Without departing from the scope of the technical solutions described in this invention, any equivalent substitutions, equivalent modifications, equivalent combinations, order adjustments, corresponding module replacements, equivalent changes in field names, equivalent acceptance of the executing entity, or equivalent changes in the carrier form that can be conceived by those skilled in the art should fall within the scope of protection of this patent; however, the above changes should not be extended to unclaimed topics, nor should they alter the substantive correspondence of the technical objects through changes in name or identifier.

Claims

1. A non-contact AR monitoring device based on facial temporal images, characterized in that, It includes the AR glasses themselves, a camera, sensors, processor, memory, communication interface, and AR display unit mounted on the AR glasses; The camera outputs a sequence of time-series images of the patient's facial area to the processor, the sensor outputs facial micro-motion signals or lighting change auxiliary signals to the processor, and the memory stores the monitoring program that the processor can call. The processor runs a monitoring program to filter valid image frames according to the facial region localization results and image frame quality conditions, extracts facial color temporal change features or facial micro-movement temporal change features from the valid image frames, and generates real-time vital sign data. When the confidence level of the patient's facial image does not meet the preset recognition conditions, the processor fuses gait information, iris information, or wristband radio frequency signals to obtain the patient's identity identifier, and calls the patient's condition information through the communication interface based on the patient's identity identifier; The processor compares the real-time vital signs data with the patient's condition information according to the vital signs field, examination and test field, and medical history field. When the comparison result meets the preset risk conditions, a risk status is generated, and the AR display unit overlays and outputs the prompt information corresponding to the risk status within the AR glasses' field of view.

2. The non-contact AR monitoring device based on facial temporal images according to claim 1, characterized in that, The processor determines candidate image frames corresponding to the patient's facial region in the temporal image sequence according to the facial region localization result, and compares the candidate image frames with the image frame quality conditions, which include facial region integrity conditions, brightness stability conditions, and adjacent image frame displacement conditions. When a candidate image frame simultaneously meets the facial region integrity conditions, the brightness stability conditions, and the adjacent image frame displacement conditions, the candidate image frame is taken as the valid image frame. When a candidate image frame does not simultaneously meet the facial region integrity conditions, the brightness stability conditions, and the adjacent image frame displacement conditions, the processor discards the candidate image frame.

3. The non-contact AR monitoring device based on facial temporal images according to claim 2, characterized in that, The processor performs a temporal continuity check on the valid image frames. The temporal continuity check includes determining whether the acquisition interval between adjacent valid image frames meets a preset time interval condition according to the acquisition time sequence, and determining whether the number of consecutive valid image frames meets a preset quantity condition. When both the acquisition interval and the number of consecutive valid image frames meet the conditions, the processor forms a vital sign analysis window with the corresponding valid image frames. When the acquisition interval or the number of consecutive valid image frames does not meet the conditions, the processor re-selects valid image frames from the candidate image frames.

4. The non-contact AR monitoring device based on facial temporal images according to claim 3, characterized in that, The processor determines the region of interest on the patient's face within the vital signs analysis window, and extracts the color channel change sequence and pixel displacement change sequence within the region of interest on the patient's face in the order of acquisition time. The processor corrects the color channel change sequence with an illumination change auxiliary signal to form facial color temporal change features, and the processor subtracts the overall facial displacement from the pixel displacement change sequence to form facial micro-motion temporal change features.

5. The non-contact AR monitoring device based on facial temporal images according to claim 4, characterized in that, The processor extracts periodic components from the temporal change features of facial color and the temporal change features of facial micro-movement, and uses the peak interval, fluctuation amplitude, and continuity of the periodic components as the basis for generating vital signs. When the periodic components corresponding to the temporal change features of facial color and the temporal change features of facial micro-movement meet the consistency condition, the processor generates real-time vital sign data including heart rate, respiratory rate, or blood oxygen saturation. When the periodic components do not meet the consistency condition, the processor marks the corresponding vital sign generation result as pending confirmation.

6. The non-contact AR monitoring device based on facial temporal images according to claim 1, characterized in that, When the processor performs identity recognition on a patient's facial image, it first matches the patient's facial image with pre-stored patient facial features and generates the identity recognition confidence score. When the identity recognition confidence score meets the preset recognition conditions, the processor uses the matched patient identity as the patient identity identifier. When the identity recognition confidence score does not meet the preset recognition conditions, the processor reads gait information, iris information, or wristband radio frequency signals according to preset priorities, and generates the patient identity identifier after verifying the consistency between the read information and the facial recognition results.

7. The non-contact AR monitoring device based on facial temporal images according to claim 1, characterized in that, When the processor calls the patient's condition information through the communication interface, it reads the medical record, examination and test records, and medical history records using the patient's identity identifier as an index. It maps the diagnosis field in the medical record, the numerical field in the examination and test records, and the past status field in the medical history records to the vital signs field, examination and test field, and medical history field. When the vital signs field has the same name as the real-time vital signs data, the processor updates the corresponding field value according to the acquisition time. When the vital signs field does not have the same name as the real-time vital signs data, the processor retains the corresponding field value as the basis for association comparison.

8. The non-contact AR monitoring device based on facial temporal images according to claim 1, characterized in that, When the processor performs correlation comparison, it writes the current vital sign value from the real-time vital sign data into the corresponding vital sign field, and sequentially compares the current vital sign value with the similar test results recorded in the examination and testing field and the previous status recorded in the medical history field. When the difference between the current vital sign value and the similar test results meets the preset risk conditions and no allowable deviation status is recorded in the previous status, the processor generates the risk status. When the difference between the current vital sign value and the similar test results does not meet the preset risk conditions, the processor uses the current vital sign value as the benchmark value for the next correlation comparison.

9. The non-contact AR monitoring device based on facial temporal images according to claim 1, characterized in that, When the AR display unit outputs a prompt message, the processor matches the risk status with a preset display layer, generates an overlay display instruction including a vital signs display layer, a medical condition summary display layer, and an operation prompt display layer, and sends the overlay display instruction to the AR display unit. When the communication interface receives remote annotation information, the processor spatially associates the remote annotation information with the patient's facial area or body positioning area within the current field of view, and controls the AR display unit to display the corresponding annotation graphic within the AR glasses' field of view.

10. A non-contact AR monitoring method based on facial temporal images, for execution by the device described in any one of claims 1 to 9, characterized in that, A series of temporal images of the patient's facial region were continuously acquired, and valid image frames were selected according to the facial region localization results and image frame quality conditions. Extract facial color temporal change features or facial micro-movement temporal change features from the effective image frames, and generate real-time vital sign data based on at least one temporal change feature. The system performs facial recognition on the patient. When the confidence level of the recognition meets the preset recognition conditions, the system retrieves the patient's medical information. When the confidence level of the recognition does not meet the preset recognition conditions, the system fuses gait information, iris information, or wristband radio frequency signals to obtain the patient's identity identifier and retrieves the patient's medical information. The real-time vital signs data and the patient's condition information are compared and correlated according to the vital signs field, examination and test field, and medical history field. When the correlation and comparison results meet the preset risk conditions, a risk status and a condition summary are generated, and prompt information corresponding to the risk status is superimposed and output in the AR glasses' field of view.

Images