A method and system for assisted health detection of a mobile tabletop robot

Abstract

The application discloses a kind of auxiliary health detection method and system of mobile desktop robot.The method includes: initiative ranging and posture adaptation step, according to the type of detection task, call target space parameter condition, drive robot to adjust pose;Wearing state pre-checking step, extract the spatial deviation information of wearing accessory before contact type collection, when exceeding the qualified range, output guide instruction and postpone collection;Timing alignment step, with the completion time of adaptation as reference, timing alignment is carried out to non-contact sensing data and contact type measurement data;Multi-source fusion and hierarchical decision step, based on cross-checking rule, risk cross-validation is carried out to single abnormal data, and output comprehensive risk assessment grade and execute multi-level response strategy.The application reduces the false detection rate and false alarm rate of family health detection through the closed-loop architecture of space condition optimization, wearing pre-checking, timing alignment and cross-validation.

Description

Technical Field

[0001] This application relates to the field of intelligent health monitoring and mobile robot technology, and in particular to a method and system for assisting health detection of a mobile desktop robot. Background Technology

[0002] Mobile robots, with their autonomous navigation and human-computer interaction capabilities, are increasingly being applied to the field of home health monitoring, and are expected to provide users with more convenient health monitoring services.

[0003] However, in real-world home environments, users often encounter inaccurate measurement data when operating mobile robots for health checks due to improper operation or lack of cooperation. This leads to repeated testing and a less-than-ideal user experience. Furthermore, the health assessments provided by these mobile robots still deviate from the users' actual health conditions, failing to fully meet expectations for accuracy and reliability in home settings.

[0004] Therefore, further improvements are still needed to enhance the overall performance of mobile robots in home health monitoring. Summary of the Invention

[0005] This application aims to provide an auxiliary health detection method and system for mobile desktop robots to solve the problems of uncontrollable detection conditions, inability to self-check the wearing specifications, one-sided single-modal evaluation, and high false alarm rate in related technologies.

[0006] To achieve the above objectives, this application provides an assisted health detection method for a mobile desktop robot, comprising: Active ranging and attitude adaptation steps: Based on the type of the first detection task to be executed, retrieve the target space parameter conditions corresponding to the first detection task type from the preset parameter configuration table; drive the mobile desktop robot relative to the user's measured space parameters until the measured space parameters meet the target space parameter conditions; Wearing status pre-detection steps: Before initiating contact-based physiological parameter acquisition, acquire wearing perception data of the user's target body area; extract spatial deviation information of the contact detection accessory relative to the standard wearing status from the wearing perception data; if the spatial deviation information exceeds the preset acceptable range, output a wearing guidance command and postpone the contact-based physiological parameter acquisition until the spatial deviation information meets the preset acceptable range; Contact-based data acquisition and time-series alignment steps: Acquire multi-source health data, which includes a first type of sensory data and a second type of measurement data; wherein, the first type of sensory data is user perception information collected during and / or after the active ranging and posture adaptation steps, and the second type of measurement data is contact-based physiological parameters collected after passing the wear status pre-detection step; using the completion time of the active ranging and posture adaptation steps as the time-series benchmark, extract and fuse benchmark features from the first type of sensory data within the time window associated with the time-series benchmark, and use the second type of measurement data as the inserted feature frame of the time-series benchmark for time-series alignment. Multi-source fusion and hierarchical decision-making steps: Based on preset cross-validation rules, perform real risk cross-validation on individual abnormal data in the time-aligned multi-source health data, and output a comprehensive risk assessment level; according to the comprehensive risk assessment level, execute a multi-level response strategy that matches the comprehensive risk assessment level.

[0007] Secondly, this application also provides an auxiliary health detection system for a mobile desktop robot, including: an active ranging and posture adaptation module, used to call target spatial parameter conditions according to the detection task type and drive the mobile desktop robot to adjust its posture; a wearing status pre-detection module, used to acquire wearing perception data and extract spatial deviation information before contact acquisition; a contact data acquisition and time-series alignment module, used to acquire multi-source health data and perform time-series alignment based on the completion time of active ranging and posture adaptation; and a multi-source fusion and hierarchical decision-making module, used to perform real risk cross-validation based on cross-validation rules and execute hierarchical responses.

[0008] Beneficial effects This application systematically solves the problems of uncontrollable detection conditions, inability to self-check wearing compliance, and one-sided evaluation with high false alarm rate in related technologies through a closed-loop architecture of active ranging and attitude adaptation, wearing status pre-detection, contact data acquisition and time alignment, and multi-source fusion and hierarchical decision-making. Specifically, the active ranging and posture adaptation steps enable the robot to adaptively adjust to the optimal spatial conditions according to different detection task types, transforming the traditional "user adapts to device" into "device actively adapts to user," significantly improving the accuracy of non-contact detection data and user experience. The wearing status pre-detection step actively extracts spatial deviation information of contact detection accessories relative to the standard wearing state before contact physiological parameter collection, and outputs guidance instructions and postpones collection when the deviation exceeds the preset acceptable range, transforming "detecting anomalies after measurement" into "ensuring conditions before measurement," effectively reducing measurement errors caused by incorrect wearing. The contact data collection and time-series alignment step uses the completion time of active ranging and posture adaptation as the time-series benchmark to uniformly align non-contact sensing data and contact measurement data in time, providing a time-consistent data foundation for subsequent fusion evaluation. The multi-source fusion and hierarchical decision-making step performs real risk cross-validation on individual abnormal data after time-series alignment based on preset cross-validation rules, and outputs a comprehensive risk assessment level to execute a matching multi-level response strategy, significantly reducing the false alarm rate of single-modal evaluation and realizing a complete health management closed loop from daily screening to emergency intervention. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A schematic diagram of the structure of an assisted health detection system for a mobile desktop robot according to an embodiment of this application is shown.

[0011] Figure 2 A flowchart illustrating an example of an assisted health detection method for a mobile desktop robot according to an embodiment of this application is shown.

[0012] Figure 3 A schematic diagram of the structure of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0014] Currently, mobile robots have entered the field of home health monitoring. Some solutions integrate infrared temperature sensors and 2D image acquisition components into mobile robot platforms, enabling non-contact measurement of user body temperature and acquisition of facial images. These solutions utilize the autonomous navigation capabilities of mobile robots, freeing health monitoring from fixed locations and improving flexibility and convenience to some extent. However, in practical applications, the spatial relationship between the robot and the user still largely depends on the user's active adjustment and cooperation. The user needs to move to a suitable distance and face the camera; the robot itself does not have the ability to adaptively adjust the optimal detection distance and posture according to the type of detection task. When the user does not notice excessive distance or angular deviation, infrared temperature measurement emissivity compensation may fail, and facial expression recognition may suffer from reduced accuracy due to incomplete facial area capture.

[0015] Other solutions further integrate contact-based physiological parameter detection functions, such as connecting blood pressure cuffs or ECG electrodes via Bluetooth to complete blood pressure measurement or ECG data acquisition under robot guidance. These solutions preliminarily combine non-contact screening with precise contact measurement, expanding the parameter coverage of home health monitoring. However, existing solutions lack proactive pre-measurement identification capabilities regarding the proper wearing of contact-based testing accessories (such as blood pressure cuffs and ECG electrodes). Misalignment, incorrect orientation, improper tightness, or inconsistency in heart height of the blood pressure cuff, as well as misalignment or poor contact of the ECG electrodes, are often only discovered after measurement due to abnormal data, requiring repeated operation by the user, and erroneous data may be mistakenly recorded as health abnormalities.

[0016] In fall detection, some solutions utilize accelerometer and gyroscope data from wearable devices, identifying fall events through threshold judgment or machine learning classifiers. Wearable device solutions offer high portability, and accelerometer data is highly sensitive to impact-induced movements. However, wearable solutions require users to wear the device continuously, and in home settings (especially during rest or bathing), users often remove the device, resulting in a loss of monitoring coverage. Other solutions employ fixed cameras or millimeter-wave radar for fall detection. Millimeter-wave radar solutions indirectly infer human posture by analyzing changes in the spectral characteristics of reflected signals. While these solutions do not require users to wear devices, the signal characteristics of millimeter-wave radar are significantly affected by environmental multipath reflections and human orientation. Everyday actions (such as quickly sitting down or squatting to tie shoelaces) may be confused with actual falls in terms of spectral characteristics, leading to a high false alarm rate for single sensors.

[0017] Furthermore, in terms of comprehensive health status assessment, existing solutions often rely on a single physiological parameter (such as body temperature or heart rate) for anomaly detection. While some solutions collect multiple physiological data, they lack effective fusion and cross-validation mechanisms between these modalities. For example, elevated body temperature may be a transient response after exercise, and increased heart rate may stem from emotional fluctuations rather than organic disease. If an anomaly alarm is triggered solely based on a single indicator exceeding a common threshold, frequent false alarms are likely to occur in a home setting. This reduces user trust in the device and may also lead to users ignoring alarms when genuine anomalies occur due to the "boy who cried wolf" effect.

[0018] In summary, current technologies have laid a certain foundation for integrating health monitoring functions into mobile robot platforms, connecting contact accessories, implementing fall detection algorithms, and acquiring multiple parameters. However, these solutions primarily focus on the aggregation of functions and the improvement of single-point capabilities. In the complex and ever-changing home application scenarios, without systematic mechanisms such as proactive optimization of spatial conditions before detection, pre-identification of the proper wearing of contact accessories, human-computer interaction confirmation of fall events, and cross-modal cross-validation of multi-source heterogeneous data, it is difficult to achieve the leap from "passively collecting data" to "proactively ensuring detection quality." Therefore, how to enable mobile robots to proactively ensure detection conditions, pre-identify proper wearing, effectively eliminate false fall alarms, and achieve fusion evaluation of multi-source data in home scenarios remains a technical problem that needs further resolution.

[0019] It should be understood that the above description of the prior art is intended only to help the public better understand the inventive spirit and motivation of this application, and is not intended to limit this application. Furthermore, the technical solutions described in the above prior art may be undisclosed technical solutions, such as those under research or in the laboratory stage.

[0020] See Figure 1 , Figure 1A schematic diagram of an example of a mobile desktop robot-assisted health detection system 100 according to an embodiment of this application is shown. The system of this application adopts a three-layer architecture design of "perception layer 101, decision layer 102, and execution layer 103", and the modules deployed in each layer and their functions are as follows.

[0021] The perception layer 101 is equipped with a visual recognition module, a non-contact detection module, and a communication interface for the contact detection module.

[0022] The visual recognition module is used to acquire 2D images and 3D depth information of the user, supporting facial key point detection, expression recognition, pose estimation, and fall detection functions. It includes a 2D image acquisition component and a depth image acquisition component, both mounted on a controllable gimbal. Based on pose adjustment commands generated by the control processing module, the gimbal can be driven to adjust the shooting angle and field of view to adapt to changes in the user's position in different scenarios.

[0023] The non-contact detection module is used to collect the user's body temperature data in a non-contact manner. It includes an infrared thermometry module that measures the infrared radiation intensity over the user's forehead area and calculates the surface temperature after emissivity compensation. The non-contact detection module works in conjunction with a visual recognition module. The visual recognition module provides spatial location information of the user's face, and the non-contact detection module uses this location information to align with the user's forehead area to collect the temperature.

[0024] The contact detection module's communication interface is used to establish a wireless communication connection with external contact detection accessories, receive measurement data, and send control commands. It supports Bluetooth and Bluetooth Low Energy protocols, and compatible contact detection accessories include blood pressure cuffs, pulse oximeters, and ECG electrodes. Through this communication interface, the system can send control commands to the detection accessories before performing contact physiological parameter acquisition and receive measurement data after acquisition is complete.

[0025] The decision-making layer 102 is equipped with a control processing module.

[0026] The control and processing module executes the processing steps of the assisted health detection method described in this application, including active ranging and posture adaptation, wearing status pre-detection, contact data acquisition and time-series alignment, and multi-source fusion and hierarchical decision-making. The control and processing module includes an edge processor and memory. The edge processor supports real-time inference of the neural network model. The edge processor is responsible for performing high-real-time level one screening tasks, including spatial relationship assessment, wearing visual recognition, and preliminary risk assessment. When the preliminary judgment result is a level two or higher abnormality, the control and processing module uploads the desensitized multi-source health data to a cloud server via a remote communication module for in-depth analysis and backup.

[0027] The execution layer 103 is equipped with a mobile platform, a voice interaction module, and a remote communication module.

[0028] The mobile platform is used to drive the robot to adjust its spatial position relative to the user based on motion control commands generated by the control processing module. It adopts a differential drive chassis, equipped with independent drive wheels and omnidirectional support wheels, and features LiDAR, inertial measurement unit, and wheeled odometer, supporting smooth movement, autonomous positioning, and target tracking in the home environment.

[0029] The voice interaction module is used to perform voice prompts, abnormal inquiries, and remote notifications. It includes a microphone array and speaker, supporting voice wake-up and voice command recognition. In the pre-detection step of wearing status, the voice interaction module outputs wearing guidance voice; in the fall detection and closed-loop confirmation step, it issues confirmation inquiries and listens for user voice feedback; in the multi-source fusion and graded decision-making step, it plays corresponding health advice or reassuring statements based on the assessment level.

[0030] The remote communication module is used to upload de-identified multi-source health data to a cloud server via a wireless network when an abnormality of level two or above is detected, and to send notification messages to preset emergency contacts or automatically dial emergency numbers.

[0031] The modules exchange data loosely using a topic-based communication mechanism within the robot operating system. Image data is transmitted in a compressed format, and ECG signals are transmitted using a custom message type, ensuring efficient and decoupled data transmission between modules.

[0032] In the three-layer architecture described above, the perception layer 101 is responsible for collecting and accessing multi-source health data, providing the system with its raw data foundation; the decision layer 102 is responsible for processing, analyzing, and fusing the perceived data to generate detection results and response strategies; and the execution layer 103 is responsible for converting the output of the decision layer 102 into specific physical actions and interacting with the user. A closed loop of "perception, decision, and execution" is formed among the three layers: the data collected by the perception layer 101, after being analyzed and evaluated by the decision layer 102, drives the execution layer 103 to take corresponding actions. The results of the actions taken by the execution layer 103 may then trigger the perception layer 101 to enter a new collection cycle, thereby achieving an intelligent closed loop where the robot proactively adapts to the detection scenario and continuously optimizes detection conditions.

[0033] Figure 2 A flowchart illustrating an example of an assisted health detection method for a mobile desktop robot according to an embodiment of this application is shown. Figure 2 As shown, the method includes the following steps.

[0034] S100: Active ranging and attitude adaptation steps.

[0035] It should be noted that in a home desktop setting, different health monitoring tasks have varying requirements for the spatial relationship between the robot and the user. For example, body temperature detection, facial expression recognition, and posture capture each require different detection distances and angle ranges. If the robot simply moves to a relatively broad, preset position without fine-tuning its pose according to the specific monitoring task, the accuracy of the non-contact monitoring data will be constrained by spatial conditions. Related solutions typically rely on the user actively adjusting their position to cooperate with the robot; when the user doesn't notice or forgets to adjust, the quality of the monitoring data significantly decreases. The core design consideration of this step is to transform the passive mode of "user adapting to device" into an active mode of "device actively adapting to user," enabling the robot to autonomously optimize spatial conditions based on the type of monitoring task to be performed.

[0036] In this step, the system retrieves the target spatial parameter conditions corresponding to the first detection task type from a preset parameter configuration table, based on the type of the first detection task to be executed. The first detection task type refers to the category of the specific non-contact detection project that the system needs to perform, including body temperature detection, facial expression recognition, and body posture recognition. The target spatial parameter conditions refer to the spatial positional relationship requirements between the robot and the user that need to be met to complete the corresponding detection task, including distance and posture conditions. Different detection task types correspond to different combinations of spatial parameter conditions.

[0037] The system acquires the user's measured spatial parameters relative to the robot. These measured spatial parameters, obtained in real-time by sensors, characterize the current spatial relationship between the user and the robot, including measured distance and measured posture. The measured distance characterizes the current distance between the user and the robot, while the measured posture characterizes the user's current orientation angle relative to the robot's camera. The optical axis of the image acquisition device refers to the axis perpendicular to the imaging plane along the center of the image acquisition device's lens, i.e., the depth axis in the image acquisition device's coordinate system. The angle between the normal vector of the facial 3D plane and the optical axis of the image acquisition device reflects the overall tilt of the user's face relative to the image acquisition device.

[0038] The system compares the measured spatial parameters with the target spatial parameters. When the measured spatial parameters do not meet the target spatial parameters, it generates motion control commands to drive the robot to adjust its pose. These motion control commands are used to drive the robot's mobile platform and / or gimbal to move, thereby changing the relative spatial position between the robot and the user. The system repeats the spatial parameter acquisition and comparison process until the measured spatial parameters meet the target spatial parameters.

[0039] This step enables the system to achieve task-driven proactive optimization of spatial conditions for detection. The robot no longer relies on user intervention to adjust its position; instead, it adaptively adjusts to the optimal detection distance and posture based on different detection task types, ensuring optimal spatial conditions for non-contact detection data before collection. This step provides a reliable spatial foundation for subsequent non-contact sensing data acquisition.

[0040] S200: Pre-inspection procedure for wearing status.

[0041] It should be noted that for contact-based physiological parameter collection, such as blood pressure measurement and electrocardiogram (ECG) monitoring, which requires users to wear external accessories, proper wearing is a prerequisite for measurement accuracy. Misalignment, incorrect orientation, improper tightness, or inconsistency in heart height of the blood pressure cuff, as well as misalignment or poor contact of the ECG electrode pads, can all lead to systematic biases in the measurement data. In related solutions, wearing problems are often only discovered after measurement is completed due to abnormal data, requiring users to perform repeated operations, and erroneous data may be mistakenly recorded as health abnormalities. The core design consideration of this step is to transform the delayed quality judgment of "discovering abnormalities after measurement" into a proactive quality control of "ensuring conditions before measurement," completing the verification of proper wearing before the initiation of contact-based physiological parameter collection.

[0042] In this step, before initiating contact-based physiological parameter acquisition, the system acquires wearing perception data of the user's target body area. The target body area refers to the predetermined wearing location of the contact detection accessory on the user's body; for example, for a blood pressure cuff, it is the upper arm area, and for ECG electrodes, it is the chest area. The wearing perception data refers to data reflecting the actual wearing status of the contact detection accessory on the user's body; for example, it can be image data of the user's target body area acquired through a visual sensor, or signal data acquired through the contact detection accessory's own sensors.

[0043] The system extracts spatial deviation information of the contact detection accessory relative to the standard wearing state from the wear perception data. The standard wearing state refers to the ideal position and posture that the contact detection accessory should achieve when worn according to medical specifications. The spatial deviation information is used to quantitatively characterize the degree of deviation between the actual wearing state and the standard wearing state in terms of spatial position and / or direction.

[0044] The system compares the spatial deviation information with a preset acceptable range. The preset acceptable range refers to the maximum allowable deviation limit for the spatial deviation information; when the spatial deviation information is within this range, the wearing status is considered to meet the measurement specifications. If the spatial deviation information exceeds the preset acceptable range, the system outputs a wearing guidance instruction. This wearing guidance instruction is used to guide the user to correct the wearing method; for example, it can be a voice prompt instruction, a screen display instruction, or a combination of both. Simultaneously, the system delays the initiation of contact-based physiological parameter acquisition, continuously monitoring the wearing status until the spatial deviation information meets the preset acceptable range, only then allowing the initiation of contact-based physiological parameter acquisition.

[0045] Through this step, the system actively identifies and guides the correction of wearing compliance before initiating contact-based physiological parameter acquisition. The output of the wearing guidance command and the delay in the timing of acquisition initiation together form a quality assurance closed loop of "detection, guidance, re-inspection, and release," reducing measurement errors and data distortion caused by incorrect wearing at the source.

[0046] S300: Contact-based data acquisition and timing alignment steps.

[0047] It should be noted that the multi-source health data involved in this application includes two types of data with fundamentally different sampling mechanisms: the first type is sensory data acquired through non-contact continuous streaming (such as body temperature, facial expressions, and posture), and the second type is measurement data acquired through contact-triggered single-shot sampling (such as blood pressure and electrocardiogram). These two types of data differ in data format, sampling frequency, and time reference. If they are fused directly without time alignment, the fusion results will be distorted due to the inconsistency in time reference. The core design consideration of this step is to establish a unified time reference for non-contact sensory data and contact measurement data, so that multi-source data with different sampling mechanisms can be aligned in the time dimension, providing a time-consistent data foundation for subsequent multi-source fusion evaluation.

[0048] In this step, the system acquires multi-source health data. This multi-source health data includes a first type of sensory data and a second type of measurement data. The first type of sensory data refers to user physiological or behavioral characteristic data collected in a non-contact manner. It is user perception information collected during and / or after the S100 active ranging and posture adaptation step, and for example, includes at least one of body temperature data, facial expression data, body posture data, and voice feature data. The second type of measurement data refers to physiological parameter data collected through a detection accessory that comes into contact with the user's body. It is contact-type physiological parameters collected after passing the S200 wearing status pre-detection step, and for example, includes at least one of blood pressure data, heart rate data, blood oxygen data, and electrocardiogram data.

[0049] The system uses the completion time of the S100 active ranging and attitude adaptation step as the time reference. This time reference refers to the time reference origin used to unify the data acquisition times of each modality. Since the completion of the S100 step signifies the stability of the spatial positional relationship between the system and the user, and subsequent data acquisition is all under stable detection conditions, this moment is selected as the time reference origin. The system extracts fusion reference features from the first type of sensing data within the time window associated with this time reference. These fusion reference features are statistical features extracted from multiple sampled values ​​of the first type of sensing data within a preset time window, representing the overall level of the data within that time window. For example, this could be the average of multiple sampled values ​​within that time window.

[0050] Simultaneously, the system uses the second type of measurement data as an inserted feature frame for time-series alignment with the time-series reference. The inserted feature frame refers to mapping the completion time of the second type of measurement data acquisition to the corresponding position on the time axis with the time-series reference as the origin, so that the second type of measurement data is inserted as a feature frame on the time axis, thereby establishing a definite sequential relationship with the fusion reference feature of the first type of sensing data on the time axis.

[0051] This step addresses the challenge of effectively fusing multi-source heterogeneous data due to differences in sampling frequencies and time bases. The establishment of the time base and the insertion of feature frames enable non-contact continuous acquisition data and contact-triggered acquisition data to be expressed on a unified time axis, providing a time-consistent data foundation for the S400 multi-source fusion and hierarchical decision-making steps.

[0052] S400: Multi-source fusion and hierarchical decision-making steps.

[0053] It should be noted that the evaluation of relevant health monitoring programs often relies on the judgment of abnormalities based on a single physiological parameter. An abnormality in a single indicator may be caused by a variety of non-pathological factors. For example, an elevated body temperature may be due to a transient reaction after exercise, and an increased heart rate may be due to emotional fluctuations. If an abnormality alarm is triggered solely based on a single indicator exceeding a common threshold, it is prone to frequent false alarms in a home setting. The core design consideration of this step is: by performing cross-modal cross-validation on time-aligned multi-source health data, the abnormality of a single indicator must be verified by other modalities before the true risk is confirmed, thereby reducing the false alarm rate of single-modal assessment; at the same time, through a tiered response strategy, abnormalities of different degrees can receive appropriate treatment measures.

[0054] In this step, the system performs true risk cross-validation on individual abnormal data in multi-source health data after S300 time-series alignment, based on preset cross-validation rules. The preset cross-validation rules refer to a pre-established set of rules used to define the correlation verification relationships between different modalities of health data. When an abnormality occurs in a single indicator of a certain modality, the rule specifies which other modalities' correlation indicators must meet preset conditions to confirm the abnormality as a true risk. True risk cross-validation refers to distinguishing between true health risks caused by pathological factors and instantaneous data fluctuations caused by non-pathological factors through cross-modal correlation verification, thereby reducing false alarms caused by a single indicator abnormality.

[0055] Based on the results of cross-validation, the system outputs a comprehensive risk assessment level. This comprehensive risk assessment level refers to a comprehensive evaluation of the user's current health risk level, determined based on the actual risk level confirmed by cross-validation. The system then executes a multi-level response strategy matching the comprehensive risk assessment level. This multi-level response strategy refers to a progressive system of handling measures corresponding to different risk assessment levels; the higher the response level, the greater the urgency and intervention intensity of the handling measures.

[0056] This step enables the system to move from single-parameter judgment to multimodal cross-validation. The cross-validation rules ensure that any abnormality in a single indicator requires cross-modal verification before confirming the true risk, significantly reducing the false alarm rate of single-modal assessments. The tiered response strategy allows the system to match appropriate measures based on the risk assessment level, achieving a complete health management closed loop from routine screening to emergency intervention.

[0057] Compared with related technologies, the above embodiments systematically solve the problems of uncontrollable detection conditions, inability to self-check wearing compliance, and one-sided evaluation with high false alarm rate in related technologies by using a closed-loop architecture of active ranging and attitude adaptation, wearing status pre-detection, contact data acquisition and time alignment, and multi-source fusion and hierarchical decision-making. Specifically, the active ranging and posture adaptation steps enable the robot to adaptively adjust to the optimal spatial conditions according to different detection task types, transforming the traditional "user adapts to device" into "device actively adapts to user," significantly improving the accuracy of non-contact detection data and user experience. The wearing status pre-detection step actively extracts spatial deviation information of contact detection accessories relative to the standard wearing state before contact physiological parameter collection, and outputs guidance instructions and postpones collection when the deviation exceeds the preset acceptable range, transforming "detecting anomalies after measurement" into "ensuring conditions before measurement," effectively reducing measurement errors caused by incorrect wearing. The contact data collection and time-series alignment step uses the completion time of active ranging and posture adaptation as the time-series benchmark to uniformly align non-contact sensing data and contact measurement data in time, providing a time-consistent data foundation for subsequent fusion evaluation. The multi-source fusion and hierarchical decision-making step performs real risk cross-validation on individual abnormal data after time-series alignment based on preset cross-validation rules, and outputs a comprehensive risk assessment level to execute a matching multi-level response strategy, significantly reducing the false alarm rate of single-modal evaluation and realizing a complete health management closed loop from daily screening to emergency intervention.

[0058] The above combination Figure 2 The overall scheme of the four steps (S100-S400) of the method of this application has been described. The specific implementation methods of the additional technical features involved in each step of this application will be further explained in detail below.

[0059] Regarding the implementation details of how to obtain the measured spatial parameters of the user relative to the robot in the S100 active ranging and posture adaptation steps, in some examples of this application, a combination of a facial key point detection model and a depth image acquisition component is used to achieve accurate quantitative calculation of the measured distance and posture. The depth image acquisition component is used to acquire depth images of the user's face, and it can be a depth sensor, a structured light sensor, or a binocular vision device with depth perception capabilities, etc. This application does not limit the specific hardware form of the depth image acquisition component.

[0060] It should be noted that the method of obtaining the measured spatial parameters directly determines the accuracy of spatial adaptation. If only the depth image acquisition component is used to average the distance across the entire scene, it is impossible to distinguish between the user's face and the background or other body parts, and the distance measurement results may be interfered with. Similarly, if only the position of a simple face detection bounding box is used to determine whether the user is facing the camera, the accurate facial orientation angle cannot be obtained. The design consideration of this embodiment is: to use a facial key point detection model to extract facial feature points with clear anatomical significance, and to calculate the measured distance and measured pose by using the depth value and two-dimensional coordinates of the key points, respectively, thus transforming the coarse-grained requirement of "aligning with the user" into quantifiable spatial parameters.

[0061] In this embodiment, the facial keypoint detection model employs a pre-trained lightweight facial landmark localization network. For example, the model can use the MediaPipe Face Mesh model. The input to this model is a two-dimensional facial image captured by a two-dimensional image acquisition component. Before being fed into the model, the input image is scaled to a preset input size, for example, scaled to 256×256 pixels, and pixel value normalization is performed, that is, linearly mapping pixel values ​​from 0-255 to the 0-1 range. The output of this model includes two parts: first, the two-dimensional coordinates of multiple facial keypoints in the image coordinate system, each keypoint represented in the form of (x,y), with coordinate values ​​normalized to the 0-1 range, i.e., the ratio of the width and height relative to the input image; second, the coordinates of each keypoint in three-dimensional space, each keypoint represented in the form of (X,Y,Z), representing the spatial position of the keypoint in a coordinate system centered on the head, in millimeters. For example, this model can output the two-dimensional and three-dimensional coordinates of 468 facial keypoints.

[0062] Regarding the acquisition of the measured distance, the system acquires a depth image of the user's face through a depth image acquisition component. Extrinsic parameter calibration is pre-performed between the depth image acquisition component and the 2D image acquisition component; that is, the rotation matrix and translation vector between the two components are obtained through calibration, ensuring that each pixel in the depth image is spatially aligned with its corresponding pixel in the 2D image. It should be noted that the depth image acquisition component and the 2D image acquisition component can be integrated into the same image acquisition device or can be separate hardware devices; this application does not impose specific limitations in this regard. For the 2D coordinates of any key point output by the facial key point detection model, after inverse normalization to the actual pixel coordinates of the 2D image, they are mapped to the corresponding pixel position in the depth image through the extrinsic parameter calibration matrix. The depth value of this pixel is then read, which represents the physical distance between the key point and the image acquisition device, in millimeters.

[0063] The system selects the center point of the forehead and the tip of the nose from multiple facial key points for distance measurement. The center point of the forehead can be obtained by taking the arithmetic mean of the two-dimensional coordinates of multiple key points in the forehead region. For example, the mean of key point indices 10, 21, and 54 is taken. These three key points are located on the left, middle, and right sides of the forehead, respectively. Taking the mean value can obtain the geometric center of the forehead region, improving the stability of the distance estimation of the target area for temperature measurement. The tip of the nose is selected from the key point corresponding to the most protruding position of the face. For example, key point index 1 is selected.

[0064] The system performs a weighted average of the depth values ​​D1 at the center point of the forehead and D2 at the tip of the nose to obtain the measured distance D: ; Where w1° ​​is the weight of the center point of the forehead and w2 is the weight of the tip of the nose, and w1° + w2 = 1. For example, w1 is set to 0.6 and w2 to 0.4, with the center point of the forehead having a higher weight than the tip of the nose. This makes the measured distance more closely approximate the actual distance to the forehead, the target area for temperature measurement, thus prioritizing the accuracy of body temperature detection. The above distance calculation is performed periodically at a preset frequency to meet the robot's real-time perception requirements for changes in the user's position.

[0065] Regarding the acquisition of the measured pose, the system uses the PnP algorithm to calculate the pose angle of the user's face relative to the camera coordinate system based on the two-dimensional and three-dimensional coordinates of multiple key points output by the facial key point detection model. The selected key points should be distributed in the upper, middle, lower, left, and right areas of the face, which can relatively completely represent the orientation of the face in three-dimensional space. For example, six key points are selected: the center of the forehead, the tip of the nose, the left corner of the eye, the right corner of the eye, the left corner of the mouth, and the right corner of the mouth.

[0066] The solution process of the PnP algorithm is as follows. Let P be the three-dimensional spatial coordinates of the i-th key point. i =(X i ,Y i Z i Its two-dimensional projection coordinates on the image plane are p. i =(u i ,v i Based on the pinhole camera model, the two satisfy the following projection relationship: ; Among them, s i Let K be the scale factor, K be the intrinsic parameter matrix of the 2D image acquisition component (obtained through pre-calibration), R be a 3×3 rotation matrix, and t be a 3×1 translation vector. [R|t] is the extrinsic parameter matrix of the camera, representing the rotation and translation relationship between the camera coordinate system and the world coordinate system. The goal of the PnP algorithm is to solve the problem given K and the 3D coordinates P of multiple keypoints. iand its corresponding two-dimensional projection coordinates p i Under the given conditions, solve for R and t.

[0067] The PnP algorithm can be solved using various numerical methods. For example, the EPnP algorithm is used as follows: First, virtual control points are selected in 3D space, and the 3D coordinates of each key point are represented as a linear combination of the control points. Then, using the constraint relationship between the control points and the 2D projected coordinates, a system of linear equations is constructed and the coordinates of the control points in the camera coordinate system are solved. Finally, based on the correspondence between the coordinates of the control points in the two coordinate systems, R and t are solved through singular value decomposition.

[0068] After obtaining the rotation matrix R, it is decomposed to obtain the measured attitude angles. Let the expansion of R be: Then the pitch angle ; Yaw angle Roll angle .

[0069] in, to Represented by the elements of the rotation matrix R, The negative sign in the image acquisition device indicates direction. The optical axis of the image acquisition device refers to the axis perpendicular to the imaging plane along the center of the lens, i.e., the Z-axis in the image acquisition device's coordinate system. The angle between the normal vector of the three-dimensional plane of the face and the optical axis of the image acquisition device reflects the overall tilt of the user's face relative to the image acquisition device. Decomposing this angle into rotational components about the three coordinate axes yields the pitch, yaw, and roll angles. The pitch angle characterizes the vertical tilt of the user's head relative to the camera, the yaw angle characterizes the horizontal rotation, and the roll angle characterizes the lateral tilt of the head. When the absolute values ​​of the pitch or yaw angles are large, the face is not directly facing the camera, which may lead to errors in the emissivity compensation of infrared thermography or incomplete facial region capture during expression recognition.

[0070] The system will output the measured distance D obtained by the above method and the measured attitude including pitch angle, yaw angle and roll angle as measured spatial parameters for use in the spatial parameter comparison stage in S100.

[0071] To facilitate understanding, examples of its application scenarios are provided: Regarding the application scenario of proactive health screening and location optimization, a family user works at their desk after waking up in the morning. A mobile desktop robot initiates a health screening task according to a preset schedule, starting non-contact detection. The system uses a facial key point detection model and depth image acquisition components to obtain the user's measured distance and posture relative to the robot, discovering that the user's current sitting posture is slightly off-center and the distance is slightly too far. The robot automatically adjusts itself to be directly in front of the user using a differential drive chassis and gimbal, and prompts through a voice interaction module, "Please keep your head upright and move closer," while displaying a detection position indicator on the screen to help the user quickly adjust to the optimal detection posture. After completing posture adaptation, the system collects the user's body temperature data through an infrared temperature measurement module and analyzes the user's facial expression features through a facial expression subnetwork. The detection results show that the user's body temperature is 37.2℃ and their facial expression is poor, so the system automatically judges it as "mildly abnormal" and suggests the user rest for a while and measure their blood pressure via voice. This application scenario demonstrates the technical effectiveness of this application in adaptive control of detection position and distance. The robot actively adjusts its posture and guides the user to cooperate, rather than relying on the user to adjust to the appropriate position on their own, thus transforming the traditional "user adapts to device" into "device actively adapts to user".

[0072] In this embodiment, the system refines the higher-level operation of "acquiring the measured spatial parameters of the user relative to the robot" in S100 into a joint calculation scheme based on depth-weighted ranging and PnP pose estimation using a facial keypoint detection model. The extrinsic parameter calibration of the 2D image acquisition component and the depth image acquisition component achieves accurate mapping from 2D keypoints to depth values; the averaging of multiple keypoints at the forehead center point improves the stability of distance estimation; the selection of six keypoints covers the main spatial orientations of the face, and the PnP algorithm converts 2D image features into quantized outputs of 3D pose angles. The above process does not depend on a specific network structure or training data. Those skilled in the art can implement this scheme using different known facial keypoint detection models according to the input / output requirements and processing steps described in the specification. It should be noted that the acquisition of depth images and the acquisition of 2D images can be completed separately by different components of the same image acquisition device, or they can be completed collaboratively by independent depth image acquisition components and 2D image acquisition components. This application does not limit the specific hardware form and integration method of the relevant components.

[0073] Regarding the S200 wearing status pre-inspection steps, for the blood pressure cuff, a contact-type detection accessory, how to extract spatial deviation information of the contact-type detection accessory relative to the standard wearing status from the wearing perception data, in some examples of this application, the cuff area and key points are extracted by a cascaded visual inspection model, and multi-dimensional spatial deviation quantification indicators are calculated to achieve an objective assessment of the standardization of blood pressure cuff wearing.

[0074] It should be noted that judging the proper wearing of a blood pressure cuff involves multiple evaluation factors. Whether the cuff is correctly positioned on the upper arm, whether its direction is along the longitudinal axis of the upper arm, whether the tightness of the cuff wrapping is appropriate, and whether the cuff is at the same level as the heart all directly affect the accuracy of blood pressure measurements. Relying solely on a single dimension (such as only detecting the presence of the cuff) cannot comprehensively assess the proper wearing. This embodiment is designed with a two-stage visual detection architecture cascaded with a target detection network and a keypoint detection network. It first locates the cuff area, then extracts fine keypoints, and quantitatively evaluates the wearing status from four dimensions: position, direction, tightness, and heart height. This transforms the subjective judgment of "whether the wearing is proper" into an objective comparison of multi-dimensional spatial deviation information.

[0075] In this embodiment, the system utilizes a first object detection network to detect the upper arm region and the blood pressure cuff region within the user's body target area. The first object detection network is used to locate the spatial positions of the upper arm and cuff in the input image and outputs a cuff location bounding box, represented in the form (x, y, w, h), where (x, y) are the coordinates of the upper left corner of the bounding box, w is the width, and h is the height. For example, the first object detection network can employ the YOLOv8 model, with its backbone network using a CSPDarknet structure, achieving the detection of targets of different sizes through multi-scale feature fusion.

[0076] After detecting the cuff region, the system uses a first keypoint detection network to extract multiple corner points of the cuff and wrist reference points within the cuff location bounding box. The first keypoint detection network is used to accurately locate the coordinates of feature points within the cuff region. For example, the first keypoint detection network can employ the HRNet model, which maintains high-resolution feature maps, achieving high spatial accuracy in keypoint localization tasks and distinguishing between similar image features such as cuff corner points and cuff surface texture. The wrist reference point is selected from the radial artery location at the wrist, used to determine the distal reference direction of the upper arm's longitudinal axis.

[0077] The system extracts the corner coordinates of the cuff and the wrist reference point coordinates from the output of the first key point detection network, and calculates the spatial deviation information in the following four dimensions.

[0078] First, directional deviation. Let the coordinates of the two corner points of the cuff along the longitudinal axis be (x1, y1) and (x2, y2), respectively. The cuff direction angle θ is calculated using the following formula: The cuff orientation angle characterizes the cuff's tilt angle within the image plane. The system compares the cuff orientation angle to the standard direction of the upper arm's longitudinal axis. The upper arm's longitudinal axis is determined by the line connecting the center point of the shoulder joint and the radial artery reference point at the wrist; this direction anatomically corresponds to the course of the brachial artery. The angle between the cuff's longitudinal axis and the upper arm's longitudinal axis is the orientation deviation. If this angle is large, it indicates that the cuff's bladder is not aligned with the brachial artery, causing a deviation in pressure transmission during inflation, which may lead to deviations in the measurement of systolic or diastolic blood pressure.

[0079] Second, positional deviation. The system calculates the Euclidean distance between the cuff center and the upper arm center as the positional deviation. The cuff center is taken as the geometric center coordinates of the cuff position frame, and the upper arm center is taken as the midpoint of the line connecting the center points of the shoulder joint and the elbow joint. The positional deviation is used to measure whether the cuff is worn in the recommended position in the middle of the upper arm, which corresponds to the main segment of the brachial artery. When the Euclidean distance exceeds a preset threshold, it indicates that the cuff has deviated from the optimal compression position and may partially or completely fail to cover the brachial artery.

[0080] Third, heart height deviation. The system calculates the vertical height difference between the cuff center and the estimated heart position. The heart position is estimated from chest key points in the human posture keypoint model. For example, the midpoint between the key point corresponding to the suprasternal notch and the key point corresponding to the xiphoid process is taken as an approximate reference point for the heart position. The vertical height difference between the cuff center and the heart position is the heart height deviation. This deviation is used to assess whether the cuff and heart are on the same horizontal plane. Inconsistency between the height of the cuff and the heart will produce hydrostatic pressure error, causing the blood pressure measurement value to deviate from the true level.

[0081] Fourth, tightness level. The system uses a pre-trained tightness classification model to classify the image features of the cuff area and outputs the tightness level. The tightness level is divided into three categories: loose, moderate, and tight.

[0082] Regarding the training of the tightness classification model, this model is a custom-trained model used in this application. Its training dataset consists of 5000 labeled images, covering arm cuff wearing scenarios under different lighting conditions (100-500 lux), different skin tones, and different clothing thicknesses. Each image's annotation includes the cuff location bounding box, cuff orientation angle (0-360°, in degrees), tightness level (loose / moderate / tight, labeled by medical professionals), and the vertical deviation of the cuff center from the heart height (in centimeters). For example, moderate corresponds to the state where "two fingers can be just inserted under the cuff," loose corresponds to the state where "more than two fingers can be easily inserted under the cuff," and tight corresponds to the state where "it is difficult to insert two fingers under the cuff." Labeling by medical professionals ensures the clinical rationality of the tightness classification. To ensure label consistency, each image was independently labeled by two medical professionals, and samples with consistent labeling were used as training data; for samples with inconsistent labeling, a third senior medical professional determined the final labeling result.

[0083] The tightness / looseness classification model employs a lightweight convolutional neural network. For example, MobileNetV3-Small is used as the backbone, followed by a global average pooling layer and a fully connected classification layer. The fully connected classification layer has 3 output nodes, corresponding to the loose, moderate, and tight categories. During training, the cross-entropy loss function is used, with the Adam optimizer employed. The initial learning rate is set to 0.001, the batch size to 64, and the training epochs to 50. The backbone network uses the h-swish activation function, and the output layer uses the Softmax activation function. The validation set loss is monitored during training, and training stops when the validation set loss does not decrease for five consecutive epochs. After training, the model parameters are fixed into an inference model and deployed on the edge processor of a mobile desktop robot.

[0084] The system uses the aforementioned directional deviation, positional deviation, heart height deviation, and tightness level as spatial deviation information for the blood pressure cuff, which is then used in the acceptable range comparison step in S200. When the deviation in any dimension exceeds the preset acceptable range, the system determines that the wearing status is unacceptable and triggers the output of a wearing guidance command.

[0085] To facilitate understanding, examples of its application scenarios are provided: Regarding the application scenario of intelligent guidance for blood pressure cuff wearing, Ms. Li, an elderly user, prepared to measure her blood pressure and had already put the cuff on her upper arm. The robot located the cuff area using a first object detection network and extracted the cuff corner points and wrist reference points using a first key point detection network. The system calculated that the cuff's direction angle was more than 10° from the upper arm's longitudinal axis, and the Euclidean distance between the cuff center and the upper arm center was more than 2cm, determining that the cuff did not completely cover the upper arm and the arrow direction was off-target. The system immediately prompted via voice interaction, "Please fit the cuff snugly against the middle of the upper arm, with the arrow pointing towards the forearm along the blood vessel direction," and displayed an arm wearing diagram and a red box indicating the abnormal position on the screen. Ms. Li adjusted the cuff according to the guidance. After re-detection, the system confirmed that the directional and positional deviations were within the preset acceptable range, the tightness level was moderate, and the heart height deviation was less than 3cm. The system automatically started blood pressure measurement, obtaining accurate systolic and diastolic blood pressure data. This application scenario demonstrates the technical effect of this application in transforming "discovering abnormalities after measurement" into "ensuring conditions before measurement" in the blood pressure measurement process. By proactively identifying improper wearing and providing guidance, the reliability of blood pressure data is ensured from the source.

[0086] In this embodiment, the system refines the higher-level operation in S200, which involves "extracting spatial deviation information from wearing perception data" for blood pressure cuffs, into a complete technical solution encompassing cascaded visual detection model construction, multi-dimensional spatial deviation calculation, tightness classification model training, and deviation information output. The cascaded architecture of the first target detection network and the first keypoint detection network allows the system to first complete coarse localization of the cuff area and then refine keypoint extraction, balancing detection speed and localization accuracy. The quantitative evaluation of four dimensions—direction deviation, position deviation, heart height deviation, and tightness level—covers the main wearing factors affecting the accuracy of blood pressure measurement, avoiding omissions that may occur when judging based on a single indicator. The disclosure of the training dataset composition, annotation method, network structure, and training process of the tightness classification model enables those skilled in the art to reproduce the training and application of the model.

[0087] Regarding the S200 wearing status pre-detection steps, specifically for ECG electrode pads (a contact-based detection accessory), this application details the implementation of extracting spatial deviation information from wearing perception data and verifying signal quality after the spatial deviation information meets a preset acceptable range. In some examples, a dual verification mechanism of "visual pre-detection + signal quality control" is employed. The visual pre-detection part calculates the standard lead position and positional deviation using a human posture key point model. The signal quality control part initiates ECG signal acquisition after the visual pre-detection is successful, evaluating signal quality through three dimensions: signal-to-noise ratio, baseline drift, and R-wave detection rate. Together, these two components constitute a complete closed loop for ECG electrode pad wearing status pre-detection.

[0088] It should be noted that the proper wearing of ECG electrode pads involves two different levels of issues. The first level is the issue of spatial positioning. This involves ensuring the electrode pads are attached to the lead positions that meet medical standards. If the electrode pads deviate too far from the standard lead positions, the acquired ECG signals may not reflect the electrical activity of the corresponding cardiac region, leading to abnormal waveform morphology or clinical interpretation errors. The second level is the issue of contact quality. Even if the electrode pads are correctly positioned, poor contact between the electrodes and the skin (such as loose pads, dry skin, or insufficient conductive adhesive) can result in excessive signal noise, baseline drift, or waveform distortion. These two levels of issues are independent: correct positioning does not equate to good contact, and good contact does not equate to correct positioning. Most existing solutions rely on user self-perception or single-dimensional detection, making it difficult to simultaneously cover both levels of verification. This embodiment is designed to combine visual recognition with signal analysis. First, a human posture key point model is used to verify the accuracy of the electrode pad position in the spatial dimension. Then, an ECG signal processing algorithm is used to verify the goodness of the contact quality in the signal dimension. These two verification steps are connected in series; failure in either step triggers a wearing reminder, thus forming a dual quality assurance system.

[0089] Regarding the implementation details of visual position pre-detection using a human posture keypoint model, the system calculates the standard lead positions of ECG electrode pads using this model. The human posture keypoint model employs a pre-trained lightweight human posture estimation network. For example, the MediaPipePose model can be used. The model's input is a 2D human image acquired by a 2D image acquisition component, and its output includes the 2D and 3D coordinates of multiple skeletal keypoints of the human torso and upper limbs. For example, the model can output 33 skeletal keypoints, covering key upper body areas such as the shoulders, chest, waist, and hips.

[0090] The system calculates the positions of standard leads based on detected key skeletal points in the chest, clavicle, and costal arch regions, according to human anatomical relationships. Taking the standard chest leads V1 to V6 as an example: lead V1 is located at the fourth intercostal space along the right sternal border; lead V2 is located at the fourth intercostal space along the left sternal border; lead V3 is located at the midpoint of the line connecting V2 and V4; lead V4 is located at the fifth intercostal space along the left midclavicular line; lead V5 is located at the same level as V4 along the left anterior axillary line; and lead V6 is located at the same level as V4 along the left midaxillary line. The system determines the position of the midclavicular line through key clavicle points, the positions of the left and right sternal borders through key sternal points, and calculates the intercostal positions through multiple key points in the rib region, thus locating the spatial coordinates of each standard lead. Simultaneously, the system obtains the actual attachment position of the current electrode pad through visual recognition and calculates the positional deviation between the actual position and the standard lead position.

[0091] The positional deviation is calculated as follows. Let the standard lead position of the i-th electrode be... The actual attachment position of the electrode sheet obtained by visual recognition is The positional deviation of the electrode plate is: .

[0092] Where Δdᵢ represents the positional deviation of the i-th electrode plate. The x-coordinate representing the standard lead position of the i-th electrode plate. The y-coordinate represents the standard lead position of the i-th electrode plate. The x-coordinate represents the actual attachment position of the i-th electrode. The y-coordinate represents the actual attachment position of the i-th electrode pad. The system uses the positional deviation of each electrode pad as the spatial deviation information corresponding to the ECG electrode pad. For example, if the positional deviation of any electrode pad exceeds 2cm, the visual pre-detection is deemed unqualified. The positional deviation refers to the Euclidean distance between the actual attachment position and the standard lead position on the user's body surface. This distance is calculated from the two-dimensional image coordinates obtained by the image acquisition device, converted into the actual physical distance through a pre-calibrated pixel-to-physical distance conversion relationship, and the unit is centimeters. The threshold is set based on medical reference data of adult chest size and lead distance. The standard distance between adjacent chest leads is usually 3-5cm. The 2cm tolerance ensures that the electrode pad is still within the effective acquisition area of ​​the target lead and will not acquire signals from adjacent leads due to excessive offset.

[0093] Regarding the implementation details of the triple quality assessment of ECG signals, after passing the visual pre-inspection, the system initiates ECG signal acquisition. A data connection is established with the ECG electrode pads via the contact detection module communication interface to acquire initial ECG signal segments. For example, the initial acquisition duration is 10 seconds, the sampling frequency is 500Hz, and a total of 5000 sampling points are collected.

[0094] The system performs signal quality assessment on the initial ECG signals acquired in the following three dimensions.

[0095] First, signal-to-noise ratio (SNR) calculation. SNR measures the power ratio of the effective signal component to the background noise component in an electrocardiogram (ECG) signal. The system first performs bandpass filtering on the initial ECG signal, for example, using a 4th-order Butterworth bandpass filter with a passband frequency range of 0.5-40Hz to filter out high-frequency electromyographic noise and low-frequency baseline drift components. Then, the QRS complex portion of the filtered signal is taken as the signal component, and the TP segment (i.e., the flat segment between the end of the T wave and the beginning of the next P wave) is taken as the noise component, and their average power is calculated separately.

[0096] Signal-to-noise ratio The unit is decibels, of which, Characterized as the average power of the signal components (QRS group portion), This is characterized by the average power of the noise component (TP segment). For example, a signal-to-noise ratio (SNR) is considered acceptable when it is greater than 15 dB. A 15 dB threshold means that the power of the effective signal is approximately 32 times the noise power; at this SNR level, R-wave detection and waveform analysis of the QRS group have sufficient reliability.

[0097] Second, baseline drift detection. Baseline drift refers to the slow shift of the overall DC component of the ECG signal over time, typically caused by slow relative movement between the electrodes and skin, chest rise and fall caused by respiration, or changes in electrode polarization voltage. The system uses a high-pass filter to remove baseline drift components. For example, a fourth-order Butterworth high-pass filter with a cutoff frequency of 0.5 Hz is used. The choice of a cutoff frequency of 0.5 Hz balances the need for effective baseline drift removal with the preservation of low-frequency ECG components such as the ST segment. If the cutoff frequency is too high, important diagnostic information from the ST segment may be filtered out; if the cutoff frequency is too low, the suppression of baseline drift caused by respiration will be insufficient. The shift in the signal before and after filtering is calculated, i.e., the difference between the DC component of the signal before filtering and the residual low-frequency components in the signal after filtering. If the drift is greater than 200 μV, it is considered baseline instability. The 200 μV threshold is based on clinical ECG signal standards. The amplitude of the R wave in a normal electrocardiogram signal is usually 0.5-2mV. When the baseline drift exceeds 200μV, low-frequency components such as the ST segment may be submerged by the drift, affecting the judgment of abnormalities such as myocardial ischemia.

[0098] Third, R-wave detection rate calculation. The R-wave is the most prominent characteristic waveform in an electrocardiogram (ECG) signal, and accurate R-wave detection is fundamental to heart rate calculation and heart rate variability analysis. The system uses the Pan-Tompkins algorithm to detect R-wave location. This algorithm first performs bandpass filtering on the signal (passband frequency 5-15Hz, covering the main energy band of the QRS complex), then enhances the slope characteristics of the QRS complex through differentiation, amplifies high-frequency components through squaring, and finally obtains the energy envelope of the QRS complex through sliding window integration. The peak position of the R-wave is then detected on the energy envelope using an adaptive threshold. The system calculates the expected number of R-waves based on the sampling frequency and initial acquisition duration. For example, for a 10-second initial acquisition segment, if the user's heart rate is 60 beats / minute, the expected number of R-waves is 10; if the heart rate is 80 beats / minute, the expected number of R-waves is approximately 13. The system divides the number of R-waves actually detected by the Pan-Tompkins algorithm by the expected number of R-waves to obtain the R-wave detection rate. For example, when the R-wave detection rate is greater than 90%, the signal quality is considered acceptable. An R-wave detection rate below 90% usually indicates that the signal noise is too great, causing the R-wave to be submerged, or that the electrode position is severely deviated, resulting in abnormal QRS group morphology that cannot be recognized by the algorithm.

[0099] Regarding the triggering logic for the dual verification of visual pre-inspection and signal quality control, the system comprehensively judges the results of visual pre-inspection and signal quality assessment. The logical relationship between visual pre-inspection and signal quality assessment is "OR": when visual pre-inspection fails (i.e., the positional deviation of any electrode exceeds a preset threshold), or when any indicator in the signal quality assessment fails to meet the preset signal quality standard (i.e., SNR is substandard, baseline drift exceeds the standard, or R wave detection rate is substandard), the system determines that the ECG electrode wearing status is unqualified and triggers a wearing reminder.

[0100] The wear reminder output includes at least one of voice prompts and screen displays. For example, the voice prompt might read, "Please place the ECG electrodes on your chest according to the marked positions and keep your skin dry," while the screen display might include a diagram of the standard lead positions and a comparison of the current electrode positions. After the user adjusts the electrode placement, the system re-performs visual pre-detection and signal acquisition assessment.

[0101] To facilitate understanding, examples of its application scenarios are provided: Regarding the application scenario of ECG electrode patch standardization inspection, Mr. Zhang underwent an ECG test. The robot first guided him via voice to expose his chest and maintain a seated posture. Then, it detected key points in the chest, clavicle, and costal arch regions using a human posture key point model, calculated the standard lead positions based on human anatomy, and obtained the actual attachment position of the current electrode patch through visual recognition. The visual pre-inspection results showed that the electrode patch position deviation was within the preset acceptable range, and the system started ECG signal acquisition. After the initial 10-second ECG signal acquisition, the system performed a triple signal quality assessment. If the signal-to-noise ratio was below 15dB, the baseline drift exceeded 200μV, and the R-wave detection rate was below 90%, the signal quality was deemed unacceptable, indicating poor electrode patch contact. The system prompted him via voice, "Please reattach the electrode patch properly. The patch should be firmly attached to the skin and not stick to clothing." After Mr. Zhang adjusted the electrode patch attachment, the system re-acquired the signal, and all indicators met the preset standards. The system output a stable ECG curve and HRV analysis results. This application scenario demonstrates the dual verification mechanism of this application in home ECG monitoring: visual pre-inspection ensures accurate electrode placement, and signal quality control ensures good contact quality. Together, they guarantee the reliability of ECG data.

[0102] In this embodiment, the system refines the higher-level operation in S200, which involves "extracting spatial deviation information from wearing perception data" for ECG electrode pads, into a dual verification scheme combining visual position pre-detection and signal quality triple evaluation. The standard lead positions calculated by the human posture key point model provide a spatial reference benchmark based on human anatomy for electrode pad attachment, making the calculation of position deviation objective. Signal quality evaluation in three dimensions—signal-to-noise ratio, baseline drift, and R-wave detection rate—examines the effectiveness of the ECG signal from the power domain, time domain, and morphological domain respectively, forming a complementary system. Signal-to-noise ratio reflects the overall noise level, baseline drift reflects the degree of low-frequency interference, and R-wave detection rate reflects the integrity of the core characteristic wave. The "OR" logic concatenation of visual pre-detection and signal quality control ensures that the system only determines the wearing status is qualified when the electrode pads are correctly attached in spatial position and meet the signal acquisition requirements in terms of contact quality. This dual-check system guarantees the reliability of ECG data from the source.

[0103] Regarding the specific implementation of the fall detection and closed-loop confirmation function in this application, in some examples, the system determines a fall event through kinematic analysis of a three-dimensional skeletal key point sequence, and provides immediate confirmation via a voice interaction unit after determination, executing tiered emergency measures based on user feedback. This function runs independently of the main health detection process of S100-S400, providing monitoring and response to sudden safety events for the elderly and children in home settings.

[0104] It should be noted that fall detection faces two core challenges in home settings. The first challenge is detection accuracy. The home environment is complex, and everyday actions such as squatting to tie shoelaces or quickly sitting down share similar motion characteristics with falls. Relying solely on a single acceleration threshold or simple body tilt angle judgment can easily misclassify everyday actions as falls, resulting in a high false alarm rate. The second challenge is the false alarm handling mechanism. If an alarm is triggered immediately upon detecting a suspected fall, it may cause unnecessary panic and waste of resources due to false alarms; however, simply recording the information without triggering an alarm may delay timely medical intervention in the event of a real fall. This embodiment is designed to: use a combination of three kinematic rules with clear geometric and physical meanings to determine the fall event and associate it with three temporal stages: rapid descent, slump, and significant tilting, thus distinguishing falls from everyday actions from a kinematic perspective; simultaneously, after a fall is detected, an immediate voice confirmation step is added, giving the user the opportunity to deny a false alarm, and a tiered response is executed based on the confirmation result, establishing a balance between "no false alarms" and "no missed alarms."

[0105] Regarding the acquisition of the 3D skeletal keypoint sequence, the system continuously acquires user images through the 2D image acquisition component of the visual perception module and calls the pose estimation model for inference. The pose estimation model employs a pre-trained lightweight human pose estimation network. For example, this model can use the MediaPipe Pose model. The input to this model is an RGB image, and the output is the 3D coordinates of multiple skeletal keypoints. For example, this model can output 33 skeletal keypoints, with the 3D coordinates of each keypoint represented in the form (X, Y, Z), where X and Y are the spatial coordinates in the image plane, and Z is the depth direction coordinate. Combining depth map information can further correct the depth values ​​of each keypoint, improving the 3D spatial positioning accuracy. The model's inference speed meets the requirements of real-time detection; for example, the inference speed is greater than 30 frames per second.

[0106] The system extracts key skeletal points for fall detection from 33 skeletal key points, including the hip center point (the midpoint between the left and right hip key points), the left knee key point, the right knee key point, the left shoulder key point, and the right shoulder key point. These key points can characterize the spatial position of the core body parts, the lower limb support status, and the trunk orientation, providing necessary kinematic information for fall detection.

[0107] Regarding fall detection based on kinematic rules, the system analyzes the acquired 3D skeletal keypoint sequence based on preset kinematic rules. A fall event is determined to have occurred when the following three conditions are met simultaneously.

[0108] Condition 1: The vertical descent height of the hip center point within a preset first time period exceeds a preset proportion of the hip height when standing. For example, the preset first time period is 1 second, and the preset proportion is 50%. This condition is used to identify the rapid descent of the body's center of gravity. The hip height when standing is based on the average hip height of the user in an upright position during the previous preset time period. When a person falls, the hips rapidly descend from standing height to near the ground within a short period, typically exceeding half of the standing hip height. In everyday movements such as squatting or bending over, the descent speed or magnitude of the hips is relatively slower.

[0109] Condition 2: The vertical coordinate of the hip key point is lower than the vertical coordinate of the knee key point, and the duration exceeds a preset second time. For example, the preset second time is 3 seconds. This condition is used to determine whether the human body is in a fallen posture and remains there. In a normal standing or walking state, the hip is always higher than the knee. When a person falls, the body hits the ground, and the hip drops to a height close to or lower than the knee, and due to physical limitations, the person cannot stand up on their own within a short time. While actions such as squatting to tie shoelaces also lower the hip, this is usually completed within a few seconds, and the person returns to a standing position.

[0110] Condition 3: The body tilt angle exceeds a preset angle threshold. For example, the preset angle threshold is 60°. The body tilt angle is calculated from the angle between the line connecting the shoulders (the line connecting the key points of the left and right shoulders) and the horizontal plane. The horizontal plane refers to a reference plane perpendicular to the direction of gravity. A body tilt angle of 0° indicates that the torso is perpendicular to the horizontal plane (upright state), and a tilt angle of 90° indicates that the torso is parallel to the horizontal plane (lying down state). The body tilt angle is used to determine whether the torso has significantly deviated from an upright posture. When a person is standing or walking normally, the body tilt angle is close to 0°. When a person falls, the torso tilts with the body, and the tilt angle increases significantly. However, in everyday actions such as bending over to pick up objects, although the torso leans forward, it usually does not exceed 60°.

[0111] The three conditions are logically related by "AND," meaning that the system will only determine a fall event if all three conditions—rapid descent, falling to the ground and stopping, and significant tilting—are met simultaneously. This logical design ensures that a fall is only recognized when the human body has experienced the complete process of "rapid descent, falling to the ground and stopping, and significant tilting," effectively distinguishing a fall from everyday actions such as sitting down quickly, bending over to pick up an object, and lying down slowly.

[0112] Regarding immediate voice confirmation and tiered response after a fall, when the system determines that a fall has occurred, it immediately sends a confirmation inquiry to the user via the voice interaction unit. "Immediately" means initiating the voice interaction within a very short time after the fall detection algorithm outputs its result, in order to establish communication with the user as quickly as possible. For example, this delay is less than 500 milliseconds. The confirmation inquiry is used to confirm with the user whether they need help; for example, the voice prompt might be, "We have detected that you may have fallen. Do you need help?"

[0113] The system opens a response monitoring window of a preset duration, continuously listening for the user's voice feedback within the window. For example, this response window is set to 15 seconds. The 15-second time window provides the user with sufficient time to react and respond, while preventing the system from waiting for extended periods and delaying potential emergency response.

[0114] The system executes tiered emergency measures based on the voice feedback in the response window. If the user explicitly responds with "No help needed" or a similar negative statement in the response window, the system only records the relevant information for this fall event and resumes normal monitoring without triggering further emergency measures. If the user does not respond in the response window, or responds with an affirmative statement such as "Yes" or "Need help," the system executes the emergency contact procedure. This emergency contact procedure includes: playing reassuring statements to the user through the voice interaction module; sending a notification message to the preset emergency contact person through the remote communication module; and automatically dialing the emergency number if the user does not respond or family members do not confirm intervention within the specified time, along with the user's current location information and a summary of the most recent health data.

[0115] Regarding the linkage between fall detection results and multi-source fusion decision-making, the system inputs the confirmation results from the fall detection and closed-loop confirmation steps as supplementary information into the multi-source fusion and hierarchical decision-making steps. When the confirmation result indicates that the user has confirmed a fall or is unresponsive, the comprehensive risk assessment level can be directly set to the highest level, triggering a level three response, without waiting for cross-validation results from other modalities, thus ensuring timeliness in emergency scenarios. When the confirmation result indicates that the user denies a fall, this confirmation result participates in cross-validation as one modality of multi-source health data, without triggering an upgrade independently.

[0116] To facilitate understanding, examples of its application scenarios are provided: Regarding the application scenario of fall detection and emergency response for the elderly, Mr. Wang, an elderly man living alone, accidentally lost his balance and fell while moving around in the kitchen. The robot obtained the three-dimensional coordinates of 33 key skeletal points of Mr. Wang in real time through a posture estimation model. The system detected that the vertical drop of the hip center within 1 second exceeded 50% of the hip height when standing, the y-coordinate of the hip key point was lower than the y-coordinate of the knee key point for more than 3 seconds, and the body tilt angle exceeded 60°. When all three conditions were met simultaneously, a fall event was determined to have occurred. Within a delay of less than 500 milliseconds, the system issued a question through the voice interaction module: "Hello, I have detected that you may have fallen. Do you need help?" Within a 15-second response window, Mr. Wang was unable to respond in time due to limited mobility. The system determined that there was no response, continued to reassure him with voice, and initiated the emergency linkage procedure, automatically sending a notification message to Mr. Wang's children and dialing 120 for emergency medical assistance, along with Mr. Wang's current location information and a summary of his most recent health data. At the same time, the fall confirmation result was input into the multi-source fusion and hierarchical decision-making module, which directly set the comprehensive risk assessment level to critical and triggered a level three response. This application scenario demonstrates the technical effectiveness of this application in fall scenarios. 3D skeletal kinematics rule determination improves detection accuracy, voice confirmation eliminates false alarms, and automatic triggering of graded emergency linkage when there is no response achieves a complete safety closed loop from anomaly detection to anomaly confirmation to anomaly handling.

[0117] This embodiment constructs a complete closed loop from "anomaly detection" to "anomaly confirmation" and then to "anomaly handling." The combined judgment of three kinematic rules with clear geometric and physical meaning transforms fall detection from traditional methods that rely on a single acceleration threshold or data-driven classification lacking interpretable kinematic criteria, to objective quantitative judgment based on human kinematic characteristics. This approach offers strong interpretability and allows for easy adjustment of threshold parameters for each condition according to actual application scenarios. The timing design of the immediate voice inquiry and response window strikes a balance between "no false alarms" and "no missed alarms." The linkage mechanism with the multi-source fusion step allows fall confirmation results to bypass the conventional cross-validation process and directly trigger the highest-level response, ensuring timeliness in emergency scenarios.

[0118] Regarding the implementation details of how to perform real risk cross-validation on multi-source health data based on preset cross-validation rules in the S400 multi-source fusion and hierarchical decision-making process, and how to output a comprehensive risk assessment level and execute a multi-level response strategy, in some examples of this application, the outputs of each modal sub-network are fused at the decision level through evidence theory fusion rules, and combined with cross-validation rule tables and personalized baseline normalization processing, a comprehensive risk assessment level is finally output, and a matching multi-level response strategy is executed.

[0119] Specifically, the formula for personalized baseline normalization is as follows: ; in, Represents the normalized value, Characterized by the current detection value, It is represented as a personalized baseline value (the mean of the previous few test results). The standard deviation is represented as the baseline value.

[0120] It should be noted that multi-source fusion faces two core challenges. The first challenge is the fusion method for heterogeneous modalities. Data from different modalities, such as body temperature, facial expression, posture, and ECG, differ fundamentally in physical meaning, data form, and output type. Body temperature is a scalar value, facial expression is a high-dimensional embedding vector, posture is a temporal skeletal coordinate sequence, and ECG is a statistical indicator derived from feature extraction of a temporal signal. These heterogeneous outputs cannot be directly fused using traditional homogeneous data fusion methods such as numerical weighting or Kalman filtering. The second challenge is the reliability of single-indicator anomalies. Single-indicator anomalies may be caused by non-pathological factors. If an alarm is triggered solely by a single indicator, frequent false alarms are likely to occur in a home setting, reducing user trust. This embodiment addresses these challenges by employing evidence-based fusion rules at decision layer 102 to fuse the independent judgment results of each modality, adapting to the inconsistent output types of heterogeneous modalities. Simultaneously, a cross-modal verification mechanism is established through a cross-validation rule table, ensuring that a single-indicator anomaly requires verification by at least one related indicator from another modality before confirming the true risk, thereby reducing the false alarm rate of single-modal assessment.

[0121] Regarding the construction and output of each modal subnetwork, the system first performs personalized baseline normalization on the data for each modality. Personalized baseline values ​​for each health indicator are established for each user. The baseline value is taken as the statistical value of the corresponding indicator in the user's previous tests; for example, the mean of the first 10 test results is used. The normalized value is equal to the difference between the current value and the baseline value divided by the standard deviation of the baseline value. For new users with fewer than the preset number of tests, a minimum-maximum scaling method is used to map the current value of each indicator to the 0-1 range, with the minimum and maximum values ​​referencing the medical reference range for that age group. Through personalized baseline normalization, anomaly detection is based on the user's own historical fluctuation range, rather than a fixed threshold for the general population, thus improving the individualized accuracy of anomaly detection.

[0122] After normalization, the modal data are input into their respective subnetworks, and each subnetwork independently outputs the basic probability assignment of that modality to each category. The basic probability assignment is a function in evidence theory used to express the degree of support of a certain piece of evidence for each category. Its value is a real number between 0 and 1, and the sum of the basic probability assignments for each category is 1.

[0123] The facial expression sub-network employs a convolutional neural network. The input is a 128-dimensional expression feature vector extracted from the user's facial image, and the output is a probability distribution for four categories: normal, pain, fear, and anxiety. The backbone of this network uses a lightweight convolutional network; for example, MobileNetV3-Small is used as the backbone, followed by a fully connected classification layer. The output nodes of the fully connected classification layer are 4, corresponding to the four categories mentioned above. The training dataset uses publicly available facial expression recognition datasets, including, for example, AffectNet and FER2013 datasets, covering facial expression samples of different ages, genders, and lighting conditions. Each sample is labeled with the corresponding expression category by professionals. During training, the cross-entropy loss function is used, with the Adam optimizer employed. The initial learning rate is set to 0.001, the batch size to 64, and the training epochs to 50. The hidden layers of this network use the ReLU activation function, and the output layer uses the Softmax activation function. The validation set loss is monitored during training, and training stops when the validation set loss does not decrease for 5 consecutive epochs. After training, the model parameters are solidified into an inference model and deployed on the edge processor of the mobile desktop robot.

[0124] The pose sub-network employs a Long Short-Term Memory (LSTM) network. The input is a 99-dimensional feature vector (33 keypoints × 3D coordinates) extracted from the user's skeletal keypoint sequence, and the output is the probability distribution of three categories: normal, curled-up, and abnormal movements. The LSM network captures the temporal dependencies in the skeletal keypoint sequence through a gating mechanism, enabling it to distinguish between transient movement changes and persistent pose abnormalities. The training dataset is constructed by collecting skeletal keypoint sequences from different users in daily life and simulated abnormal states. Each sequence is labeled with the corresponding pose category by professionals. During training, the cross-entropy loss function is used, with the Adam optimizer, an initial learning rate of 0.001, a batch size of 32, and 80 training epochs. The hidden layers of the network use the tanh activation function, and the output layer uses the Softmax activation function. The validation set loss is monitored during training, and training stops when the validation set loss does not decrease for five consecutive epochs. After training, the model parameters are solidified into an inference model and deployed on the edge processor of a mobile desktop robot.

[0125] The ECG network employs a one-dimensional convolutional neural network (CNN). The input is a 20-dimensional heart rate variability feature vector (including SDNN, RMSSD, LF / HF, etc.) extracted from ECG signal segments. The output is the probability distribution of three categories: normal, arrhythmia, and myocardial ischemia. The CNN extracts local patterns and abnormal fluctuations in the heart rate variability features by sliding the convolutional kernel along the temporal feature dimension. The training dataset uses publicly available ECG databases, exemplarily including the MIT-BIH arrhythmia database and the PTB diagnostic ECG database. Each ECG signal segment is labeled with its corresponding diagnostic category by ECG professionals. During training, a cross-entropy loss function is used, with the Adam optimizer employed. The initial learning rate is set to 0.001, the batch size to 128, and the training epochs to 60. The hidden layers of the network use the ReLU activation function, and the output layer uses the Softmax activation function. The validation set loss is monitored during training, and training stops when the validation set loss does not decrease for five consecutive epochs. After training, the model parameters are solidified into an inference model and deployed on the edge processor of a mobile desktop robot.

[0126] Regarding the execution of the evidence theory fusion rule, the system combines the basic probability assignments of each modal subnetwork output using Dempster evidence theory. Let there be modality A and modality B, with basic probability assignment functions m1 and m2 respectively. For any hypothesis H, the Dempster combination rule is: ; in, K is called the normalization coefficient, used to handle conflicts between modal evidence. When the support of each modality for the same category tends to be consistent, the K value is close to 1, and the fusion result strengthens the support for that category; when there are conflicts between the conclusions of each modality, the K value is low, and the support of the fusion result for each category is relatively balanced, reflecting that the system is in an uncertain state. m(H) represents the basic probability allocation of hypothesis H after fusion, m1(H) i The mode A is characterized as a pair of hypotheses H. i The basic probability assignment, m2(H) j The mode is characterized as mode B against hypothesis H. j The basic probability allocation.

[0127] Specifically, the system follows a preset fusion order. First, it fuses the basic probability assignments of two modalities using the Dempster combination rule to obtain an intermediate fusion result. Then, it fuses the intermediate fusion result with the basic probability assignment of a third modality, and so on, until the basic probability assignments of all modalities are included in the fusion, ultimately obtaining the fused comprehensive anomaly probability. For example, the fusion order can be determined according to the detection priority of the modalities, such as fusing body temperature and facial expression first, then fusing with posture and electrocardiogram in sequence, but this application does not impose specific restrictions on the fusion order. The comprehensive anomaly probability refers to the overall support the system provides for the hypothesis that "the user is in an abnormal health state" after fusing all modal evidence. Its value ranges from 0 to 1, with a higher value indicating a greater likelihood that the user is in an abnormal health state.

[0128] Regarding the execution of cross-validation rules, when any single indicator is abnormal, the system does not immediately make an anomaly determination. Instead, it queries the preset cross-validation rule table to determine whether there are other modal related indicators that simultaneously meet the preset conditions.

[0129] For example, the rule table includes at least one of the following rules: First, when the body temperature is greater than 37.3℃ and the facial expression of pain is greater than 0.6, the overall abnormal probability is increased to 0.85. Second, when the body temperature is greater than 37.3℃ and the posture is curled up (trunk bending angle greater than 30°), the overall abnormal probability is increased to 0.80. Third, when the heart rate is greater than 100 beats / minute and the speech rate change rate is greater than 30% faster than the baseline, the overall abnormal probability is increased to 0.75. The baseline for the speech rate change rate refers to the statistical average of the user's speech rate (number of syllables pronounced per unit time) in the previous few tests, and the speech rate change rate = (current speech rate - baseline speech rate) / baseline speech rate × 100%. The medical logic behind this rule is that an increased heart rate and abnormally increased speech rate are typical physiological behavioral manifestations of anxiety. Fourth, when the heart rate is greater than 100 beats / minute and the facial expression of fear is greater than 0.5, the overall abnormal probability is increased to 0.80.

[0130] If only one modality is abnormal while other modalities are normal, the system reduces the weight of the overall anomaly probability to avoid false alarms caused by fluctuations in a single indicator. If at least two modalities are abnormal simultaneously and meet the mutual verification conditions in the cross-validation rule table, the system confirms multimodal anomaly, multiplies the overall anomaly probability by 1.5, and upgrades the response level.

[0131] Regarding the determination of the comprehensive risk assessment level and the execution of the multi-level response strategy, the system compares the integrated anomaly probability with a preset threshold to determine the comprehensive risk assessment level.

[0132] The comprehensive risk assessment level includes four discrete levels: normal, slightly abnormal, moderately abnormal, and critical. Preset thresholds include a first threshold, a second threshold, and a third threshold, used to divide the continuous comprehensive anomaly probability into four level intervals. For example, the comprehensive anomaly probability ranges from 0 to 1, with the first threshold set to 0.2, the second threshold to 0.5, and the third threshold to 0.75. When the comprehensive anomaly probability is between 0 and the first threshold, it is determined to be at the normal level; when the comprehensive anomaly probability exceeds the first threshold but does not exceed the second threshold, it is determined to be at the slightly abnormal level; when the comprehensive anomaly probability exceeds the second threshold but does not exceed the third threshold, it is determined to be at the moderately abnormal level; and when the comprehensive anomaly probability exceeds the third threshold, it is determined to be at the critical level.

[0133] Optionally, the system can also output a continuous risk score as an auxiliary expression of the overall anomaly probability. For example, the continuous risk score ranges from 0 to 100, and can be obtained by multiplying the overall anomaly probability by 100, allowing users to more intuitively understand the level of risk. However, the determination of the overall risk assessment level is based on a direct comparison between the overall anomaly probability and a preset threshold.

[0134] The system executes a multi-level response strategy based on the comprehensive risk assessment level. When the assessment level is normal, the system only records the test data to the user's health record without triggering any prompts or notifications. When the assessment level is slightly abnormal, the system issues a Level 1 response, including playing health reminders and suggestions through the voice interaction module, recording data, and displaying a brief health report on the screen. When the assessment level is moderately abnormal, the system issues a Level 2 response, including a voice reminder, initiating a retest procedure, and notifying the preset contact person of the test results. When the assessment level is critical, the system issues a Level 3 response, including immediately notifying the preset emergency contact person, automatically dialing the emergency number, continuously monitoring and uploading the user's location information to the cloud server until rescue arrives or the user manually deactivates the alarm.

[0135] To facilitate understanding, examples of its application scenarios are provided: Regarding the application scenario of multimodal health and emotion assessment, Ms. Chen, a white-collar worker, experienced increased stress while working. The robot continuously monitored her health status through a non-contact detection module. The system detected that Ms. Chen's heart rate increased to 105 beats / minute, the LF / HF ratio in the HRV index increased, her speech rate increased by 35% compared to the baseline, and facial expression features, analyzed by the expression sub-network, showed increased tension. Each modality sub-network independently outputs a basic probability allocation, and the system combines the basic probability allocations of each modality using evidence theory fusion rules. The system queries the cross-validation rule table: if the heart rate is greater than 100 beats / minute and the speech rate change rate is greater than 30% compared to the baseline, the probability of an anxiety state increases to 0.75; if the heart rate is greater than 100 beats / minute and the fear level in the facial expression is greater than 0.5, the probability of a panic state increases to 0.80. Since the heart rate and speech rate, and the heart rate and facial expression all meet the mutual verification conditions, the system confirms the multimodal abnormality and multiplies the overall abnormality probability by 1.5. The system outputs a comprehensive risk assessment level of moderate abnormality, triggering a level-two response: a proactive reminder via the voice interaction module stating, "We have detected that you are currently under high pressure. We suggest pausing your work for five minutes and practicing deep breathing," followed by a brief health report displayed on the screen. This application scenario demonstrates the technical effectiveness of this application in integrating health and emotional assessment. Multimodal data is comprehensively judged through evidence theory fusion and cross-validation rules, avoiding false alarms caused by single indicators directly triggering alarms, and achieving a comprehensive assessment of the user's physical and mental health status.

[0136] In this embodiment, the system refines the higher-level operations in S400—"cross-validation of real risks based on preset cross-validation rules" and "outputting a comprehensive risk assessment level and executing a multi-level response strategy"—into a complete technical solution encompassing the construction and training of each modal sub-network, evidence theory fusion, cross-validation rule execution, threshold comparison to determine the level, and execution of the multi-level response strategy. The disclosure of the structure, training data, and training process of the three sub-networks enables those skilled in the art to reproduce the construction and application of each modal sub-network. The adoption of evidence theory fusion rules solves the problem of inconsistent output types across different modalities, preventing direct data-level fusion. The basic probability allocation of each modal sub-network output can be combined within a unified evidence framework. Direct comparison of the comprehensive anomaly probability with preset thresholds provides a clear quantitative basis for level determination. The establishment of the cross-validation rule table transforms the medical diagnostic logic of "multi-sign cross-validation" into automatically executable quantitative rules, ensuring that a single indicator anomaly requires cross-modal cross-validation before confirming the real risk, significantly reducing the false alarm rate caused by a single indicator anomaly. The precise mapping between the four-level classification and multi-level response strategies enables the system to match appropriate handling measures based on the severity of anomalies. Personalized baseline normalization allows anomaly detection to be based on the user's own historical fluctuation range, improving the individualized accuracy of the judgment.

[0137] Regarding the execution architecture of the computational tasks in the method of this application, some examples of this application adopt a hierarchical computing architecture that combines real-time processing on the edge with in-depth analysis in the cloud. Specifically, the high-real-time-requirement primary screening task is executed on the edge processor of the mobile desktop robot, while the computationally intensive in-depth analysis task is executed on the cloud server. The two work together through preset latency constraints and de-identification mechanisms, balancing real-time interactive experience with in-depth analysis.

[0138] It should be noted that home mobile desktop robots are limited by size, cost, and power consumption, resulting in relatively limited computing power and storage space for edge processors. However, the various computing tasks involved in this application have significantly different resource requirements: First-level screening tasks such as facial key point detection and pose estimation in the active ranging and pose adaptation steps, and visual recognition in the wearing status pre-detection step, require end-to-end latency to meet real-time interaction needs, but their model size is relatively small and can run efficiently on edge processors; tasks such as parallel inference of multiple sub-networks, long-term health data trend analysis, and detailed health report generation involved in multimodal fusion evaluation have high requirements for computing resources. If all of these are executed on the edge, it may lead to increased response latency or reduced battery life. The design considerations of this embodiment are: to classify and process tasks according to their real-time requirements and computing resource needs, retaining real-time tasks that directly affect user experience on the edge to ensure smooth interaction during the detection process; and to offload non-real-time or computationally intensive tasks to the cloud, overcoming edge computing power limitations, while protecting user privacy by utilizing cloud computing resources through a de-identification mechanism.

[0139] Regarding real-time processing at the edge, the spatial relationship assessment task in the active ranging and posture adaptation steps is executed in real-time on the edge processor of the mobile desktop robot. This spatial relationship assessment task includes inference of the facial keypoint detection model, weighted ranging calculation based on keypoint depth values, and solving the PnP posture estimation algorithm. The visual recognition task in the wearing status pre-detection step is also executed in real-time on the edge processor, including cascaded inference of the target detection network and keypoint detection network corresponding to the blood pressure cuff, inference of the tightness classification model, and inference of the human posture keypoint model corresponding to the ECG electrode pads.

[0140] The inference latency of the edge processor meets preset real-time requirements to ensure a smooth interactive experience for users during health checks. For example, the edge inference latency is less than 100 milliseconds. This latency level meets the requirements of real-time interaction, meaning that users will not perceive any noticeable system processing latency during health checks, thus enabling the real-time execution of the active ranging and attitude adaptation steps and the wearing status pre-detection steps on the edge processor. This latency metric ensures that the end-to-end processing time from image acquisition by the camera to outputting spatial parameters or wearing status results is controlled within a user-perceptible latency range, preventing users from experiencing a sense of waiting due to system processing.

[0141] Regarding cloud-based deep analysis, the multi-source fusion and hierarchical decision-making steps are performed on the edge processor for preliminary judgment, calculating the output of each modal subnetwork, performing evidence theory fusion, matching cross-validation rules, and determining the comprehensive risk assessment level. When the preliminary judgment result is an anomaly of level two or above, the system uploads the de-identified multi-source health data to the cloud server via a remote communication module.

[0142] The de-identification process refers to removing data that can directly or indirectly identify a user's personal identity before uploading. For example, this involves removing the user's facial image, retaining only irreversible features such as facial expression vectors; replacing the user's identity identifier with an anonymous identifier; and performing voiceprint de-identification on the voice data. The de-identified data retains only the physiological and behavioral characteristics required for health monitoring and does not contain original biometric data traceable to a specific individual, thus protecting user privacy while enabling cloud-based analysis.

[0143] After receiving the anonymized data, the cloud server utilizes its richer computing resources to perform in-depth analysis tasks. For example, these tasks include: long-term health trend analysis, comparing the current test data with data from the user's historical health records to identify chronic evolution trends, such as long-term blood pressure fluctuation patterns and gradual changes in heart rate variability; population comparison analysis, comparing the user's anonymized data with anonymized health statistics of the same age and gender group to assess the user's relative health level within the population; and detailed health report generation, combining the current test results and long-term trend analysis to output a structured health report for the user or doctor's reference. After the cloud analysis is completed, the results are sent back to the mobile desktop robot to update the user's personalized baseline values ​​and health records.

[0144] Regarding latency constraints between the edge and cloud, the maximum permissible latency from the completion of active ranging and attitude adaptation steps to the output of level two or higher trigger decisions in multi-source fusion and hierarchical decision-making steps meets the preset timeliness requirements for emergency decision-making. For example, this maximum permissible latency is 500 milliseconds. This latency constraint covers the entire process time from the completion of level one screening data collection, inference of each modal sub-network, evidence theory fusion, cross-validation rule matching, level determination, to triggering level two or higher response decisions. The edge processor ensures that the above processing is completed within this latency constraint through optimized task scheduling strategies and model inference acceleration technologies.

[0145] In this embodiment, the system rationally allocates computational tasks between the edge and the cloud based on real-time requirements and resource needs. Real-time processing on the edge ensures that tasks directly impacting user experience, such as spatial adaptation, pre-wearing detection, and preliminary risk assessment, are completed within a user-perceptible latency range. Deep cloud analysis overcomes edge-side computing power limitations, enabling computationally intensive tasks such as long-term trend analysis and population comparisons, providing users with more comprehensive health management services. The anonymization mechanism fully utilizes cloud computing resources while protecting user privacy. Latency constraints on edge-to-cloud decision-making ensure that when moderate to severe anomalies are detected, the system can trigger notifications and responses within a preset timeframe, guaranteeing timely responses in emergency scenarios.

[0146] Regarding the specific implementation of the multi-turn conversational health consultation enhancement scheme that integrates a large language model in the method of this application, in some examples of this application, when the comprehensive anomaly probability calculated in the multi-source fusion and hierarchical decision-making steps is within a preset fuzzy threshold range, or when there are contradictions in the evaluation conclusions of each modality, the system initiates a multi-turn conversational consultation based on a large language model, extracts structured semantic feature vectors from user responses as new modalities to participate in joint reasoning, so as to correct the comprehensive anomaly probability and make more accurate decisions.

[0147] It should be noted that uncertainties in system cognition may arise during multi-source fusion. One scenario is that the overall anomaly probability falls within a fuzzy range, neither clearly normal nor clearly abnormal. In this case, directly determining normality may result in underreporting of potential risks, while directly determining anomaly may lead to false positives. Another scenario is that the evaluation conclusions of different modalities are contradictory, making it difficult for the system to determine which modality's conclusion to accept. Related solutions typically adopt preset fixed strategies for such fuzzy anomalies, such as "better to falsely report than to miss a report" or "wait for more data before making a judgment." The former causes unnecessary user panic, while the latter may delay the discovery of physiological anomalies. The design consideration of this embodiment is that when the system is in a state of cognitive uncertainty, it introduces a large language model to conduct multiple rounds of empathetic questioning to obtain users' subjective descriptions of their symptoms and transforms them into structured semantic features that can be used for joint reasoning with physiological modalities. This allows the system to enhance the accuracy of decision-making by utilizing user complaints in a fuzzy state.

[0148] Regarding the triggering conditions for fuzzy abnormal states, the system determines that it is currently in a fuzzy abnormal state and initiates a large language model diagnostic process when one of the following two conditions is met.

[0149] First, the overall anomaly probability falls within a preset fuzzy threshold range. For example, the fuzzy threshold range is between 40% and 60% for the overall anomaly probability. When the overall anomaly probability is below 40%, the system has a high confidence level in classifying it as normal; when the overall anomaly probability is above 60%, the system has a high confidence level in classifying it as abnormal; when the overall anomaly probability is between 40% and 60%, the system is in an uncertain state and requires additional information to assist in the judgment.

[0150] Second, the evaluation conclusions of the different modalities are contradictory. For example, if the difference between the judgment results of any two modalities exceeds a preset difference threshold, it is judged as contradictory. The preset difference threshold is 30 percentage points.

[0151] Regarding the execution of multi-turn conversational consultation, the system proactively initiates multi-turn dialogues in an empathetic consultation manner based on a large language model, through a voice interaction module and a natural language processing unit. The large language model refers to a language generation model pre-trained on a large-scale text corpus, capable of understanding user natural language input and generating context-appropriate natural language responses. For example, GPT series models, LLaMA series models, or other pre-trained large language models with similar capabilities can be used. It should be noted that different versions or specifications of large language models share common technical principles in their core capabilities for multi-turn dialogue understanding and natural language generation. The technical effectiveness of this application's solution mainly relies on the large language model's ability to understand and generate natural language, rather than the specific functions of a particular model. Those skilled in the art can select a large language model with an appropriate capability level to implement this solution based on the computing power and response latency requirements of the actual deployment environment.

[0152] The consultation dialogue begins with open-ended questions, gradually focusing on the user's specific symptoms. The multi-turn consultation strategy employs a symptom-oriented dialogue framework, with a large language model dynamically generating questions based on the dialogue context. The system pre-sets core information collection targets, including symptom description, onset time, triggering factors, accompanying symptoms, nature of pain, and radiation location. The large language model flexibly adjusts the question order and expression during the dialogue to achieve the information collection targets through natural and fluent conversation. The number of dialogue rounds can be dynamically determined based on the completeness of information collection; for example, 3 to 5 rounds are used.

[0153] Regarding the extraction and joint inference of semantic feature vectors, the large language model extracts structured information from user responses after completing multiple rounds of dialogue. The extracted content includes: keywords describing symptoms and their semantic categories; temporal characteristics of symptoms; factors that trigger and alleviate symptoms; and feature descriptions with clinical diagnostic significance.

[0154] The system encodes the aforementioned structured information into semantic feature vectors. These semantic feature vectors transform unstructured natural language consultation information into fixed-dimensional numerical vectors that can be processed by machine learning models. For example, a pre-trained medical text encoder can be used to map the extracted structured information into semantic feature vectors. This medical text encoder can employ pre-trained clinical text encoding models such as ClinicalBERT or BioBERT to encode each text field in the structured information into a fixed-dimensional vector representation. These vectors are then concatenated to form the final semantic feature vector. For context processing in multi-turn dialogues, the current user response can be concatenated with the system queries and user responses from previous turns to form a complete context text, which is then input into the medical text encoder for encoding, preserving the coherent semantic information of the dialogue. The semantic feature vector can be set to 64 dimensions. This dimension effectively encodes consultation-extracted content such as symptom descriptions, temporal features, and clinical differential information, while maintaining comparability in dimensionality with other modalities such as facial expression feature vectors (128 dimensions) and speech feature vectors (64 dimensions), facilitating unified processing within the framework of evidence theory fusion.

[0155] The system inputs semantic feature vectors as a new modality into the evidence theory fusion model. Let the original multimodal fusion result be the comprehensive anomaly probability P_original. The semantic feature vectors, after being processed by an independent consultation assessment sub-network, output basic probability assignments m_text. The consultation assessment sub-network is used to transform the semantic feature vectors into basic probability assignments within the evidence theory framework. For example, the consultation assessment sub-network can employ a multilayer perceptron, taking the semantic feature vectors as input and outputting basic probability assignments for four categories: normal, mildly abnormal, moderately abnormal, and critical. The training data for this sub-network is labeled by medical professionals based on consultation texts and corresponding clinical diagnoses. During training, the cross-entropy loss function is used, with the Adam optimizer employed. The initial learning rate is set to 0.001, the batch size to 32, and the training epochs to 30. The hidden layers of this network use the ReLU activation function, and the output layer uses the Softmax activation function. During training, the validation set loss is monitored, and training stops when the validation set loss does not decrease for five consecutive epochs. After training, the model parameters are solidified into an inference model and deployed on the edge processor of a mobile desktop robot.

[0156] The system fuses m_text with the original basic probability assignments for each modality using the Dempster combination rule to obtain the revised comprehensive abnormal probability P_revised. Since the semantic feature vector carries key clinical information from the user's complaints, the fused result reflects the user's true health status better than the result relying solely on the physiological modality.

[0157] Regarding the revised decision escalation, when the revised overall anomaly probability exceeds a preset escalation threshold, the system directly triggers a higher-level response strategy. For example, the escalation threshold is set to 75%. If the revised overall anomaly probability remains at a low level, the system determines the overall risk assessment level as normal or slightly anomaly and executes the corresponding conventional response strategy without escalation.

[0158] In this embodiment, the system adds a cognitive layer to enhance decision-making through a large language model, building upon multi-source fusion decision-making. The setting of two triggering conditions—fuzzy threshold range and modal contradiction—ensures that the large language model is invoked only when the system's cognition is uncertain, balancing computational efficiency and decision accuracy. Multi-round empathic consultation upgrades the traditional one-way assessment mode in health testing, which relies solely on objective physiological signals, to an enhanced decision-making mode that fuses objective signals and subjective complaints. The introduction of semantic feature vectors allows subjective symptom information, previously only obtainable by doctors through consultation, to participate in mathematical reasoning through evidence theory fusion in a structured form. The public disclosure of the structure, training data, and training process of the consultation assessment sub-network enables those skilled in the art to reproduce the construction and application of this sub-network, achieving joint reasoning of natural language information and physiological sensor information within a unified evidence framework.

[0159] In the foregoing embodiments of this application, the technical solutions defined in this application have been described in detail. The following is a further supplementary explanation of the technical content described in this application specification that can be used as preferred embodiments or optional implementations, as follows: Regarding the preferred parameter configuration of the graded detection parameter table, in a preferred embodiment of this application, the preset parameter configuration table in the S100 active ranging and attitude adaptation step sets differentiated optimal spatial conditions for different detection task types. Specifically, the parameter configuration table includes at least one of the following configuration items.

[0160] The infrared body temperature measurement is configured with a first distance range of 30-50cm, a first pitch angle allowable range of ±10°, a first horizontal offset allowable range of ±5cm, and a forehead emissivity compensation coefficient of 0.98. This configuration takes into account the sensitivity of the infrared temperature measurement module to the forehead angle, and the small pitch angle tolerance ensures that the forehead area is within the optimal field of view of the temperature measurement module. The emissivity compensation coefficient is used to correct for infrared radiation measurement deviations caused by skin surface characteristics.

[0161] The facial expression recognition system has a second distance range of 50-100cm, a second pitch angle allowable range of ±15°, and a second horizontal offset allowable range of ±10cm, requiring complete capture of the user's facial area. This configuration provides users with a relatively large degree of spatial freedom while ensuring clear and recognizable facial images, reducing user discomfort caused by excessively close proximity.

[0162] The pose recognition requires a third distance range of 100-150cm, a third pitch angle allowable range of ±20°, and a third horizontal offset allowable range of ±20cm, while also requiring the user's entire skeleton to be visible. With this configuration, the robot moves back to a greater distance to obtain a field of view encompassing the user's entire body, thus meeting the pose estimation requirement of a complete human skeleton as input.

[0163] The fourth distance range for blood pressure measurement is 20-30cm, the fourth pitch angle is within ±10°, and the fourth horizontal offset is within ±5cm, with the cuff height difference from the heart height less than 3cm. During blood pressure measurement, the robot is positioned close to the user, facilitating close-range image acquisition of the cuff wearing by the visual recognition module, and providing favorable shooting conditions for pre-detection of the cuff's wearing status.

[0164] For ECG measurements, the fifth distance range is 15-25cm, the fifth pitch angle tolerance is ±8°, and the fifth horizontal offset tolerance is ±3cm, with the electrode attachment area located in the center of the field of view. ECG measurements require precise electrode placement on the chest region; therefore, the strictest horizontal offset tolerance is employed.

[0165] The spatial parameter conditions corresponding to each of the above detection task types can be adjusted individually according to the actual home environment and user situation, and this application does not impose specific restrictions on this.

[0166] Regarding the preferred threshold for the acceptable range of blood pressure cuff wearing, in a preferred embodiment of this application, the preset acceptable range mentioned in the blood pressure cuff wearing status pre-inspection step includes the following quantitative standards.

[0167] Regarding positional deviation, the Euclidean distance between the cuff center and the upper arm center should be less than a first threshold. For example, the first threshold is 2 cm. This threshold is set based on the typical width of the upper arm, which is approximately 8-12 cm in the middle of an adult's upper arm. A 2 cm offset is within an acceptable range relative to the upper arm width. If the offset exceeds this amount, the cuff bladder may deviate from the position of the brachial artery, affecting the accuracy of pressure transmission.

[0168] Regarding directional deviation, the angle between the cuff's longitudinal axis and the upper arm's longitudinal axis should be less than a second threshold. For example, the second threshold is 10°. The cuff's longitudinal axis should extend along the upper arm's longitudinal axis; an excessively large angle will result in uneven pressure distribution during cuff inflation, causing the airbag's compression direction on the brachial artery to deviate from the optimal angle.

[0169] Regarding cardiac height deviation, the vertical height difference between the cuff center and the estimated heart position should be less than a third threshold. For example, the third threshold is set to 3 cm. Inconsistency between the cuff and heart height will cause hydrostatic pressure error; for every 1 cm the cuff is higher or lower than the heart, the blood pressure measurement will be approximately 0.7 mmHg lower or higher. A 3 cm threshold controls the hydrostatic pressure error to within approximately 2 mmHg, having a minimal impact on clinical judgment.

[0170] Regarding tightness, the standard for acceptance is the moderate level as indicated by medical professionals.

[0171] Regarding the preferred scheme for repeated verification of ECG electrode pads, in one preferred embodiment of this application, the ECG electrode pad wearing status pre-detection step further includes a repeated verification mechanism.

[0172] Specifically, after triggering the wearing reminder, the user adjusts the electrode pads, and the system re-executes the wearing status pre-check steps, including visual pre-check and signal quality assessment. If the re-acquired signal quality is still unsatisfactory, the system outputs another wearing reminder and allows the user to continue adjusting. The maximum number of repeated verification attempts is a preset number; for example, the maximum number of attempts is set to 3, and the interval between two adjacent verifications is set to 10 seconds.

[0173] If the signal quality assessment fails for a preset number of consecutive times, the system outputs a prompt message, such as "Electrode placement failed the verification. Please check if the pads are dry and re-attach them, or contact a professional for assistance," via the voice interaction module. The prompt message is also displayed on the screen. This mechanism prevents users from getting stuck in an endless loop of repeated attempts. When objective conditions cause the verification to fail, it promptly guides the user to replace the electrode pads or seek professional assistance, preventing invalid data from entering subsequent evaluation processes.

[0174] Regarding the preferred embodiment of the fall detection and closed-loop confirmation steps, the fall detection and closed-loop confirmation steps further include the following specific measures.

[0175] When a user explicitly responds within the preset response window that they do not need help, the system only records relevant information about the fall event, including the time of the fall, the movement trajectory data of key skeletal points before and after the fall, the user's voice confirmation record, etc., and then resumes normal monitoring.

[0176] When the user does not receive a voice response within the preset response window, or receives a voice response indicating that the user needs help, the system executes the emergency contact procedure. This procedure includes: first, playing reassuring statements to the user through the voice interaction module to stabilize their emotions; second, sending a notification message to the preset emergency contact through the remote communication module, containing a brief description of the fall, the user's current home location, and a summary of their most recent health monitoring data; and finally, if the user does not respond or family members do not confirm intervention within a specified time, automatically dialing emergency services with the user's location information and a summary of their health data.

[0177] Regarding the preferred configuration of multimodal feature dimensions and temporal alignment windows, in a preferred embodiment of this application, the specific configuration of feature dimensions and temporal alignment windows for each modality in the contact data acquisition and temporal alignment steps is as follows.

[0178] The modal feature dimensions of the first type of perception data are as follows: body temperature data is a 1-dimensional feature with a sampling frequency of 1Hz; facial expression data is a 128-dimensional feature vector extracted from facial images by a pre-trained embedding model with a sampling frequency of 10Hz; pose data is a 99-dimensional feature vector consisting of the three-dimensional coordinates of 33 skeletal key points with a sampling frequency of 15Hz; and speech data is a 64-dimensional feature vector containing speech features such as speech rate, mean fundamental frequency F0, and Mel spectrum with a sampling frequency of 10Hz.

[0179] The modal feature dimensions of the second type of measurement data are as follows: blood pressure data is a 2-dimensional feature, which is a single measurement value; electrocardiogram data is a 20-dimensional feature vector, which is composed of heart rate variability features obtained by aggregating 10-second segments of 500Hz electrocardiogram time series signal, including indicators such as SDNN, RMSSD, LF / HF, etc.

[0180] For temporal alignment, the completion time of the S100 active ranging and attitude adaptation steps is taken as the temporal origin T0. The average value of the feature vectors of the first type of sensing data within a 30-second window before and after T0 is used as the fusion reference feature. The second type of measurement data is collected approximately 60 seconds after triggering and is included in the temporal sequence as the inserted feature frame at time T0+60s. The specific values ​​of the above feature dimensions and temporal windows can be adjusted according to the actual application scenario and device performance, and this application does not impose specific restrictions on them.

[0181] Regarding the optional implementation of personalized baseline normalization, in one optional implementation of this application, the multi-source fusion and hierarchical decision-making steps further include personalized baseline normalization processing of multi-source health data.

[0182] Specifically, a personalized baseline value is established for each user's various health indicators. This baseline value is taken as the statistical value of the corresponding indicator in the user's previous tests; for example, the mean of the first 10 test results is used. After obtaining the current test data, normalization is performed using a relative deviation representation relative to the personalized baseline. That is, the normalized value is equal to the difference between the current value and the baseline value divided by the standard deviation of the baseline value. For new users who have not completed a preset number of tests, a minimum-maximum scaling method is used to map the current value of each indicator to the 0-1 range. The minimum and maximum values ​​used refer to the medical reference range for that age group. When the user's number of tests reaches the preset number, the system automatically switches to the relative deviation representation based on the personalized baseline.

[0183] Through the above personalized baseline normalization process, anomaly detection is based on the user's own historical fluctuation range, rather than a fixed threshold for the general population, which improves the individualized accuracy of anomaly detection.

[0184] Regarding the supplementary explanation of the training dataset for the blood pressure cuff recognition model, the medical labeling standards for tightness levels are as follows: Moderate corresponds to the state where "two fingers can be just inserted under the cuff"; Loose corresponds to the state where "more than two fingers can be easily inserted under the cuff"; Tight corresponds to the state where "it is difficult to insert two fingers under the cuff". Labeling by medical professionals ensures the clinical rationality of the tightness classification, making the classification boundaries learned by the model consistent with clinical standards.

[0185] Regarding the supplementary explanation of the triggering conditions for large language model-based health consultation, the preset fuzzy threshold range and the criteria for determining the contradiction between modal conclusions have been described in detail in the specific implementation of the multi-turn conversational health consultation that integrates large language models, and will not be repeated here.

[0186] In a preferred embodiment of this application, the multi-source fusion and hierarchical decision-making steps described in the above embodiments can be combined with a multi-turn conversational health consultation scheme that integrates a large language model to further enhance the system's decision-making ability under fuzzy and abnormal states. The specific implementation of this improved embodiment has been described in detail above and will not be repeated here. This improved embodiment can be implemented alone or in combination with any of the foregoing embodiments.

[0187] Regarding the specific hardware implementation of the mobile desktop robot provided in this application, in some examples of this application, the robot includes a mobile platform, a visual recognition module, a non-contact detection module, a voice interaction module, a communication interface for communicating with a contact detection accessory, and a control processing module. The connection relationships and collaborative working methods between the modules are as follows.

[0188] The mobile platform drives the robot to adjust its spatial position relative to the user based on motion control commands generated by the control processing module. The platform uses a differential drive chassis, equipped with two independent drive wheels and two omnidirectional support wheels, and features a LiDAR, inertial measurement unit, and wheeled odometer, supporting smooth movement, autonomous positioning, and target tracking in a home environment. Motion control employs a dynamic window method for real-time path planning.

[0189] The visual recognition module is used to acquire 2D images and 3D depth information of the user, supporting facial key point detection, expression recognition, pose estimation, and fall detection functions. It includes a 2D image acquisition component and a depth image acquisition component, both mounted on a controllable gimbal. Based on pose adjustment commands generated by the control processing module, the gimbal can be driven to adjust the shooting angle and field of view to adapt to changes in the user's position in different scenarios.

[0190] The non-contact detection module is used to collect users' body temperature data in a non-contact manner. It includes an infrared thermometry module that measures the infrared radiation intensity over the user's forehead area and calculates the body surface temperature after emissivity compensation. The non-contact detection module works in conjunction with a visual recognition module. The visual recognition module provides spatial location information of the user's face, and the non-contact detection module uses this location information to align with the user's forehead area to collect the temperature.

[0191] The voice interaction module is used to perform voice prompts, abnormal inquiries, and remote notifications. It includes a microphone array and speaker, and supports voice wake-up and voice command recognition.

[0192] A communication interface for communicating with contact-based diagnostic accessories, supporting wireless communication connections with external contact-based diagnostic accessories via Bluetooth or Bluetooth Low Energy protocols, receiving measurement data and issuing control commands. Connectable contact-based diagnostic accessories include blood pressure cuffs, pulse oximeters, and ECG electrodes.

[0193] The control processing module is electrically connected to the mobile platform, visual recognition module, non-contact detection module, voice interaction module, and communication interface. The control processing module includes an edge processor and a memory, with the edge processor supporting real-time inference of the neural network model. The control processing module is configured to execute the steps in any of the above-described method embodiments of this application, including active ranging and posture adaptation, wearing status pre-detection, contact data acquisition and temporal alignment, multi-source fusion and hierarchical decision-making, fall detection and loop closure confirmation, edge-cloud collaborative computing, and large language model-enhanced decision-making.

[0194] The modules exchange data loosely using a topic-based communication mechanism within the robot operating system. Image data is transmitted in a compressed format, and ECG signals are transmitted using a custom message type, ensuring efficient and decoupled data transmission between modules.

[0195] Through the organic combination of the above hardware modules, the mobile desktop robot provided in this application can transform the traditional passive health detection mode of "user adapting to device" into an active health service mode of "device actively adapting to user", providing users with full-process health management from daily screening to emergency intervention in the home setting.

[0196] Through the description of the above embodiments, the mobile desktop robot-assisted health detection method and system provided in this application, centered on the technical main line of "condition perception, quality control, anomaly capture, and fusion decision-making," constructs a systematic closed loop for home health detection. The active ranging and posture adaptation steps enable the robot to adaptively adjust to the optimal spatial conditions according to different detection task types, transforming the traditional "user adapts to device" into "device actively adapts to user." The wearing status pre-detection step actively identifies the wearing specifications of contact detection accessories before contact physiological parameter collection, transforming "detecting anomalies after measurement" into "ensuring conditions before measurement." The fall detection and closed-loop confirmation steps, based on kinematic rules and combined with real-time voice inquiry and hierarchical response, achieve a safe closed loop from anomaly detection to anomaly confirmation and then to anomaly handling. The multi-source fusion and hierarchical decision-making steps, based on temporal alignment, with cross-validation rules as the core, and evidence theory fusion as a means, comprehensively perform real-world risk cross-validation using multimodal data, significantly reducing the false alarm rate of single-modal assessment. The edge-side and cloud-side collaborative computing architecture balances real-time interactive experience with in-depth analysis needs. Through a unified timeline benchmark, evidence fusion framework, and hierarchical response mapping, the various technical units have formed a collaborative technical system that systematically improves the accuracy, reliability, and user experience of home health monitoring.

[0197] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements any of the method steps described in the above method embodiments.

[0198] This application also provides a computer program product, including computer program instructions, which, when executed by a processor, implement any of the method steps described in the above method embodiments.

[0199] In some embodiments, such as Figure 3 As shown, combined with Figure 3 As shown, this disclosure also provides an electronic device, including a processor 300 and a memory 301. Optionally, the device may further include a communication interface 302 and a bus 303. The processor 300, communication interface 302, and memory 301 can communicate with each other via the bus 303. The communication interface 302 can be used for information transmission. The processor 300 can call logical instructions in the memory 301 to execute the content generation method of the above embodiments.

[0200] Furthermore, the logic instructions in the aforementioned memory 301 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0201] The memory 301, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this disclosure. The processor 300 executes functional applications and data processing by running the program instructions / modules stored in the memory 301, that is, it implements the content generation method in the above embodiments.

[0202] The memory 301 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 301 may include high-speed random access memory and may also include non-volatile memory.

[0203] The electronic device may be a mobile desktop robot, a cloud server, or a system consisting of a mobile desktop robot and a cloud server.

[0204] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for assisted health detection of a mobile desktop robot, characterized in that, include: Active ranging and attitude adaptation steps: Based on the type of the first detection task to be executed, retrieve the target space parameter conditions corresponding to the first detection task type from the preset parameter configuration table; drive the mobile desktop robot relative to the user's measured space parameters until the measured space parameters meet the target space parameter conditions; Wearing status pre-detection steps: Before initiating contact-based physiological parameter acquisition, acquire wearing perception data of the user's target body area; Spatial deviation information of the contact detection accessory relative to the standard wearing state is extracted from the wearing perception data; if the spatial deviation information exceeds the preset qualified range, a wearing guidance command is output, and the collection of contact physiological parameters is postponed until the spatial deviation information meets the preset qualified range; Contact-based data acquisition and time-series alignment steps: Acquire multi-source health data, which includes a first type of sensory data and a second type of measurement data; wherein, the first type of sensory data is user perception information collected during and / or after the active ranging and posture adaptation steps, and the second type of measurement data is contact-based physiological parameters collected after passing the wear status pre-detection step; using the completion time of the active ranging and posture adaptation steps as the time-series benchmark, extract and fuse benchmark features from the first type of sensory data within the time window associated with the time-series benchmark, and use the second type of measurement data as the inserted feature frame of the time-series benchmark for time-series alignment. Multi-source fusion and hierarchical decision-making steps: Based on preset cross-validation rules, perform real risk cross-validation on individual abnormal data in the time-aligned multi-source health data, and output a comprehensive risk assessment level; according to the comprehensive risk assessment level, execute a multi-level response strategy that matches the comprehensive risk assessment level.

2. The method according to claim 1, characterized in that, In the active ranging and attitude adaptation step, the mobile desktop robot is driven relative to the user's measured spatial parameters until the measured spatial parameters meet the target spatial parameter conditions, including: Acquire depth images of the user's face and extract depth values ​​of the center point of the forehead and the tip of the nose based on a facial key point detection model; The measured distance in the measured spatial parameters is obtained by weighted averaging the depth values ​​of the center point of the forehead and the tip of the nose; and... Based on the two-dimensional coordinates of multiple facial key points, a three-dimensional facial plane is fitted using a pose estimation algorithm. The angle between the normal vector of the three-dimensional facial plane and the optical axis of the image acquisition device is calculated to obtain the measured pose in the measured spatial parameters, which include pitch angle, yaw angle, and roll angle.

3. The method according to claim 1, characterized in that, In the pre-detection step of the wearing status, wearing perception data of the user's target area is acquired, and spatial deviation information of the contact detection accessory relative to the standard wearing status is extracted from the wearing perception data, including: The upper arm region and blood pressure cuff region in the target area of ​​the body are detected using a first target detection network, and multiple corner points of the blood pressure cuff and wrist reference points are extracted using a first key point detection network. The spatial deviation information is calculated as follows: the cuff direction angle, the Euclidean distance between the cuff center and the upper arm center, the angle between the cuff longitudinal axis and the upper arm longitudinal axis, the vertical height difference between the cuff center and the estimated heart position, and the cuff tightness level.

4. The method according to claim 1, characterized in that, In the pre-detection step of the wearing status, wearing perception data of the user's target area is acquired, and spatial deviation information of the contact detection accessory relative to the standard wearing status is extracted from the wearing perception data, including: The standard lead positions of the ECG electrode pads are calculated using a human posture key point model, and the actual position of the current electrode pads is obtained. The positional deviation between the actual position and the standard lead position is calculated as the spatial deviation information.

5. The method according to claim 4, characterized in that, The wearing status pre-detection step also includes: Once the spatial deviation information meets the preset acceptable range, ECG signal acquisition is initiated, and the signal quality of the acquired initial ECG signal is evaluated. The signal quality assessment includes signal-to-noise ratio calculation, baseline drift detection, and R-wave detection rate calculation. If any indicator in the signal quality assessment fails to meet the preset signal quality standard, the wearing status is determined to be unqualified, and a wearing reminder is triggered.

6. The method according to claim 1, characterized in that, Also includes: Fall detection and closed-loop confirmation steps: Obtain the user's 3D skeletal key point sequence, and determine whether a fall event has occurred based on preset kinematic rules; When a fall is detected, the mobile desktop robot sends a confirmation question to the user through its voice interaction unit, and executes tiered emergency measures based on the voice feedback received within the preset response window. Preferably, the determination of whether a fall event has occurred based on preset kinematic rules includes: A fall event is determined to have occurred when all three of the following conditions are met: The vertical descent height of the hip center key point in the preset first time exceeds the preset proportion of the hip height when standing; The vertical coordinates of the hip key point are lower than the vertical coordinates of the knee key point, and the duration exceeds the preset second time. The body tilt angle exceeds the preset angle threshold; More preferably, the confirmation inquiry is initiated immediately after a fall event is determined to have occurred, and the preset response window is a listening window of preset duration; Preferably, the multi-source fusion and hierarchical decision-making step further includes: Receive the fall confirmation result output from the fall detection and closed-loop confirmation steps, and use the fall confirmation result as a modality in the multi-source health data to participate in the cross-validation, or when the fall confirmation result indicates that the user has confirmed a fall or is unresponsive, directly set the comprehensive risk assessment level to the highest level.

7. The method according to claim 1, characterized in that, In the multi-source fusion and hierarchical decision-making step, based on preset cross-validation rules, real risk cross-validation is performed on individual abnormal data in the time-aligned multi-source health data, including: Using evidence theory fusion rules, the basic probability assignments of the sub-network outputs corresponding to at least two modal data in the multi-source health data are combined to obtain the comprehensive anomaly probability. The overall anomaly probability is compared with a preset threshold to determine the overall risk assessment level; Preferably, the preset cross-validation rules include at least one of the following rules: When the body temperature is abnormal and the facial expression of pain exceeds the first expression threshold, the overall abnormality probability is increased; When the heart rate is abnormal and the speech features deviate from the baseline by more than the first speech threshold, the overall abnormality probability is increased; More preferably, the comprehensive risk assessment level includes four discrete levels: normal, slightly abnormal, moderately abnormal, and critical. The multi-level response strategy includes: When the overall risk assessment level is normal, only data is recorded; When the overall risk assessment level is mildly abnormal, a voice reminder and health advice will be issued; When the overall risk assessment level is moderately abnormal, the retesting procedure is initiated and the designated contact person is notified; When the overall risk assessment level is critical, it will automatically dial emergency services and share the current location and health data summary.

8. The method according to claim 1, characterized in that, The active ranging and attitude adaptation steps and the wearing status pre-detection steps are executed in real time on the edge processor of the mobile desktop robot. The multi-source fusion and hierarchical decision-making steps include: performing a preliminary judgment on the edge processor; when the preliminary judgment result is an abnormality of level two or above, uploading the desensitized multi-source health data to the cloud server for in-depth analysis and backup. The maximum allowable delay from the completion of the active ranging and attitude adaptation step to the output of the multi-source fusion and hierarchical decision-making step for triggering decisions at level two or above meets the preset emergency decision-making timeliness requirements.

9. The method according to claim 7, characterized in that, The multi-source fusion and hierarchical decision-making steps also include: When the overall anomaly probability is within a preset fuzzy threshold range, or when there are contradictions in the evaluation conclusions of each modality in the multi-source health data, a multi-turn conversational consultation based on a large language model is initiated. Structured semantic feature vectors are extracted from the user responses of the multi-round conversational consultation, and the semantic feature vectors are used as new modal inputs into the evidence theory fusion rules to participate in joint reasoning to correct the comprehensive anomaly probability; When the corrected overall anomaly probability exceeds the preset upgrade threshold, a higher-level response strategy is triggered directly.

10. An assisted health monitoring system for a mobile desktop robot, characterized in that, include: The active ranging and attitude adaptation module is used to call the target space parameter conditions corresponding to the first detection task type from the preset parameter configuration table according to the first detection task type to be executed, and drive the measured space parameters of the mobile desktop robot relative to the user until the measured space parameters meet the target space parameter conditions. The wearing status pre-detection module is used to acquire wearing perception data of the user's target area before starting contact physiological parameter acquisition, extract spatial deviation information of the contact detection accessory relative to the standard wearing status from the wearing perception data, and output wearing guidance instructions and postpone the acquisition of contact physiological parameters when the spatial deviation information exceeds the preset qualified range. The contact-type data acquisition and timing alignment module is used to acquire multi-source health data containing first-type sensing data and second-type measurement data, and uses the completion time of the active ranging and attitude adaptation steps as the timing reference to extract and fuse reference features from the first-type sensing data and use the second-type measurement data as an insertion feature frame for timing alignment. The multi-source fusion and hierarchical decision-making module is used to perform real risk cross-validation on individual abnormal data in the time-aligned multi-source health data based on preset cross-validation rules, output a comprehensive risk assessment level, and execute a matching multi-level response strategy according to the comprehensive risk assessment level.

Images