A health and rehabilitation robot remote control method and system based on artificial intelligence

Abstract

The application belongs to the technical field of robot remote control, and discloses a kind of health-care robot remote control method and system based on artificial intelligence, the method includes, through visual sensor, obtain user video data stream, extract body center of gravity change, joint angle change and micro-motion time sequence characteristics such as limb movement trend from it;Characteristics are input into pre-trained behavior precursor prediction model, predict user potential behavior intention and confidence;Receive and analyze remote operation instruction, establish system prediction queue and remote operation queue;Combined with confidence, risk level, network delay, action execution time window and instruction urgency, the action to be executed in two queues is dynamically weighted and cross-sequenced, and the final action execution sequence is generated;Control robot to execute auxiliary action.The application solves the problem that the behavior change of the old people does not match the response of remote control, improves the safety and efficiency of remote control, and is suitable for remote control of robots in health-care scenarios.

Description

A remote control method and system for elderly care robots based on artificial intelligence Technical Field

[0001] This invention relates to the field of robot remote control technology, and more specifically, to a remote control method and system for a health and wellness robot based on artificial intelligence. Background Technology

[0002] Against the backdrop of an aging population, remote control technology for elderly care robots has become an important means of addressing the shortage of elderly care resources. Its core requirement is to provide safe assistance and posture care for elderly users through remote operation. However, existing remote control solutions for elderly care robots face significant technical bottlenecks in practical applications, struggling to adapt to the behavioral characteristics of elderly users and the inherent limitations of remote control links.

[0003] While elderly users move slowly, their behavior is highly unpredictable, often exhibiting sudden changes in posture such as turning around abruptly, standing up abruptly, or switching hands without warning. These subtle movements require extremely high levels of real-time safety assistance. However, during remote control, issues such as network latency, reduced video frame rate, operator lag, and robot execution delays are common, causing the robot's action response to often lag behind changes in user behavior, creating a safety hazard where the speed of behavior change exceeds the speed of control response.

[0004] Existing technologies mostly focus on action recognition, signal-level delay compensation, or simple safety shutdown. They can only passively identify actions that have already occurred and cannot predict the user's intentions in the next instant. Delay compensation only optimizes data transmission and lacks an understanding of human behavior. Emergency shutdown strategies not only fail to provide effective safety assistance but also easily lead to collaboration failures. Summary of the Invention

[0005] In response, the present invention provides a remote control method and system for health and wellness robots based on artificial intelligence, so as to at least partially solve the above-mentioned technical problems.

[0006] This invention provides a remote control method for a health and wellness robot based on artificial intelligence, comprising the following steps:

[0007] S1 continuously acquires the user's video data stream through a visual sensor;

[0008] S2, extract micro-motion temporal features representing changes in user posture from the video data stream in real time. The micro-motion temporal features include at least changes in body center of gravity, changes in joint angles, and limb movement trends.

[0009] S3, input the micro-action timing features into the pre-trained behavioral precursor prediction model to predict at least one potential behavioral intention of the user in the future and its corresponding confidence level.

[0010] S4 receives a sequence of control commands from a remote control terminal and parses them into executable operation commands for the robot.

[0011] S5, establish a system prediction queue and a remote operation queue. The system prediction queue generates corresponding robot preparatory auxiliary actions based on the potential behavioral intentions and confidence levels. The remote operation queue is used to cache the operation instructions.

[0012] S6. Based on the confidence level of the potential behavioral intention, the preset risk level, the current network latency, the action execution time window, and the urgency of the operation instruction, the actions to be executed in the system prediction queue and the remote operation queue are dynamically weighted and cross-sorted to generate the final action execution sequence.

[0013] S7. According to the final action execution sequence, control the health care robot to perform corresponding auxiliary actions.

[0014] Preferably, the behavioral precursor prediction model is a model trained based on a temporal fusion Transformer architecture, which is used to perform multi-step prediction of the multivariate micro-action temporal features and output the confidence interval or uncertainty measure of the potential behavioral intention.

[0015] Preferably, dynamic weight evaluation and cross-sorting are performed on the actions to be executed in the system prediction queue and the remote operation queue. Specifically, this includes: calculating a priority score for each action to be executed in the queue, wherein the priority score is a weighted function of the confidence level, the risk level, the urgency, the action execution time window, and a delay penalty factor based on the current network latency; sorting all candidate actions in descending order according to the priority score; detecting conflicts between the sorted actions in space, time, or robot degrees of freedom; and for actions with conflicts, resolving the conflict based on the difference in their priority scores, and executing the action with the higher score.

[0016] Preferably, during the conflict resolution process, if the difference in priority scores between actions in the system prediction queue and actions in the remote operation queue is less than a preset lag threshold, then the actions in the system prediction queue are executed first, or a confirmation request is sent to the remote operation terminal.

[0017] Preferably, the system monitors the network round-trip latency with the remote operating terminal in real time; when the network round-trip latency exceeds a preset safety threshold, the system automatically increases the overall weight of actions in the system prediction queue; when communication with the remote operating terminal is interrupted, the system switches to an autonomous mode that executes only actions with high confidence in the system prediction queue.

[0018] Preferably, the high-level motion commands are mapped to the robot's low-level motion primitives. Before executing the low-level motion primitives, a safety check is performed to ensure collision protection and mechanical torque constraints. If the check fails, the original motion is replaced with a preset safety backup motion.

[0019] Preferably, after the cross-sorting, if the instructions in the remote operation queue do not conflict with the actions in the system prediction queue, then the instructions and the parts of the actions involving different degrees of freedom of the robot are merged and executed synchronously.

[0020] Preferably, the system further includes a policy execution mechanism based on the confidence level: when the confidence level is higher than a first threshold, the system is allowed to automatically execute the corresponding robot preparatory assistance action and may interrupt potentially conflicting instructions in the remote operation queue; when the confidence level is between a second threshold and the first threshold, the corresponding robot preparatory assistance action is sent as a suggestion to the remote operation terminal for confirmation and execution; wherein, the first threshold is higher than the second threshold.

[0021] In another aspect, this application also provides a remote control system for a healthcare robot based on artificial intelligence, comprising:

[0022] The data acquisition module is used to continuously acquire the user's video data stream through a visual sensor;

[0023] The temporal feature extraction module is used to extract micro-motion temporal features that characterize changes in user posture from the video data stream in real time. The micro-motion temporal features include at least changes in body center of gravity, changes in joint angles, and limb movement trends.

[0024] The behavioral intent prediction module is used to input the micro-action temporal features into a pre-trained behavioral precursor prediction model to predict at least one potential behavioral intent of the user in the future and its corresponding confidence level.

[0025] The instruction parsing module is used to receive a sequence of control instructions from the remote control terminal and parse them into operation instructions that the robot can execute.

[0026] The queue establishment module is used to establish a system prediction queue and a remote operation queue. The system prediction queue generates corresponding robot preparatory auxiliary actions based on the potential behavioral intentions and confidence levels. The remote operation queue is used to cache the operation instructions.

[0027] The execution sequence generation module is used to dynamically weight and cross-sort the actions to be executed in the system prediction queue and the remote operation queue based on the confidence level of the potential behavioral intention, the preset risk level, the current network latency, the action execution time window, and the urgency of the operation instruction, so as to generate the final action execution sequence.

[0028] The execution control module is used to control the health and wellness robot to perform corresponding auxiliary actions according to the final action execution sequence.

[0029] This application also provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the artificial intelligence-based remote control method for a health and wellness robot as described above.

[0030] In another aspect, this application provides a computer-readable storage medium having computer program instructions stored thereon, which can be executed by a processor to implement the artificial intelligence-based remote control method for a health and wellness robot as described above.

[0031] This application extracts micro-motion temporal features and predicts behavioral intentions based on a temporal fusion Transformer model, enabling early detection of sudden changes in elderly postures. This overcomes the limitations of traditional technologies that can only identify actions that have already occurred, significantly reducing safety risks. The application constructs a system prediction queue and a remote operation queue, combining multi-factor dynamic weight evaluation and cross-sorting based on confidence level, risk level, network latency, and other factors. This achieves intelligent resolution of command conflicts and merges conflict-free actions for execution, avoiding the potential for remote operation delays and improving collaborative efficiency. It eliminates the need for additional dedicated sensors, adapts to network latency fluctuations and communication interruption scenarios, and balances cost control and robustness. It outperforms traditional solutions in both action response latency and remote operation collaboration efficiency, providing safer, more efficient, and more reliable remote control technology support for elderly care scenarios. Attached Figure Description

[0032] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0033] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0034] Figure 1 is a schematic diagram of a remote control method for a health care robot based on artificial intelligence provided in an embodiment of the present invention.

[0035] Figure 2 is a schematic diagram of the structure of the TFT model provided in an embodiment of the present invention.

[0036] Figure 3 is a schematic diagram of the cross sorting process provided in an embodiment of the present invention.

[0037] Figure 4 is a schematic diagram of the structure of a remote control system for a health care robot based on artificial intelligence provided in an embodiment of the present invention.

[0038] Figure 5 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0040] It should be noted that all user information (including but not limited to user device information, user personal information, object information corresponding to device usage data, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, device usage data, etc.) involved in all embodiments of this disclosure are information and data authorized by the user or fully authorized by all parties.

[0041] To address the aforementioned issues, this application proposes a remote control method for a health and wellness robot based on artificial intelligence. The technical solution of this application will be described in detail below with reference to various embodiments.

[0042] As shown in Figure 1, this embodiment of the invention discloses a schematic diagram 100 of a remote control method for a health and wellness robot based on artificial intelligence, including the following method steps:

[0043] S1 continuously acquires users' video data streams through visual sensors deployed in health and wellness environments;

[0044] S2, extract micro-motion temporal features representing changes in user posture from the video data stream in real time. The micro-motion temporal features include at least changes in body center of gravity, changes in joint angles, and limb movement trends.

[0045] S3, input the micro-action timing features into the pre-trained behavioral precursor prediction model to predict at least one potential behavioral intention of the user in the future and its corresponding confidence level.

[0046] S4 receives a sequence of control commands from a remote control terminal and parses them into executable operation commands for the robot.

[0047] S5, establish a system prediction queue and a remote operation queue. The system prediction queue generates corresponding robot preparatory auxiliary actions based on the potential behavioral intentions and confidence levels. The remote operation queue is used to cache the operation instructions.

[0048] S6. Based on the confidence level of the potential behavioral intent, the preset risk level, the current network latency, and the urgency of the operation instruction, the actions to be executed in the system prediction queue and the remote operation queue are dynamically weighted and cross-sorted to generate the final action execution sequence.

[0049] S7. According to the final action execution sequence, control the health care robot to perform corresponding auxiliary actions.

[0050] In some embodiments, for step S1, specifically, the visual sensor is first deployed in the health and wellness environment. The deployment location needs to cover the core areas of the user's daily activities, such as the bedside area in the bedroom, the area around the sofa in the living room, and the area near the dining table in the restaurant, to ensure that the user's posture changes in different scenarios such as standing, sitting, and walking can be fully captured.

[0051] In some embodiments, for step S2, firstly, the video data stream transmitted to the main control module in step S1 is subjected to frame-level preprocessing to remove noise interference, which will not be described in detail here.

[0052] According to an embodiment of the present invention, after obtaining the preprocessed user foreground image, key skeletal points of the user are extracted based on a human pose estimation network, and then the change in the body's center of gravity is calculated. Preferably, the human pose estimation network adopts the open-source OpenPose network or MediaPipe Pose network. Such networks can output 2D or 3D skeletal key point coordinates in real time, including 25-33 key skeletal points such as head, neck, shoulder, elbow, wrist, hip, knee, and ankle.

[0053] Specifically, for each frame of image, the pose estimation network outputs the coordinates (x, y) and confidence score (between 0 and 1, with higher confidence scores indicating more accurate localization) of each key skeleton point. Then, a weighted average method is used to calculate the user's body center of gravity coordinates. The weights of skeleton points in different body parts are set according to the mass proportion of each body part; for example, the head is weighted at 0.15, the torso (neck to hip) at 0.45, the upper limbs (elbow to wrist) at 0.15, and the lower limbs (knee to ankle) at 0.25. The formula for calculating the body center of gravity coordinates is as follows:

[0054]

[0055] In the formula, , These are the x-axis and y-axis coordinates of the body's center of gravity, respectively. Let i be the weight of the i-th critical bone point; , Let i be the coordinates of the i-th key skeleton point; Let n be the confidence level of the i-th key skeleton point; n is the total number of key skeleton points (e.g., 25).

[0056] After obtaining the body centroid coordinates of each frame, calculate the change in centroid coordinates over a consecutive T frames (T is determined based on the video frame rate and prediction window; for example, 30 frames corresponding to 2 seconds at 15 FPS). , and the speed of the center of gravity shift ( The time interval between two frames, such as at 15 FPS. (seconds), using these parameters as the timing characteristics of micro-movements in response to changes in the body's center of gravity.

[0057] In one embodiment, the extraction of joint angle change features is based on the key skeletal point coordinates output by the aforementioned pose estimation network, and angle calculations are performed for major human joints (such as the neck, shoulder, elbow, wrist, hip, knee, and ankle). Taking the elbow joint as an example, the angle calculation process is as follows:

[0058] First, identify the three key skeletal points that form the elbow joint: shoulder skeletal point S ( Elbow bone point E ( W wrist bone point ( ); then, calculate the vector with vector Finally, the angle of the elbow joint is calculated using the vector dot product formula. The calculation formula is as follows:

[0059]

[0060] In the formula, For vectors and The dot product; , They are vectors , The length of the module.

[0061] Using the same method described above, the joint angles of the neck (based on the skeletal points of the head, neck, and shoulders), shoulder (based on the skeletal points of the neck, shoulder, and elbow), wrist (based on the skeletal points of the elbow, wrist, and hand), hip (based on the skeletal points of the shoulder, hip, and knee), knee (based on the skeletal points of the hip, knee, and ankle), and ankle (based on the skeletal points of the knee, ankle, and foot) are calculated respectively.

[0062] After obtaining the angles of each joint in each frame, calculate the change in the angles of each joint within a consecutive T frames. and the rate of change of angle. These parameters are used as the timing features of micro-motions in response to changes in joint angles.

[0063] In one embodiment, limb movement trend features are primarily obtained by calculating the movement direction and velocity of key limb points using optical flow. Preferably, the Lucas-Kanade optical flow algorithm is employed, which can efficiently calculate the optical flow vectors of feature points in an image, reflecting the movement trend of those feature points.

[0064] Specifically, firstly, representative limb key points, such as wrist, elbow, knee, and ankle, are selected from the key skeletal points obtained in the aforementioned steps; then, for two consecutive frames (frame t and frame t-1), using these limb key points as feature points, the optical flow vector of each feature point is calculated using the Lucas-Kanade algorithm. ),in The velocity component in the x-axis direction. The velocity component along the y-axis; then, based on the direction of the optical flow vector ( ) and size ( This determines the motion trend of each limb key point. For example, when the optical flow direction of the wrist key point is 90° (vertically upward) and the speed is greater than 0.5 pixels / frame, it can be determined that the wrist has an upward lifting motion trend.

[0065] Finally, after extracting the three types of features—changes in body center of gravity, changes in joint angles, and limb movement trends—these features are integrated into a micro-motion temporal feature tensor in chronological order.

[0066] Specifically, for a continuous T-frame video data, each feature dimension (such as the change in the centroid x-axis, the change in the elbow angle, the optical flow velocity of the wrist, etc.) forms a temporal sequence of length T. Then, the temporal sequences of all feature dimensions are concatenated to form a temporal feature tensor of shape T×F, where F is the total number of feature dimensions, which is determined according to the actual number of features extracted, usually between 100 and 300 dimensions (for example, the centroid-related features of 25 bone points are 10 dimensions, the angle-related features of 7 joints are 21 dimensions, and the optical flow-related features of 8 limb key points are 24 dimensions, for a total of 55 dimensions. If the first-order difference and the second-order difference of each feature are considered, the feature dimensions can be extended to 165 dimensions).

[0067] During feature integration, all features are normalized using the Min-Max normalization method to map feature values ​​to the [0,1] interval, thus eliminating the impact of differences in feature dimensions on subsequent model predictions. The normalization formula is as follows:

[0068]

[0069] In the formula, Let be the original value of a certain feature in frame t; , These are the minimum and maximum values ​​of the feature in the training dataset or historical data, respectively. These are the normalized eigenvalues.

[0070] In some embodiments, for step S3, preferably, the behavioral precursor prediction model is a model trained based on the Temporal Fusion Transformer (TFT) architecture. This model has excellent performance in multivariate temporal prediction tasks, can effectively handle the temporal correlation and dynamic changes of different dimensions of micro-motion temporal features, and can output the confidence interval or uncertainty measure of the prediction results, providing a reliable decision basis for subsequent queue scheduling.

[0071] Specifically, please refer to Figure 2. The structure of the TFT model includes the following core modules: Input Embedding Layer: The T×F micro-action temporal feature tensor obtained in step S2 is mapped into a high-dimensional feature vector through a fully connected layer. The embedding dimension is preferably 128-256 dimensions. At the same time, the feature vector is processed by layer normalization to accelerate the convergence of model training.

[0072] Temporal Attention Layer: It calculates the correlation weights between features at different time steps through a self-attention mechanism, focusing on time steps that are important for behavior prediction (such as key frames where the user's posture begins to change). The calculation of attention weights adopts Scaled Dot-Product Attention to avoid gradient vanishing or exploding.

[0073] Gated Recurrent Unit (GRU) Layer: This layer performs temporal modeling on the features output by the temporal attention layer, capturing long-term dependencies between features. The optimal hidden layer dimension of the GRU is 64-128, and dropout regularization (dropout rate set to 0.2-0.3) is employed to prevent overfitting.

[0074] Static Variable Processing Layer: If there is static information about the user (such as age, height, weight, etc., which can be obtained through system presets or user input), this layer will fuse the static information with dynamic temporal features to improve the model's adaptability to different users.

[0075] Output Layer: Employs a fully connected layer and a softmax activation function to output the probability distribution of various potential behavioral intentions of the user over a future period (prediction horizon set to 0.5-1.5 seconds, prioritizing coverage of key time periods of behavioral changes in the elderly). Simultaneously, it outputs the confidence interval (e.g., 90% confidence interval) for each behavioral intention through quantile regression, with the confidence level taken as the maximum value of the probability distribution or the median of the confidence interval.

[0076] In one embodiment, after the model structure is determined, the model is trained using a labeled training dataset. The specific steps are as follows:

[0077] The training dataset consists of user video data collected in elderly care settings, and must include various behavioral data from users of different age groups (e.g., 60-90 years old) and different physical conditions (e.g., healthy elderly and elderly with mild mobility impairments). Each sample is composed as follows:

[0078] Input data: the micro-motion temporal feature tensor described in step S2 (T×F, where T is set to 22 frames corresponding to 1.5 seconds @ 15 FPS, and F is set to 100-300 dimensions).

[0079] Tag data: The category and time of the user's actual behavioral intention within the next 0.5-1.5 seconds. The behavioral intention categories are preset according to the needs of the health and wellness scenario, including but not limited to "stand up", "sit down", "turn around", "grasp an object", "hold onto a support", "no significant movement", etc., totaling 5-8 categories. Tag labeling adopts a combination of manual and automatic labeling. First, it is initially labeled by action recognition algorithm (such as skeleton-based action classification algorithm), and then reviewed and corrected by professional caregivers to ensure that the label accuracy rate is ≥95%.

[0080] The training dataset should be large enough to meet the model training requirements. Initially, it should contain at least 1000+ labeled behavioral event samples. The number of samples for each type of behavioral intent should be balanced (e.g., 20% for "stand up" and 20% for "sit down", 30% for "no significant action", and 10%-15% for the other categories) to avoid the model biasing towards the category with the most samples.

[0081] The model training uses the Adam optimizer with an initial learning rate of 1e-4-5e-4. The learning rate decay strategy uses cosine annealing decay, where the learning rate is halved if the validation set loss does not decrease for 5-10 consecutive epochs. The batch size is set according to the hardware computing power. If GPU (such as NVIDIA Jetson AGXXavier) is used for training, the batch size is set to 16-32. If CPU is used for training, the batch size is set to 4-8. The number of training epochs is set to 50-100, and an early stopping strategy is used. Training is stopped if the validation set loss does not decrease for 15 consecutive epochs to prevent the model from overfitting.

[0082] In one embodiment, the model's loss function is a weighted sum of multi-class cross-entropy loss and quantile loss to simultaneously optimize the prediction accuracy of behavioral intent categories and the accuracy of confidence intervals. The loss function formula is as follows:

[0083]

[0084] in, For multi-class cross-entropy loss, the difference between the predicted behavioral intent probability distribution and the true label (one-hot encoding) is calculated using the following formula: ,in The total number of behavioral intent categories. This is an indicator variable (0 or 1) for the c-th class in the actual labels. Predict the probability of class c for the model;

[0085] Quantile loss is used to optimize the estimation of confidence intervals, and the formula is as follows: ,in For the sample size, , These correspond to the lower and upper bounds of the 90% confidence interval, respectively. This represents the probability of the true behavioral intent of the i-th sample (1 if it is a certain type of behavior, 0 otherwise). To predict the quantile of the i-th sample in the model The probability of it happening;

[0086] This is the L2 regularization term, used to prevent overfitting due to excessively large model parameters. The formula is: ,in The set of all trainable parameters of the model;

[0087] , , The loss weights were determined based on experimental verification, and the preferred value was [value missing]. , , .

[0088] Subsequently, the constructed training dataset is divided into a training set, a validation set, and a test set in a 7:2:1 ratio. The training set is used for updating model parameters, the validation set is used for adjusting model hyperparameters (such as learning rate and loss weights), and the test set is used for evaluating the final performance of the model. These details will not be elaborated further here.

[0089] In some embodiments, after the model training is completed and deployed to the main control module of the health and wellness robot, real-time behavioral precursor prediction and inference can be performed. The specific process is as follows:

[0090] First, the micro-motion temporal feature tensor (T×F) extracted in real time in step S2 is input into a pre-trained behavioral precursor prediction model (a lightweight distillation model can be used for edge deployment). Then, the model outputs the probability distribution of various potential behavioral intentions of the user in the next 0.5-1.5 seconds through the collaborative calculation of modules such as the input embedding layer, temporal attention layer, and GRU layer (e.g., {"Stand up": 0.72, "Turn around": 0.18, "Hold onto support": 0.10}) and the corresponding 90% confidence interval (e.g., the confidence interval for the "Stand up" behavior is [0.65, 0.80]). Finally, the behavioral intention with the highest probability value in the probability distribution is determined as the primary potential behavioral intention, and its probability value is used as the confidence level of the behavioral intention (e.g., the confidence level for the "Stand up" behavior is 0.72). At the same time, other behavioral intentions with a probability value greater than 0.1 are retained as secondary potential behavioral intentions for multi-scenario adaptation in subsequent queue construction.

[0091] In some embodiments, for step S4, a dedicated control application can be installed on the remote operating terminal. For example, the application provides a graphical user interface, including robot motion control buttons (such as "lift the robot arm", "move the chassis", "open the gripper"), an emergency stop button, a motion speed adjustment slider, etc. The operator generates a sequence of control commands by clicking the buttons or sliding the slider.

[0092] Subsequently, the main control module of the health and wellness robot continuously listens for the control command sequence sent by the remote operator through a preset communication port (such as UDP port 5005). The control command sequence is encapsulated in JSON format and includes fields such as command type, command parameters, sending timestamp, operator ID, and emergency flag. An example command format is as follows:

[0093] {

[0094] "command_type": "arm_control",

[0095] "parameters": {

[0096] "arm_side": "right",

[0097] "target_angle": 45.0,

[0098] "velocity": 10.0

[0099] },

[0100] "timestamp": 1699999999.123,

[0101] "operator_id": "caregiver_001",

[0102] "emergency_flag": false

[0103] }

[0104] in," The field indicates the instruction type (e.g., " "For robotic arm control, " "For chassis control, " The "for gripper control" field contains the specific parameters of the command (e.g., the "parameters" field for robotic arm control). "Indicates the side of the robotic arm, " "" indicates the target angle, "velocity" indicates the movement speed); the "timestamp" field is the timestamp of the command sent (accurate to milliseconds); "This field is used to identify the operator's identity, facilitating access control." The field is for emergency indicators (true indicates an emergency command, such as an emergency stop; false indicates a normal command).

[0105] Upon receiving the control command sequence, the main control module's command parsing module parses and processes the commands, integrating the parsed command type, standardized parameters, sending timestamp, and emergency flag into executable operation commands for the robot. These operation commands are stored in a structure format for easy subsequent queue caching and scheduling. An example operation command structure is shown below:

[0106] typedef struct {

[0107] enum CommandType type; / / Command type: ARM_CONTROL, BASE_CONTROL, etc.

[0108] float params

[10] ; / / Standardized parameters, supporting up to 10 parameters

[0109] double timestamp; / / Command sending timestamp

[0110] bool is_emergency; / / Whether it is an emergency instruction

[0111] int priority; / / Instruction priority (1-5, 5 being the highest)

[0112] OperationCommand;

[0113] Among them, instruction priority is based on instruction type and " "The priority of emergency control commands is set to 5, the priority of robotic arm support commands (such as "robotic arm supports user") is set to 4, the priority of regular movement commands (such as "chassis moves slowly") is set to 2-3, and the priority of adjustment commands (such as "gripper fine-tuning") is set to 1. The priority setting provides a basis for the weight evaluation of subsequent queue scheduling.

[0114] In some embodiments, for step S5, the system prediction queue is used to store robot pre-assistive actions generated based on the potential behavioral intentions predicted in step S3. The generation of pre-assistive actions needs to combine the type and confidence level of the behavioral intention to ensure the safety and relevance of the actions. The specific construction process is as follows:

[0115] First, based on the common behavioral intentions of users in health and wellness scenarios, a pre-set auxiliary action library for the robot is established. Each action in the action library is associated with a specific behavioral intention type and includes information such as action description, action parameter range, and risk level.

[0116] In some embodiments, based on the potential behavioral intent and confidence level output in step S3, a corresponding pre-assisted action is matched from the pre-assisted action library, and the action parameters are adjusted according to the user's current posture and confidence level. For example, the specific rules are as follows:

[0117] For the primary potential behavioral intention (the behavioral intention with the highest confidence): if the confidence p ≥ 0.75 (first threshold), the corresponding preparatory auxiliary action is directly generated, and the action parameters are adjusted to the optimal range (e.g., the robotic arm support angle is 45° and the support force is 8N for the "stand up" action), and the risk level of the action is marked (e.g., level 3).

[0118] If the confidence level is 0.5≤p<0.75 (the second threshold ≤p<the first threshold, where the first threshold>the second threshold), then the corresponding preparatory auxiliary action is generated, but the action parameters are set to a conservative range (e.g., the "stand up" action corresponds to a robotic arm support angle of 30° and a support force of 5N), and the action is marked as "action to be confirmed". Whether to execute it needs to be determined later based on remote commands or changes in confidence level.

[0119] If the confidence level p < 0.5, then no main preparatory auxiliary action will be generated, and only the "attitude monitoring + standby" action (risk level 0) will be generated.

[0120] For secondary potential behavioral intentions (other behavioral intentions with a probability value > 0.1): If the confidence level p of the secondary behavioral intention is ≥ 0.5, a corresponding preparatory auxiliary action is generated, the action parameters are set to a conservative range, and the action is marked as an "alternative action". When the confidence level of the primary behavioral intention decreases or the user's posture changes, the action can be quickly switched to the primary action; if the confidence level p of the secondary behavioral intention is < 0.5, only the behavioral intention is recorded, and no corresponding preparatory auxiliary action is generated.

[0121] In some embodiments, the system prediction queue adopts a first-in, first-out (FIFO) queue structure. Each queue element is a preparatory auxiliary action unit, containing fields such as action ID, action type, action parameters, generation timestamp, confidence level, risk level, and action status (pending execution, in execution, completed, canceled). An exemplary queue element structure is as follows:

[0122] typedef struct {

[0123] int action_id; / / Action ID, a unique identifier

[0124] enum ActionType type; / / Action type: ARM_SUPPORT, BASE_STABILIZE, etc.

[0125] float params

[10] ; / / Action parameters

[0126] double generate_time; / / Generate timestamp for the action

[0127] float confidence; / / Confidence level corresponding to the behavioral intent

[0128] int risk_level; / / Risk level (0-3)

[0129] enum ActionStatus status; / / Action status: PENDING, EXECUTING,COMPLETED, CANCELLED

[0130] PredictedAction;

[0131] Optionally, during queue construction, the generated preparatory auxiliary action units are added to the system prediction queue in the order of their generation timestamps. A queue length limit is set (e.g., 5-10 action units). When the queue length exceeds the limit, the earliest generated action unit with a status of "pending execution" is deleted to avoid queue redundancy and scheduling delays. Furthermore, if a newly generated preparatory auxiliary action has the same type as an existing action in the queue and the parameter difference is less than 10%, the parameters, confidence level, and generation timestamp of the existing action are updated, and it is not added to the queue again to reduce action redundancy.

[0132] In some embodiments, the remote operation queue is used to cache the robot's executable operation instructions parsed in step S4, ensuring the orderly storage and scheduling of remote instructions. The specific construction process is as follows:

[0133] The remote operation queue also adopts a FIFO queue structure, with each queue element being an operation command unit. For example, based on the Operation Command structure generated in step S4, it is expanded to add fields such as command reception timestamp and command status (pending execution, executing, completed, canceled). An example queue element structure is as follows:

[0134] typedef struct {

[0135] int cmd_id; / / Command ID, unique identifier

[0136] enum CommandType type; / / Command type: ARM_CONTROL, BASE_CONTROL, etc.

[0137] float params

[10] ; / / Standardized instruction parameters

[0138] double send_time; / / Command sending timestamp (remote operator)

[0139] double recv_time; / / Command reception timestamp (bot side)

[0140] int priority; / / Instruction priority (1-5)

[0141] bool is_emergency; / / Whether it is an emergency instruction

[0142] enum CmdStatus status; / / Command status: PENDING, EXECUTING, COMPLETED,CANCELLED

[0143] char operator_id

[20] ; / / Operator ID

[0144] OperatorCommand;

[0145] Among them, the instruction reception timestamp ( The transmission delay is recorded by the robot's main control module when it receives an instruction and is used to calculate the instruction's transmission delay. This provides a basis for subsequent scheduling delay penalties; the instruction status is initially set to "PENDING", updated to "EXECUTING" when the instruction is scheduled for execution, updated to "COMPLETED" after execution is completed, and updated to "CANCELLED" if interrupted by a high-priority action.

[0146] In some embodiments, for step S6, the core of dynamic weight evaluation is to calculate priority scores (denoted as S) for the preparatory auxiliary actions (hereinafter referred to as "predicted actions") in the system prediction queue and the operation instructions to be executed (hereinafter referred to as "operation instructions") in the remote operation queue. The priority scores comprehensively reflect the urgency, safety and timeliness of the actions / instructions, providing a quantitative basis for cross-sorting.

[0147] In one embodiment, the priority score is calculated based on the following five core evaluation parameters, the definitions and acquisition methods of each parameter are as follows:

[0148] Prediction confidence (p): Only for predicted actions, it is the confidence (0-1) of the corresponding potential behavioral intent output in step S3, directly obtained from the action units in the system prediction queue; for operation instructions, p is the operator's historical operation confidence for the instruction (obtained by statistically analyzing the operator's past execution success rate of instructions, such as ≥95% for p=0.9, 85%-95% for p=0.8, and so on), and if it is a new operator, p=0.75;

[0149] Risk level (r): For the predicted action, the risk level (0-3) preset for step S5 is obtained from the action units in the system prediction queue; for the operation instruction, the risk level is determined according to the instruction type.

[0150] Operational urgency (u): Only applies to operational instructions, determined by the "is_emergency" field and priority of the instruction. Urgent instructions have u=1.0, and regular instructions have u=priority / 5.0 (mapping priorities 1-5 to 0.2-1.0). For predicted actions, u is determined by the risk level: r=3, u=1.0; r=2, u=0.8; r=1, u=0.5; r=0, u=0.2.

[0151] Current network latency (L): The round-trip latency (ms) between the remote control terminal and the robot is obtained by periodically sending ping packets or calculating the average transmission latency of commands (the average of the transmission latency of the last 10 commands). If there is no remote command transmission at present, the historical latency average (the latency average of the last 1 minute) is used.

[0152] Action execution time window (Δt): For predicted actions, it is the predicted start time (s, i.e. the time when the user may start the action) of the potential behavioral intention corresponding to the action, which is obtained from the prediction results of step S3; for operation instructions, Δt is the expected execution time of the instruction (preset according to the instruction type, such as Δt=1.0s for the robotic arm lifting instruction and Δt=2.0s for the chassis moving instruction).

[0153] The priority score S is calculated using a weighted summation formula, while also incorporating a latency penalty function to reflect the impact of network latency on instruction execution timeliness. The specific formula is as follows:

[0154]

[0155] in, , , , The weighting coefficients for prediction confidence, normalized risk level, operational urgency, and delay penalty are determined through experimental verification, and the optimal values ​​are [values ​​to be filled in]. , , , And satisfy (The total weight can be greater than 1 to highlight the combined effect of each factor.)

[0156] To determine the normalized risk level, r(0-3) is mapped to the 0-1 interval, and the calculation formula is as follows: ; This is a latency penalty function used to quantify the impact of network latency on the safety of action / instruction execution. When excessive network latency prevents an action from being executed before a change in user behavior, a larger penalty is imposed. The formula is as follows:

[0157]

[0158] In the formula, L is in milliseconds (ms) and Δt is in seconds (s), so Δt needs to be converted to ms (multiplied by 1000). The value range of the delay penalty function is 0-1. When L≤30%Δt (ms), the delay has no effect on execution, and the penalty is 0; when L≥Δt (ms), the delay causes the action to be unable to be executed in time, and the penalty is 1; when 30%Δt < L < Δt, the penalty increases linearly with the increase of L.

[0159] In some possible embodiments, please refer to Figure 3, a flowchart of cross-sorting. Cross-sorting is the process of merging the "pending" actions / instructions in the system prediction queue and the remote operation queue, arranging them in descending order according to priority scores, and performing conflict resolution and action synthesis to finally generate an execution sequence.

[0160] S301, Construct a candidate action / instruction set. First, select predicted actions with a status of "PENDING" from the system prediction queue, and select operation instructions with a status of "PENDING" from the remote operation queue, and merge them to form a candidate set C.

[0161] S302, sort by priority score in descending order. Specifically, for each element (predicted action or operation instruction) in the candidate set C, sort it in descending order according to the calculated priority score S to obtain a preliminary sorting result. If the difference in priority scores between two elements ΔS < 0.05 (a preset hysteresis threshold used to prevent frequent switching), then the original order in the queue is maintained (i.e., the element that entered the queue first is placed at the front), avoiding frequent action switching due to small differences in scores, which would affect the stability of robot execution.

[0162] S303, Conflict resolution for actions / instructions. Specifically, conflict resolution is used to handle conflicts between sorted elements in space, time, or robot degrees of freedom, ensuring that the robot performs actions without collisions or mechanical interference. For example, the specific resolution rules are as follows:

[0163] Conflict type definition, specifically including, spatial conflict: two actions / instructions require the robot actuator (such as robotic arm, chassis) to reach overlapping spatial positions, which may lead to collision (such as robotic arm raised to 45° and robotic arm lowered to 30°).

[0164] Time conflict: Two actions / instructions are required to be executed within the same time period, and the robot actuator cannot process them in parallel (such as two consecutive angle control instructions of the same robotic arm).

[0165] Degree of freedom conflict: Two actions / instructions involve the control of the same degree of freedom of the robot, and the control objectives are opposite (such as the chassis moving forward and the chassis moving backward).

[0166] In some possible embodiments, the conflict between local emergency actions and other actions is resolved as follows: if a local emergency action (such as fall protection) conflicts with other actions, the other actions are directly suspended or canceled, the local emergency action is executed first, and a prompt message "Executing local emergency action, other instructions have been suspended" is sent to the remote operating terminal.

[0167] The solution to the conflict between predicted actions and operational instructions is as follows: if the priority score of the predicted action is... Priority scoring of operation instructions If the (hysteresis threshold) is reached, the predicted action will be executed, the operation command will be paused or canceled, and the remote operation terminal will be notified that "a high-priority safety action has been detected and your command has been paused".

[0168] If the priority of the operation instruction is scored If the operation command is executed, the predicted action will be paused or canceled, and the remote operation terminal will be notified that "your command has higher priority and has been executed".

[0169] like If the delay threshold is reached, a confirmation request will be sent to the remote control terminal, displaying the content of the predicted action and operation instruction, as well as risk warnings (such as "The system suggests performing a standing assistance action. Your robotic arm lifting instruction may conflict. Do you want to continue?"). If the operator responds with "yes" within 1000ms, the operation instruction will be executed; if the operator responds with "no", the predicted action will be executed; if no response is received within the time limit, the predicted action will be executed by default (prioritizing user safety).

[0170] The conflict resolution method between predicted actions is as follows: prioritize the predicted action with the higher priority score. If the score difference is ≤0.05, then execute the action with the more recent generation time (which better reflects the user's current behavior trend).

[0171] The method for resolving conflicts between operation instructions is as follows: the operation instruction with the higher priority score is executed first. If the score difference is ≤0.05, the instruction with the newer reception time is executed (which better reflects the operator's current intention).

[0172] For example, if there is a spatial conflict between B (predicted action: S=0.86) and C (operation instruction: S=0.745) in the sorting results, and ΔS=0.115>0.05, then B will be executed, C will be canceled, and a prompt message will be sent to the remote operation terminal.

[0173] S304, Action / Instruction Combined Execution: Specifically, for actions / instructions that do not conflict after sequencing, if they involve different degrees of freedom of the robot (such as robotic arm actions and chassis actions), they can be combined for execution to improve the robot's response efficiency. For example,

[0174] Two actions / instructions involving non-overlapping robot motion degrees of freedom (such as the shoulder joint movement of a robotic arm and the translational movement of the chassis), and whose execution time window overlap ratio is >70%, are merged; specifically, the motion trajectories of the two actions / instructions are synchronized in time to generate a compound trajectory for parallel execution, ensuring that the two actions start and end synchronously in time, or the start time is adjusted according to the length of the action execution time (such as a short-time action waiting for a long-time action, or a long-time action and a short-time action partially running in parallel).

[0175] S305, Generate the final action execution sequence. Specifically, after completing conflict resolution and merging, the processed actions / instructions are organized into a final action execution sequence according to the execution order (serial actions are ordered according to the sorting order, and parallel actions are marked as simultaneous). Each element in the sequence contains information such as the action / instruction ID, type, execution start time, execution end time, execution parameters, and status (to be executed). Preferably, after the final action execution sequence is generated, it is stored in the robot's execution instruction buffer and synchronized to the remote control terminal, allowing the operator to view the robot's execution plan in real time.

[0176] In some embodiments, for step S7, the actions / instructions in the final action execution sequence are mostly high-level descriptions (such as "stand-up assistance" and "robotic arm lift"), which are mapped to low-level action primitives that can be directly executed by the robot actuators. The low-level action primitives are based on the robot's hardware structure presets and cover the basic control units of actuators such as robotic arms, chassis, and grippers.

[0177] Specifically, the underlying motion primitive library contains the basic motions of each actuator of the robot. Each primitive defines the control parameters, execution constraints and hardware interface of the motion.

[0178] In some possible embodiments, based on the type and parameters of the high-level action, it is decomposed into one or more low-level action primitives according to a preset mapping rule. The mapping rule is combined with the robot's kinematic model to ensure that the action after the primitive combination is consistent with the high-level action description. Examples are as follows:

[0179] High-level motion: "Stand-up assistance" (predicted motion, parameters: right robotic arm support angle 45°, support force 8N, chassis stability), mapping result:

[0180] Right robotic arm joint angle control primitives: Joint 1 (shoulder joint) target angle 45°, speed 10° / s, acceleration 5° / s²;

[0181] Right-side gripper clamping force control primitives: target clamping force 8N, clamping speed 5mm / s;

[0182] Chassis translation control primitives: direction 0° (forward), distance 0.1m, speed 0.05m / s (fine-tune chassis position, move closer to user);

[0183] Chassis rotation control primitives: rotation angle 0° (no rotation), angular velocity 0° / s (maintain directional stability).

[0184] High-level action: "Robotic arm lifts up" (operation command, parameters: right robotic arm lifts up to 60°, speed 15° / s), mapping result:

[0185] Right robotic arm joint angle control primitives: Joint 1 (shoulder joint) target angle 60°, speed 15° / s, acceleration 8° / s²;

[0186] Right robotic arm joint angle control primitives: Joint 2 (elbow joint) target angle 30°, speed 15° / s, acceleration 8° / s² (in coordination with shoulder joint movement to ensure stable robotic arm posture).

[0187] During the mapping process, if the high-level motion parameters exceed the execution constraints of the low-level primitives (e.g., the target angle of the robotic arm is 95°, which is outside the 0-90° range), the parameters will be automatically adjusted to the constraint range (e.g., adjusted to 90°), and a prompt message "Motion parameters exceed hardware constraints, automatically adjusted" will be sent to the remote operator.

[0188] In some possible embodiments, to ensure the safety of the robot when performing actions, collision protection and mechanical torque constraint safety checks are performed before the underlying motion primitives are sent to the actuator.

[0189] Specifically, for collision protection, a collision detection algorithm based on a three-dimensional environment model is adopted. For example, the specific steps are as follows:

[0190] Construct a 3D model of the robot actuator (based on URDF format, including the geometry and linkage of the robotic arm, chassis, gripper) and a 3D model of the health and wellness environment (reconstructed in real time through visual sensors, including static obstacles such as furniture and walls).

[0191] Based on the motion trajectory of the underlying motion primitives, the motion process of the robot actuator in the three-dimensional environment is simulated, and the position of the actuator at each time step (e.g., 10ms) is generated.

[0192] Check the distance between the actuator and environmental obstacles, and between the actuator's own links, at each time step. If the distance is less than the safe distance (the safe distance between the robotic arm and obstacles is 10cm, and the safe distance between the robotic arm's own links is 5cm), then it is considered a collision risk.

[0193] If a collision risk exists, adjust the parameters of the underlying motion primitive (such as reducing the speed of the robotic arm or adjusting the position of the chassis), and re-perform the collision detection. If a collision risk still exists after three adjustments, cancel the action and send a message to the remote control terminal that says "Collision risk detected, action canceled".

[0194] Mechanical torque constraint verification is used to prevent hardware damage or harm to users caused by excessive torque in robot actuators. For example, the specific steps are as follows:

[0195] Based on the dynamic model of the robot actuator, the theoretical output torque of each joint motor during the execution of each underlying motion primitive is calculated. The calculation formula is as follows:

[0196]

[0197] In the formula, This provides torque output to the joint motor. The inertia matrix; Joint angular acceleration; The matrix represents the Coriolis force and the centrifugal force. Joint angular velocity; The gravity matrix; It is a Jacobian matrix; External forces (such as the support force of the gripper on the user).

[0198] Check whether the calculated theoretical output torque is less than the maximum rated torque of the motor (e.g., the maximum rated torque of the motor of the robotic arm joint is 5 N·m). If the theoretical output torque of a certain joint motor is greater than 90% of the maximum rated torque, it is determined that the torque is over-limit.

[0199] If the torque exceeds the limit, reduce the speed and acceleration parameters of the underlying motion primitive (e.g., reduce the speed from 15° / s to 10° / s), recalculate the torque, and if it still exceeds the limit after 3 adjustments, cancel the action and send a prompt message "Motor torque is about to exceed the limit, action has been canceled" to the remote control terminal.

[0200] After the security verification is passed, the underlying action primitives are sent to the corresponding actuators in the order of execution time to control the health care robot to perform the corresponding auxiliary actions.

[0201] In some possible embodiments, preferably, when network latency changes, queue weights can be adjusted or execution modes can be switched, as specifically implemented as follows:

[0202] The preset network latency safety threshold is 200ms. When L continuously exceeds 200ms for 3 monitoring periods (300ms), the overall weight of actions in the system's prediction queue is automatically increased. Specifically, the weight of prediction confidence is adjusted as follows: Adjusted from 0.5 to 0.6; Operational urgency weight. Adjusted from 0.2 to 0.1; Delay penalty function weight The priority score was adjusted from 0.4 to 0.5; at the same time, the priority score of regular instructions in the remote operation queue was reduced (multiplied by a coefficient of 0.8) to reduce the risk of instruction delays due to latency.

[0203] When a communication interruption is detected with the remote control terminal, the control system automatically switches to autonomous mode. In autonomous mode, only high-confidence safety actions (such as standing assistance and fall prevention support) with a confidence level p ≥ 0.75 and a risk level r ≥ 2 in the system's prediction queue are executed. Actions with a confidence level p < 0.75 and all remote commands are temporarily suspended.

[0204] In some possible embodiments, different execution strategies are adopted based on the confidence level of the behavioral aura prediction, specifically including:

[0205] The first threshold is preset to 0.75 and the second threshold is 0.5. These thresholds can be dynamically adjusted according to the safety requirements of the actual health and wellness scenario and the accuracy of the model prediction (for example, if the nursing home scenario has higher safety requirements, the first threshold can be adjusted to 0.8 and the second threshold can be adjusted to 0.55).

[0206] When the system predicts that the confidence level p of a certain preparatory auxiliary action in the prediction queue is ≥0.75 and the risk level r≥2, the system automatically increases the priority score of the action by 20% (e.g., from 0.86 to 1.03), allowing it to be executed automatically without waiting for confirmation from the remote operator; if the action conflicts with the instructions to be executed in the remote operation queue, the system can directly interrupt the execution of the remote instructions (except for emergency instructions) and send a prompt to the remote operator.

[0207] When the confidence level of the preparatory auxiliary action meets the condition 0.5≤p<0.75, the system will not automatically execute the action, but will send it as a "suggested action" to the remote operation terminal. The type, parameters, confidence level and risk level of the suggested action (e.g. "suggested to execute the hand grip auxiliary action, confidence level 0.68, risk level: medium") will be executed after confirmation.

[0208] When the confidence level p of the preparatory auxiliary action is less than 0.5, the system marks the action as a "low confidence action" and keeps it only in the queue for reference, without participating in priority sorting and scheduling; at the same time, the overall weight of the system prediction queue is reduced and the weight of the remote operation queue is increased.

[0209] Please refer to Figure 4, which is a schematic diagram of the structure of a remote control system 4 for a health and wellness robot based on artificial intelligence provided in an embodiment of this application. As shown in Figure 4, the system 400 includes:

[0210] The data acquisition module 401 is used to continuously acquire the user's video data stream through a visual sensor;

[0211] The temporal feature extraction module 402 is used to extract micro-motion temporal features that characterize changes in user posture from the video data stream in real time. The micro-motion temporal features include at least changes in body center of gravity, changes in joint angles, and limb movement trends.

[0212] The behavioral intention prediction module 403 is used to input the micro-action temporal features into a pre-trained behavioral precursor prediction model to predict at least one potential behavioral intention of the user in the future and its corresponding confidence level.

[0213] The instruction parsing module 404 is used to receive a sequence of control instructions from a remote operator and parse them into operation instructions that the robot can execute.

[0214] The queue establishment module 405 is used to establish a system prediction queue and a remote operation queue. The system prediction queue generates corresponding robot preparatory auxiliary actions based on the potential behavioral intentions and confidence levels. The remote operation queue is used to cache the operation instructions.

[0215] The execution sequence generation module 406 is used to dynamically weight and cross-sort the actions to be executed in the system prediction queue and the remote operation queue based on the confidence level of the potential behavioral intention, the preset risk level, the current network latency, the action execution time window, and the urgency of the operation instruction, so as to generate the final action execution sequence.

[0216] The execution control module 407 is used to control the health care robot to perform corresponding auxiliary actions according to the final action execution sequence.

[0217] Those skilled in the art will clearly understand that the technical solutions of the embodiments of this application can be implemented by means of software and / or hardware. In this specification, "unit" and "module" refer to software and / or hardware that can independently complete or cooperate with other components to complete a specific function, wherein the hardware may be, for example, a field-programmable gate array (FPGA), an integrated circuit (IC), etc.

[0218] Each processing unit and / or module in the embodiments of this application can be implemented by an analog circuit that implements the functions described in the embodiments of this application, or by software that executes the functions described in the embodiments of this application.

[0219] Please refer to Figure 5, which shows a schematic diagram of the structure of an electronic device according to an embodiment of this application. This electronic device can be used to implement the method in the embodiment shown in Figure 1. As shown in Figure 5, the electronic device 500 may include:

[0220] The system includes at least one processor 501, at least one network interface 504, a user interface 503, a memory 505, and at least one communication bus 502. The communication bus 502 is used to enable connection and communication between the components. The user interface 503 may include buttons, and optionally include a standard wired or wireless interface. The network interface 504 may include, but is not limited to, a Bluetooth module, an NFC module, a Wi-Fi module, etc.

[0221] The processor 501 may include one or more processing cores and connect to various parts within the device 500 via various interfaces and lines. It implements the various functions and data processing of the device 500 by running or executing instructions, programs, code sets, or instruction sets stored in the memory 505, and by accessing data in the memory 505. Optionally, the processor 501 may be implemented using at least one hardware form of DSP, FPGA, or PLA. The processor 501 may also integrate one or more combinations of CPU, GPU, and modem. The CPU is mainly used to handle the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display on the screen; and the modem is used for wireless communication. It is understood that the modem may not be integrated into the processor 501, but may be implemented through a separate chip.

[0222] The memory 505 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 505 includes a non-transitory computer-readable medium that can be used to store instructions, programs, code, code sets, or instruction sets. The memory 505 may be divided into a program storage area and a data storage area, wherein the program storage area can be used to store instructions for implementing an operating system, instructions for implementing at least one function (such as touch function, audio playback function, image playback function, etc.), and instructions for implementing the aforementioned method embodiments; the data storage area can be used to store data involved in the relevant method embodiments. The memory 505 may also be at least one storage device located away from the processor 501. As shown in FIG5, the memory 505, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and program instructions.

[0223] Another embodiment of this application provides a computer-readable storage medium having computer program instructions stored thereon, which can be executed by a processor to implement the methods and / or technical solutions of any one or more embodiments of this application described above.

[0224] The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0225] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0226] Furthermore, it should be noted that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in the system claims may also be implemented by a single unit or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0227] In some possible embodiments, preferably, when network latency changes, queue weights can be adjusted or execution modes can be switched, as specifically implemented as follows:

[0228] The preset network latency safety threshold is 200ms. When L continuously exceeds 200ms for 3 monitoring periods (300ms), the overall weight of actions in the system's prediction queue is automatically increased. Specifically, the weight of prediction confidence is adjusted as follows: Adjusted from 0.5 to 0.6; Operational urgency weight. Adjusted from 0.2 to 0.1; Delay penalty function weight The priority score was adjusted from 0.4 to 0.5; at the same time, the priority score of regular instructions in the remote operation queue was reduced (multiplied by a coefficient of 0.8) to reduce the risk of instruction delays due to latency.

[0229] When a communication interruption is detected with the remote control terminal, the control system automatically switches to autonomous mode. In autonomous mode, only high-confidence safety actions (such as standing assistance and fall prevention support) with a confidence level p ≥ 0.75 and a risk level r ≥ 2 in the system's prediction queue are executed. Actions with a confidence level p < 0.75 and all remote commands are temporarily suspended.

[0230] In some possible embodiments, different execution strategies are adopted based on the confidence level of the behavioral aura prediction, specifically including:

[0231] The first threshold is preset to 0.75 and the second threshold is 0.5. These thresholds can be dynamically adjusted according to the safety requirements of the actual health and wellness scenario and the accuracy of the model prediction (for example, if the nursing home scenario has higher safety requirements, the first threshold can be adjusted to 0.8 and the second threshold can be adjusted to 0.55).

[0232] When the system predicts that the confidence level p of a certain preparatory auxiliary action in the prediction queue is ≥0.75 and the risk level r≥2, the system automatically increases the priority score of the action by 20% (e.g., from 0.86 to 1.03), allowing it to be executed automatically without waiting for confirmation from the remote operator; if the action conflicts with the instructions to be executed in the remote operation queue, the system can directly interrupt the execution of the remote instructions (except for emergency instructions) and send a prompt to the remote operator.

[0233] When the confidence level of the preparatory auxiliary action meets the condition 0.5≤p<0.75, the system will not automatically execute the action, but will send it as a "suggested action" to the remote operation terminal. The type, parameters, confidence level and risk level of the suggested action (e.g. "suggested to execute the hand grip auxiliary action, confidence level 0.68, risk level: medium") will be executed after confirmation.

[0234] When the confidence level p of the preparatory auxiliary action is less than 0.5, the system marks the action as a "low confidence action" and keeps it only in the queue for reference, without participating in priority sorting and scheduling; at the same time, the overall weight of the system prediction queue is reduced and the weight of the remote operation queue is increased.

[0235] Please refer to Figure 4, which is a schematic diagram of the structure of a remote control system 4 for a health and wellness robot based on artificial intelligence provided in an embodiment of this application. As shown in Figure 4, the system 400 includes:

[0236] The data acquisition module 401 is used to continuously acquire the user's video data stream through a visual sensor;

[0237] The temporal feature extraction module 402 is used to extract micro-motion temporal features that characterize changes in user posture from the video data stream in real time. The micro-motion temporal features include at least changes in body center of gravity, changes in joint angles, and limb movement trends.

[0238] The behavioral intention prediction module 403 is used to input the micro-action temporal features into a pre-trained behavioral precursor prediction model to predict at least one potential behavioral intention of the user in the future and its corresponding confidence level.

[0239] The instruction parsing module 404 is used to receive a sequence of control instructions from a remote operator and parse them into operation instructions that the robot can execute.

[0240] The queue establishment module 405 is used to establish a system prediction queue and a remote operation queue. The system prediction queue generates corresponding robot preparatory auxiliary actions based on the potential behavioral intentions and confidence levels. The remote operation queue is used to cache the operation instructions.

[0241] The execution sequence generation module 406 is used to dynamically weight and cross-sort the actions to be executed in the system prediction queue and the remote operation queue based on the confidence level of the potential behavioral intention, the preset risk level, the current network latency, the action execution time window, and the urgency of the operation instruction, so as to generate the final action execution sequence.

[0242] The execution control module 407 is used to control the health care robot to perform corresponding auxiliary actions according to the final action execution sequence.

[0243] Those skilled in the art will clearly understand that the technical solutions of the embodiments of this application can be implemented by means of software and / or hardware. In this specification, "unit" and "module" refer to software and / or hardware that can independently complete or cooperate with other components to complete a specific function, wherein the hardware may be, for example, a field-programmable gate array (FPGA), an integrated circuit (IC), etc.

[0244] Each processing unit and / or module in the embodiments of this application can be implemented by an analog circuit that implements the functions described in the embodiments of this application, or by software that executes the functions described in the embodiments of this application.

[0245] Please refer to Figure 5, which shows a schematic diagram of the structure of an electronic device according to an embodiment of this application. This electronic device can be used to implement the method in the embodiment shown in Figure 1. As shown in Figure 5, the electronic device 500 may include:

[0246] The system includes at least one processor 501, at least one network interface 504, a user interface 503, a memory 505, and at least one communication bus 502. The communication bus 502 is used to enable connection and communication between the components. The user interface 503 may include buttons, and optionally include a standard wired or wireless interface. The network interface 504 may include, but is not limited to, a Bluetooth module, an NFC module, a Wi-Fi module, etc.

[0247] The processor 501 may include one or more processing cores and connect to various parts within the device 500 via various interfaces and lines. It implements the various functions and data processing of the device 500 by running or executing instructions, programs, code sets, or instruction sets stored in the memory 505, and by accessing data in the memory 505. Optionally, the processor 501 may be implemented using at least one hardware form of DSP, FPGA, or PLA. The processor 501 may also integrate one or more combinations of CPU, GPU, and modem. The CPU is mainly used to handle the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display on the screen; and the modem is used for wireless communication. It is understood that the modem may not be integrated into the processor 501, but may be implemented through a separate chip.

[0248] The memory 505 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 505 includes a non-transitory computer-readable medium that can be used to store instructions, programs, code, code sets, or instruction sets. The memory 505 may be divided into a program storage area and a data storage area, wherein the program storage area can be used to store instructions for implementing an operating system, instructions for implementing at least one function (such as touch function, audio playback function, image playback function, etc.), and instructions for implementing the aforementioned method embodiments; the data storage area can be used to store data involved in the relevant method embodiments. The memory 505 may also be at least one storage device located away from the processor 501. As shown in FIG5, the memory 505, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and program instructions.

[0249] Another embodiment of this application provides a computer-readable storage medium having computer program instructions stored thereon, which can be executed by a processor to implement the methods and / or technical solutions of any one or more embodiments of this application described above.

[0250] The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0251] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0252] Furthermore, it should be noted that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in the system claims may also be implemented by a single unit or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

Claims

1. A remote control method for a health and wellness robot based on artificial intelligence, characterized in that, The method includes the following steps: S1, continuously acquiring the user's video data stream through a visual sensor; S2, extracting micro-motion temporal features representing changes in the user's posture from the video data stream in real time, wherein the micro-motion temporal features include at least changes in body center of gravity, changes in joint angles, and limb movement trends; S3, inputting the micro-motion temporal features into a pre-trained behavioral aura prediction model to predict at least one potential behavioral intention of the user in the future and its corresponding confidence level; S4, receiving a sequence of control instructions from a remote control terminal and parsing it into operation instructions that the robot can execute; S5, establishing a system prediction queue and a remote operation queue, wherein the system prediction queue generates corresponding robot preparatory auxiliary actions based on the potential behavioral intentions and confidence levels, and the remote operation queue is used to cache the operation instructions; S6, based on the confidence level of the potential behavioral intentions, a preset risk level, current network latency, and action execution time... Based on the window and the urgency of the operation instructions, dynamic weight evaluation and cross-sorting are performed on the actions to be executed in the system prediction queue and the remote operation queue to generate a final action execution sequence. Specifically, the dynamic weight evaluation and cross-sorting of the actions to be executed in the system prediction queue and the remote operation queue includes: calculating a priority score for each action to be executed in the queue, where the priority score is a weighted function of the confidence level, the risk level, the urgency, the action execution time window, and a delay penalty factor based on the current network latency; sorting all candidate actions in descending order according to the priority scores; detecting conflicts between the sorted actions in space, time, or robot degrees of freedom; for conflicting actions, conflict resolution is performed based on the difference in their priority scores, and the action with the higher score is executed; S7, according to the final action execution sequence, the health and wellness robot is controlled to perform corresponding auxiliary actions.

2. The remote control method for a health and wellness robot based on artificial intelligence according to claim 1, characterized in that, The behavioral precursor prediction model is a model trained based on a temporal fusion Transformer architecture. It is used to perform multi-step prediction of the multivariate micro-action temporal features and output the confidence interval or uncertainty measure of the potential behavioral intention.

3. The remote control method for a health and wellness robot based on artificial intelligence according to claim 1, characterized in that, During the conflict resolution process, if the difference in priority scores between actions in the system prediction queue and actions in the remote operation queue is less than a preset lag threshold, then the actions in the system prediction queue are executed first, or a confirmation request is sent to the remote operation terminal.

4. The remote control method for a health and wellness robot based on artificial intelligence according to claim 1, characterized in that, The system monitors the network round-trip latency with the remote operating terminal in real time; when the network round-trip latency exceeds a preset safety threshold, it automatically increases the overall weight of actions in the system prediction queue; when communication with the remote operating terminal is interrupted, the system switches to an autonomous mode that executes only high-confidence actions in the system prediction queue.

5. The remote control method for a health and wellness robot based on artificial intelligence according to claim 1, characterized in that, The process of controlling the health and wellness robot to perform corresponding auxiliary actions based on the final action execution sequence specifically includes: mapping high-level action instructions to the robot's low-level action primitives; performing safety checks on collision protection and mechanical torque constraints before executing the low-level action primitives; if the checks fail, replacing the original action with a preset safety backup action.

6. The remote control method for a health and wellness robot based on artificial intelligence according to claim 1, characterized in that, After the cross-sorting, if the instructions in the remote operation queue do not conflict with the actions in the system prediction queue, the instructions and the parts of the actions involving different degrees of freedom of the robot are merged and executed synchronously.

7. The remote control method for a health and wellness robot based on artificial intelligence according to claim 1, characterized in that, It also includes a policy execution mechanism based on the confidence level: when the confidence level is higher than a first threshold, the system is allowed to automatically execute the corresponding robot preparatory assistance action and may interrupt potentially conflicting instructions in the remote operation queue; when the confidence level is between a second threshold and the first threshold, the corresponding robot preparatory assistance action is sent as a suggestion to the remote operation terminal and executed after confirmation; wherein, the first threshold is higher than the second threshold.

8. A remote control system for a health and wellness robot based on artificial intelligence, characterized in that, include: The data acquisition module is used to continuously acquire the user's video data stream through a visual sensor; The system includes a temporal feature extraction module for extracting micro-motion temporal features representing changes in user posture from the video data stream in real time. These micro-motion temporal features include at least changes in body center of gravity, joint angle changes, and limb movement trends. A behavior intention prediction module is used to input the micro-motion temporal features into a pre-trained behavior precursor prediction model to predict at least one potential behavioral intention of the user within a future period and its corresponding confidence level. An instruction parsing module receives a sequence of control instructions from a remote control terminal and parses it into executable operation instructions for the robot. A queue establishment module is used to establish a system prediction queue and a remote operation queue. The system prediction queue generates corresponding robot preparatory auxiliary actions based on the potential behavioral intentions and confidence levels, while the remote operation queue caches the operation instructions. The execution sequence generation module is used to dynamically weight and cross-sort the actions to be executed in the system prediction queue and the remote operation queue based on the confidence level of the potential behavioral intention, the preset risk level, the current network latency, the action execution time window, and the urgency of the operation instruction, to generate a final action execution sequence. Specifically, the dynamic weight evaluation and cross-sorting of the actions to be executed in the system prediction queue and the remote operation queue includes: calculating a priority score for each action to be executed in the queue, where the priority score is a weighted function of the confidence level, the risk level, the urgency, the action execution time window, and a delay penalty factor based on the current network latency; sorting all candidate actions in descending order according to the priority scores; detecting conflicts between the sorted actions in space, time, or robot degrees of freedom; and for conflicting actions, resolving the conflict based on the difference in their priority scores, executing the action with the higher score. The execution control module is used to control the rehabilitation robot to perform corresponding auxiliary actions according to the final action execution sequence.

9. An electronic device, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

Images