Voice control intelligent accompanying robot service method and system

Abstract

The invention discloses a voice control intelligent accompanying robot service method and system, and relates to the technical field of medical care automation, and the method comprises the steps: receiving a patient service demand, and generating an accompanying task instruction; according to the accompanying task instruction, a visual force coupling feedback control system of the robot is controlled to execute accompanying actions; the accompanying action comprises a liquid control action and / or a human body contact action, the visual force coupling feedback control system adopts a binocular camera to obtain a depth image to construct a liquid flow model and a human body biomechanical model, and contact force data is collected in real time through a force sensor array; a force control threshold curve is dynamically generated through a liquid flow model and / or a human body biomechanical model to adjust the output force of the mechanical arm; and transmitting the execution information of the accompanying action and the force control threshold curve to a monitoring terminal. According to the invention, accurate recognition of voice semantics is realized, the problem of accurate control during liquid control and human body contact is solved, and the accuracy, safety and adaptability of nursing service are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical care automation, and in particular to a voice-controlled intelligent accompanying robot service method and system. Background Art

[0002] Automation technology in the field of medical care has significantly changed the traditional medical service model. Especially in the field of intelligent nursing robots, the introduction of voice interaction and automatic service functions has greatly improved the efficiency of nursing. At present, medical institutions generally use intelligent nursing systems based on voice recognition to identify and respond to patient needs, and monitor patient status through visual perception technology. These systems usually use independent voice modules for command recognition, use visual sensors for environmental perception, and perform corresponding nursing actions through robotic arms. In terms of human-computer interaction, existing technologies mainly rely on preset service modes and fixed action sequences, and trigger corresponding service processes through simple voice command word matching.

[0003] However, existing intelligent nursing systems still have significant limitations when dealing with complex service scenarios. First, traditional speech recognition methods often separate semantic features from acoustic features, making it difficult to accurately understand patients' real needs in different situations. Second, existing nursing robots lack an effective feedback mechanism when performing tasks that require precise force control, such as liquid control and human contact, which can easily cause problems such as liquid overflow or improper contact force. In addition, most solutions use a single decision-making model and a fixed execution strategy, and are unable to dynamically adjust their behavior according to different service scenarios, resulting in unstable service quality and difficulty in ensuring safety. Summary of the invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the present invention provides a voice-controlled intelligent companion robot service method and system, which can solve the problems mentioned in the background technology.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: a voice-controlled intelligent accompanying robot service method, comprising: receiving patient service needs and generating accompanying task instructions; according to the accompanying task instructions, controlling the robot's visual force coupling feedback control system to perform accompanying actions; wherein the accompanying actions include liquid control actions and / or human contact actions, the visual force coupling feedback control system uses a binocular camera to acquire depth images to construct a liquid flow model and a human biomechanical model, collects contact force data in real time through a force sensor array, and uses the liquid flow model and / or the human biomechanical model to dynamically generate a force control threshold curve to adjust the output force of the robotic arm; the execution information of the accompanying action and the force control threshold curve are transmitted to the monitoring terminal.

[0007] As a preferred solution of the voice-controlled intelligent accompanying robot service method described in the present invention, receiving patient service needs and generating accompanying task instructions include the following steps: collecting patient voice signals, extracting features of the voice signals to obtain voice feature vectors, and inputting the voice feature vectors into a deep neural network that integrates an attention mechanism to identify service demand types; wherein service demand types include basic care, adverse event handling, vital sign monitoring, and rehabilitation guidance; according to the service demand type, obtaining service scenario parameters through a depth camera; and generating accompanying task instructions containing an action sequence according to the service demand type and the service scenario parameters.

[0008] As a preferred solution of the voice-controlled intelligent accompanying robot service method described in the present invention, the service scenario parameters include the ambient light intensity, the relative position relationship between the robot and the patient, and the patient's posture parameters; the acquisition of the patient's posture parameters includes applying a human skeleton key point extraction algorithm to the human body area in the depth image acquired by the depth camera, dividing the human body into multiple key nodes, calculating the three-dimensional spatial coordinates and connection relationship of the key nodes to obtain a static skeleton structure, continuously collecting multiple frames of images to calculate the displacement and angle change of the key nodes to obtain motion characteristics, and combining the static skeleton structure and the motion characteristics to form the patient's posture parameters; the acquisition of the relative position relationship includes constructing a polar coordinate system between the current position of the robot and the center point of the patient's torso, The radial distance and azimuth of the robot in the polar coordinate system are calculated to obtain the relative position relationship; after obtaining the relative position relationship, a dynamic path planning algorithm is used to generate an obstacle avoidance path in combination with the position information of fixed obstacles in the environment, and the obstacle avoidance path is mapped to the global coordinate system to obtain a service path planning diagram; an association analysis is performed on the patient posture parameters and the service demand type, specifically when the service demand type is basic care or rehabilitation guidance, the motion displacement and angular velocity of the key nodes of the trunk and limbs are calculated; when the service demand type is adverse event handling, the acceleration and tilt angle of the key nodes of the head and trunk in the vertical direction are calculated; when the service demand type is vital signs monitoring, the vibration frequency and amplitude of the key nodes of the chest and abdomen are calculated.

[0009] As a preferred solution of the voice-controlled intelligent accompanying robot service method described in the present invention, wherein: a accompanying task instruction containing an action sequence is generated according to the service demand type and the service scenario parameters, including the following steps: determining the robot motion mode according to the service demand type, specifically when the service demand type is basic care, selecting the position constraint motion mode; when the service demand type is adverse event handling, selecting the emergency approach motion mode; when the service demand type is vital sign monitoring or rehabilitation guidance, selecting the space reservation motion mode; generating a corresponding approach path according to the robot motion mode, wherein the position constraint motion mode generates an approach path constrained by the relative position relationship, the emergency approach motion mode generates an approach path constrained by the shortest distance of the path, and the space reservation motion mode generates an approach path constrained by the patient's activity space reservation; according to the service scenario The execution height is set according to the patient posture parameters in the parameters, specifically, when the static bone structure in the patient posture parameters is horizontally distributed and the vertical coordinate of the trunk center point is less than the sum of the bed height and the human trunk thickness, the execution height is set to the bed height detected in the service path planning diagram; when the static bone structure is vertically distributed and the vertical coordinate of the trunk center point is greater than the sum of the bed height and the human trunk thickness, the execution height is set to the seat height detected in the service path planning diagram; the action sequence is generated based on the robot motion mode and the execution height, specifically, a robot motion trajectory is generated based on the execution height and the robot motion mode, and a light source compensation instruction is inserted at the path turning point and the service execution point of the robot motion trajectory according to the ambient light intensity, and the robot motion trajectory and the light source compensation instruction are combined in time sequence to generate a nursing task instruction containing the action sequence.

[0010] As a preferred solution of the voice-controlled intelligent accompanying robot service method described in the present invention, wherein: according to the accompanying task instruction, the robot's visual force coupling feedback control system is controlled to perform the accompanying action, including the following steps: identifying the type of accompanying task and determining the visual force control strategy; constructing the liquid flow model and the human biomechanical model; collecting force feedback data in real time and performing force control compensation; generating a force control threshold curve; wherein the accompanying task type is obtained by parsing the accompanying task instruction; the accompanying action includes liquid control action and / or human contact action; if it is a liquid control action, a liquid control strategy including a dumping velocity constraint and an angular acceleration constraint is generated based on the liquid surface deformation variable, flow velocity vector and action torque in the liquid flow model; if it is a human contact action, a human contact strategy including a contact force magnitude constraint and a force direction constraint is generated based on the human configuration vector, center of mass position, static stability index and dynamic stability index in the human biomechanical model; if it includes both liquid control action and human contact action, a fusion control strategy that satisfies the dual constraints is generated based on the liquid control strategy and the human contact strategy.

[0011] As a preferred solution of the voice-controlled intelligent accompanying robot service method described in the present invention, wherein: the real-time collection of force feedback data and force control compensation specifically calculates the required compensation force according to the accompanying task instructions, including: when executing the liquid control action, collecting container force data through the force sensor array, and calculating the liquid flow state parameters based on the liquid flow model; wherein the liquid flow state parameters include the liquid level fluctuation amplitude, the liquid center of mass offset and the container wall pressure distribution; the robot arm tipping angular velocity compensation is calculated according to the product of the liquid level fluctuation amplitude and the liquid center of mass offset, and the robot arm acceleration compensation is calculated according to the uniformity of the container wall pressure distribution; when executing the human body contact action, the contact force data is collected through the force sensor array, and the human body stability parameters are calculated based on the human body biomechanical model; wherein the human body stability parameters include the center of gravity projection position, the support polygon area and the angle momentum change trend; calculating the robot contact force compensation amount according to the overlapping area of ​​the center of gravity projection position and the supporting polygon area, and taking the reverse vector of the angular momentum change trend as the robot force compensation direction; when executing the liquid control action and the human body contact action at the same time, weightedly fusion the liquid flow state parameters and the human body stability parameters to obtain the force compensation amount; wherein, the weighted fusion includes: taking the square sum of the liquid level fluctuation amplitude, the liquid center of mass offset and the container wall pressure distribution as the liquid control instability, taking the square sum of the center of gravity projection position and the angular momentum change trend as the human body stability risk, calculating the quotient of the liquid control instability and the human body stability risk as the weight coefficient, multiplying the weight coefficient with the robot arm tipping angular velocity compensation amount, the robot arm acceleration compensation amount, and the robot arm contact force compensation amount, and superimposing the multiplication results to obtain the force compensation amount.

[0012] As a preferred solution of the voice-controlled intelligent companion robot service method described in the present invention, wherein: a force control threshold curve is calculated according to the liquid flow state parameters and the human body stability parameters, including: when executing the liquid control action, the product of the liquid level fluctuation amplitude and the liquid center of mass offset is used as the torque factor, and the upper and lower limits of the liquid force are calculated according to the torque factor as the first force control range; when executing the human body contact action, the overlapping area of ​​the center of gravity projection position and the supporting polygon area is used as the stability factor, and the angular momentum change trend is used as the dynamic compensation factor, and the upper and lower limits of the human body supporting force are calculated according to the stability factor and the dynamic compensation factor as the second force control range; when executing the liquid control action and the human body contact action at the same time, the overlapping part of the first force control range and the second force control range is compared, and the upper and lower limits of the overlapping interval are selected as the final force control threshold curve, and the output force of the robotic arm is adjusted according to the force control threshold curve.

[0013] To further solve the above technical problems, the present invention provides the following technical solutions: a voice-controlled intelligent accompanying robot service system, comprising: a service demand analysis module, used to receive patient service needs and generate accompanying task instructions; an action execution control module, used to control the robot's visual force coupling feedback control system to perform accompanying actions according to the accompanying task instructions; wherein the accompanying actions include liquid control actions and / or human contact actions, the visual force coupling feedback control system uses a binocular camera to acquire depth images to construct a liquid flow model and a human biomechanical model, collects contact force data in real time through a force sensor array, and uses the liquid flow model and / or the human biomechanical model to dynamically generate a force control threshold curve to adjust the output force of the robotic arm; an execution information transmission module is used to transmit the execution information of the accompanying action and the force control threshold curve to the monitoring terminal.

[0014] A computer device includes a memory and a processor, wherein the memory stores a computer program, and is characterized in that when the processor executes the computer program, the steps of the voice-controlled intelligent companion robot service method as described above are implemented.

[0015] A computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of the voice-controlled intelligent companion robot service method as described above are implemented.

[0016] Beneficial effects of the present invention: The present invention realizes adaptive weight allocation of semantic features and acoustic features by introducing a fusion attention mechanism in the voice interaction link, overcomes the recognition bias caused by feature segmentation processing in traditional methods, and can more accurately understand the service needs of patients in different situations. In terms of environmental perception, the present invention performs multi-dimensional fusion analysis on ambient light intensity, relative position relationship and patient posture parameters, and adopts differentiated parameter collection strategies according to different service types, so as to realize comprehensive perception and prediction of service scenes. In terms of execution control, the present invention constructs a dual-model system of liquid flow model and human biomechanics model, and realizes precise adjustment of liquid control and human contact through visual force coupling feedback control mechanism. This control mechanism can not only handle single-type tasks separately, but also realize the synergy of the two models through the dynamic generation of force control threshold curves in complex tasks, ensuring the smooth control of liquid while maintaining human stability. In addition, the present invention adopts a hierarchical decision-making method to generate accompanying task instructions, and improves the adaptability and safety of the system in complex service environments through the combined optimization of motion mode selection and execution height setting, as well as dynamic obstacle avoidance based on service path planning. The organic combination of these technical features enables the present invention to provide more intelligent and reliable nursing services for unattended wards. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0018] Figure 1 This is a schematic diagram of the overall process of a voice-controlled intelligent accompanying robot service method proposed by the present invention;

[0019] Figure 2 This is a schematic diagram of the overall structure of a voice-controlled intelligent companion robot service system proposed by the present invention;

[0020] Figure 3 This is a diagram of computer equipment in a voice-controlled intelligent companion robot service method proposed by the present invention. DETAILED DESCRIPTION

[0021] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0022] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0023] Example 1, reference Figure 1 , which is an embodiment of the present invention, provides a voice-controlled intelligent companion robot service method.

[0024] Figure 1 The overall process diagram of a voice-controlled intelligent accompanying robot service method is shown, including the following steps:

[0025] S1: Receive patient service requirements and generate accompanying task instructions.

[0026] S1.1: Collect the patient’s voice signal, extract the features of the voice signal to obtain a voice feature vector, and input the voice feature vector into a deep neural network integrated with an attention mechanism to identify the type of service demand.

[0027] Specifically, the acquisition of speech feature vectors includes: framing and windowing the speech signal, extracting acoustic features including fundamental frequency parameters, resonance peak parameters and energy parameters, and semantic features including word vectors and part-of-speech tagging information.

[0028] The fusion attention mechanism includes semantic attention weights and acoustic attention weights. Among them, the weight calculation of the speech feature vector includes: calculating the cosine similarity of the word vectors between adjacent words to obtain the semantic relevance, calculating the semantic attention weight based on the semantic relevance, calculating the covariance matrix eigenvalues ​​of the fundamental frequency parameters, resonance peak parameters and energy parameters to obtain the acoustic feature distribution, and calculating the acoustic attention weight based on the acoustic feature distribution. In this embodiment, the deep neural network of the fusion attention mechanism adopts the currently widely used Transformer architecture, specifically including a multi-head self-attention layer and a feedforward neural network layer. In terms of speech processing, this embodiment uses the standard MFCC (Mel Frequency Cepstral Coefficient) feature extraction method to obtain acoustic features, and uses the Word2Vec model to extract word vectors as semantic features.

[0029] The identification of service demand types includes: assigning the product of semantic attention weight and acoustic attention weight as fusion weight to speech feature vector, inputting weighted speech feature vector into deep neural network for multi-classification operation, and mapping the operation result into one of four service demand types: basic care, adverse event handling, vital signs monitoring and rehabilitation guidance. The present invention adopts standard deep neural network multi-classification architecture to identify service demand types. The last layer of the network uses Softmax activation function to output a four-dimensional vector, and each dimension corresponds to the probability value of a service demand type. Specifically, a standard demand dictionary is first constructed, which contains keywords and phrases in various service scenarios. For basic care type, it includes voice instructions related to daily care such as "pouring water" and "infusion"; for adverse event handling, it includes voice instructions related to emergency situations such as "falling" and "discomfort"; for vital signs monitoring, it includes voice instructions related to monitoring such as "measurement" and "inspection"; for rehabilitation guidance, it includes voice instructions related to rehabilitation such as "exercise" and "activity". The voice input is matched with the demand dictionary, and the probability distribution of each type is calculated by neural network in combination with acoustic features such as voice pitch and speech speed. The category with the highest probability exceeding 0.85 can be selected as the final service demand type. When the highest probability value is lower than 0.85, the patient is asked to restate the demand through voice feedback to ensure the accuracy of service type recognition.

[0030] It should be noted that the four types of service requirements in the present invention are all identified based on sensor data that can be objectively detected and quantified. For basic care needs, the acoustic features (such as fundamental frequency, sound intensity and other physical quantities) and semantic features (such as numerical features such as word vector distance) of the speech signal are analyzed for identification. For example, when the patient issues a voice command of "help me pour a glass of water", the fundamental frequency features (the frequency range is usually 80-250Hz) and sound intensity features (normal speaking intensity is 50-70dB) of the speech signal are first extracted, and the word vector of the keyword "pour water" in the speech is calculated at the same time, and the cosine similarity is calculated with the standard word vector in the service demand dictionary. When the similarity is greater than 0.9 and the speech features are within the normal conversation range, the demand is identified as a pouring water service in the basic care type. The need for adverse event handling is triggered by analyzing the kinematic parameters (such as position, velocity, acceleration and other physical quantities) of the key points of the human skeleton in the depth image. For example, the spatial position of the center point of the patient's trunk is tracked in real time by a depth camera. When the vertical speed of the center point of the trunk is detected to exceed 1.5m / s, and the angle between the trunk and the ground changes sharply from a vertical state (about 90 degrees) to a horizontal state (about 0 degrees), it is determined to be a fall risk event. At the same time, the relative acceleration of the key point of the head relative to the center point of the trunk is calculated. When the acceleration exceeds the gravitational acceleration g, the occurrence of the fall event is further confirmed. The need for vital sign monitoring is based on observable periodic motion characteristics in the image sequence (such as objective physical quantities such as chest and abdominal displacement caused by breathing). For example, by analyzing the periodic motion of the chest and abdominal regions in the image sequence, the system extracts breathing-related physiological parameters. Specifically, the key points of the sternum and abdomen are selected, and their displacement changes in the vertical direction are calculated. During normal breathing, these key points will show periodic ups and downs of about 12-20 times / minute, with a displacement amplitude of about 0.5-1.5cm. The system extracts the frequency and amplitude characteristics of this periodic motion through Fourier transform to evaluate the respiratory state. The rehabilitation guidance needs are evaluated by analyzing the kinematic parameters of the human motion chain composed of skeletal key points and mathematical indicators such as the Euclidean distance between the standard action template. For example: when conducting upper limb rehabilitation training, the system first establishes a motion chain model containing shoulder, elbow, and wrist key points. For the standard arm-raising movement, the shoulder joint is required to perform 0-180 degrees of flexion and extension in the sagittal plane, while the elbow joint remains straight (joint angle greater than 160 degrees). The Euclidean distance between the angle value of each joint and the standard template during the patient's actual movement is calculated in real time. When the distance value is less than 15 degrees, it is judged as a standard action, and when it is greater than 30 degrees, it is judged to need adjustment, and voice correction guidance is generated accordingly.

[0031] S1.2: According to the service requirement type, obtain service scenario parameters through the depth camera.

[0032] Among them, the service scenario parameters include ambient light intensity, the relative position relationship between the robot and the patient, and the patient posture parameters. Specifically, the acquisition of ambient light intensity includes: using a deep image processing algorithm to analyze the image captured by the depth camera, and calculating the ambient light intensity through the image grayscale value distribution. It should be noted that the deep image processing in this embodiment uses mature computer vision algorithms, such as Faster R-CNN for human body detection and OpenPose for skeletal key point detection. The present invention uses these open source frameworks and pre-trained models as a basis, and fine-tunes the nursing scene data to adapt to the actual application environment.

[0033] The acquisition of patient posture parameters includes: applying the human skeleton key point extraction algorithm to the human body area in the depth image, dividing the human body into 25 key nodes including the head, trunk, and limbs, calculating the three-dimensional spatial coordinates and connection relationship of the key nodes to obtain the static bone structure, continuously collecting multiple frames of images to calculate the displacement and angle change of the key nodes to obtain the motion characteristics, and combining the static bone structure and motion characteristics to form the patient posture parameters; performing correlation analysis on the patient posture parameters and the service demand type: when the service demand type is basic care or rehabilitation guidance, calculating the motion displacement and angular velocity of the key nodes of the trunk and limbs; when the service demand type is adverse event handling , calculate the acceleration and tilt angle of the key nodes of the head and torso in the vertical direction; when the service demand type is vital signs monitoring, calculate the vibration frequency and amplitude of the key nodes of the chest and abdomen; the acquisition of the relative position relationship includes: constructing a polar coordinate system with the current position of the robot and the center point of the patient's torso, and calculating the radial distance and azimuth of the robot in the polar coordinate system to obtain the relative position relationship; after obtaining the relative position relationship, generate a service path planning map based on the relative position relationship, specifically, combine the position information of fixed obstacles in the environment, use a dynamic path planning algorithm to generate an obstacle avoidance path, and map the obstacle avoidance path to the global coordinate system to obtain the service path planning map.

[0034] It should be noted that in the present invention, dynamic path planning uses the classic RRT (rapid random expansion tree) algorithm for basic path search. First, the environment is modeled as a two-dimensional grid map, and the state of each grid can be idle, occupied by a fixed obstacle, or occupied by a dynamic object. The position of fixed obstacles (such as beds, tables, chairs, etc.) is acquired by a depth camera and marked in the grid map. On this basis, the center point of the patient's torso is taken as the polar coordinate origin, the current position of the robot is taken as the path starting point, and the target service position is taken as the end point. The RRT algorithm gradually constructs a search tree from the starting point to the target point by randomly sampling expansion nodes in the free space. For each sampling point, the algorithm checks whether the path from the nearest node to the point intersects with an obstacle. After the path is generated, the path is smoothed using a B-spline curve to ensure the continuity of the robot's motion. Finally, the smoothed path is mapped from the polar coordinate system to the global Cartesian coordinate system with the fixed reference point of the room as the origin through coordinate transformation to form the final service path planning diagram. This RRT-based path planning method can effectively handle path search problems in complex environments and has a high planning success rate and execution efficiency.

[0035] Preferably, the present invention integrates the ambient light intensity, relative position relationship and patient posture parameters in multiple dimensions, and adopts differentiated parameter collection strategies according to different service demand types, so as to overcome the defects of the prior art that the environmental perception and human body monitoring are often separated. In the process of acquiring the patient's posture parameters, the present invention not only considers the static skeletal structure, but also introduces the motion feature analysis, and designs specific focus points for different service types, such as focusing on motion displacement and angular velocity in basic care, focusing on acceleration and tilt angle in adverse event handling, and focusing on vibration frequency and amplitude in vital sign monitoring. This multi-level parameter acquisition and analysis method enables the present invention to understand the service scenario more comprehensively and predict possible situations in advance. For example, when performing infusion services, the system can not only accurately locate the patient's position, but also monitor the patient's subtle movement changes in real time, adjust the service strategy in time, and improve the safety and adaptability of the service. At the same time, the relative position relationship expression method based on the polar coordinate system, combined with the dynamic path planning algorithm, can better handle the obstacle avoidance problem in a complex environment and ensure the safety and efficiency of the service path.

[0036] S1.3: Generate a care task instruction including an action sequence according to the service demand type and service scenario parameters.

[0037] Specifically, the generation of escort task instructions is divided into determining the robot's motion mode and setting the execution height.

[0038] Firstly, the robot motion mode is determined according to the service demand type. When the service demand type is basic care, the position constraint motion mode is selected; when the service demand type is adverse event handling, the emergency approach motion mode is selected; when the service demand type is vital sign monitoring or rehabilitation guidance, the space reservation motion mode is selected; according to the robot motion mode, the corresponding approach path is generated. The position constraint motion mode generates an approach path constrained by the relative position relationship, the emergency approach motion mode generates an approach path constrained by the shortest distance of the path, and the space reservation motion mode generates an approach path constrained by the patient's activity space reservation.

[0039] Secondly, the execution height is set according to the patient posture parameters in the service scenario parameters. When the static bone structure in the patient posture parameters is horizontally distributed and the vertical coordinate of the trunk center point is less than the sum of the bed height and the human trunk thickness, the execution height is set to the bed height detected in the service path planning map; when the static bone structure is vertically distributed and the vertical coordinate of the trunk center point is greater than the sum of the bed height and the human trunk thickness, the execution height is set to the seat height detected in the service path planning map.

[0040] Finally, an action sequence is generated based on the robot motion mode and execution height, specifically including: generating the robot motion trajectory based on the execution height and the robot motion mode, and inserting light source compensation instructions at the path turning points and service execution points of the robot motion trajectory according to the ambient light intensity, and combining the robot motion trajectory and the light source compensation instructions in time sequence to generate an accompanying task instruction containing an action sequence.

[0041] Preferably, the present invention adopts a hierarchical decision-making method when generating accompanying task instructions, first determining the motion mode, then setting the execution height, and finally generating the action sequence. This hierarchical decision-making structure overcomes the problem that the existing technology often adopts a single decision-making method, resulting in insufficient flexibility of service. In particular, in the selection of motion modes, different constraints are adopted according to the type of service requirements, such as position constraints, shortest path constraints and space reservation constraints. This differentiated constraint mechanism can better adapt to the needs of different service scenarios. For example, when dealing with emergencies such as falls, the shortest path is preferred to approach the patient; while when conducting rehabilitation guidance, sufficient activity space is reserved. In addition, the present invention also combines ambient light compensation with motion planning, and dynamically adjusts the light source compensation at the path turning point and the service execution point, which not only ensures the visual perception quality of the service process, but also improves the accuracy of the service. The generation process of the entire action sequence takes into account the constraints of multiple dimensions, which can achieve more intelligent and humanized service interaction, and significantly improve the quality and reliability of the accompanying service.

[0042] S2: According to the accompanying task instructions, control the robot's visual force coupling feedback control system to perform the accompanying action.

[0043] Among them, the accompanying actions include liquid control actions and / or human contact actions. The visual force coupling feedback control system uses a binocular camera to obtain depth images to construct a liquid flow model and a human biomechanical model, and collects contact force data in real time through a force sensor array. The liquid flow model and / or the human biomechanical model are used to dynamically generate a force control threshold curve to adjust the output force of the robotic arm.

[0044] S2.1: Identify the type of caregiving task and determine the visual force control strategy.

[0045] Specifically, the task type is obtained by parsing the accompanying task instruction, wherein the accompanying action includes a liquid control action and / or a human body contact action.

[0046] If it is a liquid control action, a liquid control strategy including tipping velocity constraints and angular acceleration constraints is generated based on the liquid surface deformation, flow velocity vector and action torque in the liquid flow model; if it is a human contact action, a human contact strategy including contact force magnitude constraints and force direction constraints is generated based on the human body configuration vector, center of mass position, static stability index and dynamic stability index in the human biomechanical model; if it includes both liquid control action and human body contact action, a fusion control strategy that meets the dual constraints is generated based on both the liquid control strategy and the human body contact strategy.

[0047] For example, when the accompanying task is "pouring service", parameters are extracted from the liquid flow model for control. Specifically, by analyzing the liquid surface deformation, it is found that when the water cup is tilted at an angle of more than 30 degrees, the liquid surface will fluctuate significantly, and the flow velocity vector shows that the liquid has a tendency to overflow. At this time, according to the change in the torque, the pouring speed is constrained within the range of 0.1-0.2 rad / s, and the angular acceleration is limited to 0.05 rad / s 2Below, ensure the smooth flow of liquid. When the accompanying task is "supporting the patient", control is performed according to the human biomechanical model. For example, the human body configuration vector detects that the patient is in a state of preparing to stand up (the hip joint bending angle is about 100 degrees), and the center of mass position shows a trend of the center of gravity moving forward. At this time, the static stability index (the distance between the center of gravity projection and the support polygon) is less than 10cm, and the dynamic stability index shows that the angular momentum change rate increases. Based on this, the contact force control strategy is generated: the vertical support force is controlled within 30% of the patient's weight, the horizontal thrust is kept below 50N, and the direction of the force is always along the movement trend of the patient's trunk. In the "assisting eating" task, it is necessary to control the pouring of the spoon and the support of the patient's head at the same time. Based on the liquid flow model, the flow state of the liquid in the spoon is detected, and the pouring speed is controlled below 0.05rad / s; at the same time, the human biomechanical model monitors the patient's head stability index to ensure that the support force for the head is kept within the range of 20-30N. The two sets of constraints are integrated to ensure that the spoon can be poured smoothly while supporting the head.

[0048] Compared with the prior art, traditional nursing robots usually adopt preset fixed action modes and lack detailed identification and differentiated processing of task types. The present invention determines the specific action type by parsing the task instructions, and adopts different control strategies based on different action types, thereby achieving more accurate task execution. When processing complex tasks, the present invention can simultaneously consider the constraints of liquid control and human contact, and generate a fusion control strategy that meets the dual constraints. This strategy generation mechanism not only improves the accuracy of task execution, but also provides a clear optimization target for subsequent force control compensation, effectively avoiding the problem of unstable service quality caused by the single control strategy in traditional methods.

[0049] S2.2: Construct fluid flow models and human biomechanics models.

[0050] Specifically, the construction of the liquid flow model includes: using a binocular camera to obtain a depth image of the liquid in the container, extracting a liquid surface edge contour point set based on the image grayscale gradient and region growing algorithm, and calculating the contour point displacement between adjacent frames to obtain the liquid flow velocity vector; establishing a corresponding relationship between the inclination angle of the liquid container and the liquid surface deformation according to the contour point set and the flow velocity vector, calculating the torque acting on the container wall by the liquid surface tension, and combining the liquid surface deformation, flow velocity vector and torque to construct the liquid flow model.

[0051] The construction of the human biomechanical model includes: detecting the position of human joints based on depth images, modeling the human body as a multi-link motion chain structure, in which the position and angle parameters of each joint constitute the human body configuration vector; calculating the relative position and posture matrix of each link through the DH parameter method, and calculating the center of mass position and inertia tensor of the entire motion chain based on the mass distribution parameters of various parts of the human body; establishing the dynamic equation of the zero moment point based on the center of mass position, calculating the relationship between the projection of the center of mass of the human body and the supporting polygon to obtain the static stability index, and at the same time obtaining the dynamic stability index based on the rate of change of the human angular momentum, and combining the human body configuration vector, center of mass position, static stability index and dynamic stability index to construct the human biomechanical model.

[0052] Preferably, traditional nursing robots either focus only on liquid control or only on human contact, and few systems can handle these two types of tasks at the same time. The present invention uses deep image processing technology to construct a liquid flow model and a human biomechanics model respectively, and incorporates multiple key parameters into each model. In the liquid flow model, by analyzing the temporal changes of the liquid surface contour point set, not only can the flow trend of the liquid be predicted, but also the stability of the liquid can be evaluated by the torque of the surface tension. This modeling method based on physical properties can more accurately predict and control liquid behavior compared to traditional pure visual feedback control. Similarly, in the human biomechanics model, the present invention adopts a multi-link motion chain structure and realizes an accurate description of human motion through the DH parameter method. In particular, the zero-moment point dynamic equation is introduced to consider static stability and dynamic stability in a unified manner. Such a comprehensive modeling method enables the system to better understand and predict the human motion state, thereby providing safer and more reliable contact services.

[0053] S2.3: Collect force feedback data in real time and perform force control compensation.

[0054] Specifically, the required compensation force is calculated according to the escort task instruction. When executing the liquid control action, the force data of the container is collected through the force sensor array, and the liquid flow state parameters are calculated based on the liquid flow model. The liquid flow state parameters include the liquid surface fluctuation amplitude, the liquid center of mass offset and the container wall pressure distribution; the robot arm tipping angular velocity compensation is calculated according to the product of the liquid surface fluctuation amplitude and the liquid center of mass offset, and the robot arm acceleration compensation is calculated according to the uniformity of the container wall pressure distribution.

[0055] It should be noted that the liquid level fluctuation amplitude refers to the height change of the liquid surface contour line extracted from the image sequence captured by the depth camera. The liquid level fluctuation amplitude can be obtained based on the edge detection and target tracking technology in computer vision. The liquid center of mass offset refers to the position offset of the liquid center of mass relative to the center of the container obtained by the center of mass calculation formula through the liquid surface shape and the container inclination. The calculation of the liquid center of mass offset is based on the basic principles of fluid statics. The container wall pressure distribution refers to the pressure data directly measured by the force sensor array, which is a basic mechanical measurement parameter.

[0056] When performing human contact actions, contact force data is collected through the force sensor array, and the human body stability parameters are calculated based on the human biomechanical model, where the human body stability parameters include the center of gravity projection position, the support polygon area and the angular momentum change trend; the robot arm contact force compensation amount is calculated according to the overlapping area between the center of gravity projection position and the support polygon area, and the reverse vector of the angular momentum change trend is used as the robot arm force compensation direction.

[0057] It should be noted that the center of gravity projection position refers to the center of gravity calculated by the mass distribution and spatial position of each part of the human body, and then projected to the ground. The calculation of the center of gravity projection position is a basic calculation method in human biomechanics. The support polygon area refers to the support area obtained by constructing a convex hull by detecting the contact points between the human body and the ground (such as the position of the feet). The support polygon area is a standard parameter in gait analysis. The angular momentum change trend refers to the calculation of the time derivative of angular momentum by the change of the human body configuration between consecutive frames, which is based on the angular momentum theorem in classical mechanics. Exemplarily, in human biomechanics, the angular momentum change trend is an important parameter to describe the rotational motion state of the human body. The angular momentum L of the human body can be expressed as: L = I × ω, where I is the moment of inertia of the human body and ω is the angular velocity. The angular momentum change trend dL / dt represents the derivative of angular momentum with respect to time, reflecting the change of the rotational motion of the human body. Take the scenario of assisting a patient as an example: when the patient gets up from the bed, the human biomechanical model will detect the change in angular momentum throughout the process: first, the patient's upper body leans forward, and the human body generates positive angular momentum around the hip joint; then, during the standing process, due to the upward shift of the center of gravity, the human body may tend to lean back, generating negative angular momentum. If a sudden increase in angular momentum is detected, it means that the patient may be unbalanced. At this time, the robot needs to apply a supporting force in the opposite direction of the change in angular momentum, that is: when the patient leans forward (positive angular momentum), the robot applies a backward supporting force on the patient's chest; when the patient leans back (negative angular momentum), the robot applies a forward supporting force on the patient's back. This force control based on the trend of angular momentum changes can effectively prevent patients from falling.

[0058] When executing liquid control actions and human body contact actions at the same time, the liquid flow state parameters and human body stability parameters are weightedly fused, wherein the weighted fusion includes: taking the square sum of the liquid level fluctuation amplitude, the liquid center of mass offset and the container wall pressure distribution as the liquid control instability, taking the square sum of the center of gravity projection position and the angular momentum change trend as the human body stability risk, calculating the quotient of the liquid control instability and the human body stability risk as the weight coefficient, multiplying the weight coefficient with the robot arm tipping angular velocity compensation, the robot arm acceleration compensation, and the robot arm contact force compensation, respectively, and superimposing the multiplication results to obtain the final force compensation.

[0059] Preferably, the traditional force control system usually adopts a single force feedback loop, which is difficult to cope with complex nursing scenarios. The present invention designs a composite force control compensation mechanism and designs special compensation strategies for liquid control and human contact respectively. In terms of liquid control, by analyzing the amplitude of liquid surface fluctuation and the center of mass offset, the present invention can accurately calculate the required angular velocity and acceleration compensation to effectively prevent liquid overflow. In terms of human contact, by analyzing the projection position of the center of gravity and the trend of angular momentum change, the present invention can timely adjust the size and direction of the contact force to ensure the safety of the service process. Especially when dealing with complex tasks, the present invention proposes a weighted fusion method based on instability and danger, and realizes a dynamic balance of the two compensation requirements by calculating their quotient as the weight coefficient. This compensation mechanism not only improves the adaptability of the system, but also significantly reduces the risk in the service process.

[0060] S2.4: Generate a force control threshold curve.

[0061] Specifically, a force control threshold curve is calculated based on liquid flow state parameters and human body stability parameters, wherein: when executing a liquid control action, the product of the liquid surface fluctuation amplitude and the liquid center of mass offset is used as a torque factor, and the upper and lower limits of the liquid force are calculated based on the torque factor as a first force control range; when executing a human body contact action, the overlapping area of ​​the center of gravity projection position and the supporting polygon area is used as a stability factor, and the angular momentum change trend is used as a dynamic compensation factor, and the upper and lower limits of the human body supporting force are calculated based on the stability factor and the dynamic compensation factor as a second force control range; when executing a liquid control action and a human body contact action at the same time, the overlapping parts of the first force control range and the second force control range are compared, and the upper and lower limits of the overlapping interval are selected as the final force control threshold curve, and the output force of the robotic arm is adjusted according to the force control threshold curve.

[0062] It should be noted that for the calculation of the first force control range: the torque factor reflects the movement state of the liquid in the container, and it comprehensively considers the forces exerted on the container by the liquid level fluctuation and the center of mass shift. Specifically, when the liquid level fluctuates greatly and the center of mass shifts significantly, the torque factor increases, which means that the liquid is more likely to overflow; conversely, when the liquid level is stable and the center of mass is centered, the torque factor is small and the liquid is in a stable state. The present invention determines the range of force that the robot arm needs to apply when performing the pouring action, that is, the first force control range, by establishing a corresponding relationship between the torque factor and the stability of the container.

[0063] For the calculation of the second force control range: the stability factor and the dynamic compensation factor jointly determine the stability of the human body posture. The stability factor reflects the stability of the human body's center of gravity position. When the overlapping area of ​​the center of gravity projection position and the supporting polygon is large, it indicates that the human body posture is relatively stable; the dynamic compensation factor reflects the movement trend of the human body and predicts the possible instability direction through the change of angular momentum. The present invention determines the support force range that the robot arm needs to provide, that is, the second force control range, based on the combination of these two factors. This dual constraint based on static stability and dynamic trend ensures the rationality and safety of the support force.

[0064] Preferably, conventional systems usually use fixed force control thresholds, which are difficult to adapt to dynamically changing service environments. The present invention establishes a complete force control range calculation system by introducing torque factors, stability factors, and dynamic compensation factors. When processing complex tasks, the present invention calculates the overlapping intervals of two force control ranges to find an optimal force control interval that can simultaneously meet liquid stability and human safety. This dynamic threshold generation mechanism enables the present invention to minimize risks while ensuring service quality. For example, when it is detected that the liquid is about to overflow, the present invention will appropriately shrink the force control range to ensure more cautious actions; when it is detected that the human body posture is unstable, the present invention will adjust the force control range accordingly to provide more reliable support.

[0065] S3: Transmitting the execution information of the accompanying action and the force control threshold curve to the monitoring terminal.

[0066] Specifically, the transmission of execution information includes: real-time collection of liquid flow state parameters and human body stability parameters as state monitoring data, comparing the output force of the robot arm with the force control threshold curve as force control execution data, and generating an accompanying action execution report based on the state monitoring data and the force control execution data; at the end of each accompanying action execution cycle, the accompanying action execution report and the force control threshold curve are sent to the monitoring terminal for medical staff to evaluate service quality and analyze possible abnormal situations.

[0067] In summary, the present invention realizes the adaptive weight allocation of semantic features and acoustic features by introducing a fusion attention mechanism in the voice interaction link, overcomes the recognition bias caused by feature segmentation processing in traditional methods, and can more accurately understand the service needs of patients in different situations. In terms of environmental perception, the present invention performs multi-dimensional fusion analysis of ambient light intensity, relative position relationship and patient posture parameters, and adopts differentiated parameter collection strategies according to different service types, so as to realize comprehensive perception and prediction of service scenes. In terms of execution control, the present invention constructs a dual-model system of liquid flow model and human biomechanics model, and realizes precise adjustment of liquid control and human contact through visual force coupling feedback control mechanism. This control mechanism can not only handle single-type tasks separately, but also realize the synergy of the two models through the dynamic generation of force control threshold curves in complex tasks, ensuring the smooth control of liquid while maintaining human stability. In addition, the present invention adopts a hierarchical decision-making method to generate accompanying task instructions, and improves the adaptability and safety of the system in complex service environments through the combined optimization of motion mode selection and execution height setting, as well as dynamic obstacle avoidance based on service path planning. The organic combination of these technical features enables the present invention to provide more intelligent and reliable nursing services for unattended wards.

[0068] Example 2, reference Figure 2 , as an embodiment of the present invention, provides a voice-controlled intelligent companion robot service system.

[0069] like Figure 2 The figure shows the overall structure of the system, which also shows the interaction process of each module, including:

[0070] The service demand analysis module is used to receive patient service demands and generate accompanying task instructions;

[0071] An action execution control module is used to control the robot's visual force coupling feedback control system to perform accompanying actions according to accompanying task instructions; wherein the accompanying actions include liquid control actions and / or human contact actions, and the visual force coupling feedback control system uses a binocular camera to obtain depth images to construct a liquid flow model and a human biomechanics model, collects contact force data in real time through a force sensor array, and uses the liquid flow model and / or the human biomechanics model to dynamically generate a force control threshold curve to adjust the output force of the robot arm;

[0072] The execution information transmission module is used to transmit the execution information of the accompanying action and the force control threshold curve to the monitoring terminal.

[0073] Example 3, reference Figure 3, is an embodiment of the present invention, which is different from the previous embodiment in that: if the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0074] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0075] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0076] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0077] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A voice-controlled intelligent companion robot service method, characterized in that: include: Receive patient service requirements and generate accompanying task instructions; According to the escort task instruction, the visual force coupling feedback control system of the robot is controlled to perform the escort action; wherein the escort action includes a liquid control action and / or a human body contact action, and the visual force coupling feedback control system uses a binocular camera to obtain a depth image to construct a liquid flow model and a human body biomechanics model, collects contact force data in real time through a force sensor array, and uses the liquid flow model and / or the human body biomechanics model to dynamically generate a force control threshold curve to adjust the output force of the robot arm; The execution information of the accompanying action and the force control threshold curve are transmitted to the monitoring terminal.

2. The voice-controlled intelligent accompanying robot service method according to claim 1, characterized in that: Receiving patient service requirements and generating accompanying task instructions includes the following steps: Collecting patient voice signals, extracting features from the voice signals to obtain voice feature vectors, and inputting the voice feature vectors into a deep neural network fused with an attention mechanism to identify service demand types; wherein the service demand types include basic care, adverse event handling, vital sign monitoring, and rehabilitation guidance; According to the service requirement type, obtaining service scenario parameters through a depth camera; Generate a care task instruction including an action sequence according to the service demand type and the service scenario parameters.

3. The voice-controlled intelligent accompanying robot service method according to claim 2, characterized in that: The service scenario parameters include ambient light intensity, the relative position relationship between the robot and the patient, and the patient's posture parameters; The acquisition of the patient posture parameters includes applying a human skeleton key point extraction algorithm to the human body area in the depth image acquired by the depth camera, dividing the human body into a plurality of key nodes, calculating the three-dimensional spatial coordinates and connection relationships of the key nodes to obtain a static skeleton structure, continuously collecting multiple frames of images to calculate the displacement and angle change of the key nodes to obtain motion features, and combining the static skeleton structure and the motion features to form the patient posture parameters; The acquisition of the relative position relationship includes constructing a polar coordinate system with the current position of the robot and the center point of the patient's torso, and calculating the radial distance and azimuth of the robot in the polar coordinate system to obtain the relative position relationship; after obtaining the relative position relationship, combining the position information of fixed obstacles in the environment, using a dynamic path planning algorithm to generate an obstacle avoidance path, and mapping the obstacle avoidance path to a global coordinate system to obtain a service path planning diagram; Performing correlation analysis on the patient posture parameters and the service demand type, specifically, when the service demand type is basic care or rehabilitation guidance, calculating the motion displacement and angular velocity of the key nodes of the trunk and limbs; When the service requirement type is adverse event handling, the acceleration and tilt angle of the key nodes of the head and torso in the vertical direction are calculated; when the service requirement type is vital sign monitoring, the vibration frequency and amplitude of the key nodes of the chest and abdomen are calculated.

4. The voice-controlled intelligent accompanying robot service method according to claim 3, characterized in that: Generating a care task instruction including an action sequence according to the service requirement type and the service scenario parameter comprises the following steps: Determine the robot motion mode according to the service demand type, specifically, when the service demand type is basic care, select the position constraint motion mode; when the service demand type is adverse event handling, select the emergency approach motion mode; When the service demand type is vital sign monitoring or rehabilitation guidance, a space reservation movement mode is selected; Generate a corresponding approach path according to the robot motion mode, wherein the position constraint motion mode generates an approach path constrained by the relative position relationship, the emergency approach motion mode generates an approach path constrained by the shortest path distance, and the space reservation motion mode generates an approach path constrained by patient activity space reservation; The execution height is set according to the patient posture parameters in the service scenario parameters, specifically, when the static bone structure in the patient posture parameters is horizontally distributed and the vertical coordinate of the trunk center point is less than the sum of the bed height and the human trunk thickness, the execution height is set to the bed height detected in the service path planning map; when the static bone structure is vertically distributed and the vertical coordinate of the trunk center point is greater than the sum of the bed height and the human trunk thickness, the execution height is set to the seat height detected in the service path planning map; The action sequence is generated based on the robot motion mode and the execution height, specifically, the robot motion trajectory is generated based on the execution height and the robot motion mode, and light source compensation instructions are inserted at the path turning points and service execution points of the robot motion trajectory according to the ambient light intensity, and the robot motion trajectory and the light source compensation instructions are combined in time sequence to generate an accompanying task instruction containing the action sequence.

5. The voice-controlled intelligent accompanying robot service method according to claim 4, characterized in that: According to the escort task instruction, the visual force coupling feedback control system of the robot is controlled to perform the escort action, including the following steps: Identify the type of caregiving tasks and determine visual force control strategies; Constructing the liquid flow model and the human body biomechanics model; Collect force feedback data in real time and perform force control compensation; Generate force control threshold curve; The escort task type is obtained by parsing the escort task instruction; the escort action includes a liquid control action and / or a human body contact action; If it is a liquid control action, a liquid control strategy including a dumping velocity constraint and an angular acceleration constraint is generated based on the liquid surface deformation, the flow velocity vector and the action torque in the liquid flow model; If it is a human contact action, a human contact strategy including a contact force magnitude constraint and a force action direction constraint is generated based on the human body configuration vector, the center of mass position, the static stability index and the dynamic stability index in the human biomechanics model; If both the liquid control action and the human body contact action are included, a fusion control strategy that satisfies the dual constraints is generated based on the liquid control strategy and the human body contact strategy.

6. The voice-controlled intelligent accompanying robot service method according to claim 5, characterized in that: The real-time collection of force feedback data and the force control compensation are specifically to calculate the required compensation force according to the accompanying task instruction, including: When executing the liquid control action, the force data of the container is collected through the force sensor array, and the liquid flow state parameters are calculated based on the liquid flow model; wherein the liquid flow state parameters include the liquid level fluctuation amplitude, the liquid center of mass offset and the container wall pressure distribution; the robot arm tipping angular velocity compensation amount is calculated according to the product of the liquid level fluctuation amplitude and the liquid center of mass offset, and the robot arm acceleration compensation amount is calculated according to the uniformity of the container wall pressure distribution; When performing the human body contact action, the contact force data is collected through the force sensor array, and the human body stability parameters are calculated based on the human body biomechanics model; wherein the human body stability parameters include the center of gravity projection position, the support polygon area and the angular momentum change trend; the mechanical arm contact force compensation amount is calculated according to the overlapping area of ​​the center of gravity projection position and the support polygon area, and the reverse vector of the angular momentum change trend is used as the mechanical arm force compensation direction; When the liquid control action and the human body contact action are performed simultaneously, the liquid flow state parameter and the human body stability parameter are weightedly fused to obtain a force compensation amount; Among them, the weighted fusion includes: taking the liquid level fluctuation amplitude, the liquid center of mass offset and the square sum of the container wall pressure distribution as the liquid control instability, taking the center of gravity projection position and the square sum of the angular momentum change trend as the human body stability risk, calculating the quotient of the liquid control instability and the human body stability risk as the weight coefficient, multiplying the weight coefficient with the robot arm tipping angular velocity compensation, the robot arm acceleration compensation, and the robot arm contact force compensation respectively, and superimposing the multiplication results to obtain the force compensation.

7. The voice-controlled intelligent accompanying robot service method according to claim 6, characterized in that: Calculating a force control threshold curve according to the liquid flow state parameter and the human body stability parameter includes: When executing the liquid control action, the product of the liquid level fluctuation amplitude and the liquid mass center offset is used as a torque factor, and the upper and lower limits of the liquid force are calculated according to the torque factor as a first force control range; When performing the human body contact action, the overlapping area of ​​the center of gravity projection position and the supporting polygon area is used as a stability factor, the angular momentum change trend is used as a dynamic compensation factor, and the upper and lower limits of the human body support force are calculated according to the stability factor and the dynamic compensation factor as a second force control range; When the liquid control action and the human body contact action are executed simultaneously, the overlapping parts of the first force control range and the second force control range are compared, and the upper and lower limits of the overlapping interval are selected as the final force control threshold curve, and the output force of the robotic arm is adjusted according to the force control threshold curve.

8. A voice-controlled intelligent accompanying robot service system, based on the voice-controlled intelligent accompanying robot service method according to any one of claims 1 to 7, characterized in that: include, The service demand analysis module is used to receive patient service demands and generate accompanying task instructions; An action execution control module is used to control the robot's visual force coupling feedback control system to perform an accompanying action according to the accompanying task instruction; wherein the accompanying action includes a liquid control action and / or a human body contact action, and the visual force coupling feedback control system uses a binocular camera to obtain a depth image to construct a liquid flow model and a human body biomechanics model, collects contact force data in real time through a force sensor array, and uses the liquid flow model and / or the human body biomechanics model to dynamically generate a force control threshold curve to adjust the output force of the robot arm; The execution information transmission module is used to transmit the execution information of the accompanying action and the force control threshold curve to the monitoring terminal.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the voice-controlled intelligent companion robot service method described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the voice-controlled intelligent companion robot service method described in any one of claims 1 to 7 are implemented.

Images