Control method for support robot, support robot, electronic device, computer storage medium, and computer program product

Abstract

A control method for a support robot, the method comprising: in response to recognizing that a target object has an intent to require support, controlling a support robot to move toward the target object, and when the support robot moves to a target movement position, controlling a robot arm to move to a target joint position, wherein the target movement position and the target joint position are both determined on the basis of the relative position between the target object and a target sensor; and in the process of the robot arm moving to the target joint position or after the robot arm moves to the target joint position, when it is detected by means of a tactile sensor that an external force is applied to the robot arm, performing compliance control on the support robot. Further provided are a support robot, an electronic device, a computer storage medium, and a computer program.

Description

Control methods for assistive robots, assistive robots, electronic devices, computer storage media, and computer program products

[0001] Cross-reference to related applications

[0002] This application is based on and claims priority to Chinese Patent Application No. 202410163810.2, filed on February 2, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the fields of robotics and artificial intelligence, and in particular to a control method for a assisted robot, the assisted robot itself, electronic equipment, computer storage media, and computer programs. Background Technology

[0004] In daily life, assistive devices are widely used in various scenarios, such as daily rehabilitation, to enhance balance, provide support, and improve stability when users are standing or walking. Currently, commonly used assistive devices are generally canes or other simple walking aids, and the level of intelligence in these devices needs to be improved. Summary of the Invention

[0005] The following is an overview of the subject matter described in detail in this application. This overview is not intended to limit the scope of the claims.

[0006] This application provides a control method for a assisted robot, the assisted robot itself, an electronic device, a computer storage medium, and a computer program product, which can automatically perform assisted tasks, improving the accuracy and efficiency of the assisted robot in performing assisted tasks.

[0007] On one hand, embodiments of this application provide a control method for an assistive robot, the method being executed by an electronic device. The assistive robot is equipped with a robotic arm and a target sensor for object perception. The robotic arm is equipped with a tactile sensor. The control method includes:

[0008] In response to the detection that the target object needs assistance, the robot is controlled to move toward the target object. When the robot moves to the target moving position, the robotic arm is controlled to move to the target joint position. The target moving position and the target joint position are both based on the relative position between the target object and the target sensor.

[0009] During or after the robotic arm moves to the target joint position, when the tactile sensor detects that an external force has been applied to the robotic arm, the assisting robot is subjected to compliant control.

[0010] On the other hand, this application provides a support robot, which is equipped with a control module, a robotic arm, and a target sensor for object perception. The robotic arm is equipped with a tactile sensor.

[0011] The control module is used to respond to the recognition that the target object needs assistance, control the assistance robot to move towards the target object, control the robotic arm to move to the target joint position when the assistance robot moves to the target moving position, and control the robotic arm to move to the target joint position during or after the robotic arm moves to the target joint position, when the tactile sensor detects that the robotic arm is subjected to external force, perform compliant control on the assistance robot.

[0012] The target movement position and the target joint position are both determined by the relative position change between the target object and the target sensor.

[0013] On the other hand, embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described control method for the assisted robot.

[0014] On the other hand, embodiments of this application also provide a computer-readable storage medium storing a computer program, which is executed by a processor to implement the above-described control method for the assisted robot.

[0015] On the other hand, embodiments of this application also provide a computer program product, which includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the control method for implementing the aforementioned assisted robot.

[0016] The embodiments of this application include at least the following beneficial effects: In response to the recognition that a target object needs assistance, the assistance robot is controlled to move towards the target object. When the assistance robot moves to the target movement position, the robotic arm is controlled to move to the target joint position. Both the target movement position and the target joint position are based on the relative position changes between the target object and the target sensor, thereby enabling the perception of the target object's state and automatically performing the assistance task according to the target object's state. Furthermore, during or after the robotic arm moves to the target joint position, when the tactile sensor detects that the robotic arm is subjected to external force, compliant control is applied to the robotic arm, allowing it to follow the target object's posture to perform the assistance task, thereby improving the comfort of the target object during the assistance process. Therefore, the control method provided in this application can interact with the target object in diverse ways, enabling the assistance robot to automatically perform assistance tasks, improving the accuracy and efficiency of the assistance robot's assistance task execution.

[0017] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the application. Attached Figure Description

[0018] The accompanying drawings are used to provide a further understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0019] Figure 1 is a schematic diagram of an implementation environment provided in an embodiment of this application;

[0020] Figure 2 is a flowchart of a control method for a assisted robot provided in an embodiment of this application;

[0021] Figure 3 is a schematic diagram of the target object to be assisted provided in an embodiment of this application;

[0022] Figure 4 is a first schematic diagram of an embodiment of this application, showing that the target object has the intention to need assistance.

[0023] Figure 5 is a second schematic diagram provided by an embodiment of this application for identifying that the target object has the intention to need assistance;

[0024] Figure 6 is a third schematic diagram provided by an embodiment of this application for identifying that the target object has the intention to need assistance;

[0025] Figure 7 is a fourth schematic diagram provided by an embodiment of this application for identifying that the target object has the intention to need assistance;

[0026] Figure 8 is a schematic diagram of the position transformation relationship provided in an embodiment of this application;

[0027] Figure 9 is a schematic diagram of the data processing of pose data obtained by pose sensor measurement according to an embodiment of this application;

[0028] Figure 10 is a schematic diagram of the center point location provided in an embodiment of this application;

[0029] Figure 11 is a schematic diagram showing the transformation of the coordinate system of the joints of the assistive robot provided in this embodiment to the coordinate system of the tactile unit;

[0030] Figure 12 is a schematic diagram of the compliance control provided in an embodiment of this application;

[0031] Figure 13 is a schematic diagram of obstacle avoidance by the assisting robot provided in an embodiment of this application;

[0032] Figure 14 is a schematic diagram of a first overall process of the control method provided in an embodiment of this application;

[0033] Figure 15 is a schematic diagram of a second overall process of the control method provided in an embodiment of this application;

[0034] Figure 16 is a schematic diagram of a third overall process of the control method provided in an embodiment of this application;

[0035] Figure 17 is a schematic diagram of the first structure of the assisting robot provided in an embodiment of this application;

[0036] Figure 18 is a schematic diagram of the second structure of the assisting robot provided in an embodiment of this application;

[0037] Figure 19 is a structural schematic diagram of the assisting robot provided in an embodiment of this application from another perspective;

[0038] Figure 20 is a schematic diagram of the internal structure of the control cabinet of the assistive robot provided in the embodiment of this application;

[0039] Figure 21 is a schematic diagram of the structural connection of the assisting robot provided in the embodiment of this application. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0041] It should be noted that in various specific embodiments of this application, when processing data related to the characteristics of the target object, such as target object attribute information or attribute information sets, is required, the permission or consent of the target object will be obtained first. Furthermore, the collection, use, and processing of this data will comply with relevant laws, regulations, and standards. The target object can be a user. In addition, when embodiments of this application need to obtain target object attribute information, separate permission or consent from the target object will be obtained through pop-ups or redirection to a confirmation page. Only after obtaining the target object's separate permission or consent will the necessary target object-related data for the normal operation of the embodiments of this application be obtained.

[0042] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0043] To facilitate understanding of the technical solutions provided in the embodiments of this application, some key terms used in the embodiments of this application will be explained below:

[0044] 1) Target sensor: A device for detecting and measuring the physical quantities, state, position, and other parameters of a specific target or object. These sensors are typically integrated into automated systems, robots, and various detection devices to acquire and identify target information.

[0045] 2) Tactile sensors: Sensors used in robots to mimic tactile functions. They can be classified by function into contact sensors, force-torque sensors, pressure sensors, and slip sensors, etc.

[0046] 3) Pose: This describes the position and orientation of an object (such as a coordinate) in a specified coordinate system. For example, pose is often used to describe the position and orientation of a robot in a spatial coordinate system. Position refers to the location of a rigid body in space. The position of a rigid body can be represented by a 3×1 matrix, that is, the position of the center of the rigid body coordinate system in the base coordinate system. Orientation refers to the orientation of a rigid body in space. The orientation of a rigid body can be represented by a 3×3 matrix, that is, the orientation of the rigid body coordinate system in the base coordinate system.

[0047] 4) Pose sensor: A sensor used to detect and measure the position and attitude of an object in space. It can provide three-dimensional position information (such as x, y, z coordinates) and attitude information (such as pitch angle, roll angle, and yaw angle).

[0048] In daily life, assistive devices are widely used in various scenarios, such as daily rehabilitation, to enhance balance, provide support, and improve stability when standing or walking. Currently, commonly used assistive devices are generally canes or other simple walking aids, and the level of intelligence in these devices needs improvement.

[0049] To address the aforementioned issues, this application provides a control method for a assisted robot, the assisted robot itself, and electronic equipment, enabling the assisted robot to automatically perform assisted tasks and improving the accuracy and efficiency of its assisted task execution.

[0050] Referring to Figure 1, which is a schematic diagram of an implementation environment provided by an embodiment of this application, the implementation environment includes a control module for a assisted robot. The control module is capable of communicating with sensors installed on the assisted robot. After obtaining sensor data from the sensors, the control module can control the assisted robot to work based on the obtained sensor data.

[0051] For example, if the control module identifies that the target object intends to be helped through the obtained sensor data, the control module can control the help robot to move towards the target object based on the obtained sensor data. When the help robot moves to the target movement position, the control module can control the robotic arm to move to the target joint position. The target movement position and the target joint position are both based on the relative position change between the target object and the target sensor, thereby sensing the state of the target object and automatically performing the help task according to the state of the target object. On this basis, during or after the robotic arm moves to the target joint position, when the control module senses that the robotic arm where the tactile sensor is located is subjected to external force, it can obtain the interaction force data detected by the tactile sensor and perform compliant control on the help robot based on the interaction force data, so that the help robot can follow the posture of the target object, thereby improving the comfort of the help. It can be seen that the control method provided by this application can perform diverse interactions with the target object, enabling the help robot to automatically perform the help task, improving the accuracy and efficiency of the help robot in performing the help task.

[0052] In addition, the control module can communicate with the server, which will update the algorithm package corresponding to the control method to the control module in real time.

[0053] In some embodiments, the control module is located in a terminal device, which may be mounted on the robot, or the terminal device may be connected to a communication module on the robot via a short-range communication technology (e.g., ZigBee, Bluetooth, and Wi-Fi). Alternatively, the robot and the terminal device may be connected via a wired connection (e.g., the robot may have a serial communication COM interface or USB interface, a data cable may be inserted into the interface and connected to the terminal device).

[0054] For example, sensors on the robot collect environmental information and send it to a terminal device. The terminal device then sends the environmental information to a server via the network. The server determines the robot's motion control commands based on the environmental information of the robot's environment. The server sends the motion control commands to the terminal device, which then sends the motion control commands to the robot. The robot then executes the corresponding actions based on the motion control commands.

[0055] Figure 2 is a flowchart of a control method for a assisted robot provided in an embodiment of this application. The control method for the assisted robot can be executed by the control module of the assisted robot, or it can be executed by the control module of the assisted robot in cooperation with the server. In this embodiment of the application, the control method of the assisted robot is described as being executed by the control module of the assisted robot. The control method of the assisted robot includes, but is not limited to, the following steps 201 to 202.

[0056] Step 201: In response to the recognition that the target object has the intention to need assistance, control the assistance robot to move towards the target object. When the assistance robot moves to the target movement position, control the robotic arm to move to the target joint position.

[0057] In one possible implementation, the target object can refer to a user object that the assistance robot can perceive and that intends to receive assistance. This intention can be the target object's desire for help from the assistance robot, such as an object within the robot's computer vision range or an object linked to a terminal via biometric information. The assistance robot can perceive and identify the target object through its installed target sensors and assess whether the perceived target object needs assistance. For the assistance robot, the target user's intention to provide assistance can manifest in various ways, such as through voice, body posture, physical interaction, or signal commands. The specific method can be based on the perception provided by the assistance robot's target sensors.

[0058] In one possible implementation, the target movement position is a location within a preset range around the target object. This preset range can be set according to the actual needs of the target object. For example, based on the target object's height and arm span, the target object's arm movement range (a circular area centered on the target object with the arm span as its diameter) is determined, and this arm movement range is used as the preset range for the target movement position.

[0059] The target movement position can refer to the location that the assistive robot needs to reach to complete the task of assisting the target object, such as approaching the target object that intends to be assisted, or the target movement position can refer to the position of the assistive robot relative to the target object. The target joint position can refer to the specific posture or joint angle that the robotic arm of the assistive robot needs to achieve to complete the task of assisting the target object, or the target joint position can refer to the position of the robotic arm relative to the target object. Both the target movement position and the target joint position change based on the relative position between the target sensor and the target object. Essentially, as the assistive robot moves towards the target object, it can perceive the state of the target object in real time through the target sensor, correct the relative position between the target sensor and the target object, and thus adjust the target movement position and target joint position in real time. This allows it to automatically adjust the assistance task to better suit the current state of the target object, improving the comfort of the assistance.

[0060] In one possible implementation, during the process of the assisting robot performing the assistance task for the target object, since the state of the target object changes in real time, the target movement position and target joint position can be adjusted according to the different intentions of the target object to provide assistance. For example, the assistance posture (i.e., target movement position and target joint position) required by the assisting robot for two different assistance needs, namely assisting the target object to walk and assisting the target object to stand, are different. By automatically sensing the state of the target object, the target movement position and target joint position of the assisting robot can be adjusted in real time to provide different interaction methods, movement characteristics and assistance methods, and conduct diversified interactions with the target object, so that the assisting robot can automatically perform the assistance task, thereby improving the accuracy and efficiency of the assisting robot in performing the assistance task.

[0061] In one possible implementation, referring to Figure 3, which is a schematic diagram of a target object to be assisted according to an embodiment of this application, when the assisting robot needs to assist the target object to walk, the target movement position of the assisting robot can be located to the side and rear of the target object. During the assistance process, the state of the target object is sensed in real time, and the target movement position is updated synchronously with the movement of the target object, so that the assisting robot continues to move towards the target object, maintaining the relative position stability between the target sensor and the target object, and achieving the effect of the assisting robot following the target object to provide assistance. In addition, the target joint position can be adjusted in real time so that the assistance point of the robotic arm is maintained at the waist, hip, and wrist of the target object, mimicking the human assistance posture to guide the target object's walking direction and maintain the walking rhythm, thereby improving the comfort of assistance.

[0062] In one possible implementation, when the assistive robot needs to help a target object stand up, the robot's target movement position can be located in front of the target object, and this position is fixed relative to the target object to help maintain balance. That is, the assistive robot moves towards the target object and remains stable after reaching the target movement position. During the assistance process, the robot senses the target object's state in real time and adjusts the target joint positions to maintain the robotic arm's assistance point on the target object's upper body, such as the waist and arms, to support the target object's weight, allowing the target object to change from a sitting or squatting position to a standing position.

[0063] In one possible implementation, the intention of the target object to require assistance can be determined by sensing the state of the target object, i.e., the relative position between the target object and the target sensor. The relative position between the target object and the target sensor can include the relative positions of the target object's center of gravity, joints, and torso with the target sensor. For example, if the target object's center of gravity is at a low position relative to the target sensor and does not move significantly in consecutive frames, it can be assumed that the target object is attempting to stand up, thus determining that the target object has the intention to require assistance to stand up. If the target object's center of gravity changes continuously relative to the target sensor in consecutive frames, and the leg joints have a continuous displacement in a certain direction relative to the target sensor, it can be assumed that the object is walking, thus determining that the target object has the intention to require assistance to walk.

[0064] Step 202: During or after the robotic arm moves to the target joint position, when the tactile sensor detects that an external force has been applied to the robotic arm, the assisting robot is subjected to compliant control.

[0065] For example, compliance control obtains control signals from force sensors and uses these signals to control the robot, causing it to move in response to changes in those signals. Compliance is divided into two categories: active compliance and passive compliance. Passive compliance control involves a robot using auxiliary compliance mechanisms to naturally conform to external forces when in contact with its environment. Active compliance control involves a robot actively controlling forces using force feedback information and specific control strategies.

[0066] In one possible implementation, when the robotic arm moves to the target joint position, or after the robotic arm has moved to the target joint position, and a tactile sensor detects that an external force has been applied to the robotic arm, it can be considered that the target object is in contact with the assisting robot. Therefore, by implementing compliant control, the assisting robot can respond appropriately according to the movement and posture of the target object. For example, the assisting robot can be controlled to maintain a fixed output force unaffected by external pressure, or it can be controlled to change its position and speed according to a preset impedance characteristic in response to external force, thus maintaining a gentle interaction with the target object. Alternatively, it can provide feedback on touch or contact while satisfying a specific posture. Specifically, the compliant control of the assisting robot can be achieved by adjusting the parameters of all joints of the assisting robot (including the upper and lower limbs), such as adjusting the target joint position, target movement position, and the movement speed and acceleration of each joint, to reduce the occurrence of impacts or excessive pressure on the target object during the assistance process, thereby improving the comfort of assistance.

[0067] In one possible implementation, during the assistance process, the robotic arm of the assistance robot can be compliantly controlled. Specifically, parameters such as torque, position, speed, and acceleration of each joint of the robotic arm can be adjusted to change the movement trajectory and position of the robotic arm. For example, when a tactile sensor detects that an external force is applied to the robotic arm, the torque output of the corresponding joint can be adjusted in real time according to the location and direction of the applied force to maintain the posture of the target object or to interact gently with the target object. At the same time, the dynamic impedance parameters of the corresponding joints can be increased to make the robotic arm's movements smoother and slower, conforming to the posture of the target object. In addition to compliant control of the robotic arm, the assistance robot can also be made compliant throughout its entire body. By adjusting all the joints of the assistance robot to conform to the posture of the target object, the assistance robot can better mimic the characteristics of human limb movement, improving comfort during the assistance process. For example, when an assistive robot is a wheeled robot with a robotic arm, it can adjust the degrees of freedom of the omnidirectional wheels (such as the forward and backward distances and rotation angles) and the acceleration of the omnidirectional wheels to coordinate with the compliant movements of the robotic arm when the tactile sensors detect that an external force is applied to the robotic arm. When an assistive robot is a legged robot with a robotic arm, it can adjust the height between the waist and feet by adjusting the joint controllers between the feet and waist when the tactile sensors detect that an external force is applied to the robotic arm, thus coordinating with the compliant movements of the robotic arm. Compliant movements of a robotic arm refer to its ability to adjust its trajectory, speed, or force in real time based on the force and position of the contact when it comes into contact with the external environment or object, in order to adapt to changes in the external environment and object, achieving a smooth and precise interaction effect.

[0068] In one possible implementation, the target sensors can be of various types, specifically including vision sensors and pose sensors. The pose sensor can detect the spatial position and orientation of the target object, as well as the spatial position and orientation of the assistive robot. When the pose sensor detects a change in the position of the assistive robot relative to the target object, it can adjust the target movement position of the assistive robot in real time to achieve or maintain the desired relative pose. Therefore, the target movement position can be determined based on the relative position between the target object and the pose sensor. The vision sensor can identify the target object and its posture. When the relative posture (i.e., the relative position) between the target object and the vision sensor changes, the movement position of the assistive robot's robotic arm can be adjusted in real time to respond to the change in the target object's posture. Therefore, the target joint position is determined based on the relative position between the target object and the vision sensor.

[0069] In one possible implementation, at least one vision sensor and a pose sensor are provided. By integrating data from multiple sensors, the measurement errors of individual sensors can be compensated for, improving the accuracy and reliability of object perception. For example, the pose sensor may include an inertial measurement unit, LiDAR, ultrasonic sensors, etc., so the relative position between the target object and the pose sensor can be determined by integrating the relative positions of each pose sensor with respect to the target object. Specifically, by collecting sensor data from each pose sensor at the same time, filtering out noise from all sensor data, and then using a pre-trained neural network model to fuse and analyze all sensor data, the relative position between the target object and the pose sensor is determined. Correspondingly, the vision sensor may include a depth camera, a stereo camera, an infrared camera, etc. By integrating multiple vision sensors, the three-dimensional pose of the target object can be determined more accurately, thereby determining the relative position between the vision sensor and the target object.

[0070] In one possible implementation, the assistive robot can be equipped with visual sensors, such as cameras. The terminal can use these sensors to perceive and identify people within the computer vision range and recognize the posture of the perceived target object. The visual sensors can capture real-time image data of the robot's environment and analyze the data to determine if the target object is present. Specifically, human posture recognition algorithms, such as the Mediapipe framework, can be used to identify human figures in the image data, pinpointing the location of skeletal points. This allows for keypoint detection of the target object. After identifying these keypoints, posture recognition can be performed to determine the target object's posture and whether it intends to provide assistance. The human posture recognition algorithm can be selected based on the sampling frequency of the visual sensor, choosing algorithms with different response rates and complexities. For example, if the video stream frame rate of the visual sensor is 30 frames per second, a model with moderate complexity can be selected to recognize human postures, ensuring that each video node identifies human postures at a frequency of 28Hz, thus achieving real-time human recognition of the current image data from the visual sensor. If the robot detects that the target object does not intend to require assistance, it can continue to perform the current assistance task or continue to perform posture recognition on the target object to determine whether it intends to require assistance.

[0071] For example, Computer Vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes for target recognition and measurement, and then performs image processing to create images more suitable for human observation or transmission to instruments for detection. In this embodiment, a visual sensor performs key point detection and pose recognition of the target object on the image data, improving the control efficiency of the assisted robot.

[0072] Referring to Figure 4, which is a schematic diagram of identifying a target object's intention to require assistance according to an embodiment of this application, the terminal can deploy a pre-trained posture estimation network model. After obtaining image data captured by the visual sensor, the terminal can extract features from the image data to obtain first image data. The first image data is then input into the posture estimation network model to obtain human body key points. Based on the human body key points, the current posture of the target object 401 can be determined. The current posture of the target object is then matched with the posture of the person requiring assistance. If the current posture matches the posture, the terminal can consider that the target object intends to require assistance. As shown in Figure 4, the posture requiring assistance can be a user's body posture such as half-squatting, sitting, walking, or a specific gesture. It should be noted that if there are multiple user objects in the image data, the posture of each user object can be identified, and the user object whose current posture matches the posture requiring assistance is identified as the target object.

[0073] In addition, historical image data captured by the visual sensor at the previous moment can be acquired. Feature extraction is performed on the historical image data at the previous moment to obtain the second image data. The first image data and the second image data are stitched together to obtain the third image data. The third image data is input into the pose estimation network model to obtain the target key points. The estimated pose of the target object is determined based on the target key points. By combining the action changes in consecutive frames, a more accurate object pose can be obtained. If the estimated pose matches the object that needs assistance, the terminal can consider that the target object has the intention to need assistance.

[0074] In one possible implementation, the assisting robot can also be equipped with acoustic sensors, such as microphones. The terminal can use the acoustic sensors on the assisting robot to perceive the voice information emitted by the target object, and perform voice recognition to determine whether the voice information indicates an intention to ask for assistance.

[0075] In some embodiments, prior to step 201, the following processes are performed: detecting the speech information of the target object; performing feature extraction processing on the speech information to obtain speech features; performing classification processing based on the speech features to obtain a predicted type of the speech features; and determining that the target object has an intention to need assistance in response to the predicted type indicating that the target object has an intention to need assistance.

[0076] For example, the voice information of the target object can be detected by an acoustic sensor installed in the assistance robot. The acoustic sensor converts the user's voice from a voice signal to an electrical signal, and then from an electrical signal to an analog signal. The voice information is represented as an analog signal. The voice information is processed by a neural network model, which can be achieved as follows: the analog voice signal is converted into a digital signal through sampling and quantization. Features are extracted from the digitized voice signal. Common feature extraction methods include Mel-frequency cepstral coefficients (MFCC), filter banks, and spectral features. The neural network model can be a convolutional neural network or a recurrent neural network. The gating unit in the neural network model classifies the voice features to obtain the predicted type of the voice features. The predicted type includes: the target object intends to need assistance, and the target object does not intend to need assistance. When the predicted type is that the target object intends to need assistance, step 201 is executed, that is, the assistance robot is controlled to assist the user.

[0077] For example, referring to Figure 5, which is a schematic diagram of identifying the intention of a target object to require assistance according to an embodiment of this application, when a user says a sentence with the semantic meaning of needing assistance, such as "Please help me to the room" or "I need assistance," it can be considered that the target object has the intention to need assistance, so as to control the assistance robot to perform the assistance task. For example, in Figure 5, the assistance robot 502 recognizes the user's intention based on speech recognition. When the target object 501 says "Please help me to the room," in response to recognizing the intention of the target object 501 to need assistance, the assistance robot 502 performs the assistance task.

[0078] Artificial intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In this embodiment, a neural network model is used to detect the user's voice information and determine whether the user intends to receive assistance based on that information, thereby improving the robot's control efficiency.

[0079] In this embodiment, by detecting whether a user needs assistance through voice, the convenience of controlling the robot can be improved, the robot's response speed can be increased, the robot can be made easier for users to use, and the versatility of the robot in practical application scenarios can be enhanced.

[0080] In one possible implementation, referring to Figure 6, which is a schematic diagram of identifying a target object's intention to require assistance according to an embodiment of this application, the target object 601's hand is placed on the robotic arm of robot 602. The terminal can sense the externally applied interaction force through the tactile sensor set on the robotic arm of the assistance robot. As shown in Figure 6, if the terminal detects an external force applied to the robotic arm through the tactile sensor while not controlling the assistance robot to perform the assistance task, the object with the smallest relative distance to the target sensor can be taken as the target object, and it is considered that the target object has the intention to require assistance. Alternatively, the terminal can call the visual sensor (if the assistance robot is also equipped with a visual sensor such as a camera) to perceive and identify the posture of each object within the computer vision range, and take the object that meets the posture to be assisted as the target object, and consider the target object to have the intention to require assistance.

[0081] In one possible implementation, referring to Figure 7, which is a schematic diagram of identifying the intention of a target object to need assistance provided in an embodiment of this application, the assistance robot may be equipped with a help button or a remote controller that can be arranged separately from the assistance robot. The terminal can sense the signal generated when the help button or the remote controller is triggered, and then it can be considered that the target object that triggered the help button or the remote controller has the intention to need assistance.

[0082] In some embodiments, prior to step 201, the following processing is performed: in response to receiving a first signal for the assistance robot, the first signal is detected and processed to obtain a detection result; in response to the detection result indicating that the first signal was issued by a pre-configured remote controller, it is determined that the target object has the intention to require assistance.

[0083] For example, the first signal is emitted by a remote controller that is detachably arranged from the assisting robot. The first signal carries the device identifier of the remote controller. The terminal in the assisting robot parses the first signal to obtain the device identifier of the remote controller. In response to the presence of a pre-configured device identifier of the remote controller in the first signal, it is determined that the first signal is used to indicate that the target object has the intention to need assistance, and step 201 is executed.

[0084] For example, in Figure 7, the target object 701 holds a remote controller 703, and the target object 701 sends a signal by triggering the remote controller 703. The receiver in the robot 702 receives the signal and moves towards the target object 701 based on the signal. After moving to the target position, the robot 702 performs the task of assisting the target object 701.

[0085] Alternatively, the terminal can use visual sensors (if the robot is also equipped with visual sensors such as cameras) to perceive and identify the postures of various objects within the computer vision range, select objects that meet the posture to be assisted as target objects, and assume that the target objects have the intention to need assistance.

[0086] In this embodiment, a remote controller can control the robot's status information according to user needs, enabling the robot to adjust and react in a timely manner and ensuring that the robot works as expected. Controlling the robot through a specific remote controller reduces the difficulty of operating the robot for users and improves the robot's versatility.

[0087] In one possible implementation, object-related sensor data is captured by a target sensor. For example, feature extraction and stitching are performed using image data captured by a vision sensor and pose data detected by a pose sensor to obtain multi-source sensor data. This multi-source sensor data is then imported into a pre-trained deep learning model, such as a convolutional neural network model, or a real-time object detection algorithm, such as YOLO (You Only Look Once) or SSD (Single Shot Multi-Bo10 Detector). This allows for the rapid detection of human objects in sensor data (such as images) and estimation of their positions. Pose estimation models, such as OpenPose or AlphaPose based on human keypoint detection, can be used to analyze the poses of the human objects in the sensor data, analyzing their behavioral patterns and intentions to determine if they intend to request assistance. When an intention to request assistance is identified, a dynamic path planning algorithm can be used to calculate the optimal path for the assistance robot to reach the target location, controlling the robot to move towards the object requiring assistance. The target sensor can detect not only human objects but also non-human objects, i.e., obstacles. For example, it uses LiDAR and vision sensors to perceive the surrounding environment of the assistance robot. During movement, the target sensor can detect obstacles along the path to achieve obstacle avoidance. Simultaneously, it uses the target sensor to perceive the target object in real time, correct the relative distance between the target object and the sensor, and adjust the target object's movement position. After the assistance robot reaches the target movement position, the robotic arm can be controlled to move to the human support point, i.e., the target joint position. At the same time, it senses the triggering status of the tactile sensors on the robotic arm and the relative distance between the target sensor and the target object, determines the target object's state, and adjusts the position and force of the robotic arm in real time to adapt to the target object's movement and balance.

[0088] Specifically, the assistive robot and its control method provided in this application embodiment can be applied to various human-computer interaction scenarios such as medical rehabilitation, daily care, public assistance, and rescue and disaster relief. The application scenarios are wide-ranging. For example, in hospitals or rehabilitation centers, the assistive robot can help patients or the elderly with posture changes such as walking, standing, and sitting. Because it can sense the state of the target object, it can adjust the assistance plan in real time according to the target object's state and needs, improving the comfort of assistance while assisting the object's rehabilitation and reducing the workload of physical therapists. Another example is controlling the assistive robot to detect the elderly's activity status in real time, helping them perform various exercises indoors or outdoors (such as going up and down stairs, standing, and walking) and providing timely assistance in emergencies (such as helping them up after a fall). Yet another example is in public places such as stations or shopping malls, helping passengers or customers with mobility difficulties reach their destinations. Specifically, in a station scenario, it can assist passengers with mobility difficulties from the waiting area to the side of the bus door and help them board the bus, or help them sit down inside the bus. In a shopping mall scenario, it can assist customers up the steps of an escalator.

[0089] In one possible implementation, during the process of controlling the assisted robot to move toward the target object, the current target pose data of the pose sensor can be acquired, and the target pose data can be transformed based on a preset first transformation matrix to obtain the target movement position; the assisted robot can then be controlled to move toward the target object based on the target movement position.

[0090] In one possible implementation, the pose sensor can detect the human pose data of the target object. This target pose data can be the human pose data measured by the pose sensor at the current moment, or it can be calculated based on the measured human pose data. The preset first transformation matrix can be a pose transformation matrix that transforms the human spatial coordinates to the spatial coordinates of the assisting robot. This matrix represents the pose transformation relationship between the target movement position of the assisting robot and the position of the target object. Thus, by transforming the spatial coordinates of the target object, the desired position of the assisting robot relative to the target object, i.e., the target movement position, is obtained. This allows the assisting robot to be controlled to move to the target movement position near the target object to perform the assistance task.

[0091] In one possible implementation, multiple first candidate transformation matrices can be pre-set to adapt to different postures or different assistance needs of the target object. For example, different transformation matrices can be selected based on the movement mode and posture of the target object to determine the corresponding target movement position of the assistance robot, ensuring the safety and comfort of assistance. After determining the target movement position, during the process of controlling the assistance robot to move towards the target object based on the target movement position, the pose data of the target object can be detected in real time, and the target movement position of the assistance robot can be updated in real time based on the first transformation matrix. Alternatively, the pose data of the target object can be detected in real time to determine the movement mode and posture of the target object, and the first transformation matrix can be updated in real time to adjust the target movement position of the assistance robot.

[0092] Referring to Figure 8, which is a schematic diagram of position transformation relationship provided in an embodiment of this application, the target pose data of the target object at the current moment can be detected by a pose sensor. From the target pose data, the state data of the target object, namely its position and orientation, can be extracted, such as the position coordinates (x, y) of the target object relative to the assisted robot. h y h , 1), and the angle θ of the target object relative to the assisting robot. h Assuming the target object's need for assistance is to walk, the first transformation matrix corresponding to walking can be determined from multiple first candidate transformation matrices, thus based on the preset first transformation matrix [0.92, 0, 1]. T By transforming the position of the target object, the target movement position of the assisting robot is obtained. Therefore, the assisting robot can be controlled to move from its original position towards the target object and to the target movement position, thereby performing the task of assisting the target object. Specifically, the target movement position (x...) base y base The calculation process for 1) is shown in the following formula:

[0093] Therefore, it can be seen that the target movement position of the assisting robot is related to the position of the target object relative to the assisting robot. As the assisting robot moves continuously, the target pose data detected by the pose sensor changes continuously, and the determined target movement position also changes accordingly. That is, the target movement position of the assisting robot changes with the relative position between the target object and the pose sensor. It should be noted that the target movement position can be determined based on the assisting robot's own spatial coordinate system. Therefore, although the target movement position changes continuously, the target movement position at each moment can be relatively fixed relative to the world coordinate system or the coordinate system of the target object.

[0094] In one possible implementation, the target pose data can be obtained by integrating pose data measured by the pose sensor at multiple moments. Specifically, the first pose data and the first covariance matrix obtained by predicting the pose sensor at the previous moment can be acquired. The current second pose data can be predicted based on the first pose data, and the current second covariance matrix can be predicted based on the first covariance matrix. The actual pose data currently acquired by the pose sensor can be acquired. The target gain can be determined based on the actual pose data and the second covariance matrix. The second pose data can be corrected based on the target gain to obtain the current target pose data of the pose sensor.

[0095] In one possible implementation, due to measurement errors in the pose sensor, it is easy to confuse the target object (such as a person) with non-target objects (such as obstacles), resulting in the inability to perform the assistance task. Therefore, it is necessary to correct the human pose data directly measured by the pose sensor to obtain highly accurate target pose data. Referring to Figure 9, which is a schematic flowchart of the data processing of pose data measured by the pose sensor according to an embodiment of this application, as shown in Figure 9, the pose data correction process first uses the pose data from the previous moment for prediction, including state prediction estimation of the pose data and prediction estimation of the covariance matrix corresponding to the predicted state quantity. The predicted state value is corrected using the measurement value at the current moment, including calculating the target gain, then correcting the predicted state value based on the target gain, and finally updating the previously predicted covariance matrix. Human pose data may include the angles and directions of human joints, the three-dimensional coordinate positions of key body points, etc. Specifically, the human pose data detected by the pose sensor can be understood as the motion state of the target object, including the position and velocity of the target object, i.e. The first pose data is the pose data predicted from the pose data measured by the pose sensor at the previous moment. Since the motion state of the target object is not affected by external control input, the external control vector and external control input matrix can be ignored based on the state equation of the corrected system. Therefore, the second pose data predicted at the current moment can be obtained by multiplying the first pose data predicted at the previous moment with the state transition matrix, that is, the state of the target object at the current moment is predicted. The state transition matrix describes the relationship of the corrected system from the previous moment to the current moment. Specifically, the state transition matrix is:

[0096] Where ΔT represents the time interval of the correction system, which is the time difference between the previous moment and the current moment, and is used to correct the changes in state variables caused by the time interval.

[0097] Based on the state equation of the corrected system after ignoring the influence of external control input, the calculation formula for the second pose data can be obtained, as shown in the following formula:

[0098] in, Let x represent the second pose data predicted at the current moment. k-1 Let ω represent the first pose data predicted at the previous time step. k This represents the process noise of the corrected system that follows a Gaussian distribution.

[0099] In one possible implementation, the first covariance matrix is ​​a matrix that measures the uncertainty of the predicted first pose data obtained from the pose sensor at the previous time step. It represents the uncertainty of the state estimation at the previous time step in the correction system. Specifically, the first covariance matrix expresses the error distribution of the first pose data. The diagonal elements of the first covariance matrix represent the variance of each state variable (i.e., the element of the first pose data), while the off-diagonal elements represent the covariance between different state variables, reflecting the correlation between them. Based on the correction system, the second covariance matrix corresponding to the second pose data at the current time step can be obtained by multiplying the state transition matrix and the first covariance matrix. Specifically, the formula for calculating the second covariance matrix is ​​as follows:

[0100] in, Represented as the second pose data The corresponding second covariance matrix, P k-1 Represented as the first pose data x k-1 The corresponding first covariance matrix, Q, represents the process noise ω of the corrected system. k The corresponding noise covariance matrix.

[0101] In one possible implementation, the target gain is used to optimize the prediction accuracy by combining the predicted state (i.e., the second pose data) obtained from the pose sensor's prediction with the observations (i.e., the actual pose data) measured by the pose sensor. This balances the predicted pose from the second pose with the actual pose data, corrects the predicted second pose data, and yields the target pose data. Specifically, the feedback gain matrix is ​​calculated using the second covariance matrix and a pre-set observation matrix in the correction system. The target gain is then calculated using the actual pose data and the feedback gain matrix. The formula for calculating the feedback gain matrix is ​​as follows:

[0102] Among them, K kLet H be the feedback gain matrix, H be the observation matrix, and R be the observation noise matrix. Specifically, the observation matrix can be represented as:

[0103] After obtaining the feedback gain matrix, the actual pose data and the second pose data can be weighed using the feedback gain matrix to obtain the target gain. The specific calculation formula is shown in the following equation.

[0104] Where B represents the target gain, z k This represents the actual pose data. It should be noted that the actual pose data can be obtained by low-pass correction of the initial pose data directly measured by the pose sensor. Specifically, the correction calculation formula for the actual pose data is as follows:

[0105] Where α represents the correction factor, and the correction factor represents the low-pass correction strength. This represents the second initial pose data directly measured by the pose sensor at the current moment, z. k-1 This represents the first initial pose data directly measured by the pose sensor at the previous moment. Historical actual pose data obtained by performing low-pass correction.

[0106] By correcting the second pose data using the target gain, the current target pose data of the pose sensor can be obtained, denoted as x. k The specific correction formula can be shown in the following equation:

[0107] After correcting the second pose data of the predicted state using the target gain, the second covariance matrix corresponding to the second pose data can be updated. The specific matrix update formula is as follows:

[0108] Among them, P k Represented as the updated second covariance matrix, it is obtained by correcting the system's feedback gain matrix K. k The second covariance matrix obtained by comparing the observation matrix H with the predicted covariance matrix This allows for the use of the updated second covariance matrix P. k Predict the target pose data for the next moment.

[0109] In one possible implementation, the relationship between the predicted state and the observations in the correction system can be represented by an observation equation, specifically, the observation equation can be expressed as follows: z k =Hx k +v k(9)

[0110] Among them, v k This represents the observation noise in the correction system that follows a Gaussian distribution.

[0111] In one possible implementation, during the process of controlling the robotic arm to move to the target joint position, the current image data of the vision sensor can be acquired first to determine the key point positions of multiple key points of the target object in the image data; the coordinate system where the key point positions are located can be transformed to the coordinate system where the assisting robot is located; based on the multiple transformed key point positions, the center point position of multiple key points can be determined; the center point position can be transformed based on the preset second transformation matrix to obtain the target joint position; and the robotic arm movement can be controlled based on the target joint position.

[0112] In one possible implementation, there can be multiple key points for the target object. Specifically, key points can include salient features on the human body such as joints like the eyes, ears, shoulders, waist, elbows, wrists, hips, knees, and ankles. When performing key point detection on image data, key points can be determined from multiple joints based on the main body of the target object displayed in the image data. For example, if the main body of the target object displayed in the image data is the upper limb, joints such as the shoulders and waist can be selected as key points; if the main body of the target object displayed in the image data is the lower limb, joints such as the hips and knees can be selected as key points. Alternatively, when performing key point detection on image data, key points can be determined from multiple joints based on the target object's need for assistance. For example, if the target object's need for assistance is to be helped to stand up, joints such as the shoulders, elbows, and wrists can be selected as key points; if the target object's need for assistance is to be helped to walk, joints such as the waist, hips, and elbows can be selected as key points.

[0113] In one possible implementation, the key points of the target object can be fixed joints on the human body. When multiple fixed joints of the target object cannot be detected from the image data, the target movement is re-determined based on the relative position between the pose sensor and the target object. After the assisting robot is re-controlled to reach the new target movement position, the image data from the vision sensor is re-acquired for analysis. For example, the key points of the target object can be four key points on the human body: left shoulder, right shoulder, left hip, and right hip. When the image data captured by the vision sensor cannot completely detect the above four key points of the target object, it can be considered that the assisting robot cannot accurately perceive the target object and is prone to perception errors. At the same time, it can also be considered that if the relative position between the assisting robot and the target object is too close or too far, it is difficult to ensure safe execution of the assistance task. Therefore, the target movement position can be re-determined based on the relative position between the pose sensor and the target object, and the assisting robot can be controlled to move to the updated target movement position. Then, the current image data is captured by the vision sensor until the four key points of the target object (left shoulder, right shoulder, left hip, and right hip) can be completely detected in the new image data.

[0114] In one possible implementation, the keypoint location can refer to the coordinates of each keypoint in the image data in a pixel coordinate system. This pixel coordinate system can have its origin at the top left corner of the image, with the horizontal direction to the right as the positive direction of the x-axis and the vertical direction downwards as the positive direction of the y-axis. After identifying multiple keypoints through image recognition, image registration can be used to determine the keypoint locations in the image data, i.e., the two-dimensional coordinates in the pixel coordinate system. The coordinate system of the keypoint locations (pixel coordinate system) is then transformed to the coordinate system of the assisting robot. The transformed keypoint locations are then represented as three-dimensional coordinates of each keypoint in the assisting robot's coordinate system. The arithmetic mean of the transformed keypoint locations is calculated to obtain the center point location of the multiple keypoints, i.e., the coordinates of the center point. A preset second transformation matrix is ​​used to transform the center point location to obtain the target joint location. This second transformation matrix can be a transformation matrix that changes the center point location to the desired assisting position of the robotic arm; the center point location serves as a reference point for the desired assisting position of the robotic arm to determine the target joint location of the robotic arm.

[0115] In one possible implementation, referring to Figure 10, which is a schematic diagram of the center point position provided in an embodiment of this application, image data of the current environment of the assistive robot is captured by a vision sensor. As shown in Figure 10, due to the limitation of the vision sensor's shooting angle, if the image data captured by the vision sensor is required to capture the main body of the target object, the relative distance between the vision sensor and the target object needs to be greater than the minimum shooting distance of the vision sensor. For example, if the vision sensor is a depth camera and the minimum shooting distance of the depth camera is 50 centimeters, then when the relative distance between the depth camera and the target object reaches 50 centimeters, the image data captured by the vision sensor will show the main body of the target object. Otherwise, it is necessary to adjust the relative position between the assistive robot and the target object and re-capture the image data. If the image data contains the main body of the target object, key point detection of the target object can be performed on the image data. The relative position between the target object and the vision sensor is determined by four key points: the left shoulder (marked point A in Figure 10), the right shoulder (marked point B in Figure 10), the left hip (marked point C in Figure 10), and the right hip (marked point D in Figure 10). Simultaneously, the distance between the target object and the vision sensor can be determined. By calculating the center point position of the four key points (marked point E in Figure 10), the center point position is located at the waist of the target object shown in Figure 10. Therefore, the waist of the target object can be used as the reference point for the desired support position of the robotic arm of the assisting robot, i.e., the reference point for the target joint position. By transforming the center point position using the second transformation matrix, the support positions located on both sides of the waist can be obtained as the target joint positions (marked point F in Figure 10).

[0116] In one possible implementation, the intrinsic parameter matrix of the vision sensor can be obtained by calibrating the vision sensor parameters; the installation position of the vision sensor in the assistive robot is determined, and the extrinsic parameter matrix of the vision sensor is determined based on the installation position; the coordinate system of the key point position is transformed to the coordinate system of the target movement position based on the intrinsic parameter matrix and the extrinsic parameter matrix.

[0117] In one possible implementation, the intrinsic parameter matrix of the vision sensor describes its internal optical characteristics, including the focal length and the imaging center (optical center). The focal length of the vision sensor can be represented by the 10-axis focal length and the y-axis focal length in pixels, respectively, while the optical center can be represented by its projection position on the image plane. For example, after parameter calibration of the vision sensor, the intrinsic parameter matrix can be obtained, which can be represented as:

[0118] Among them, f xExpressed as a 10-axis focal length, f y Expressed as focal length along the y-axis, (c x c y ) represents the position of the optical center.

[0119] In one possible implementation, the extrinsic matrix of the vision sensor defines the spatial relationship between the vision sensor coordinate system and the coordinate system of the assistive robot (base). The extrinsic matrix includes a rotation matrix and a translation matrix. The rotation matrix is ​​typically a 3x3 matrix representing the rotation of the vision sensor in the assistive robot (base) coordinate system, while the translation matrix is ​​typically a 3x1 vector representing the position of the vision sensor relative to the assistive robot (base). For example, based on the installation position of the vision sensor in the assistive robot, the extrinsic matrix can be obtained, specifically represented as follows:

[0120] Where R represents the rotation matrix and T represents the translation matrix.

[0121] In one possible implementation, the coordinate system of the key point location is transformed to the coordinate system of the target movement location. This can be done by first transforming from the key point location's coordinate system (pixel coordinate system) to the vision sensor's coordinate system, and then from the vision sensor's coordinate system to the support robot's (base's) coordinate system. The specific calculation process can be shown in the following formula:

[0122] Where (u, v) represents the pixel coordinate system, that is, the coordinate system in which the key point is located; (X c Y c Z c The coordinate system is denoted as ( ), and the origin of the coordinate system is the optical center of the vision sensor. The Z-axis is parallel to the optical axis of the vision sensor. c This refers to the shooting direction of the visual sensor; (X) B Y B Z B ) represents the coordinate system in which the support robot (base) is located.

[0123] In one possible implementation, kinematic modeling of the assisted robot based on its joint variables (such as displacement, velocity, acceleration, and position) yields a dynamic model describing the relationship between joint torques and joint accelerations. For example, a dynamic model is a mathematical model used to describe the robot's motion and dynamic behavior under external forces. Dynamic models are typically based on principles of classical mechanics, such as the Newton-Euler equations and the Lagrange equations. The establishment of a robot's dynamic model can be used to understand and predict the robot's motion response, design controllers, and perform simulation analysis.

[0124] By using tactile sensors mounted on the robotic arm of the assisted robot, external force data is acquired. A transformation matrix is ​​then used to convert the force information at the applied points into the coordinate system of the corresponding joints of the assisted robot. This yields the torque required for the robot's joints to respond to the external force, i.e., the target joint torque. This allows the assisted robot to generate sufficient force to counteract the external force while maintaining compliant movement and assisted stability. Based on the obtained target joint torque and the established dynamic model, the target joint acceleration of the assisted robot is determined, and then compliant control of the robot is performed based on this target joint acceleration.

[0125] In one possible implementation, the mass coefficient can be determined first based on the mass of the assisted robot, the joint position of the assisted robot, and the joint velocity of the assisted robot; the friction coefficient can be determined based on the friction force during the movement of the assisted robot and the joint position of the assisted robot; the gravity coefficient can be determined based on the gravitational acceleration of the assisted robot and the joint position of the assisted robot; the first product between the mass coefficient and the joint acceleration of the assisted robot can be determined; and the kinematic model of the assisted robot can be performed based on the sum of the first product, the friction coefficient, and the gravity coefficient to obtain the dynamic model of the assisted robot.

[0126] In one possible implementation, the mass coefficient can represent the inertial properties of each joint of the assisted robot. The mass coefficient can be understood as the mass matrix in the robot's dynamics model, used to describe the mass distribution of each joint. Specifically, the mass coefficient of each joint in the assisted robot can be determined by the mass, joint position, and joint velocity of the corresponding joint. The joint position of the assisted robot can be determined by the position encoder of each joint, and the mass coefficient can change with the joint position. The friction coefficient can be understood as the friction matrix in the dynamics model, representing the influence of friction on each joint during the assisted robot's movement. The direction and magnitude of friction are different for joints at different positions; therefore, the friction coefficient is different for joints at different positions. The gravity coefficient can be understood as the gravity matrix in the dynamics model, used to represent the influence of gravity on each joint of the assisted robot. The direction of gravity on joints at different positions is different, affecting the posture changes of each joint. The first product, obtained by multiplying the mass coefficient by the joint acceleration of the corresponding joint, represents the inertial force caused by the joint acceleration. This is the inertial force that each joint must counteract during movement, ensuring a timely response to external forces. The kinematic model of the assisting robot, obtained by summing the first product, the friction coefficient, and the gravity coefficient, can be represented by the following formula:

[0127] Where M(q) represents the mass coefficient of the joint at joint position q. Represented as the first product, This represents the joint velocity at the joint position. This represents the coefficient of friction of a joint at its joint position. Let q represent the joint velocity at joint position q, g(q) represent the gravity coefficient of the joint at joint position q, and τ represent the joint torque of the joint at joint position q.

[0128] Based on the dynamic model, the target joint acceleration generated under the action of external force can be calculated. The specific formula for calculating the target joint acceleration is as follows:

[0129] After obtaining the target joint acceleration, an adaptive control strategy can be generated using a proportional-integral-derivative (PID) controller to adjust the torque output of the corresponding joint in response to external forces, thereby controlling the movement of the assisted robot based on the target joint acceleration.

[0130] In one possible implementation, the tactile sensor includes multiple tactile units. Therefore, a third transformation matrix can be determined first to transform the coordinate system of the assisted robot to the coordinate system of the assisted robot's joints, and a fourth transformation matrix can be determined to transform the coordinate system of the assisted robot's joints to the coordinate system of the tactile units. Based on the third and fourth transformation matrices, a transformation function between the coordinate system of each tactile unit and the coordinate system of the assisted robot is determined. The derivative of the transformation function with respect to the joint position of the assisted robot is determined to obtain the Jacobian matrix corresponding to each tactile unit. The second product between the external force data collected by each tactile unit and the corresponding Jacobian matrix is ​​determined, and the target joint torque is obtained based on the sum of multiple second products.

[0131] For example, the Jacobian matrix is ​​a square matrix composed of a set of partial derivatives that describes the rate of change of each component of one vector-valued function relative to each component of another vector-valued function. In this embodiment, the Jacobian matrix is ​​used to describe the relationship between the joint velocities of a robotic arm and the velocities of its end effector. The Jacobian matrix maps joint angular velocities or angular accelerations to the linear velocities or linear accelerations of the end effector and is a key component in motion planning.

[0132] In one possible implementation, assuming the assisting robot needs to perform an assisting task, the task space and joint space of the assisting task have a non-linear relationship. That is, the coordinates of the target joint position required to perform the assisting task are located in the coordinate system of the assisting robot, while the coordinates of the external force data that performs the assisting task are located in the coordinate system of the haptic unit. It is not possible to directly use the external force data to adjust the target joint position to achieve compliant control. Therefore, a transformation function is needed, determined by the third and fourth transformation matrices. The transformation function describes how to transform the coordinate system of each haptic unit to the coordinate system of the assisting robot. By performing differential operations on the joint position of the assisting robot through the transformation function, the two different coordinate systems can be associated, and the Jacobian matrix of the corresponding joint can be obtained. The Jacobian matrix of the corresponding joint can be expressed as follows:

[0133] Here, f represents the transformation function, and J(q) represents the Jacobian matrix of the joint at joint position q. The Jacobian matrix of the corresponding joint can be obtained by taking the partial derivative of the transformation function f with respect to joint position q.

[0134] The derivative of the transformation function with respect to the joint positions of the assisted robot is used as the Jacobian matrix of each tactile unit to represent the influence of the magnitude and direction of the external force on each tactile unit on different joint positions. Specifically, it is assumed that the direction of the rotation axis of each joint i uses a unit vector. z r iThis means that the position of the origin of each joint coordinate system can be represented by the vector p. i Let J be the Jacobian matrix of joint i. i (q) can be represented as:

[0135] Among them, (x b y b Let q represent the coordinate system in which the assisting robot is located. k Let represent the k-th joint, 1 < k ≤ n, where n represents the total number of joints in the robot, that is, the sum of the number of joints in the robotic arm and the number of joints in the base.

[0136] Since each joint of the assisting robot rotates around the z-axis, the pose transformation matrix T of each joint i is... i It can be:

[0137] Substituting the pose transformation matrix of the joint corresponding to each haptic unit into the Jacobian matrix of the corresponding joint yields the Jacobian matrix for each haptic unit. Multiplying the external force data collected by each haptic unit by its corresponding Jacobian matrix yields a second product. Using the Jacobian matrix of each haptic unit, the external force data received by that unit is converted to the overall joint. In other words, the second product represents the force transmitted from each haptic unit to the overall joint. Furthermore, by calculating the sum of these second products, the force applied by the external environment (i.e., the target object) during the interaction with the robotic arm during the assistance process can be represented, thus mapping the target joint torque. The formula for calculating the target joint torque τ is as follows:

[0138] in, Let Jacobian matrix be the tactile unit j of robotic arm joint i. This represents the external force data of the tactile unit j at joint i of the robotic arm. Specifically, due to the external force f z Applying force in the z-axis direction, therefore external force data It can be expressed in the following form:

[0139] In one possible implementation, the joints of the assisting robot include multiple sub-arms and joint components of the robotic arm, as well as the base joint of the assisting robot. Therefore, compliant control can be achieved by separately controlling the joint torque and joint acceleration of each sub-arm and joint component of the robotic arm. When the assisting robot is a wheeled robot, the base joint of the assisting robot may include omnidirectional wheels mounted on the base. The joint torque of the omnidirectional wheel can be represented by the rotation angle of the omnidirectional wheel, and the joint acceleration of the omnidirectional wheel can be represented by the angular velocity, angular acceleration, and torque of the motor driving the omnidirectional wheel to rotate. Thus, compliant control can be achieved by adjusting the rotation direction, rotation speed, and rotation acceleration of the omnidirectional wheel to conform to the movement trend of the assisted object. When the assisting robot is a legged robot, the base joint of the assisting robot may include ankle joints, knee joints, hip joints, etc. Similarly, compliant control can be achieved by separately controlling the joint torque and joint acceleration of the ankle joint, knee joint, and hip joint.

[0140] In one possible implementation, to determine the fourth transformation matrix used to transform the coordinate system of the joints of the assisted robot to the coordinate system of the tactile units, it is necessary to perform kinematic modeling for each tactile unit of the tactile sensor relative to its position on the robotic arm, establishing the relationship between the tactile units and the kinematic chains on the robotic arm. Each joint of the robotic arm can be approximated as a cylinder; therefore, the multiple tactile units of the tactile sensor can be considered to be cylindrically distributed on the surface of the robotic arm, with a first distance equal between any two adjacent tactile units. This first distance represents the distance between two adjacent tactile units along the length of the robotic arm link. Simultaneously, the orientation of the coordinate system of the tactile unit relative to the coordinate system of the assisted robot is the same as the orientation of the coordinate system of the joint corresponding to that tactile unit relative to the coordinate system of the assisted robot. Therefore, based on the first distance corresponding to the tactile unit, the distance between each tactile unit and the starting end of its respective joint can be determined, thereby allowing the determination of the fourth transformation matrix used to transform the coordinate system of the assisted robot's joints to the coordinate system of the tactile units.

[0141] Referring to Figure 11, which is a schematic diagram of the transformation of the coordinate system of the joints of the assistive robot provided in this embodiment to the coordinate system of the tactile unit, assuming the first distance is 0.015 meters, as shown in Figure 11, the robotic arm can be regarded as a cylinder. Since the first distance between any two adjacent tactile units is equal, it is equivalent to dividing the multiple tactile units distributed cylindrically on the surface of the robotic arm into multiple rings along the height of the cylinder. The distance between two adjacent rings is fixed at 0.015 meters, and the tactile units on the same ring can be modeled as the same unit. Therefore, based on the first distance of the tactile units and the position of the tactile unit (the ring it is located in) on the robotic arm (cylinder), the transformation relationship between the tactile unit and the joint can be obtained, and thus the fourth transformation matrix can be obtained. The fourth transformation matrix can be specifically expressed as:

[0142] As shown in Figure 11, t i This can be understood as the origin of the coordinate system of joint i where the tactile unit is located, and This can be represented as the origin of the coordinate system of tactile unit j in joint i, and (0.015×j) can be represented as the distance between the origin of the coordinate system of tactile unit j and the origin of the coordinate system of joint i. It can be represented as the fourth transformation matrix used to transform the coordinate system of the joints of the assisted robot to the coordinate system of the tactile unit.

[0143] In one possible implementation, referring to Figure 12, which is a schematic flowchart of compliance control provided in an embodiment of this application, full-body compliance control based on a tactile sensor can be achieved through the following steps.

[0144] In step 1201, kinematic modeling of the tactile unit is performed.

[0145] For example, kinematic modeling is performed on each tactile unit of the tactile sensor for its position on the robotic arm.

[0146] In step 1202, a dynamic model of the assisting robot is established.

[0147] Dynamic modeling of the assist robot was performed to construct a dynamic model.

[0148] In step 1203, the Jacobian matrix of the tactile unit is calculated.

[0149] Modeling the tactile units and the dynamic model of the assistive robot allows for the calculation of the Jacobian matrix for each tactile unit.

[0150] In step 1204, the external force data is mapped to the joint.

[0151] External force data received by tactile sensors can be transformed and mapped to the force conditions of various joints of the robotic arm and the robot through Jacobian matrix conversion.

[0152] In step 1205, compliant control is performed on all joints throughout the body.

[0153] Based on the force conditions of each joint, the joint accelerations of the entire robot can be obtained for compliant control.

[0154] In one possible implementation, based on the perceived need for assistance from the target object, the assistance robot can be controlled to follow the target object and perform assistance tasks. For example, if the robot is perceived to be assisting the target object to walk, it can be controlled to update the target's movement position in real time based on the relative distance between the robot and the target object using a pose sensor. Based on the updated target movement position, the robot can be controlled to move towards the target object, thus enabling the assistance robot to follow the target object and perform assistance tasks. During the movement of the assistive robot following the target object, obstacles can be detected by target sensors. When an obstacle is identified within a preset angle range, feature extraction can be performed on the data obtained by the target sensors to determine the obstacle's outline and calculate its target size. Specifically, environmental data of the surrounding environment can be obtained using pose sensors (such as LiDAR), and point cloud segmentation can be performed on the environmental data to identify key features such as obstacles and the ground. The segmented point cloud data can be clustered to form "clusters" of obstacles, allowing for further analysis of obstacle size. Alternatively, depth images of the current environment can be obtained using visual sensors (such as depth cameras). The distance data of obstacles can be determined by calculating the depth of field in the depth image. A pre-trained neural network model can then be used to identify and estimate the size of obstacles based on this distance data. More accurate environmental data can be obtained by combining data from multiple sensors. For example, sensor data from both pose sensors and visual sensors can be imported into a corrector or particle corrector for multi-source data fusion analysis to determine the obstacle's size. The target size of the obstacle is then compared with a preset size range. If the target size of the obstacle exceeds the preset size range, it means that the obstacle may interfere with the movement path of the assistance robot and affect the safety of the assistance. Therefore, it is necessary to control the assistance robot to avoid the obstacle.

[0155] In one possible implementation, a second distance between the assisting robot and an obstacle can be detected by target sensors such as pose sensors and / or vision sensors installed on the assisting robot. When the second distance is less than or equal to a preset distance threshold, it can be considered that the obstacle may affect the movement path of the assisting robot and the target object. Therefore, the assisting robot can be controlled to avoid the obstacle. At the same time, the robotic arm of the assisting robot can be compliantly controlled to provide information feedback to the target object (such as applying external force, issuing sound prompts, etc.) to guide the target object to avoid the obstacle.

[0156] In one possible implementation, referring to Figure 13, which is a schematic diagram of obstacle avoidance for the assistive robot provided in this embodiment, if an obstacle is considered to affect the movement path of the assistive robot 1301, a virtual space can be created, and sensor data sensed by the target sensor of the assistive robot can be mapped into the virtual space. This allows the obstacle's position and the endpoint of the assistive robot's movement path to be represented in the virtual space. Both the obstacle and the endpoint can be modeled as spheres. Based on the obstacle's position and corresponding mass, a first force vector (i.e., the obstacle's repulsive force) can be determined. This first force vector guides the assistive robot to move away from the obstacle. Based on the endpoint position and its corresponding preset mass, a second force vector is determined for the endpoint position. This second force vector guides the assistive robot to move towards the endpoint position. The first force vector decreases as the second distance increases, while the second force vector decreases as the distance between the current position of the assistive robot and the endpoint position increases. The first force vector and the second force vector are combined to obtain the target force vector. Specifically, the two vectors can be added or weighted to obtain the target force vector. The speed and direction of the assisting robot can be controlled based on the target force vector, so that the assisting robot can avoid obstacles and move towards the set endpoint position.

[0157] It should be noted that when there are multiple obstacles whose second distance from the assisting robot is less than or equal to a preset distance threshold, multiple obstacles and their corresponding positions can be determined in the virtual space, and the first force vector corresponding to each obstacle can be obtained. Then, all the first force vectors and the second force vectors can be combined to obtain the target force vector.

[0158] In one possible implementation, during the process of assisting the target object, the target sensor can also be used to detect changes in the surrounding environment of the assisting robot in real time, so as to respond to sudden events such as environmental changes and ensure the safety of the user. For example, when the assisting robot is assisting the target object to cross the zebra crossing, the assisting robot can perceive and identify dynamic obstacles such as vehicles and pedestrians in the surrounding environment, as well as static obstacles such as road shoulders and flower beds, and guide the target object to adjust the walking speed, stop walking, or change the walking path to avoid potential collisions.

[0159] In one possible implementation, the tactile sensor includes multiple tactile units that are evenly distributed on the surface of the robotic arm. The tactile sensor can collect external force data in real time. Based on the detection principles of different tactile sensors, triggering can be determined through changes in resistance, capacitance, piezoelectric effect, etc. When the tactile sensor detects an applied force on the robotic arm, the electrical signals of the tactile units change. These signals can be converted into identifiable external force data. Simultaneously, it can analyze which tactile units are activated, thereby determining the number of activated tactile units based on the external force data. Specifically, a sensitivity or activation threshold can be set to determine whether a tactile unit is active. When the applied external force exceeds the sensitivity or activation threshold of the tactile unit, it is considered active. When the tactile sensor detects that an external force is applied to the robotic arm, it can continuously acquire external force data. When the detected external force data indicates that the current external force is greater than or equal to the external force threshold, and the number of activations is greater than the preset threshold, it can be considered that the target object is currently engaging in large-area force interaction with the robotic arm. Therefore, it can be considered that the target object is currently being supported by the assisting robot, or that the target object is currently experiencing a sudden situation such as instability of the center of gravity. Therefore, the assisting robot can be compliantly controlled to adjust the target joint position and target joint acceleration of the robotic arm to conform to the posture of the target object.

[0160] In one possible implementation, when the external force data indicates that the external force received by the tactile sensor is greater than or equal to the external force threshold, and the number of activations is less than the preset number threshold, it can be considered that the target object is interacting with the robotic arm. However, the stability of the assistance is low at this time. If the current posture of the assistance robot is changed, the target object may easily lose its support balance. Therefore, the assistance robot can be controlled to move to the target movement position or the robotic arm can be controlled to move to the target joint position according to the preset control strategy, without the need to perform compliant control on the assistance robot.

[0161] In one possible implementation, when the external force data indicates that the change in external force received by the tactile sensor is greater than or equal to the first external force change threshold within a first preset time period, and the change in external force is less than the second external force change threshold within a second preset time period after the first preset time period, it can be considered that the target object has now engaged in force interaction with the robotic arm, and the current assisted posture of the assisted robot and the assisted posture of the target object are stable, so the assisted robot can be compliantly controlled.

[0162] The control method of the assisted robot provided in the embodiments of this application is described in detail below.

[0163] Referring to Figure 14, which is a schematic flowchart of a control method provided in an embodiment of this application, the control method can be executed by a terminal and includes, but is not limited to, the following steps 1401 to 1410:

[0164] Step 1401: In response to the recognition that the target object has the intention to need assistance, obtain the first pose data and the first covariance matrix obtained by the pose sensor in the previous moment.

[0165] Step 1402: Predict the current second pose data based on the first pose data, and predict the current second covariance matrix based on the first covariance matrix.

[0166] Step 1403: Obtain the actual pose data currently acquired by the pose sensor, and determine the target gain based on the actual pose data and the second covariance matrix.

[0167] Step 1404: Correct the second pose data based on the target gain to obtain the current target pose data of the pose sensor.

[0168] Step 1405: Transform the target pose data based on the preset first transformation matrix to obtain the target movement position.

[0169] Step 1406: Control the assisted robot to move towards the target object based on the target's movement position.

[0170] Step 1407: When the assisting robot moves to the target moving position, acquire the current image data of the vision sensor and determine the key point positions of multiple key points of the target object in the image data.

[0171] Step 1408: Transform the coordinate system of the key point location to the coordinate system of the assisting robot, and determine the center point location of multiple key points based on the transformed key point locations.

[0172] Step 1409: Transform the center point position based on the preset second transformation matrix to obtain the target joint position, and control the robot arm movement based on the target joint position.

[0173] In this step, the target movement position and the target joint position are both determined based on the relative position between the target object and the target sensor;

[0174] Step 1410: During or after the robotic arm moves to the target joint position, when the tactile sensor detects that an external force has been applied to the robotic arm, the assisting robot is subjected to compliant control.

[0175] Referring to Figure 15, which is a schematic diagram of an overall flow of a control method provided in an embodiment of this application, the control method can be executed by a terminal and includes, but is not limited to, the following steps 1501 to 1512:

[0176] Step 1501: In response to the recognition that the target object has the intention to need assistance, control the assistance robot to move towards the target object.

[0177] Step 1502: When the assisting robot moves to the target movement position, control the robotic arm to move to the target joint position.

[0178] In this step, the target movement position and the target joint position are both determined based on the relative position between the target object and the target sensor.

[0179] Step 1503: During or after the robotic arm moves to the target joint position, when the tactile sensor detects that the robotic arm is subjected to an external force, the mass coefficient is determined based on the mass of the assisted robot, the joint position of the assisted robot, and the joint speed of the assisted robot.

[0180] Step 1504: Determine the friction coefficient based on the friction force during the movement of the assisting robot and the joint position of the assisting robot.

[0181] Step 1505: Determine the gravity coefficient based on the gravitational acceleration of the assisting robot and the joint position of the assisting robot.

[0182] Step 1506: Determine the first product between the mass coefficient and the joint acceleration of the assisted robot. Based on the sum of the first product, the friction coefficient, and the gravity coefficient, perform kinematic modeling on the assisted robot to obtain its dynamic model.

[0183] In this step, the dynamic model is used to indicate the relationship between the joint torques and joint accelerations of the assisted robot.

[0184] Step 1507: Obtain the current external force data of the tactile sensor.

[0185] Step 1508: Determine the third transformation matrix used to transform the coordinate system in which the assisted robot is located to the coordinate system in which the joints of the assisted robot are located.

[0186] Step 1509: Based on the first distance corresponding to the tactile unit, determine the fourth transformation matrix used to transform the coordinate system of the joints of the assisted robot to the coordinate system of the tactile unit.

[0187] In this step, multiple tactile units are distributed in a cylindrical shape on the robotic arm, with the first distance between any two adjacent tactile units being equal.

[0188] Step 1510: Based on the third and fourth transformation matrices, determine the transformation function between the coordinate system of each tactile unit and the coordinate system of the assisting robot, determine the derivative of the transformation function with respect to the joint position of the assisting robot, and obtain the Jacobian matrix corresponding to each tactile unit.

[0189] Step 1511: Determine the second product between the external force data collected by each tactile unit and the corresponding Jacobian matrix, and obtain the target joint torque based on the sum of multiple second products.

[0190] Step 1512: Determine the target joint acceleration of the assisted robot based on the target joint torque and dynamic model, and perform compliant control of the assisted robot based on the target joint acceleration.

[0191] Referring to Figure 16, which is a schematic diagram of an overall flow of a control method provided in an embodiment of this application, the control method can be executed by a terminal and includes, but is not limited to, the following steps 1601 to 1609:

[0192] Step 1601: In response to the recognition that the target object has the intention to need assistance, control the assistance robot to move towards the target object.

[0193] Step 1602: During the process of the assisting robot following the target object, when an obstacle is detected within the preset angle range, the target size of the obstacle is detected.

[0194] Step 1603: When the target size is outside the preset size range, detect the second distance between the assisting robot and the obstacle.

[0195] Step 1604: When the second distance is less than or equal to the preset distance threshold, construct the virtual space where the assisting robot is located.

[0196] Step 1605: Determine the location of obstacles and the set endpoint of the assisting robot in the virtual space.

[0197] Step 1606: Determine the first force vector of the obstacle based on the obstacle's position and mass, and determine the second force vector of the endpoint based on the endpoint position and the preset mass of the endpoint position.

[0198] Step 1607: Combine the first force vector and the second force vector to obtain the target force vector, and control the assisting robot to avoid obstacles according to the target force vector.

[0199] Step 1608: When the assisting robot moves to the target movement position, control the robotic arm to move to the target joint position.

[0200] In this step, the target movement position and the target joint position are both determined based on the relative position between the target object and the target sensor.

[0201] Step 1609: During or after the robotic arm moves to the target joint position, when the tactile sensor detects that an external force has been applied to the robotic arm, the assisting robot is subjected to compliant control.

[0202] It is understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated in this embodiment, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0203] Referring to Figure 17, which is a schematic diagram of a supporting robot provided in an embodiment of this application, the supporting robot 1700 includes:

[0204] The robotic arm 1701 is equipped with tactile sensors;

[0205] Target sensor 1702 is used for object perception;

[0206] The control module 1703 is used to respond to the recognition that the target object needs assistance, control the assistance robot 1700 to move towards the target object, control the robotic arm 1701 to move to the target joint position when the assistance robot 1700 moves to the target moving position, and control the assistance robot 1700 to perform compliant control when the tactile sensor detects that the robotic arm is subjected to external force during or after the robotic arm 1701 moves to the target joint position; wherein, the target moving position and the target joint position are both determined based on the relative position between the target object and the target sensor 1702.

[0207] In one possible implementation, the control module 1703 is also used for:

[0208] The target pose data of the pose sensor is acquired, and the target pose data is transformed based on the preset first transformation matrix to obtain the target movement position;

[0209] Based on the target's movement position, the 1700 assisted robot moves towards the target object.

[0210] In one possible implementation, the control module 1703 is also used for:

[0211] Obtain the first pose data and the first covariance matrix obtained from the previous moment by predicting the pose sensor, predict the current second pose data based on the first pose data, and predict the current second covariance matrix based on the first covariance matrix.

[0212] Acquire the actual pose data currently collected by the pose sensor, and determine the target gain based on the actual pose data and the second covariance matrix;

[0213] The second pose data is corrected based on the target gain to obtain the current target pose data of the pose sensor.

[0214] In one possible implementation, the control module 1703 is also used for:

[0215] Acquire the current image data from the vision sensor and determine the key point locations of multiple key points of the target object in the image data;

[0216] Transform the coordinate system of the key point locations to the coordinate system of the assisted robot 1700, and determine the center point locations of multiple key points based on the transformed key point locations;

[0217] The center point position is transformed based on the preset second transformation matrix to obtain the target joint position, and the robot arm 1701 is controlled to move based on the target joint position.

[0218] In one possible implementation, the control module 1703 is also used for:

[0219] The parameters of the vision sensor are calibrated to obtain the intrinsic parameter matrix of the vision sensor;

[0220] Determine the installation location of the vision sensor in the assisted robot 1700, and determine the extrinsic parameter matrix of the vision sensor based on the installation location;

[0221] Based on the intrinsic and extrinsic parameter matrices, the coordinate system of the key point location is transformed to the coordinate system of the target movement location.

[0222] In one possible implementation, the control module 1703 is also used for:

[0223] The target object's posture is recognized based on various key points. The results of the posture recognition include whether the target object intends to need assistance or not.

[0224] In one possible implementation, the control module 1703 is also used for:

[0225] A kinematic model of the assisted robot 1700 is performed to obtain a dynamic model of the assisted robot 1700. The dynamic model is used to indicate the relationship between the joint torque and the joint acceleration of the assisted robot 1700.

[0226] Acquire the current external force data from the tactile sensor, convert the external force data, and obtain the target joint torque;

[0227] The target joint acceleration of the assisted robot 1700 is determined based on the target joint torque and dynamic model, and the assisted robot 1700 is then subjected to compliant control based on the target joint acceleration.

[0228] In one possible implementation, the control module 1703 is also used for:

[0229] The mass coefficient is determined based on the mass of the assisted robot 1700, the joint position of the assisted robot 1700, and the joint speed of the assisted robot 1700.

[0230] The friction coefficient is determined based on the friction force during the movement of the assisted robot 1700 and the joint position of the assisted robot 1700.

[0231] The gravity coefficient is determined based on the gravitational acceleration of the Assist Robot 1700 and the joint position of the Assist Robot 1700.

[0232] Determine the first product between the mass coefficient and the joint acceleration of the assisted robot 1700. Based on the sum of the first product, the friction coefficient, and the gravity coefficient, perform kinematic modeling on the assisted robot 1700 to obtain the dynamic model of the assisted robot 1700.

[0233] In one possible implementation, the control module 1703 is also used for:

[0234] A third transformation matrix is ​​determined to transform the coordinate system of the assisted robot 1700 to the coordinate system of the joints of the assisted robot 1700, and a fourth transformation matrix is ​​determined to transform the coordinate system of the joints of the assisted robot 1700 to the coordinate system of the tactile unit.

[0235] Based on the third and fourth transformation matrices, the transformation function between the coordinate system of each tactile unit and the coordinate system of the assisted robot 1700 is determined, the derivative of the transformation function with respect to the joint position of the assisted robot 1700 is determined, and the Jacobian matrix corresponding to each tactile unit is obtained.

[0236] Determine the second product between the external force data collected by each tactile unit and the corresponding Jacobian matrix, and obtain the target joint torque based on the sum of multiple second products.

[0237] In one possible implementation, multiple tactile units are arranged in a cylindrical shape on the robotic arm 1701, with a first distance equal between any two adjacent tactile units. The control module 1703 is further used for:

[0238] Based on the first distance corresponding to the tactile unit, a fourth transformation matrix is ​​determined to transform the coordinate system of the joints of the assisted robot 1700 to the coordinate system of the tactile unit.

[0239] In one possible implementation, the control module 1703 is also used for:

[0240] During the process of the Assist Robot 1700 following the target object, when an obstacle is detected within a preset angle range, the target size of the obstacle is detected.

[0241] When the target size is outside the preset size range, control the assisted robot 1700 to avoid obstacles.

[0242] In one possible implementation, the control module 1703 is also used for:

[0243] The second distance between the assist robot 1700 and the obstacle is detected. When the second distance is less than or equal to a preset distance threshold, a virtual space is constructed in which the assist robot 1700 is located.

[0244] In virtual space, determine the location of obstacles and the set endpoint for the assisted robot 1700;

[0245] The first force vector of the obstacle is determined based on the obstacle's position and mass, and the second force vector of the endpoint is determined based on the endpoint position and the preset mass of the endpoint position.

[0246] The first force vector and the second force vector are combined to obtain the target force vector, and the assisted robot 1700 is controlled to avoid obstacles according to the target force vector.

[0247] In one possible implementation, the tactile sensor is provided with multiple tactile units, and the control module 1703 is further used for:

[0248] When the tactile sensor detects that an external force has been applied to the robotic arm, the current external force data of the tactile sensor is acquired, and the number of activated tactile units is determined based on the external force data.

[0249] When the external force data indicates that the external force received by the tactile sensor is greater than or equal to the external force threshold, and the number of activations is greater than or equal to the preset number threshold, the assisted robot 1700 is subjected to compliant control.

[0250] In one possible implementation, referring to Figure 18, which is a structural schematic diagram of the assistive robot 1700 provided in an embodiment of this application. As shown in Figure 18, the assistive robot 1700 includes a main body 1801 and a robotic arm 1701 including a first sub-arm 1802 and a second sub-arm 1803. One end of the first sub-arm 1802 is movably connected to the main body 1801, and the other end of the first sub-arm 1802 is movably connected to the second sub-arm 1803. Both the first sub-arm 1802 and the second sub-arm 1803 are equipped with tactile sensors. In other words, the first sub-arm 1802 can move relative to the main body 1801, while the second sub-arm 1803 can move relative to the first sub-arm 1802. The first sub-arm 1802 can serve as the main part of the robotic arm 1701 and can bear greater force than the second sub-arm 1803, making it easier to assist and help users. The second sub-arm 1803 acts as an extension of the first sub-arm 1802 and can perform more complex and delicate actions than the first sub-arm 1802, such as fine-tuning the assistance force or direction. Furthermore, the link length of the second sub-arm 1803 can be shorter than that of the first sub-arm 1802, thereby reducing the motion inertia of the second sub-arm 1803 and achieving more accurate motion control. Therefore, through the multi-stage connection of the first sub-arm 1802 and the second sub-arm 1803, a wider range of motion for the robotic arm 1701 can be provided, better mimicking the functions of human limbs, assisting the target object in movement, and providing support and protection. It should be noted that tactile sensors can be respectively installed on the first sub-arm 1802 and the second sub-arm 1803, thereby obtaining more comprehensive external force data and more accurately identifying the posture of the target object for compliant control of the assistance robot 1700.

[0251] [Corrected according to Rule 91 07.02.2025] In one possible implementation, the first sub-arm 1802 is connected to a first joint component 1804 and a second joint component 1805 at both ends, the first joint component 1804 is connected to a third joint component 1806 via a first connecting shaft, and the third joint component 1806 is connected to the main body 1801 via a second connecting shaft; one end of the second sub-arm 1803 is connected to the second joint component 1805 via a third connecting shaft, and the other end of the second sub-arm 1803 is connected to a fourth joint component 1807 via a fourth connecting shaft; the fourth joint component 1807 is connected to a fifth joint component 1808 via a fifth connecting shaft; the fifth joint component 1808 is connected to a sixth joint component 1809 via a sixth connecting shaft; and the sixth joint component 1809 is connected to an end effector 1810. As shown in Figure 18, the robotic arm 1701 of the assistive robot 1700 may include six joint components, achieving six degrees of freedom. Specifically, the third joint component 1806 can rotate relative to the main body 1801 about the second connecting axis, and the first joint component 1804 can rotate relative to the third joint component 1806 about the first connecting axis, allowing the first sub-arm 1802 and the second joint component 1805 to move synchronously relative to the third joint component 1806. The second sub-arm 1803 can rotate relative to the second joint component 1805 about the third connecting axis, the fourth joint component 1807 can rotate relative to the second sub-arm 1803 about the fourth connecting axis, the fifth joint component 1808 can rotate relative to the fourth joint component 1807 about the fifth connecting axis, and the sixth joint component 1809 can rotate relative to the fifth joint component 1808 about the sixth connecting axis.

[0252] In one possible implementation, the sixth joint component 1809 is connected to an end effector 1810, which is detachably mounted on the sixth joint component 1809. The end effector 1810 can be a robotic gripper, which facilitates the transfer of items such as heavy objects, backpacks, and canes. The end effector 1810 can also be a safety restraint device, such as a seatbelt, to improve stability and safety during the movement of the target object. The end effector 1810 can also be a handle or armrest, allowing the target object to grip and providing stable support. The end effector 1810 can also be a task controller for setting assistance tasks. The task controller can communicate with the control module 1703. The control module 1703 can trigger the set assistance task based on the task controller to control the assistance robot 1700. For example, if the original assistance task was to help the target object walk, upon reaching the target object's designated location (such as next to a seat), the task controller of the end effector 1810 can be triggered to reset the assistance task to helping the target object sit down. This allows the control module 1703 to change the assistance task and control the assistance robot 1700 to help the target object sit down in the seat.

[0253] In addition, the assisting robot 1700 can be equipped with multiple robotic arms 1701, as shown in Figure 18. The assisting robot 1700 can be equipped with two robotic arms 1701. In order to reduce collisions between multiple robotic arms 1701 and to achieve adjustments in different directions, the robotic arms 1701 can be installed on both sides of the main body 1801 respectively.

[0254] As shown in Figure 18, the assistive robot 1700 also includes a vision sensor 1811, a pose sensor, and a base 1812 for movement. The vision sensor 1811 can be installed in the front of the main body 1801, and can be located between the two robotic arms 1701, thereby obtaining a wider field of view to capture image data and identify human posture. The pose sensor can be installed in the front of the base 1812 to perceive the current environmental conditions of the assistive robot 1700, such as the position of the target object and the position of obstacles. When there are multiple pose sensors, they can be arranged on the four sides of the base 1812 to improve the perception range and obtain more accurate environmental data.

[0255] As shown in Figure 19, Figure 19 is a structural schematic diagram of the assistive robot 1700 provided in an embodiment of this application from another perspective. A pose sensor, a control cabinet 1901, an antenna 1902, a base display screen 1905 for interactively displaying the status of the base 1812, and a control display screen 1906 for interactively displaying the status of the robotic arm 1701 can be installed in the base 1812. Both the base display screen 1905 and the control display screen 1906 are mounted on the outside of the control cabinet 1901 for easy interaction with the user. The pose sensor may include a three-dimensional LiDAR 1903 for detecting obstacles and a two-dimensional LiDAR 1904 for detecting the position of the target object and the position of obstacles. The two-dimensional LiDAR 1904 can be mounted on the side wall of the base 1812, the three-dimensional LiDAR 1903 can be mounted above the control cabinet 1901, and the antenna 1902 can be mounted close to the control cabinet 1901 on one side of the base 1812. As shown in Figure 19, the base 1812 can be a Mecanum mobile platform, and the joints of the lower limbs of the assisted robot 1700 can be movable connection mechanisms between the omnidirectional wheels 1813 and the base 1812. The Mecanum mobile platform is a type of mobile robot platform that uses Mecanum wheels (also known as Swiss wheels) to achieve omnidirectional movement. This platform can move in any direction on a horizontal plane without steering, greatly enhancing the robot's maneuverability and flexibility. A Mecanum wheel is a large wheel composed of multiple smaller rollers, which are angled (usually 45 degrees) to the axle of the large wheel. When the large wheel rotates, these smaller rollers can roll, allowing the large wheel to move in any direction on a horizontal plane without changing its orientation. Each Mecanum wheel can be driven independently, and complex motion trajectories can be achieved by controlling the speed and direction of each wheel.

[0256] The main body 1801 can be located in front of the control cabinet 1901. Due to the weight of the robotic arm 1701 and the main body 1801, the center of gravity of the assisting robot 1700 moves forward. Therefore, the control cabinet 1901 can be placed behind the main body 1801, and a counterweight can be added to the base 1812 to balance the center of gravity of the assisting robot 1700 and improve the stability of the assisting robot 1700's movement.

[0257] As shown in Figure 20, Figure 20 is a schematic diagram of the internal structure of the control cabinet 1901 of the assistive robot 1700 provided in this embodiment of the application. The assistive robot 1700 also includes a switch 2001 connected to the antenna 1902 and the control module 1703 respectively, a battery 2002 for power supply, and an inverter 2003 for converting the voltage of the battery 2002 for power supply. The control module 1703, the switch 2001, the battery 2002 and the inverter 2003 are installed in the control cabinet 1901. The control module 1703 can interact with the server through the switch 2001 and the antenna 1902, for example, uploading sensor data to the server or downloading updated algorithm programs from the server.

[0258] As shown in Figure 21, which is a structural connection diagram of the assistive robot 1700 provided in this embodiment of the application, the battery 2002 is connected to the inverter 2003, thereby converting the 48V voltage of the battery 2002 into a 221V AC voltage to power the electrical load of the assistive robot 1700. The control module 1703 can communicate with the tactile sensor, the pose sensor (including the three-dimensional LiDAR 1903 and the two-dimensional LiDAR 1904), the vision sensor 1811, the robotic arm control cabinet 2101, the base display screen 1905, and the control display screen 1906, respectively. Thus, the control module 1703 can acquire data from each sensor and interactive data from each display screen. The control module 1703 can generate control commands based on the acquired data and send them to the robotic arm control cabinet 2101, thereby controlling the robotic arm 1701 and the base 1812 through the robotic arm control cabinet 2101. Specifically, the control module 1703 can establish connections with various components using different data transmission methods. For example, the control module 1703 can transmit data with the robotic arm 1701 and the base 1812 via the TCP / IP protocol, or transmit data with the tactile sensor and the vision sensor 1811 via the USB 3.0 transmission protocol, or transmit data with the pose sensor, the base display screen 1905 and the pose display screen via the HDMI transmission protocol.

[0259] The control module 1703 can be an industrial control computer used to execute the control methods of the aforementioned embodiments. In response to the recognition that the target object needs assistance, it controls the assistance robot to move towards the target object. When the assistance robot moves to the target movement position, it controls the robotic arm to move to the target joint position. The target movement position and the target joint position are determined based on the relative position between the target object and the target sensor, thereby enabling the perception of the target object's state and automatic execution of the assistance task based on the target object's state. Furthermore, during or after the robotic arm moves to the target joint position, when the tactile sensor detects that the robotic arm is subjected to external force, compliant control is applied to the robotic arm, allowing it to follow the target object's posture, thus improving the comfort of the assistance. Therefore, the control method provided in this application can interact with the target object in diverse ways, enabling the assistance robot to automatically perform assistance tasks, improving the accuracy and efficiency of the assistance robot's assistance task execution.

[0260] This application also provides an electronic device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the control methods of various embodiments.

[0261] This application also provides a computer-readable storage medium for storing a computer program for executing the control methods of the foregoing embodiments.

[0262] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the control method described above.

[0263] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate to describe embodiments of this application, for example, those that can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatuses.

[0264] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0265] It should be understood that in the description of the embodiments of this application, "multiple" means two or more, "greater than", "less than", "exceeding" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself.

[0266] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0267] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0268] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0269] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0270] It should also be understood that the various implementation methods provided in this application can be combined arbitrarily to achieve different technical effects.

[0271] The above provides a detailed description of the preferred embodiments of this application. However, this application is not limited to the above-described embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A control method for an assistive robot, the method being executed by an electronic device, the assistive robot being equipped with a robotic arm and a target sensor for object perception, the robotic arm being equipped with a tactile sensor, the control method comprising: In response to the detection that the target object needs assistance, the robot is controlled to move toward the target object. When the robot moves to the target moving position, the robotic arm is controlled to move to the target joint position. The target moving position and the target joint position are both determined based on the relative position between the target object and the target sensor. During or after the robotic arm moves to the target joint position, when the tactile sensor detects that an external force has been applied to the robotic arm, the assisting robot is subjected to compliant control.

2. The control method according to claim 1, wherein, The target sensors include visual sensors and pose sensors. The target movement position is determined based on the relative position between the target object and the pose sensor, and the target joint position is determined based on the relative position between the target object and the visual sensor.

3. The control method according to claim 2, wherein, The control of the assisting robot to move toward the target object includes: The target pose data of the pose sensor is acquired, and the target pose data is transformed based on a preset first transformation matrix to obtain the target movement position. Based on the target's movement position, the assisting robot is controlled to move towards the target object.

4. The control method according to claim 3, wherein, The step of acquiring the current target pose data of the pose sensor includes: The first pose data and the first covariance matrix obtained by predicting the pose sensor at the previous moment are acquired. The current second pose data are predicted based on the first pose data, and the current second covariance matrix is ​​predicted based on the first covariance matrix. Obtain the actual pose data currently acquired by the pose sensor, and determine the target gain based on the actual pose data and the second covariance matrix; The second pose data is corrected based on the target gain to obtain the current target pose data of the pose sensor.

5. The control method according to any one of claims 2 to 4, wherein, The control of the robotic arm to move to the target joint position includes: Acquire the current image data of the vision sensor and determine the key point positions of multiple key points of the target object in the image data; Transform the coordinate system of the key point location to the coordinate system of the assisting robot, and determine the center point location of the key point based on the transformed key point locations; The center point position is transformed based on a preset second transformation matrix to obtain the target joint position, and the robotic arm movement is controlled based on the target joint position.

6. The control method according to claim 5, wherein, The step of transforming the coordinate system of the key point location to the coordinate system of the target movement location includes: The parameters of the vision sensor are calibrated to obtain the intrinsic parameter matrix of the vision sensor; Determine the installation position of the vision sensor in the assistive robot, and determine the extrinsic parameter matrix of the vision sensor based on the installation position; Based on the intrinsic parameter matrix and the extrinsic parameter matrix, the coordinate system of the key point location is transformed to the coordinate system of the target movement location.

7. The control method according to claim 5, wherein, Before controlling the assisting robot to move toward the target object in response to recognizing the target object's intention to require assistance, the control method further includes: The target object is subjected to posture recognition based on each of the key points, wherein the result of the posture recognition includes whether the target object has the intention to need assistance or whether the target object does not have the intention to need assistance.

8. The control method according to claim 5, wherein, Before controlling the assisting robot to move toward the target object in response to recognizing the target object's intention to require assistance, the control method further includes: Detect the voice information of the target object; The speech information is subjected to feature extraction processing to obtain speech features; Based on the speech features, classification processing is performed to obtain the predicted type of the speech features; In response to the prediction type being that the target object has an intention to require assistance, it is determined that the target object has an intention to require assistance.

9. The control method according to claim 5, wherein, Before controlling the assisting robot to move toward the target object in response to recognizing the target object's intention to require assistance, the control method further includes: In response to receiving a first signal for the assisting robot, the first signal is detected and processed to obtain a detection result; In response to the detection result indicating that the first signal was issued by a pre-configured remote controller, it is determined that the target object intends to require assistance.

10. The control method according to any one of claims 1 to 9, wherein, The compliant control of the assisting robot includes: A kinematic model of the assisting robot is performed to obtain a dynamic model of the assisting robot, wherein the dynamic model is used to indicate the relationship between the joint torque and the joint acceleration of the assisting robot; The current external force data of the tactile sensor is acquired, and the external force data is converted to obtain the target joint torque; The target joint acceleration of the assisted robot is determined based on the target joint torque and the dynamic model, and the assisted robot is then subjected to compliant control based on the target joint acceleration.

11. The control method according to claim 10, wherein, The process of performing kinematic modeling on the assisting robot to obtain its dynamic model includes: The mass coefficient is determined based on the mass of the assisting robot, the joint position of the assisting robot, and the joint speed of the assisting robot; The friction coefficient is determined based on the friction force during the movement of the assisting robot and the joint position of the assisting robot; The gravity coefficient is determined based on the gravitational acceleration of the assisting robot and the joint position of the assisting robot; A first product between the mass coefficient and the joint acceleration of the assisting robot is determined. Based on the sum of the first product, the friction coefficient, and the gravity coefficient, a kinematic model of the assisting robot is performed to obtain the dynamic model of the assisting robot.

12. The control method according to claim 10, wherein, The tactile sensor is equipped with multiple tactile units. The conversion of the external force data to obtain the target joint torque includes: A third transformation matrix is ​​determined to transform the coordinate system in which the assisting robot is located to the coordinate system in which the joints of the assisting robot are located, and a fourth transformation matrix is ​​determined to transform the coordinate system in which the joints of the assisting robot are located to the coordinate system in which the tactile unit is located. Based on the third transformation matrix and the fourth transformation matrix, the transformation function between the coordinate system of each tactile unit and the coordinate system of the assisting robot is determined, and the derivative of the transformation function with respect to the joint position of the assisting robot is determined to obtain the Jacobian matrix corresponding to each tactile unit. The second product between the external force data collected by each of the tactile units and the corresponding Jacobian matrix is ​​determined, and the target joint torque is obtained based on the sum of multiple second products.

13. The control method according to claim 12, wherein, The plurality of tactile units are distributed cylindrically on the robotic arm. The first distance between any two adjacent tactile units is equal. Determining the fourth transformation matrix for transforming the coordinate system of the joints of the assistive robot to the coordinate system of the tactile units includes: Based on the first distance corresponding to the tactile unit, a fourth transformation matrix is ​​determined for transforming the coordinate system of the joints of the assisting robot to the coordinate system of the tactile unit.

14. The control method according to any one of claims 1 to 11, wherein, The control method further includes: During the process of the assisting robot following the target object, when an obstacle is detected within a preset angle range, the target size of the obstacle is detected. When the target size is outside the preset size range, the assisting robot is controlled to avoid the obstacle.

15. The control method according to claim 14, wherein, The control of the assisting robot to avoid the obstacle includes: The second distance between the assisting robot and the obstacle is detected. When the second distance is less than or equal to a preset distance threshold, a virtual space in which the assisting robot is located is constructed. The obstacle positions of the obstacles and the set endpoint positions of the assisting robot are determined in the virtual space. A first force vector of the obstacle is determined based on the obstacle's position and its mass, and a second force vector of the endpoint is determined based on the endpoint position and a preset mass at the endpoint position. The first force vector and the second force vector are combined to obtain the target force vector, and the assisting robot is controlled to avoid the obstacle according to the target force vector.

16. The control method according to any one of claims 1 to 15, wherein, The tactile sensor is equipped with multiple tactile units. When the tactile sensor detects that an external force has been applied to the robotic arm, the compliant control of the assisting robot includes: When the tactile sensor detects that an external force has been applied to the robotic arm, the current external force data of the tactile sensor is acquired, and the number of activated tactile units is determined based on the external force data. When the external force data indicates that the external force received by the tactile sensor is greater than or equal to the external force threshold, and the number of activations is greater than or equal to a preset number threshold, the assisting robot is subjected to compliant control.

17. A support robot, the support robot being provided with a control module, a robotic arm, and a target sensor for object perception, wherein the robotic arm is provided with a tactile sensor: The control module is used to respond to the recognition that the target object needs assistance, control the assistance robot to move towards the target object, control the robotic arm to move to the target joint position when the assistance robot moves to the target moving position, and control the robotic arm to move to the target joint position during or after the robotic arm moves to the target joint position, when the tactile sensor detects that the robotic arm is subjected to external force, perform compliant control on the assistance robot. in, The target movement position and the target joint position are both determined based on the relative position between the target object and the target sensor.

18. The assistive robot according to claim 17, wherein: The assisting robot includes a main body, and the robotic arm includes a first sub-arm and a second sub-arm. One end of the first sub-arm is movably connected to the main body, and the other end of the first sub-arm is movably connected to the second sub-arm. The first sub-arm and the second sub-arm are both equipped with the tactile sensor.

19. The assistive robot according to claim 18, wherein: The first subarm is connected to a first joint component and a second joint component at both ends, and the other end of the first joint component is connected to a third joint component, which is connected to the main body. One end of the second sub-arm is connected to the second joint component, the other end of the second sub-arm is connected to the fourth joint component, the fourth joint component is connected to the fifth joint component, the fifth joint component is connected to the sixth joint component, and the sixth joint component is connected to the end effector.

20. An electronic device comprising a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the control method according to any one of claims 1 to 16.

21. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the control method according to any one of claims 1 to 16.

22. A computer program product comprising a computer program that, when executed by a processor, implements the control method according to any one of claims 1 to 16.

Images