Control method for support robot, support robot, electronic device, computer storage medium, and computer program product

Abstract

A control method for a support robot, the method comprising: in response to recognizing that a target object has an intent to require support, controlling a support robot to move toward the target object, and when the support robot moves to a target movement position, controlling a robot arm to move to a target joint position, wherein the target movement position and the target joint position are both determined on the basis of the relative position between the target object and a target sensor; and in the process of the robot arm moving to the target joint position or after the robot arm moves to the target joint position, when it is detected by means of a tactile sensor that an external force is applied to the robot arm, performing compliance control on the support robot. Further provided are a support robot, an electronic device, a computer storage medium, and a computer program.

Description

Control method of support robot, support robot, electronic device, computer storage medium and computer program product

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on the Chinese patent application with application number 202410163810.2 and application date of February 2, 2024, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby introduced into this application as a reference. Technical Field

[0003] The present application relates to the field of robots and artificial intelligence technology, and in particular to a control method for an assisting robot, an assisting robot, an electronic device, a computer storage medium, and a computer program. Background Art

[0004] In daily life, assistive mobility devices are widely used in various scenarios, including daily rehabilitation, to enhance balance, provide support, and improve stability when standing or walking. Currently, commonly used assistive mobility devices are generally crutches or other simple walking aids, and the intelligence level of these assistive mobility devices needs to be improved. Summary of the Invention

[0005] The following is an overview of the subject matter described in detail in this application. This overview is not intended to limit the scope of protection of the claims.

[0006] The embodiments of the present application provide a control method for an assisting robot, an assisting robot, an electronic device, a computer storage medium, and a computer program product, which can automatically perform assisting tasks and improve the accuracy and efficiency of the assisting robot in performing assisting tasks.

[0007] In one aspect, an embodiment of the present application provides a control method for a support robot, the method being performed by an electronic device. The support robot is provided with a robotic arm and a target sensor for object perception, the robotic arm being provided with a tactile sensor, and the control method comprising:

[0008] In response to recognizing that the target object has an intention to require assistance, controlling the assistance robot to move toward the target object, and when the assistance robot moves to a target movement position, controlling the robotic arm to move to a target joint position, wherein both the target movement position and the target joint position change according to the relative position between the target object and the target sensor;

[0009] During the process of the robotic arm moving to the target joint position or after the robotic arm moves to the target joint position, when the tactile sensor detects that an external force is applied to the robotic arm, the supporting robot is compliantly controlled.

[0010] On the other hand, an embodiment of the present application provides an assisting robot, which is provided with a control module, a robotic arm, and a target sensor for object perception, wherein the robotic arm is provided with a tactile sensor:

[0011] The control module is configured to, in response to recognizing that the target object has an intention to require assistance, control the assistance robot to move toward the target object, control the robotic arm to move to a target joint position when the assistance robot moves to a target movement position, and perform compliant control on the assistance robot when the tactile sensor detects that an external force is applied to the robotic arm during or after the robotic arm moves to the target joint position;

[0012] The target moving position and the target joint position both change according to the relative position between the target object and the target sensor.

[0013] On the other hand, an embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned control method of the supporting robot when executing the computer program.

[0014] On the other hand, an embodiment of the present application further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is executed by a processor to implement the above-mentioned control method of the supporting robot.

[0015] In another aspect, embodiments of the present application further provide a computer program product, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to implement the aforementioned control method for the assisting robot.

[0016] The embodiments of the present application include at least the following beneficial effects: in response to identifying that the target object has the intention to require assistance, the assistance robot is controlled to move toward the target object; when the assistance robot moves to the target moving position, the robotic arm is controlled to move to the target joint position, wherein both the target moving position and the target joint position change according to the relative position between the target object and the target sensor, so that the state of the target object can be sensed, and the assistance task can be automatically performed according to the state of the target object; on this basis, during the process of the robotic arm moving to the target joint position or after the robotic arm moves to the target joint position, when the tactile sensor detects that the robotic arm is applied with external force, the robotic arm is compliantly controlled so that the robotic arm can follow the posture of the target object to perform the assistance task, thereby improving the comfort of the target object during the assistance process; it can be seen that the control method provided by the present application can interact with the target object in a variety of ways, so that the assistance robot can automatically perform the assistance task, thereby improving the accuracy and efficiency of the assistance robot in performing the assistance task.

[0017] Other features and advantages of the present application will be set forth in the following description, and in part will be apparent from the description, or may be understood by practicing the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to provide a further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0019] FIG1 is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0020] FIG2 is a flow chart of a control method for a supporting robot provided in an embodiment of the present application;

[0021] FIG3 is a schematic diagram of a target object to be supported according to an embodiment of the present application;

[0022] FIG4 is a first schematic diagram of identifying a target object's intention to require assistance according to an embodiment of the present application;

[0023] FIG5 is a second schematic diagram of identifying a target object's intention to require assistance according to an embodiment of the present application;

[0024] FIG6 is a third schematic diagram of identifying a target object's intention to require assistance according to an embodiment of the present application;

[0025] FIG7 is a fourth schematic diagram of identifying a target object's intention to require assistance according to an embodiment of the present application;

[0026] FIG8 is a schematic diagram of a position transformation relationship provided in an embodiment of the present application;

[0027] FIG9 is a schematic diagram of a process for processing posture data measured by a posture sensor according to an embodiment of the present application;

[0028] FIG10 is a schematic diagram of the center point position provided in an embodiment of the present application;

[0029] FIG11 is a schematic diagram of converting the coordinate system of the joints of the assisting robot provided by an embodiment of the present application to the coordinate system of the tactile unit;

[0030] FIG12 is a schematic diagram of a flow chart of compliance control provided in an embodiment of the present application;

[0031] FIG13 is a schematic diagram of an assisting robot avoiding obstacles provided by an embodiment of the present application;

[0032] FIG14 is a schematic diagram of a first overall flow chart of a control method provided in an embodiment of the present application;

[0033] FIG15 is a schematic diagram of a second overall flow chart of a control method provided in an embodiment of the present application;

[0034] FIG16 is a schematic diagram of a third overall flow chart of the control method provided in an embodiment of the present application;

[0035] FIG17 is a schematic diagram of a first structure of the assisting robot provided in an embodiment of the present application;

[0036] FIG18 is a schematic diagram of a second structure of the assisting robot provided in an embodiment of the present application;

[0037] FIG19 is a schematic structural diagram of the assisting robot provided in an embodiment of the present application from another perspective;

[0038] FIG20 is a schematic diagram of the internal structure of a control cabinet of the assisting robot provided in an embodiment of the present application;

[0039] Figure 21 is a schematic diagram of the structural connection of the assisting robot provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0041] It should be noted that, in each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to the characteristics of the target object such as target object attribute information or attribute information set, the permission or consent of the target object will be obtained first, and the collection, use and processing of these data will comply with relevant laws, regulations and standards. Among them, the target object can be a user. In addition, when the embodiment of the present application needs to obtain target object attribute information, the target object's separate permission or separate consent will be obtained by means of a pop-up window or jumping to a confirmation page. After clearly obtaining the target object's separate permission or separate consent, the necessary target object-related data for enabling the normal operation of the embodiment of the present application will be obtained.

[0042] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0043] To facilitate understanding of the technical solutions provided in the embodiments of the present application, some key terms used in the embodiments of the present application are explained here:

[0044] 1) Target sensor: A device that detects and measures the physical quantity, state, position, and other parameters of a specific target or object. These sensors are often integrated into automation systems, robots, and various inspection equipment to collect and identify target information.

[0045] 2) Tactile sensors: These are sensors used in robots to mimic the sense of touch. They can be categorized by function as contact sensors, force-torque sensors, pressure sensors, and slip sensors.

[0046] 3) Pose: It describes the position and posture of an object (such as coordinates) in a specified coordinate system. For example, pose is often used to describe the position and posture of a robot in a spatial coordinate system. Among them, position refers to the positioning of a rigid body in space. The position of a rigid body can be represented by a 3×1 matrix, that is, the position of the center of the rigid body coordinate system in the base coordinate system; and posture refers to the orientation of a rigid body in space. The posture of a rigid body can be represented by a 3×3 matrix, that is, the posture of the rigid body coordinate system in the base coordinate system.

[0047] 4) Posture sensor: A sensor used to detect and measure the position and posture of an object in space. It can provide the object's three-dimensional position information (such as x, y, z coordinates) and posture information (such as pitch, roll, and yaw angles).

[0048] In daily life, in order to enhance balance, provide support and improve stability when standing or walking, related assistive devices are widely used in various scenarios such as daily rehabilitation. At present, commonly used assistive devices are generally crutches or other simple walkers. The intelligence level of such assistive devices needs to be improved.

[0049] In order to solve the above problems, the embodiments of the present application provide a control method for an assisting robot, an assisting robot and an electronic device, so that the assisting robot can automatically perform assisting tasks, thereby improving the accuracy and efficiency of the assisting robot in performing assisting tasks.

[0050] Refer to Figure 1, which is a schematic diagram of an implementation environment provided by an embodiment of the present application. The implementation environment includes a control module of the supporting robot, which can communicate with the sensor provided on the supporting robot. After obtaining the sensor data of the sensor, the control module can control the operation of the supporting robot based on the obtained sensor data.

[0051] Exemplarily, if the control module identifies through the obtained sensor data that the target object has the intention of requiring assistance, the control module can control the supporting robot to move toward the target object based on the obtained sensor data. When the supporting robot is controlled to move to the target moving position, the robotic arm can be controlled to move to the target joint position, wherein the target moving position and the target joint position both change according to the relative position between the target object and the target sensor, so that the state of the target object can be sensed and the supporting task can be automatically performed according to the state of the target object. On this basis, in the process of the robotic arm moving to the target joint position or after the robotic arm moves to the target joint position, when the control module senses that the robotic arm where the tactile sensor is located is applied with external force, it can obtain the interactive force data detected by the tactile sensor, and perform compliant control of the supporting robot based on the interactive force data, so that the supporting robot can follow the posture of the target object, thereby improving the comfort of the support. It can be seen that the control method provided in the present application can interact with the target object in a variety of ways, so that the supporting robot can automatically perform the supporting task, thereby improving the accuracy and efficiency of the supporting robot in performing the supporting task.

[0052] In addition, the control module can communicate with the server, and the server updates the algorithm package corresponding to the control method to the control module in real time.

[0053] In some embodiments, the control module is provided in a terminal device, which can be provided on the robot, or the terminal device is connected to a communication module on the robot via a short-range communication technology (e.g., ZigBee, Bluetooth, and Wi-Fi). Alternatively, the robot and the terminal device are connected by wire (e.g., a serial communication COM port or USB port provided on the robot, a data cable inserted into the port, and connected to the terminal device).

[0054] For example, the sensors installed on the robot collect environmental information and send the environmental information to the terminal device. The terminal device sends the environmental information to the server through the network. The server determines the robot's motion control instructions based on the environmental information of the robot's environment. The server sends the motion control instructions to the terminal device. The terminal device sends the motion control instructions to the robot, and the robot performs corresponding actions according to the motion control instructions.

[0055] Figure 2 is a flowchart of a control method for an assisting robot provided in an embodiment of the present application. The control method for the assisting robot can be executed by a control module of the assisting robot, or can be executed by the control module of the assisting robot and a server in cooperation. In the embodiment of the present application, the control method for the assisting robot is executed by the control module of the assisting robot as an example for illustration. The control method for the assisting robot includes but is not limited to the following steps 201 to 202.

[0056] Step 201: In response to recognizing that the target object has the intention of requiring assistance, the assistance robot is controlled to move toward the target object. When the assistance robot moves to the target moving position, the robotic arm is controlled to move to the target joint position.

[0057] In one possible implementation, the target object may refer to a user object that can be perceived by the support robot and has the intention to request assistance, and the support intention may refer to the purpose for which the target object expects assistance from the support robot, such as an object within the computer vision range of the support robot, or an object associated with the terminal through biometric information. The support robot can perceive and identify the target object through the installed target sensor, and evaluate whether the perceived target object requires assistance. For the support robot, the target user's support intention can be expressed in various ways, for example, through voice language, body posture, physical interaction, signal instructions, etc. The specific method can be based on the perception method provided by the support robot's target sensor.

[0058] In one possible implementation, the target moving position is within a preset range around the target object. The preset range can be set based on the actual needs of the target object. For example, based on the target object's height and arm outstretched length, the target object's arm range (a circular area with the arm outstretched length as the diameter and the target object as the center) is determined, and the arm range is used as the preset range for the target moving position.

[0059] The target movement position may refer to the location that the assisting robot needs to reach in order to complete the task of assisting the target object, such as the location close to the target object that requires assistance, or the target movement position may refer to the position of the assisting robot relative to the target object; and the target joint position may refer to the specific posture or joint angle that the robotic arm of the assisting robot needs to achieve in order to complete the task of assisting the target object, or the target joint position may refer to the position of the robotic arm relative to the target object. Both the target movement position and the target joint position change according to the relative position between the target sensor and the target object. This is equivalent to the fact that, in the process of the assisting robot moving toward the target object, the target sensor can be used to sense the state of the target object in real time, and the relative position between the target sensor and the target object can be corrected, thereby adjusting the target movement position and the target joint position in real time, and then automatically adjusting the assisting task to better fit the current state of the target object, thereby improving the comfort of assistance.

[0060] In one possible implementation, in the process of the supporting robot performing the supporting task of the target object, since the state of the target object changes in real time, the target moving position and target joint position can also be adjusted accordingly according to the different supporting intentions of the target object. For example, the two different supporting needs of supporting the target object to walk and supporting the target object to stand require different supporting postures (i.e., target moving position and target joint position) of the supporting robot. By automatically sensing the state of the target object and adjusting the target moving position and target joint position of the supporting robot in real time, different interaction methods, movement characteristics and assistance methods are provided, and diversified interactions are carried out with the target object, so that the supporting robot can automatically perform the supporting task, thereby improving the accuracy and efficiency of the supporting robot in performing the supporting task.

[0061] In one possible implementation, refer to FIG3 , which is a schematic diagram of the target object to be supported provided by an embodiment of the present application. When the supporting robot needs to support the target object to walk, the target moving position of the supporting robot can be located to the side and rear of the target object, and the state of the target object is sensed in real time during the supporting process. The target moving position is updated synchronously with the movement of the target object, so that the supporting robot keeps moving toward the target object, maintains the relative position of the target sensor and the target object stable, and achieves the effect of the supporting robot following the target object to provide support. In addition, the target joint position can also be adjusted in real time to maintain the power point of the robotic arm at the waist, hip and wrist of the target object, imitating the human supporting posture to guide the target object's walking direction and maintain the walking rhythm, thereby improving the comfort of the support.

[0062] In one possible implementation, when the support robot needs to help a target person stand, the support robot's target movement position can be located in front of the target person and fixed relative to the target person, helping the target person maintain balance. Specifically, the support robot moves toward the target person and maintains stability after reaching the target movement position. During the support process, the robot senses the target person's state in real time and adjusts the target joint position so that the robot arm's force points remain on the target person's upper body, such as the waist and arms, supporting the target person's weight, allowing the target person to move from a sitting or squatting position to a standing position.

[0063] In one possible implementation, the target object's intention to require assistance can be determined by sensing the state of the target object, that is, the relative position of the target sensor and the target object. The relative position of the target object and the target sensor may include the relative positions of the target object's center of gravity position, joint position, torso position, etc. to the target sensor. For example, if the target object's center of gravity position is at a lower position relative to the target sensor and there is no significant movement in consecutive frames, it can be considered that the target object is trying to stand, and thus it can be determined that the target object has the intention to require assistance to stand; if the target object's center of gravity position relative to the target sensor changes continuously in consecutive frames, and the leg joint position has a coherent displacement in a certain direction relative to the target sensor, it can be considered that the object is walking, and thus it can be determined that the target object has the intention to require assistance to walk.

[0064] Step 202 : During or after the robotic arm moves to the target joint position, when the tactile sensor detects that an external force is applied to the robotic arm, the supporting robot is compliantly controlled.

[0065] For example, compliance control is a control method that obtains control signals from force sensors and uses these signals to control the robot so that it moves in response to changes in the signal. Compliance is categorized into two types: active compliance and passive compliance. Passive compliance occurs when a robot uses auxiliary compliance mechanisms to naturally comply with external forces when in contact with the environment. Active compliance occurs when a robot uses force feedback information and adopts a specific control strategy to actively control the forces acting on it.

[0066] In one possible implementation, when the tactile sensor detects that an external force is applied to the robotic arm during or after the robotic arm moves to the target joint position, it can be considered that the target object is in contact with the support robot. Therefore, by performing compliant control on the support robot, the compliant control can enable the support robot to respond appropriately according to the target object's movement posture. For example, the support robot can be controlled to maintain a fixed output force without being affected by external pressure, or the support robot can be controlled to change position and speed in accordance with a preset impedance characteristic in response to external force to maintain a soft interaction with the target object, or to generate feedback on touch or contact while meeting a specific posture. Specifically, the compliant control of the support robot can be to adjust the parameters of the support robot's entire body joints (including upper and lower limbs), such as by adjusting the target joint position, target movement position, movement speed and acceleration of each joint, etc., to reduce the occurrence of impact or excessive pressure on the target object during the support process, thereby improving the comfort of support.

[0067] In one possible implementation, during the assistance process, the robotic arm of the assistance robot can be compliantly controlled. Specifically, the torque, position, velocity, acceleration, and other parameters of each joint of the robotic arm can be adjusted to change the trajectory and position of the robotic arm of the assistance robot. For example, when the tactile sensor detects that an external force is applied to the robotic arm, the torque output of the corresponding joint can be adjusted in real time based on the position and direction of the applied external force to maintain the posture of the target object or gently interact with the target object. At the same time, the dynamic impedance parameters of the corresponding joint are increased to make the movement of the robotic arm smoother and slower, conforming to the posture of the target object. In addition to compliant control of the robotic arm, the entire body of the assistance robot can also be compliant. By adjusting the joints of the entire body of the assistance robot to conform to the posture of the target object, the assistance robot can better imitate the characteristics of human limb movement and improve the comfort during the assistance process. For example, when the supporting robot is a wheeled robot with a robotic arm, it can adjust the degrees of freedom of the omnidirectional wheels (such as the forward distance, backward distance, and rotation angle of the omnidirectional wheels) and the movement acceleration of the omnidirectional wheels to coordinate with the flexible movement of the robotic arm when the tactile sensor detects that an external force is applied to the robotic arm. When the supporting robot is a footed robot with a robotic arm, it can adjust the height between the waist and feet by adjusting the joint controllers between the feet and the waist to coordinate with the flexible movement of the robotic arm when the tactile sensor detects that an external force is applied to the robotic arm. The flexible movement of the robotic arm refers to the ability of the robotic arm to adjust its own movement trajectory, speed, or strength in real time according to the force, position, and other information of the contact when it contacts the external environment or object, so as to adapt to changes in the external environment and objects and achieve a smooth and precise interaction effect.

[0068] In one possible implementation, the target sensors include multiple types, specifically visual sensors and posture sensors. The posture sensor can detect the spatial position and orientation of the target object, as well as the spatial position and orientation of the support robot. When the posture sensor detects a change in the support robot's position relative to the target object, it can adjust the support robot's target movement position in real time to achieve or maintain the desired relative posture. Therefore, the target movement position can be determined based on the relative position between the target object and the posture sensor. The visual sensor can identify the target object and its posture. When the relative posture, i.e., relative position, between the target object and the visual sensor changes, the motion position of the support robot's arm can be adjusted in real time to respond to the change in the target object's posture. Therefore, the target joint position is determined based on the relative position between the target object and the visual sensor.

[0069] In one possible implementation, at least one visual sensor and a posture sensor is provided. By integrating data from multiple sensors, the measurement errors of individual sensors can be compensated for, thereby improving the accuracy and reliability of object perception. For example, the posture sensor may include an inertial measurement unit, a lidar, an ultrasonic sensor, etc., so that the relative position between the target object and the posture sensor can be determined based on the relative positions of each posture sensor and the target object. Specifically, by collecting sensor data from each posture sensor at the same time, noise is filtered out of all sensor data, and all sensor data is fused and analyzed using a pre-trained neural network model to determine the relative position between the target object and the posture sensor. Accordingly, the visual sensor may include a depth camera, a stereo camera, an infrared camera, etc. By integrating multiple visual sensors, the three-dimensional posture of the target object can be more accurately determined, thereby determining the relative position between the visual sensor and the target object.

[0070] In one possible implementation, the support robot can be equipped with a visual sensor, such as a camera. The terminal can use the visual sensor to sense and identify people within the computer's visual range and identify the posture of the perceived target object. The visual sensor can capture real-time image data of the support robot's current environment and analyze the image data to determine whether the main body of the target object exists in the image data. Specifically, a human posture recognition algorithm, such as the Mediapipe recognition framework, can be used to perform human body recognition on the image data to determine the location of human skeletal points, thereby detecting key points of the target object in the image data. After determining the key points of the target object, posture recognition can be performed on the target object to determine the target object's human posture, and then determine whether the target object intends to assist and perform the assistance task. Human posture recognition algorithms with different response rates and complexities can be selected based on the sampling frequency of the visual sensor. For example, if the visual sensor's video stream frame rate is 30 frames per second, a model with medium model complexity can be selected to recognize human posture, so that each video node recognizes human posture at a frequency of 28 Hz, achieving real-time human recognition of the current image data of the visual sensor. When it is detected that the target object has no intention of requiring assistance, the assistance robot can continue to perform the current assistance task, or continue to perform posture recognition on the target object to determine whether there is an intention of requiring assistance.

[0071] For example, computer vision (CV) technology is the study of how machines can "see." Specifically, it refers to using cameras and computers to replace the human eye in identifying and measuring objects, and further processing the images to make them more suitable for human observation or transmission to instruments. In the embodiments of this application, a visual sensor performs key point detection and posture recognition on image data, thereby improving the control efficiency of the support robot.

[0072] Referring to Figure 4, Figure 4 is a schematic diagram of identifying the target object's intention to require assistance provided by an embodiment of the present application. The terminal can be deployed with a pre-trained posture estimation network model. After obtaining the image data captured by the visual sensor, the image data can be feature extracted to obtain first image data. The first image data is input into the posture estimation network model to obtain human body key points. The current posture of the target object 401 can be determined based on the human body key points, and then the current posture of the target object is matched with the object to be assisted that has the intention to require assistance. If the current posture matches the object to be assisted, the terminal can believe that the target object has the intention to require assistance. As shown in Figure 4, the posture to be assisted can be a body posture of the user making a half-squat, squatting, walking or a specific gesture. It should be noted that if there are multiple user objects in the image data, the posture of each user object can be identified, and the user object whose current posture matches the posture to be assisted is determined as the target object.

[0073] In addition, it is also possible to obtain historical image data taken by the visual sensor at the previous moment, perform feature extraction on the historical image data at the previous moment to obtain second image data, splice the first image data and the second image data to obtain third image data, input the third image data into the posture estimation network model to obtain target key points, and determine the estimated posture of the target object based on the target key points. By combining the motion changes in consecutive frames, a more accurate object posture can be obtained. If the estimated posture matches that of the object to be assisted with the intention of requiring assistance, the terminal can assume that the target object has the intention of requiring assistance.

[0074] In one possible implementation, the support robot may also be provided with an acoustic sensor, such as a microphone. The terminal may perceive the voice information emitted by the target object through the acoustic sensor provided by the support robot, and perform voice recognition on the voice information to determine whether the voice information contains the intention to request support.

[0075] In some embodiments, before step 201, the following processing is performed: detecting the voice information of the target object; performing feature extraction processing on the voice information to obtain voice features; performing classification processing based on the voice features to obtain a predicted type of the voice features; in response to the predicted type being that the target object has the intention to require assistance, determining that the target object has the intention to require assistance.

[0076] For example, an acoustic sensor installed in the support robot can detect the target object's voice information. The acoustic sensor converts the user's voice signal from a voice signal into an electrical signal, and then from the electrical signal into an analog signal. The voice information is represented as an analog signal. The voice information is processed by a neural network model. This can be achieved by converting the analog voice signal into a digital signal, sampling and quantizing it, and extracting features from the digitized voice signal. Common feature extraction methods include Mel-Frequency Cepstral Coefficients (MFCCs), filter banks, spectral features, etc. The neural network model can be a convolutional neural network or a recurrent neural network. The gate control unit in the neural network model classifies the voice features to obtain a predicted type of the voice features. The predicted types include: the target object has an intention to require support, and the target object does not have an intention to require support. If the predicted type is that the target object has an intention to require support, the process of step 201 is executed, that is, the support robot is controlled to provide support to the user.

[0077] For example, referring to FIG5 , FIG5 is a schematic diagram of an embodiment of the present application for identifying a target object's intention to require assistance. When a user says a sentence with the semantics of requiring assistance, such as "Please help me walk to the room" or "I need assistance," it can be considered that the target object has the intention to require assistance, thereby controlling the assistance robot to perform the assistance task. For example, in FIG5 , the assistance robot 502 recognizes the user's intention based on speech, and the target object 501 says "Please help me walk to the room." In response to recognizing that the target object 501 has the intention to require assistance, the assistance robot 502 performs the assistance task.

[0078] Artificial Intelligence (AI) is the theory, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In this embodiment, a neural network model is used to detect the user's voice information and, based on that information, determine whether the user intends to assist, thereby improving the robot's control efficiency.

[0079] In the embodiment of the present application, by detecting by voice whether the user needs assistance, the convenience of controlling the robot can be improved, the response speed of the robot can be improved, the user's use can be facilitated, and the versatility of the robot in actual application scenarios can be improved.

[0080] In one possible implementation, referring to FIG6 , FIG6 is a schematic diagram of an embodiment of the present application for identifying the target object's intention to require assistance. The hand of the target object 601 is placed on the robotic arm of the robot 602, and the terminal can sense the interaction force applied by the outside world through the tactile sensor provided on the robotic arm of the assisting robot. As shown in FIG6 , if the terminal detects that an external force is applied to the robotic arm through the tactile sensor without controlling the assisting robot to perform the assisting task, the object with the smallest relative distance to the target sensor can be used as the target object, and it is considered that the target object has the intention to require assistance. Alternatively, the terminal can call the visual sensor (if the assisting robot is also provided with a visual sensor such as a camera) to sense and identify the posture of each object within the computer vision range, and use the object that meets the posture to be assisted as the target object, and it is considered that the target object has the intention to require assistance.

[0081] In one possible implementation, referring to Figure 7, Figure 7 is a schematic diagram of identifying the target object's intention to require assistance provided in an embodiment of the present application. A help button may be provided on the assistance robot, or a remote controller that can be separated from the assistance robot. The terminal can sense the signal generated when the help button or the remote controller is triggered, and it can be considered that the target object that triggered the help button or the remote controller has the intention to require assistance.

[0082] In some embodiments, before step 201, the following processing is performed: in response to receiving a first signal for the assisting robot, the first signal is detected and processed to obtain a detection result; in response to the detection result indicating that the first signal is sent by a preconfigured remote controller, it is determined that the target object has the intention to require assistance.

[0083] For example, the first signal is sent by a remote controller that is detachable from the assisting robot. The first signal carries the device identification of the remote controller. The terminal in the assisting robot parses the first signal to obtain the device identification of the remote controller. In response to the presence of the pre-configured device identification of the remote controller in the first signal, it is determined that the first signal is used to indicate that the target object has the intention to require assistance, and step 201 is executed.

[0084] For example, in Figure 7 , a target object 701 holds a remote controller 703. Target object 701 triggers remote controller 703 to send a signal. A receiver in robot 702 receives the signal and moves toward target object 701 based on the signal. After reaching the target location, robot 702 performs the task of supporting target object 701.

[0085] Alternatively, the terminal can call on visual sensors (if the supporting robot is also equipped with visual sensors such as cameras) to perceive and identify the postures of various objects within the computer vision range, and take the objects that meet the posture to be supported as target objects, and consider that the target objects have the intention to require support.

[0086] In the embodiments of the present application, the remote controller can be used to control the robot's status information according to the user's needs, allowing the robot to adjust and respond in a timely manner to ensure that the robot operates as expected. Controlling the robot through a specific remote controller reduces the difficulty of user operation of the robot and improves the robot's versatility.

[0087] In one possible implementation, target sensors capture object-related sensor data, such as image data captured by a visual sensor and pose data detected by a pose sensor. Feature extraction and concatenation are performed to generate multi-source sensor data. This multi-source sensor data is then fed into a pre-trained deep learning model, such as a convolutional neural network model, or a real-time object detection algorithm, such as the YOLO (You Only Look Once) or SSD (Single Shot MultiBody Detector) object detection algorithm. This allows for rapid detection of human objects in the sensor data (e.g., images) and estimation of their positions. Pose estimation models, such as the OpenPose human pose estimation algorithm or the AlphaPose pose estimation algorithm based on key point detection, can be used to analyze the poses of human objects in the sensor data, analyze their behavioral patterns and movement intentions, and determine whether they intend to assist. Once the target object's intention to assist is identified, a dynamic path planning algorithm can be used to calculate the optimal path for the assisting robot to reach the target moving position, controlling the assisting robot to move toward the target object. Among them, the target sensor can not only sense human objects but also non-human objects, namely obstacles. For example, laser radar and visual sensors can sense the surrounding environment of the supporting robot. Therefore, during the movement process, the target sensor can also be used to sense obstacles on the moving path to achieve obstacle avoidance. At the same time, during the movement process, the target sensor is used to sense the target object in real time, correct the relative distance between the target object and the target sensor, and adjust the target moving position of the supporting target object. After the supporting robot reaches the target moving position, it can control the movement of the robotic arm to the human support point, namely the target joint position. At the same time, it can sense the triggering status of the tactile sensor on the robotic arm and the relative distance between the target sensor and the target object, determine the state of the target object, and adjust the position and strength of the robotic arm in real time to adapt to the movement and balance state of the target object.

[0088] Specifically, the assistance robot and its control method provided in the embodiments of the present application can be applied to various human-computer interaction scenarios such as medical rehabilitation scenarios, daily care scenarios, public assistance scenarios, rescue and disaster relief scenarios, and have a wide range of application scenarios. For example, in a hospital or rehabilitation center, the assistance robot can be controlled to help patients or elderly people walk, stand, sit down, and perform other posture changes. Since it can sense the state of the target object, the assistance plan can be adjusted in real time according to the state and needs of the target object, thereby improving the comfort of assistance while assisting the object in rehabilitation, and reducing the workload of physical rehabilitation therapists. For example, by controlling the assistance robot to detect the activity status of the elderly in real time, the elderly can be helped to perform various exercises indoors or outdoors (such as going up and down stairs, standing, walking) and provide timely assistance in emergency situations (such as helping to get up after falling down); for example, in public places such as stations or shopping malls, passengers or customers with limited mobility can be helped to reach their destination. Specifically, in the station scenario, passengers with limited mobility can be helped to move from the waiting area to the side of the car door, and passengers can be helped to board the door pedal area of ​​the bus, or passengers can be helped to sit in the car; and in the shopping mall scenario, customers can be helped to climb the steps of the escalator.

[0089] In one possible implementation, in the process of controlling the supporting robot to move toward the target object, the current target posture data of the posture sensor can be obtained, and the target posture data can be transformed based on a preset first transformation matrix to obtain the target moving position; based on the target moving position, the supporting robot is controlled to move toward the target object.

[0090] In one possible implementation, the posture sensor can detect the human posture data of the target object, that is, the target posture data can be the human posture data of the target object measured by the posture sensor at the current moment, or can be calculated based on the measured human posture data. The preset first transformation matrix can refer to a posture transformation matrix that converts from the spatial coordinates of the human body to the spatial coordinates of the supporting robot, which is used to represent the posture transformation relationship between the target moving position of the supporting robot and the position of the target object. The desired position of the supporting robot relative to the target object, that is, the target moving position, is obtained by converting the spatial coordinates of the target object, and the supporting robot is then controlled to move to the target moving position near the target object to perform the target object supporting task.

[0091] In one possible implementation, multiple first candidate transformation matrices may be pre-set to accommodate different postures of the target object or different assistance requirements. For example, different transformation matrices may be selected based on the target object's motion pattern and posture to determine the corresponding target movement position of the assistance robot, thereby ensuring the safety and comfort of the assistance. After determining the target movement position, in the process of controlling the assistance robot to move toward the target object based on the target movement position, the position and posture data of the target object may be detected in real time, and the target movement position of the assistance robot may be updated in real time based on the first transformation matrix. Alternatively, the position and posture data of the target object may be detected in real time, the motion pattern and posture of the target object may be determined, and the first transformation matrix may be updated in real time to adjust the target movement position of the assistance robot.

[0092] Referring to FIG8 , FIG8 is a schematic diagram of the position transformation relationship provided by an embodiment of the present application. The target pose data of the target object at the current moment can be detected by the pose sensor, wherein the state data of the target object, i.e., the position and direction, can be extracted from the target pose data, such as the position coordinates (x h ,y h , 1), and the angle θ of the target object relative to the supporting robot h Assuming that the target object's assistance requirement is to assist walking, a first transformation matrix corresponding to the assistance walking can be determined from a plurality of first candidate transformation matrices, thereby based on the preset first transformation matrix [0.92, 0, 1] T The target moving position of the supporting robot is obtained by transforming the position of the target object. Therefore, the supporting robot can be controlled to move from the original position to the target object and to the target moving position, thereby performing the task of supporting the target object. Specifically, the target moving position (x base ,y base , the calculation process of 1) is shown in the following formula:

[0093] Therefore, it can be seen that the target movement position of the supporting robot is related to the position of the target object relative to the supporting robot. As the supporting robot continues to move, the target pose data detected by the pose sensor continuously changes, and thus the determined target movement position also changes accordingly. In other words, the target movement position of the supporting robot changes with the relative position between the target object and the pose sensor. It should be noted that the target movement position can be determined based on the supporting robot's own spatial coordinate system. Therefore, although the target movement position is constantly changing, the target movement position at each moment can be relatively fixed relative to the world coordinate system or the coordinate system of the target object.

[0094] In one possible implementation, the target posture data can be obtained by synthesizing the posture data measured by the posture sensor at multiple moments. Specifically, the first posture data and the first covariance matrix predicted by the posture sensor at the previous moment can be obtained, and the current second posture data can be predicted based on the first posture data, and the current second covariance matrix can be predicted based on the first covariance matrix; the actual posture data currently collected by the posture sensor is obtained, and the target gain is determined based on the actual posture data and the second covariance matrix; the second posture data is corrected based on the target gain to obtain the current target posture data of the posture sensor.

[0095] In one possible implementation, due to measurement errors in the posture sensor, it is easy to confuse the target object (such as a person) with a non-target object (such as an obstacle), resulting in the inability to perform the support task. Therefore, it is necessary to correct the human posture data directly measured by the posture sensor to obtain highly accurate target posture data. Referring to Figure 9, Figure 9 is a flow chart of data processing of the posture data measured by the posture sensor provided in an embodiment of the present application. As shown in Figure 9, the posture data is corrected and processed. First, the posture data at the previous moment is used for prediction, including state prediction estimation of the posture data, and prediction estimation of the covariance matrix corresponding to the predicted state quantity. The predicted state value is corrected using the measurement value at the current moment, including calculating the target gain, and then correcting the predicted state value based on the target gain, and finally updating the previously predicted covariance matrix. The human posture data may include the angle, direction, three-dimensional coordinate position of the key points of the human joints, etc. Specifically, the human posture data detected by the posture sensor can be understood as the motion state of the target object, including the position and speed of the target object, that is, The first pose data is the pose data obtained by predicting the pose data measured by the pose sensor at the previous moment. Since the motion state of the target object is not affected by the external control input, the external control vector and the external control input matrix can be ignored based on the state equation of the correction system. Therefore, the second pose data predicted at the current moment can be obtained by multiplying the first pose data predicted at the previous moment with the state transfer matrix, that is, the state of the target object at the current moment is predicted. Among them, the state transfer matrix is ​​used to describe the relationship between the change of the correction system from the previous moment to the current moment. Specifically, the state transfer matrix is:

[0096] Wherein, ΔT represents the time interval for correcting the system, which indicates the time difference between the previous moment and the current moment, and is used to correct the change of state variables caused by the time interval.

[0097] According to the state equation of the corrected system after ignoring the influence of external control input, the calculation formula of the second pose data can be obtained, which can be shown as follows:

[0098] in, Represents the second pose data predicted at the current moment, x k-1 Represents the first pose data predicted at the previous moment, ω k It is expressed as the process noise of the modified system satisfying Gaussian distribution.

[0099] In one possible implementation, the first covariance matrix refers to a matrix that measures the uncertainty of the predicted first pose data when the pose sensor predicts the first pose data at the previous moment, and is used to represent the uncertainty of the state estimation at the previous moment in the correction system. Among them, the first covariance matrix is ​​an expression of the error distribution of the first pose data, and the diagonal elements in the first covariance matrix represent the variance of each state variable of the first pose data (i.e., the elements of the first pose data), while the non-diagonal elements represent the covariance between different state variables, which are used to reflect the correlation between different state variables. Based on the correction system, the second covariance matrix corresponding to the second pose data at the current moment can be obtained by multiplying the state transfer matrix with the first covariance matrix. Specifically, the calculation formula of the second covariance matrix can be shown as follows:

[0100] in, Expressed as the second pose data The corresponding second covariance matrix, P k-1 Expressed as the first pose data x k-1 The corresponding first covariance matrix, Q, is expressed as the process noise ω of the corrected system k The corresponding noise covariance matrix.

[0101] In one possible implementation, the target gain is used to optimize the accuracy of the prediction by combining the predicted state (i.e., the second pose data) predicted by the pose sensor with the observed value (i.e., the actual pose data) measured by the pose sensor, thereby weighing the second pose predicted pose and the actual pose data, correcting the second pose predicted data, and obtaining the target pose data. Specifically, the feedback gain matrix is ​​calculated by the second covariance matrix and the observation matrix pre-set in the correction system, and then the target gain is calculated by the actual pose data and the feedback gain matrix. The calculation formula of the feedback gain matrix can be shown as follows:

[0102] Among them, K kIt is represented as the feedback gain matrix, H is represented as the measurement matrix, and R is represented as the observation noise matrix. Specifically, the measurement matrix can be expressed as:

[0103] After obtaining the feedback gain matrix, the actual posture data and the second posture data can be weighed by the feedback gain matrix to obtain the target gain. The specific calculation formula can be shown as follows:

[0104] Where B represents the target gain, z k It is represented as actual posture data. It should be noted that the actual posture data can be obtained by low-pass correction based on the initial posture data directly measured by the posture sensor. Specifically, the correction calculation formula of the actual posture data can be shown as follows:

[0105] Among them, α is the correction coefficient, and the correction coefficient is the low-pass correction strength. It is represented by the second initial pose data directly measured by the pose sensor at the current moment, z k-1 Represents the first initial pose data directly measured by the pose sensor at the previous moment The historical actual posture data is obtained by low-pass correction.

[0106] By using the target gain to correct the second pose data, the current target pose data of the pose sensor can be obtained. The target pose data x k The specific correction formula can be shown as follows:

[0107] After correcting the second posture data of the predicted state using the target gain, the second covariance matrix corresponding to the second posture data can be updated. The specific matrix update formula can be shown as follows:

[0108] Among them, P k Expressed as the updated second covariance matrix, by modifying the system's feedback gain matrix K k The second covariance matrix predicted by the observation matrix H Update, so that the updated second covariance matrix P can be used k Predict the target pose data at the next moment.

[0109] In one possible implementation, the relationship between the predicted state and the observed quantity in the correction system can be expressed by an observation equation. Specifically, the observation equation can be expressed as follows: k =Hx k +v k(9)

[0110] Among them, v k It is expressed as the observation noise in the modified system that satisfies the Gaussian distribution.

[0111] In one possible implementation, in the process of controlling the movement of the robotic arm to the target joint position, the current image data of the visual sensor can be obtained first to determine the key point positions of multiple key points of the target object in the image data; the coordinate system where the key point positions are located is converted to the coordinate system where the supporting robot is located, and based on the multiple converted key point positions, the center point positions of the multiple key points are determined; the center point positions are transformed based on a preset second transformation matrix to obtain the target joint position, and the movement of the robotic arm is controlled based on the target joint position.

[0112] In one possible implementation, the target object can have multiple key points. Specifically, the key points of the target object can include prominent feature points on the human body, such as eyes, ears, shoulders, waist, elbows, wrists, hips, knees, ankles, and other joints. When performing key point detection on image data, key points can be determined from multiple joints based on the main body of the target object displayed in the image data. For example, if the main body of the target object displayed in the image data is the target object's upper limbs, joints such as the shoulders and waist can be selected as key points; if the main body of the target object displayed in the image data is the target object's lower limbs, joints such as the hips and knees can be selected as key points. Alternatively, when performing key point detection on image data, key points can be determined from multiple joints based on the target object's assistance requirement. For example, if the target object's assistance requirement is assistance in standing up, joints such as the shoulders, elbows, and wrists can be selected as key points; if the target object's assistance requirement is assistance in walking, joints such as the waist, hips, and elbows can be selected as key points.

[0113] In one possible implementation, the key points of the target object may be fixed joints on the human body. When the target object's multiple fixed joints cannot be detected from the image data, the target movement is re-determined based on the relative position between the pose sensor and the target object. After the assisting robot is re-controlled to reach the new target movement position, the visual sensor image data is re-acquired for analysis. For example, the key points of the target object may be the left shoulder, right shoulder, left hip, and right hip on the human body. When the image data captured by the visual sensor cannot fully detect these four key points of the target object, it can be assumed that the assisting robot is currently unable to accurately perceive the target object and is prone to perception errors. It can also be assumed that the relative position between the assisting robot and the target object is too close or too far, making it difficult to ensure safe execution of the assisting task. Therefore, the target movement position can be re-determined based on the relative position between the pose sensor and the target object, and the assisting robot can be controlled to move to the updated target movement position and then capture current image data using the visual sensor until the four key points of the target object, left shoulder, right shoulder, left hip, and right hip, can be fully detected in the new image data.

[0114] In one possible implementation, the key point positions may refer to the coordinates of each key point in the image data in a pixel coordinate system, where the pixel coordinate system may be defined with the upper left corner of the image as its origin, with the horizontal rightward direction being the positive direction of the abscissa axis, and the vertical downward direction being the positive direction of the ordinate axis. After performing image recognition on the image data to determine multiple key points, image registration may be used to determine the key point positions of each key point in the image data, i.e., the two-dimensional coordinates of each key point in the pixel coordinate system. The coordinate system in which the key point positions reside, i.e., the pixel coordinate system, is transformed to the coordinate system of the support robot. The transformed key point positions are then represented as the three-dimensional coordinates of each key point in the coordinate system of the support robot. The arithmetic mean of the multiple transformed key point positions is calculated to obtain the center point positions of the multiple key points, i.e., the coordinates of the center points of the multiple key points. The center point positions are then transformed using a preset second transformation matrix to obtain the target joint position. The second transformation matrix may be a transformation matrix that transforms the center point position to the desired support position of the robotic arm. In other words, the center point position serves as a reference point for the desired support position of the support robot to determine the target joint position of the robotic arm.

[0115] In one possible implementation, referring to FIG10 , FIG10 is a schematic diagram of the center point position provided in an embodiment of the present application. Image data of the current environment of the assisting robot is captured by a visual sensor, as shown in FIG10 . Due to the limitation of the visual sensor's shooting angle, if the image data captured by the visual sensor is required to capture the main body of the target object, the relative distance between the visual sensor and the target object needs to be greater than the minimum shooting distance of the visual sensor. For example, if the visual sensor is a depth camera and the minimum shooting distance of the depth camera is 50 cm, then when the relative distance between the depth camera and the target object reaches 50 cm, the main body of the target object will be displayed in the image data captured by the visual sensor. Otherwise, the relative position between the assisting robot and the target object needs to be adjusted and the image data needs to be recaptured. If the image data shows the main body of the target object, the target object's key points can be detected on the image data. The relative position between the target object and the visual sensor can be determined by using the four key points of the human body: the left shoulder (marked point A as shown in Figure 10), the right shoulder (marked point B as shown in Figure 10), the left hip (marked point C as shown in Figure 10), and the right hip (marked point D as shown in Figure 10). At the same time, the distance between the target object and the visual sensor can be determined. By calculating the center point position of the four key points of the left shoulder, right shoulder, left hip, and right hip (marked point E as shown in Figure 10), at this time, for the target object shown in Figure 10, the center point position is located at the target object's waist. Therefore, the target object's waist can be used as the reference point for the desired support position of the robot's mechanical arm, that is, the reference point for the target joint position. By transforming the center point position using the second transformation matrix, the support positions located on both sides of the waist can be obtained as the target joint positions (marked point F as shown in Figure 10).

[0116] In one possible implementation, the visual sensor is calibrated to obtain an intrinsic parameter matrix of the visual sensor; the installation position of the visual sensor in the assisting robot is determined, and the extrinsic parameter matrix of the visual sensor is determined based on the installation position; and the coordinate system of the key point position is converted to the coordinate system of the target moving position based on the intrinsic parameter matrix and the extrinsic parameter matrix.

[0117] In one possible implementation, the intrinsic parameter matrix of the visual sensor describes the internal optical characteristics of the visual sensor, including the focal length and the imaging center (optical center). The focal length of the visual sensor can be represented by the x-axis focal length and the y-axis focal length in pixels, respectively, and the optical center can be represented by the projection position on the image plane. For example, after calibrating the parameters of the visual sensor, the intrinsic parameter matrix of the visual sensor can be obtained. The intrinsic parameter matrix can be expressed as:

[0118] Among them, f xExpressed as 10-axis focal length, f y Expressed as the y-axis focal length, (c x , c y ) represents the optical center position.

[0119] In one possible implementation, the extrinsic parameter matrix of the vision sensor defines the spatial relationship between the coordinate system of the vision sensor and the coordinate system of the support robot (base). The extrinsic parameter matrix includes a rotation matrix and a translation matrix. The rotation matrix is ​​usually a 3x3 matrix, which represents the rotation of the vision sensor in the coordinate system of the support robot (base), and the translation matrix is ​​usually a 3x1 vector, which is used to represent the position of the vision sensor relative to the support robot (base). For example, according to the installation position of the vision sensor in the support robot, the extrinsic parameter matrix can be obtained, which can be specifically expressed as:

[0120] Among them, R represents the rotation matrix and T represents the translation matrix.

[0121] In one possible implementation, the coordinate system of the key point position is converted to the coordinate system of the target moving position. The coordinate system of the key point position, i.e., the pixel coordinate system, can be first converted to the coordinate system of the visual sensor. Then, the coordinate system of the visual sensor is converted to the coordinate system of the support robot (base). The specific calculation process can be shown as follows:

[0122] Among them, (u, v) represents the pixel coordinate system, that is, the coordinate system where the key point position is located; (X c , Y c , Z c ) is the coordinate system where the vision sensor is located, and the origin of the coordinate system where the vision sensor is located is the optical center of the vision sensor. The Z axis is parallel to the optical axis of the vision sensor. c That is the shooting direction of the visual sensor; (X B , Y B , Z B ) represents the coordinate system where the supporting robot (base) is located.

[0123] In one possible implementation, kinematic modeling of the supporting robot is performed based on the supporting robot's joint variables (such as displacement, velocity, acceleration, and position), and a dynamic model of the supporting robot can be obtained, which is used to describe the relationship between the joint torque of the supporting robot and the joint acceleration of the supporting robot. For example, a dynamic model is a mathematical model used to describe the motion laws and dynamic behavior of a robot when subjected to external forces. Dynamic models are usually based on the principles of classical mechanics, such as the Newton-Euler equations and the Lagrange equations. The establishment of a dynamic model of a robot can be used to understand and predict the robot's motion response, design a controller, and perform simulation analysis.

[0124] The support robot acquires external force data using tactile sensors installed on its robotic arms. This information is then converted to the coordinate system of the robot's corresponding joints using a transformation matrix. This results in the torque required for the robot's joints to comply with the external force information, known as the target joint torque. This allows the robot to generate sufficient force to counteract the external force while ensuring compliance and support stability. The target joint acceleration is then determined based on the target joint torque and the established dynamic model, and compliance control of the robot is then implemented based on this target joint acceleration.

[0125] In one possible implementation method, the mass coefficient can be first determined based on the mass of the supporting robot, the joint position of the supporting robot, and the joint speed of the supporting robot; the friction coefficient can be determined based on the friction force when the supporting robot moves and the joint position of the supporting robot; the gravity coefficient can be determined based on the gravitational acceleration of the supporting robot and the joint position of the supporting robot; the first product between the mass coefficient and the joint acceleration of the supporting robot can be determined, and the kinematic modeling of the supporting robot can be performed based on the sum of the first product, the friction coefficient, and the gravity coefficient to obtain the dynamic model of the supporting robot.

[0126] In one possible implementation, the mass coefficient can represent the inertial properties of each joint of the support robot. The mass coefficient can be understood as the mass matrix in the robot's dynamics model, used to describe the mass distribution of each joint of the support robot. Specifically, the mass coefficient of each joint in the support robot can be determined by the mass, joint position, and joint velocity of the corresponding joint of the support robot. The joint position of the support robot can be determined by the position encoder of each joint, and the mass coefficient can change with the change of joint position. The friction coefficient can be understood as the friction matrix in the dynamics model, which represents the influence of friction on each joint when the support robot moves. The direction and magnitude of friction corresponding to joints at different joint positions are different. Therefore, the friction coefficient corresponding to joints at different joint positions are different. The gravity coefficient can be understood as the gravity matrix in the dynamics model, which is used to represent the influence of gravity on each joint of the support robot. The direction of gravity on joints at different joint positions is different, affecting the posture transformation of each joint. The first product, obtained by multiplying the mass coefficient by the joint acceleration of the corresponding joint, represents the inertial force caused by the joint acceleration, that is, the inertial force that each joint needs to resist when moving, ensuring a timely response when subjected to external forces. The kinematic model of the support robot is constructed by summing the first product, the friction coefficient, and the gravity coefficient. The specific formula of the resulting dynamic model can be shown as follows:

[0127] Where M(q) represents the mass coefficient of the joint at joint position q, Expressed as the first product, The joint velocities of the joints expressed as joint positions, The friction coefficient of the joint expressed as the joint position, is represented as the joint velocity of the joint at the joint position, g(q) is represented as the gravity coefficient of the joint at the joint position q, and τ is represented as the joint torque of the joint at the joint position q.

[0128] Based on the dynamic model, the target joint acceleration generated by the external force can be calculated. The specific calculation formula of the target joint acceleration can be shown as follows:

[0129] After obtaining the target joint acceleration, an adaptive control strategy can be generated through a proportional-integral-derivative controller, namely a PID controller, to adjust the torque output of the corresponding joint in response to the external force, thereby controlling the movement of the assisting robot based on the target joint acceleration.

[0130] In one possible implementation, the tactile sensor includes multiple tactile units. Therefore, the third transformation matrix for converting the coordinate system of the supporting robot to the coordinate system of the joints of the supporting robot can be determined first, and the fourth transformation matrix for converting the coordinate system of the joints of the supporting robot to the coordinate system of the tactile unit can be determined; based on the third transformation matrix and the fourth transformation matrix, the conversion function between the coordinate system of each tactile unit and the coordinate system of the supporting robot is determined, and the differential of the conversion function with respect to the joint position of the supporting robot is determined to obtain the Jacobian matrix corresponding to each tactile unit; the second product between the external force data collected by each tactile unit and the corresponding Jacobian matrix is ​​determined, and the target joint torque is obtained based on the sum of multiple second products.

[0131] For example, the Jacobian matrix is ​​a square matrix consisting of a set of partial derivatives, which describes the rate of change of each component of a vector-valued function relative to each component of another vector-valued function. In the embodiment of the present application, the Jacobian matrix is ​​used to describe the relationship between the joint velocity and the end effector velocity of the robotic arm. The Jacobian matrix maps the joint angular velocity or angular acceleration to the linear velocity or linear acceleration of the end effector and is a key part in motion planning.

[0132] In one possible implementation, it is assumed that the supporting robot needs to perform an supporting task. At this time, the task space of the supporting task is nonlinearly related to the joint space, that is, the coordinate point of the target joint position required to perform the supporting task is located in the coordinate system where the supporting robot is located, while the coordinate point of the external force data received during the supporting task is located in the coordinate system where the tactile unit is located. It is impossible to directly use the external force data to adjust the target joint position to achieve compliant control. Therefore, it is necessary to obtain a conversion function determined by the third transformation matrix and the fourth transformation matrix. The conversion function can describe how to convert the coordinate system where each tactile unit is located to the coordinate system where the supporting robot is located. By performing differential operations on the joint positions of the supporting robot through the conversion function, the two different coordinate systems can be associated, and the Jacobian matrix of the corresponding joint can be obtained. The Jacobian matrix of the corresponding joint can be expressed as follows:

[0133] Where f is the transfer function, J(q) is the Jacobian matrix of the joint at the joint position q, and the Jacobian matrix of the corresponding joint can be obtained by taking the partial derivative of the transfer function f with respect to the joint position q.

[0134] The differential of the transfer function to the joint position of the assisting robot is used as the Jacobian matrix of each tactile unit to represent the influence of the magnitude and direction of the external force on each tactile unit on the position of different joints. Specifically, it is assumed that the direction of the axis of each joint i is represented by the unit vector z r iIndicates that the position of the origin of each joint coordinate system can be represented by vector p i Represents that the Jacobian matrix J of joint i i (q) can be expressed as:

[0135] Among them, (x b ,y b ) represents the coordinate system where the supporting robot is located, q k represents the kth joint, 1<k≤n, and n represents the total number of joints of the supporting robot, that is, the sum of the number of joints of the robotic arm and the number of joints of the base.

[0136] Since each joint of the supporting robot rotates around the z-axis, the pose transformation matrix T of each joint i is i It can be:

[0137] Substituting the pose transformation matrix of the joint corresponding to each tactile unit into the Jacobian matrix of the corresponding joint can obtain the Jacobian matrix corresponding to each tactile unit. The external force data collected by each tactile unit is multiplied by the corresponding Jacobian matrix to obtain the second product. The Jacobian matrix of each tactile unit is used to convert the external force data received by the corresponding tactile unit to the overall joint, that is, the second product can represent the force transmitted by each tactile unit to the overall joint. Then, by calculating the sum of each second product, the force applied by the outside world (i.e., the target object) when interacting with the robotic arm during the support process can be represented, thereby mapping the target joint torque. The calculation formula of the target joint torque τ can be shown as follows:

[0138] in, Denoted as the Jacobian matrix of the tactile unit j of the robot joint i, It is represented as the external force data of the tactile unit j of the robot joint i. Specifically, due to the external force f z In the z-axis direction, the external force data It can be expressed as follows:

[0139] In one possible implementation, the joints of the supporting robot include multiple sub-arms and joint components of the robotic arm, as well as the base joints of the supporting robot. Therefore, the joint torque and joint acceleration of each sub-arm and joint component of the robotic arm can be separately controlled to achieve compliant control; when the supporting robot is a wheeled robot, that is, the base joints of the supporting robot may include an omnidirectional wheel mounted on the base, wherein the joint torque of the omnidirectional wheel can be represented by the rotation angle of the omnidirectional wheel, and the joint acceleration of the omnidirectional wheel can be represented by the angular velocity, angular acceleration, and torque of the motor driving the omnidirectional wheel to rotate, thereby achieving compliant control by adjusting the rotation direction, rotation speed, and rotation acceleration of the omnidirectional wheel to adapt to the movement trend of the assisted object; when the supporting robot is a footed robot, the base joints of the supporting robot may include ankle joints, knee joints, hip joints, etc. Similarly, compliant control can be achieved by separately controlling the joint torque and joint acceleration of the ankle joints, knee joints, and hip joints.

[0140] In one possible implementation, to determine a fourth transformation matrix for converting the coordinate system of the assisting robot's joints to the coordinate system of the tactile units, it is necessary to perform kinematic modeling for each tactile unit of the tactile sensor relative to its position on the robotic arm, and establish a relationship between the tactile unit and the kinematic chain on the robotic arm. The joints of the robotic arm can be approximately viewed as cylinders. Therefore, the multiple tactile units of the tactile sensor can be considered to be cylindrically distributed on the surface of the robotic arm, with the first distance between any two adjacent tactile units being equal, where the first distance is represented by the distance between the two adjacent tactile units along the length of the robotic arm's connecting rod. Furthermore, the pose of the coordinate system of the tactile unit relative to the coordinate system of the assisting robot is the same as the pose of the coordinate system of the joint corresponding to the tactile unit relative to the coordinate system of the assisting robot. Therefore, based on the first distance corresponding to the tactile unit, the spacing between each tactile unit and the starting end of the joint in which it is located can be determined, and the fourth transformation matrix for converting the coordinate system of the assisting robot's joints to the coordinate system of the tactile unit can be determined.

[0141] Referring to Figure 11, Figure 11 is a schematic diagram of the conversion of the coordinate system of the joints of the assisting robot provided by the embodiment of the present application to the coordinate system of the tactile unit. Assuming that the first distance is 0.015 meters, as shown in Figure 11, the robotic arm can be regarded as a cylinder. Since the first distance between any two adjacent tactile units is equal, it is equivalent to that the multiple tactile units distributed in a cylindrical shape on the surface of the robotic arm can be divided along the height of the cylinder to obtain multiple rings, and the spacing between two adjacent rings is fixed at 0.015 meters. The tactile units on the same ring can be modeled as the same unit. Therefore, based on the first distance of the tactile unit and the position of the tactile unit (the ring it is in) on the robotic arm (cylinder), the conversion relationship between the tactile unit and the joint it is located in can be obtained, and then the fourth transformation matrix can be obtained. The fourth change matrix can be specifically expressed as:

[0142] As shown in Figure 11, t i It can be understood as the origin of the coordinate system of the joint i where the tactile unit is located, and It can be expressed as the origin of the coordinate system of the tactile unit j in the joint i, and (0.015×j) can be expressed as the distance between the origin of the coordinate system of the tactile unit j and the origin of the coordinate system of the joint i. It can be expressed as a fourth transformation matrix for converting the coordinate system where the joints of the assistance robot are located to the coordinate system where the tactile unit is located.

[0143] In one possible implementation, referring to Figure 12, which is a schematic diagram of a flow chart of compliance control according to an embodiment of the present application, whole-body compliance control based on a tactile sensor can be achieved through the following steps.

[0144] In step 1201 , haptic unit kinematics is modeled.

[0145] For example, kinematic modeling is performed on the position of each tactile unit of the tactile sensor on the robotic arm.

[0146] In step 1202, a dynamic model of the supporting robot is established.

[0147] Carry out dynamic modeling of the support robot and construct a dynamic model.

[0148] In step 1203 , the haptic unit Jacobian matrix is ​​calculated.

[0149] By modeling the tactile unit and the dynamic model of the supporting robot, the Jacobian matrix can be calculated for each tactile unit.

[0150] In step 1204 , the external force data is mapped to the joints.

[0151] The external force data received by the tactile sensor can be mapped to the force conditions of the joints of the robotic arm and supporting robot through Jacobian matrix conversion.

[0152] In step 1205, all joints of the body are compliantly controlled.

[0153] Based on the force conditions of each joint, the joint acceleration corresponding to the entire body joints of the supporting robot can be obtained for compliant control.

[0154] In one possible implementation, based on sensing the target object's need for assistance, the supporting robot can be controlled to follow the target object to perform the assistance task. For example, if the target object's intention to assist in walking is sensed, the supporting robot can be controlled to update the target moving position in real time based on the relative distance between the posture sensor and the target object, and the supporting robot can be controlled to move toward the target object based on the updated target moving position, so that the supporting robot follows the target object to perform the assistance task. In the process of following the target object, the assisting robot can detect obstacles through target sensors. When an obstacle is identified within a preset angle range, the obstacle outline can be determined by extracting features from the data obtained by the target sensor, and then the target size of the obstacle can be calculated. Specifically, environmental data of the surrounding environment can be obtained by using a posture sensor (such as a lidar), and the environmental data can be segmented into point clouds to identify key features such as obstacles and the ground. The segmented point cloud data can be clustered to form obstacle "clusters", so that the size of the obstacle can be further analyzed. Alternatively, a depth image of the current environment can be obtained through a visual sensor (such as a depth camera), and the depth of field of the depth image can be calculated to determine the distance data of the obstacle. The distance data can be identified and the size of the obstacle can be estimated using a pre-trained neural network model. More accurate environmental data can be obtained by combining data from multiple sensors. For example, the sensor data obtained by both the posture sensor and the visual sensor can be imported into a corrector or particle corrector for multi-source data fusion analysis to determine the size of the obstacle. The target size of the obstacle is compared with a preset size range. If the target size of the obstacle exceeds the preset size range, it means that the obstacle may interfere with the moving path of the supporting robot and affect the safety of the supporting robot. Therefore, it is necessary to control the supporting robot to avoid the obstacle.

[0155] In one possible implementation, the target sensors such as the posture sensor and / or visual sensor provided by the supporting robot can detect the second distance between the supporting robot and the obstacle. When the second distance is less than or equal to a preset distance threshold, it can be considered that the obstacle may affect the moving path of the supporting robot and the target object. Therefore, the supporting robot can be controlled to avoid the obstacle. At the same time, the target object can be guided to avoid the obstacle by providing information feedback to the target object (such as applying external force, issuing sound prompts, etc.) by flexibly controlling the robotic arm of the supporting robot.

[0156] In one possible implementation, referring to FIG13 , FIG13 is a schematic diagram of an obstacle avoidance system for an assisting robot according to an embodiment of the present application. When it is assumed that an obstacle affects the movement path of the assisting robot 1301, a virtual space can be created, and the sensor data sensed by the target sensor of the assisting robot can be mapped into the virtual space, so that the obstacle position of the obstacle and the end position of the movement path of the assisting robot can be represented in the virtual space, wherein the obstacle and the key position can be modeled to form a sphere. Based on the obstacle position and the obstacle mass corresponding to the obstacle, a first force vector of the obstacle, i.e., the repulsive force of the obstacle, can be determined. The first force vector is used to guide the assisting robot to move in a direction away from the obstacle position. Based on the end position and the corresponding preset mass, a second force vector of the end position is determined. The second force vector is used to guide the assisting robot to move toward the end position. The first force vector decreases as the second distance increases, and the second force vector decreases as the distance between the current position of the assisting robot and the end position increases. The first force vector and the second force vector are synthesized to obtain a target force vector. Specifically, the two vectors can be added or weighted added to obtain the target force vector. The speed and direction of the supporting robot can be controlled according to the target force vector, so that the supporting robot avoids obstacles while moving towards the set end position.

[0157] It should be noted that when the second distance between multiple obstacles and the supporting robot is less than or equal to the preset distance threshold, multiple obstacles and corresponding obstacle positions can be determined in the virtual space, and the first force vector corresponding to each obstacle can be obtained respectively. Then, all the first force vectors and the second force vectors can be synthesized to obtain the target force vector.

[0158] In one possible implementation, during the process of assisting the target object, the target sensor can also be used to detect changes in the environment surrounding the assisting robot in real time, so as to respond to emergencies such as environmental changes and ensure the safety of the user. For example, during the process of assisting the target object by the assisting robot across the zebra crossing, the assisting robot can perceive and identify dynamic obstacles such as vehicles and pedestrians in the surrounding environment, as well as static obstacles such as road shoulders and flower beds, and guide the target object to adjust walking speed, pause walking, or change walking path to avoid potential collisions.

[0159] In one possible implementation, the tactile sensor includes multiple tactile units, which can be evenly distributed on the surface of the robotic arm. The tactile sensor can collect external force data in real time. Based on the detection principles of different tactile sensors, it can be determined whether the tactile sensor is triggered by resistance changes, capacitance changes, piezoelectric effects, etc. When the tactile sensor detects that the robotic arm is subjected to external force, the electrical signals of the tactile units will change. These signals can be converted into recognizable external force data. At the same time, it can be analyzed which tactile units are activated, thereby determining the number of activated tactile units based on the external force data. Specifically, it can be determined whether a tactile unit is in an activated state by setting a sensitivity or activation threshold. When the applied external force exceeds the sensitivity or activation threshold of the tactile unit, the tactile unit can be in an activated state. Among them, when the tactile sensor detects that the robotic arm is subjected to external force, the external force data can be continuously obtained through the tactile sensor. When the detected external force data indicates that the current external force is greater than or equal to the external force threshold, and the activation quantity is greater than the preset quantity threshold, it can be considered that the current target object and the robotic arm are performing large-area force interaction behavior. Therefore, it can be considered that the target object is currently being supported by the supporting robot, or that the target object is currently experiencing an emergency such as an unstable center of gravity. Therefore, the supporting robot can be compliantly controlled to adjust the target joint position and target joint acceleration of the robotic arm to adapt to the posture of the target object.

[0160] In one possible implementation, when the external force data indicates that the external force applied to the tactile sensor is greater than or equal to the external force threshold, and the activation number is less than the preset number threshold, it can be considered that the target object is interacting with the robotic arm, but the stability of the support is low at this time. If the current posture of the supporting robot is changed, it is easy for the target object to lose support balance. Therefore, the supporting robot can be controlled to move to the target moving position or the robotic arm can be controlled to move to the target joint position according to the pre-set control strategy, without the need for compliant control of the supporting robot.

[0161] In one possible implementation, when the external force data indicates that the external force change value received by the tactile sensor is greater than or equal to a first external force change threshold within a first preset time period, and the external force change value within a second preset time period after the first preset time period is less than a second external force change threshold, it can be considered that the target object is currently performing force interaction with the robotic arm, and the current supporting posture of the supporting robot is stable with the supported posture of the target object, and the supporting robot can be compliantly controlled.

[0162] The control method of the supporting robot provided in the embodiment of the present application is described in detail below.

[0163] 14 , which is a schematic diagram of an overall flow chart of a control method provided in an embodiment of the present application, wherein the control method can be executed by a terminal, and the control method includes but is not limited to the following steps 1401 to 1410:

[0164] Step 1401: In response to recognizing that the target object has an intention to require assistance, obtaining the first position data and the first covariance matrix obtained by predicting the position sensor at the previous moment.

[0165] Step 1402: Predict the current second pose data based on the first pose data, and predict the current second covariance matrix based on the first covariance matrix.

[0166] Step 1403: Acquire actual posture data currently collected by the posture sensor, and determine the target gain based on the actual posture data and the second covariance matrix.

[0167] Step 1404: Correct the second posture data based on the target gain to obtain the current target posture data of the posture sensor.

[0168] Step 1405: transform the target posture data based on a preset first transformation matrix to obtain the target moving position.

[0169] Step 1406: Control the supporting robot to move toward the target object based on the target moving position.

[0170] Step 1407: When the supporting robot moves to the target moving position, the current image data of the visual sensor is obtained, and the key point positions of multiple key points of the target object in the image data are determined.

[0171] Step 1408: Convert the coordinate system where the key point positions are located to the coordinate system where the supporting robot is located, and determine the center point positions of the multiple key points based on the multiple converted key point positions.

[0172] Step 1409: transform the center point position based on a preset second transformation matrix to obtain a target joint position, and control the movement of the robotic arm based on the target joint position.

[0173] In this step, the target movement position and target joint position are determined based on the relative position between the target object and the target sensor;

[0174] Step 1410: During the process of the robotic arm moving to the target joint position or after the robotic arm moves to the target joint position, when the tactile sensor detects that an external force is applied to the robotic arm, the supporting robot is compliantly controlled.

[0175] 15 , which is a schematic diagram of an overall flow chart of a control method provided in an embodiment of the present application, wherein the control method can be executed by a terminal, and the control method includes but is not limited to the following steps 1501 to 1512:

[0176] Step 1501: In response to identifying that a target object has an intention to require assistance, controlling the assistance robot to move toward the target object.

[0177] Step 1502: When the supporting robot moves to the target moving position, control the robotic arm to move to the target joint position.

[0178] In this step, the target movement position and the target joint position are both determined based on the relative position between the target object and the target sensor.

[0179] Step 1503: During the process of the robotic arm moving to the target joint position or after the robotic arm moves to the target joint position, when the tactile sensor detects that the robotic arm is subjected to external force, the mass coefficient is determined according to the mass of the supporting robot, the joint position of the supporting robot, and the joint speed of the supporting robot.

[0180] Step 1504: Determine the friction coefficient based on the friction force during the movement of the supporting robot and the joint position of the supporting robot.

[0181] Step 1505: Determine the gravity coefficient according to the gravity acceleration of the supporting robot and the joint positions of the supporting robot.

[0182] Step 1506: Determine the first product between the mass coefficient and the joint acceleration of the supporting robot, perform kinematic modeling on the supporting robot according to the first product, the friction coefficient, and the sum of the gravity coefficient to obtain a dynamic model of the supporting robot.

[0183] In this step, the dynamic model is used to indicate the relationship between the joint torque of the supporting robot and the joint acceleration of the supporting robot.

[0184] Step 1507: Acquire current external force data of the tactile sensor.

[0185] Step 1508: Determine a third transformation matrix for transforming the coordinate system of the supporting robot into the coordinate system of the joints of the supporting robot.

[0186] Step 1509: Based on the first distance corresponding to the tactile unit, determine a fourth transformation matrix for converting the coordinate system of the joint of the assisting robot to the coordinate system of the tactile unit.

[0187] In this step, the plurality of tactile units are distributed in a cylindrical shape on the robotic arm, and the first distance between any two adjacent tactile units is equal.

[0188] Step 1510: Based on the third transformation matrix and the fourth transformation matrix, determine the conversion function between the coordinate system of each tactile unit and the coordinate system of the supporting robot, determine the differential of the conversion function with respect to the joint position of the supporting robot, and obtain the Jacobian matrix corresponding to each tactile unit.

[0189] Step 1511: Determine the second product between the external force data collected by each tactile unit and the corresponding Jacobian matrix, and obtain the target joint torque based on the sum of multiple second products.

[0190] Step 1512: Determine the target joint acceleration of the supporting robot according to the target joint torque and the dynamic model, and perform compliant control on the supporting robot based on the target joint acceleration.

[0191] 16 , which is a schematic diagram of an overall flow chart of a control method provided in an embodiment of the present application, wherein the control method can be executed by a terminal, and the control method includes but is not limited to the following steps 1601 to 1609:

[0192] Step 1601: In response to identifying that the target object has the intention of requiring assistance, controlling the assistance robot to move toward the target object.

[0193] Step 1602: When the assisting robot is following the target object and an obstacle is identified within a preset angle range, the target size of the obstacle is detected.

[0194] Step 1603: When the target size is outside the preset size range, detect the second distance between the supporting robot and the obstacle.

[0195] Step 1604: When the second distance is less than or equal to the preset distance threshold, construct a virtual space where the supporting robot is located.

[0196] Step 1605: Determine the obstacle position of the obstacle and the set end position of the supporting robot in the virtual space.

[0197] Step 1606: Determine a first force vector of the obstacle according to the obstacle position and the obstacle mass of the obstacle, and determine a second force vector of the end position according to the end position and the preset mass of the end position.

[0198] Step 1607: synthesize the first force vector and the second force vector to obtain a target force vector, and control the supporting robot to avoid obstacles according to the target force vector.

[0199] Step 1608: When the supporting robot moves to the target moving position, control the robotic arm to move to the target joint position.

[0200] In this step, the target movement position and the target joint position are both determined based on the relative position between the target object and the target sensor.

[0201] Step 1609: During the process of the robotic arm moving to the target joint position or after the robotic arm moves to the target joint position, when the tactile sensor detects that an external force is applied to the robotic arm, the supporting robot is compliantly controlled.

[0202] It will be appreciated that, although the various steps in the above-mentioned flow charts are shown in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless clearly stated in the present embodiment, the execution of these steps is not strictly limited in order, and these steps may be performed in other orders. Moreover, at least a portion of the steps in the above-mentioned flow charts may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but may be performed at different times, and the execution order of these steps or stages is not necessarily performed in sequence, but may be performed in turn or alternately with at least a portion of the steps or stages in other steps or other steps.

[0203] 17 , which is a schematic structural diagram of an assisting robot according to an embodiment of the present application, the assisting robot 1700 includes:

[0204] The robotic arm 1701 is provided with a tactile sensor;

[0205] Target sensor 1702, for object perception;

[0206] The control module 1703 is used to control the assisting robot 1700 to move toward the target object in response to recognizing that the target object has the intention to require assistance, and when the assisting robot 1700 moves to the target moving position, control the robotic arm 1701 to move to the target joint position, and during the process of the robotic arm 1701 moving to the target joint position or after the robotic arm 1701 moves to the target joint position, when the tactile sensor detects that the robotic arm is applied with external force, the assisting robot 1700 is compliantly controlled; wherein, the target moving position and the target joint position are both determined based on the relative position between the target object and the target sensor 1702.

[0207] In a possible implementation, the control module 1703 is further configured to:

[0208] Obtaining current target posture data from the posture sensor, and transforming the target posture data based on a preset first transformation matrix to obtain a target moving position;

[0209] The supporting robot 1700 is controlled to move toward the target object based on the target movement position.

[0210] In a possible implementation, the control module 1703 is further configured to:

[0211] Obtaining the first pose data and the first covariance matrix obtained by predicting the pose sensor at the previous moment, predicting the current second pose data based on the first pose data, and predicting the current second covariance matrix based on the first covariance matrix;

[0212] Obtain actual posture data currently collected by the posture sensor, and determine the target gain based on the actual posture data and the second covariance matrix;

[0213] The second posture data is corrected based on the target gain to obtain the current target posture data of the posture sensor.

[0214] In a possible implementation, the control module 1703 is further configured to:

[0215] Obtain the current image data of the visual sensor and determine the key point positions of multiple key points of the target object in the image data;

[0216] Convert the coordinate system where the key point positions are located to the coordinate system where the assisting robot 1700 is located, and determine the center point positions of the multiple key points based on the multiple converted key point positions;

[0217] The center point position is transformed based on a preset second transformation matrix to obtain a target joint position, and the movement of the robotic arm 1701 is controlled based on the target joint position.

[0218] In a possible implementation, the control module 1703 is further configured to:

[0219] Calibrate the parameters of the visual sensor to obtain the internal parameter matrix of the visual sensor;

[0220] Determine the installation position of the visual sensor in the assisting robot 1700, and determine the external parameter matrix of the visual sensor according to the installation position;

[0221] Based on the intrinsic parameter matrix and the extrinsic parameter matrix, the coordinate system of the key point position is converted to the coordinate system of the target moving position.

[0222] In a possible implementation, the control module 1703 is further configured to:

[0223] The target object is subjected to posture recognition according to each key point, wherein a result of the posture recognition includes whether the target object intends to require assistance or whether the target object does not intend to require assistance.

[0224] In a possible implementation, the control module 1703 is further configured to:

[0225] Performing kinematic modeling on the supporting robot 1700 to obtain a dynamic model of the supporting robot 1700 , wherein the dynamic model is used to indicate the relationship between the joint torque of the supporting robot 1700 and the joint acceleration of the supporting robot 1700 ;

[0226] Obtain the current external force data of the tactile sensor, convert the external force data, and obtain the target joint torque;

[0227] The target joint acceleration of the supporting robot 1700 is determined according to the target joint torque and the dynamic model, and the supporting robot 1700 is compliantly controlled based on the target joint acceleration.

[0228] In a possible implementation, the control module 1703 is further configured to:

[0229] Determining a mass coefficient according to the mass of the supporting robot 1700, the joint positions of the supporting robot 1700, and the joint speeds of the supporting robot 1700;

[0230] Determining the friction coefficient based on the friction force during the movement of the supporting robot 1700 and the joint positions of the supporting robot 1700;

[0231] Determine the gravity coefficient according to the gravity acceleration of the supporting robot 1700 and the joint positions of the supporting robot 1700;

[0232] The first product between the mass coefficient and the joint acceleration of the supporting robot 1700 is determined, and the kinematic model of the supporting robot 1700 is performed according to the sum of the first product, the friction coefficient and the gravity coefficient to obtain a dynamic model of the supporting robot 1700.

[0233] In a possible implementation, the control module 1703 is further configured to:

[0234] Determine a third transformation matrix for converting the coordinate system of the assisting robot 1700 to the coordinate system of the joints of the assisting robot 1700, and determine a fourth transformation matrix for converting the coordinate system of the joints of the assisting robot 1700 to the coordinate system of the tactile unit;

[0235] Determine the conversion function between the coordinate system of each tactile unit and the coordinate system of the assisting robot 1700 based on the third and fourth conversion matrices, determine the differential of the conversion function with respect to the joint positions of the assisting robot 1700, and obtain the Jacobian matrix corresponding to each tactile unit;

[0236] The second product between the external force data collected by each tactile unit and the corresponding Jacobian matrix is ​​determined, and the target joint torque is obtained based on the sum of multiple second products.

[0237] In a possible implementation, the plurality of tactile units are distributed in a cylindrical shape on the robotic arm 1701, and the first distance between any two adjacent tactile units is equal. The control module 1703 is further configured to:

[0238] Based on the first distance corresponding to the tactile unit, a fourth transformation matrix for converting the coordinate system where the joints of the assisting robot 1700 are located to the coordinate system where the tactile unit is located is determined.

[0239] In a possible implementation, the control module 1703 is further configured to:

[0240] When the assisting robot 1700 identifies an obstacle within a preset angle range while following the target object, it detects the target size of the obstacle.

[0241] When the target size is outside the preset size range, the assisting robot 1700 is controlled to avoid obstacles.

[0242] In a possible implementation, the control module 1703 is further configured to:

[0243] Detecting a second distance between the assisting robot 1700 and the obstacle, and when the second distance is less than or equal to a preset distance threshold, constructing a virtual space where the assisting robot 1700 is located;

[0244] Determining the obstacle position of the obstacle and the set end position of the assisting robot 1700 in the virtual space;

[0245] determining a first force vector of the obstacle according to the obstacle position and the obstacle mass of the obstacle, and determining a second force vector of the end position according to the end position and the preset mass of the end position;

[0246] The first force vector and the second force vector are synthesized to obtain a target force vector, and the assisting robot 1700 is controlled to avoid obstacles according to the target force vector.

[0247] In a possible implementation, the tactile sensor is provided with multiple tactile units, and the control module 1703 is further configured to:

[0248] When the tactile sensor detects that an external force is applied to the robotic arm, obtaining current external force data of the tactile sensor, and determining the number of activations of the tactile unit according to the external force data;

[0249] When the external force data indicates that the external force applied to the tactile sensor is greater than or equal to the external force threshold, and the activation quantity is greater than or equal to the preset quantity threshold, the supporting robot 1700 is compliantly controlled.

[0250] In one possible implementation, refer to FIG. 18 , which is a schematic structural diagram of an assisting robot 1700 provided in an embodiment of the present application. As shown in Figure 18, the supporting robot 1700 includes a main trunk 1801, and the robotic arm 1701 includes a first sub-arm 1802 and a second sub-arm 1803. One end of the first sub-arm 1802 is movably connected to the main trunk 1801, and the other end of the first sub-arm 1802 is movably connected to the second sub-arm 1803. Tactile sensors are provided on both the first sub-arm 1802 and the second sub-arm 1803, which means that the first sub-arm 1802 can move relative to the main trunk 1801, and the second sub-arm 1803 can move relative to the first sub-arm 1802. The first sub-arm 1802 can serve as the main part of the robotic arm 1701 and can bear greater force than the second sub-arm 1803, making it easier to support and assist the user; and the second sub-arm 1803 acts as an extension of the first sub-arm 1802 and can perform more complex and delicate movements relative to the first sub-arm 1802, such as fine-tuning the support strength or direction. In addition, the connecting rod length of the second sub-arm 1803 can be smaller than the connecting rod length of the first sub-arm 1802, thereby reducing the motion inertia of the second sub-arm 1803 and achieving more accurate motion control. Therefore, through the multi-level movable connection between the first sub-arm 1802 and the second sub-arm 1803, a wider range of motion of the robotic arm 1701 can be provided, better imitating the functions of human limbs, assisting the target object in moving, and providing support and protection. It should be noted that tactile sensors can be respectively provided on the first sub-arm 1802 and the second sub-arm 1803, so that more comprehensive external force data can be obtained, and then the posture of the target object can be more accurately identified to perform compliant control of the assisting robot 1700.

[0251] [Corrected 07.02.2025 according to Rule 91] In one possible implementation, the two ends of the first sub-arm 1802 are respectively connected to the first joint component 1804 and the second joint component 1805, the first joint component 1804 is connected to the third joint component 1806 through the first connecting axis, and the third joint component 1806 is connected to the main trunk 1801 through the second connecting axis; one end of the second sub-arm 1803 is connected to the second joint component 1805 through the third connecting axis, and the other end of the second sub-arm 1803 is connected to the fourth joint component 1807 through the fourth connecting axis, the fourth joint component 1807 is connected to the fifth joint component 1808 through the fifth connecting axis, the fifth joint component 1808 is connected to the sixth joint component 1809 through the sixth connecting axis, and the sixth joint component 1809 is connected to the end effector 1810. As shown in FIG18 , the robotic arm 1701 of the assisting robot 1700 may include six joint components, achieving six degrees of freedom. The third joint component 1806 may rotate relative to the main body 1801 around the second connecting axis, and the first joint component 1804 may rotate relative to the third joint component 1806 around the first connecting axis, so that the first sub-arm 1802 and the second joint component 1805 may move synchronously relative to the third joint component 1806. Furthermore, the second sub-arm 1803 may rotate relative to the second joint component 1805 around the third connecting axis, the fourth joint component 1807 may rotate relative to the second sub-arm 1803 around the fourth connecting axis, the fifth joint component 1808 may rotate relative to the fourth joint component 1807 around the fifth connecting axis, and the sixth joint component 1809 may rotate relative to the fifth joint component 1808 around the sixth connecting axis.

[0252] In a possible implementation, the sixth joint component 1809 is connected to an end effector 1810 , and the end effector 1810 is detachably mounted on the sixth joint component 1809 . The end effector 1810 can be a robot gripper, which can facilitate the transfer of objects by users, such as heavy objects, backpacks, crutches, etc.; the end effector 1810 can also be a safety restraint device, such as a seat belt, which can improve the stability and safety of the target object during movement; the end effector 1810 can also be a support handle or armrest, which is convenient for the target object to grasp to provide stable support for the target object; the end effector 1810 can also be a task controller for setting the assistance task. The task controller can be communicated with the control module 1703. The control module 1703 can control the assistance robot 1700 based on the assistance task set by the task controller. For example, the original assistance task is to assist the target object to walk. When the target object reaches the designated position (such as next to the seat), the task controller of the end effector 1810 can be triggered to reset the assistance task to assist the target object to sit down. Then, the control module 1703 can change the assistance task and control the assistance robot 1700 to assist the target object to sit down on the seat.

[0253] In addition, the supporting robot 1700 can be provided with multiple robotic arms 1701. As shown in Figure 18, the supporting robot 1700 can be provided with two robotic arms 1701, and in order to reduce collisions between the multiple robotic arms 1701 and achieve adjustments in different directions, the robotic arms 1701 can be installed on both sides of the main trunk 1801 respectively.

[0254] As shown in FIG18 , the assisting robot 1700 further includes a visual sensor 1811, a posture sensor, and a base 1812 for movement. The visual sensor 1811 can be mounted on the front side of the main trunk 1801 and can be located between the two robotic arms 1701, thereby enabling a wider field of view to capture image data and identify human postures. The posture sensor can be mounted on the front side of the base 1812 to sense the current environment of the assisting robot 1700, such as the location of the target object and obstacles. If there are multiple posture sensors, they can be arranged on the four side walls of the base 1812 to increase the sensing range and obtain more accurate environmental data.

[0255] As shown in Figure 19, Figure 19 is a schematic structural diagram of the assisting robot 1700 provided in an embodiment of the present application from another perspective. The base 1812 may be provided with a posture sensor, a control cabinet 1901, an antenna 1902, a base display screen 1905 for interactively displaying the status of the base 1812, and a control display screen 1906 for interactively displaying the status of the robotic arm 1701. Both the base display screen 1905 and the control display screen 1906 are mounted on the outside of the control cabinet 1901 to facilitate user interaction. The posture sensor may include a three-dimensional laser radar 1903 for detecting obstacles and a two-dimensional laser radar 1904 for detecting the position of target objects and obstacles. The two-dimensional laser radar 1904 may be mounted on the side wall of the base 1812, the three-dimensional laser radar 1903 may be mounted above the control cabinet 1901, and the antenna 1902 may be mounted on the side of the base 1812 near the control cabinet 1901. As shown in Figure 19, the base 1812 can be a Mecanum mobile platform, and the joints in the lower limb portion of the assisting robot 1700 can be a movable connection mechanism between the omnidirectional wheels 1813 and the base 1812. A Mecanum mobile platform is a mobile robot platform that uses Mecanum wheels (also known as Swiss wheels) to achieve omnidirectional mobility. This platform can move in any direction on a horizontal plane without turning, greatly enhancing the robot's maneuverability and flexibility. A Mecanum wheel is a large wheel composed of multiple smaller rollers, which are arranged at a certain angle (usually 45 degrees) to the axis of the large wheel. When the large wheel rotates, these small rollers can roll, allowing the large wheel to move in any direction on a horizontal plane without changing the wheel's orientation. Each Mecanum wheel can be driven independently, and by controlling the speed and direction of each wheel, complex motion trajectories can be achieved.

[0256] Among them, the main trunk 1801 can be located on the front side of the control cabinet 1901. Due to the weight of the robotic arm 1701 and the main trunk 1801, the center of gravity of the assisting robot 1700 moves forward. Therefore, the control cabinet 1901 can be placed on the back side of the main trunk 1801 and the base 1812 can be used to add counterweights to balance the center of gravity position of the assisting robot 1700 and improve the stability of the movement of the assisting robot 1700.

[0257] As shown in FIG20 , FIG20 is a schematic diagram of the internal structure of a control cabinet 1901 of an assistance robot 1700 provided in an embodiment of the present application. The assistance robot 1700 further includes a switch 2001 connected to an antenna 1902 and a control module 1703, a battery 2002 for powering the robot, and an inverter 2003 for converting the voltage of the battery 2002 to supply power. The control module 1703, the switch 2001, the battery 2002, and the inverter 2003 are installed in the control cabinet 1901. The control module 1703 can exchange data with a server through the switch 2001 and the antenna 1902, for example, uploading sensor data to the server or downloading updated algorithm programs from the server.

[0258] As shown in FIG21 , FIG21 is a schematic diagram of the structural connection of the assisting robot 1700 provided in an embodiment of the present application. The battery 2002 is connected to the inverter 2003, thereby converting the 48V voltage of the battery 2002 into a 221V AC voltage to power the electrical load of the assisting robot 1700. The control module 1703 can communicate with the tactile sensor, posture sensor (including the 3D laser radar 1903 and the 2D laser radar 1904), the visual sensor 1811, the manipulator control cabinet 2101, the base display 1905, and the control display 1906, respectively. The control module 1703 can obtain data from each sensor and interactive data from each display. The control module 1703 can generate control instructions based on the obtained data and send them to the manipulator control cabinet 2101, thereby controlling the manipulator 1701 and the base 1812 through the manipulator control cabinet 2101. Specifically, the control module 1703 can use different data transmission methods to establish connections with various components. For example, the control module 1703 can transmit data with the robotic arm 1701 and the base 1812 through the TCP / IP protocol, or transmit data with the tactile sensor and visual sensor 1811 through the USB3.0 transmission protocol, or transmit data with the posture sensor, base display 1905 and posture display through the HDMI transmission protocol.

[0259] The control module 1703 may be an industrial computer, which is used to execute the control methods of the aforementioned embodiments, and in response to identifying that the target object has the intention to require assistance, the assisting robot is controlled to move toward the target object. When the assisting robot moves to the target moving position, the robotic arm is controlled to move to the target joint position, wherein the target moving position and the target joint position are both determined based on the relative position between the target object and the target sensor, so that the state of the target object can be sensed, and the assisting task can be automatically performed according to the state of the target object. On this basis, during the process of the robotic arm moving to the target joint position or after the robotic arm moves to the target joint position, when the tactile sensor detects that the robotic arm is subjected to external force, the robotic arm is compliantly controlled so that the robotic arm can follow the posture of the target object, thereby improving the comfort of assistance. It can be seen that the control method provided in the present application can interact with the target object in a variety of ways, so that the assisting robot can automatically perform the assisting task, thereby improving the accuracy and efficiency of the assisting robot in performing the assisting task.

[0260] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the control methods of various embodiments when executing the computer program.

[0261] An embodiment of the present application further provides a computer-readable storage medium, which is used to store a computer program, and the computer program is used to execute the control methods of the aforementioned embodiments.

[0262] The present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to implement the above-mentioned control method.

[0263] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the numbers used in this way can be interchanged where appropriate to describe the embodiments of the present application, for example, they can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0264] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0265] It should be understood that in the description of the embodiments of the present application, multiple (or multiple items) means more than two, greater than, less than, exceed, etc. are understood to exclude the number itself, and above, below, within, etc. are understood to include the number itself.

[0266] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0267] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0268] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0269] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0270] It should also be understood that the various implementation methods provided in the embodiments of the present application can be combined arbitrarily to achieve different technical effects.

[0271] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the above implementation mode. Technical personnel familiar with the art can also make various equivalent modifications or substitutions under the shared conditions that do not violate the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A control method for a support robot, the method being executed by an electronic device, wherein the support robot is provided with a robotic arm and a target sensor for object perception, the robotic arm being provided with a tactile sensor, the control method comprising: In response to recognizing that the target object has an intention to require assistance, controlling the assistance robot to move toward the target object, and when the assistance robot moves to a target movement position, controlling the robotic arm to move to a target joint position, wherein the target movement position and the target joint position are both determined based on the relative position between the target object and the target sensor; During the process of the robotic arm moving to the target joint position or after the robotic arm moves to the target joint position, when the tactile sensor detects that an external force is applied to the robotic arm, the supporting robot is compliantly controlled.

2. The control method according to claim 1, wherein: The types of target sensors include visual sensors and posture sensors. The target moving position is determined based on the relative position between the target object and the posture sensor, and the target joint position is determined based on the relative position between the target object and the visual sensor.

3. The control method according to claim 2, wherein: The controlling the assisting robot to move toward the target object comprises: Acquire current target posture data of the posture sensor, and transform the target posture data based on a preset first transformation matrix to obtain the target moving position; The supporting robot is controlled to move toward the target object based on the target movement position.

4. The control method according to claim 3, wherein: The obtaining of the current target posture data of the posture sensor includes: Acquire first pose data and a first covariance matrix obtained by predicting the pose sensor at a previous moment, predict current second pose data based on the first pose data, and predict current second covariance matrix based on the first covariance matrix; Acquire actual posture data currently collected by the posture sensor, and determine a target gain based on the actual posture data and the second covariance matrix; The second posture data is corrected based on the target gain to obtain current target posture data of the posture sensor.

5. The control method according to any one of claims 2 to 4, wherein: The controlling the robot arm to move to a target joint position includes: Acquire current image data of the visual sensor, and determine key point positions of multiple key points of the target object in the image data; Converting the coordinate system where the key point positions are located to the coordinate system where the assisting robot is located, and determining the center point positions of the plurality of key points based on the converted key point positions; The center point position is transformed based on a preset second transformation matrix to obtain the target joint position, and the robot arm movement is controlled based on the target joint position.

6. The control method according to claim 5, wherein: The step of converting the coordinate system of the key point position to the coordinate system of the target moving position includes: Calibrate the visual sensor to obtain an internal parameter matrix of the visual sensor; Determining an installation position of the visual sensor in the assisting robot, and determining an external parameter matrix of the visual sensor according to the installation position; The coordinate system where the key point position is located is converted to the coordinate system where the target moving position is located based on the intrinsic parameter matrix and the extrinsic parameter matrix.

7. The control method according to claim 5, wherein: In response to identifying that the target object has an intention to require assistance, before controlling the assistance robot to move toward the target object, the control method further includes: The target object is subjected to posture recognition according to each of the key points, wherein a result of the posture recognition includes whether the target object has an intention to require assistance or whether the target object has no intention to require assistance.

8. The control method according to claim 5, wherein: In response to identifying that the target object has an intention to require assistance, before controlling the assistance robot to move toward the target object, the control method further includes: Detecting voice information of the target object; Performing feature extraction processing on the voice information to obtain voice features; Performing classification processing based on the speech feature to obtain a predicted type of the speech feature; In response to the prediction type being that the target object intends to require assistance, it is determined that the target object intends to require assistance.

9. The control method according to claim 5, wherein: In response to identifying that the target object has an intention to require assistance, before controlling the assistance robot to move toward the target object, the control method further includes: In response to receiving a first signal directed to the supporting robot, detecting and processing the first signal to obtain a detection result; In response to the detection result indicating that the first signal is sent by a preconfigured remote controller, it is determined that the target object intends to require assistance.

10. The control method according to any one of claims 1 to 9, wherein: The compliant control of the supporting robot comprises: Performing kinematic modeling on the supporting robot to obtain a dynamic model of the supporting robot, wherein the dynamic model is used to indicate a relationship between a joint torque of the supporting robot and a joint acceleration of the supporting robot; Acquiring current external force data of the tactile sensor, converting the external force data to obtain a target joint torque; The target joint acceleration of the supporting robot is determined according to the target joint torque and the dynamic model, and the supporting robot is compliantly controlled based on the target joint acceleration.

11. The control method according to claim 10, wherein: The kinematic modeling of the supporting robot to obtain the dynamic model of the supporting robot includes: determining a mass coefficient according to the mass of the supporting robot, the joint positions of the supporting robot, and the joint speeds of the supporting robot; determining a friction coefficient according to the friction force during the movement of the supporting robot and the joint positions of the supporting robot; determining a gravity coefficient according to the gravitational acceleration of the supporting robot and the joint positions of the supporting robot; A first product between the mass coefficient and the joint acceleration of the supporting robot is determined, and kinematic modeling of the supporting robot is performed according to the sum of the first product, the friction coefficient, and the gravity coefficient to obtain a dynamic model of the supporting robot.

12. The control method according to claim 10, wherein: The tactile sensor is provided with a plurality of tactile units, and the external force data is converted to obtain a target joint torque, including: Determine a third transformation matrix for converting the coordinate system of the supporting robot to the coordinate system of the joints of the supporting robot, and determine a fourth transformation matrix for converting the coordinate system of the joints of the supporting robot to the coordinate system of the tactile unit; Determine, based on the third transformation matrix and the fourth transformation matrix, a conversion function between a coordinate system where each tactile unit is located and a coordinate system where the supporting robot is located, determine the differential of the conversion function with respect to a joint position of the supporting robot, and obtain a Jacobian matrix corresponding to each tactile unit; Determine a second product between the external force data collected by each tactile unit and the corresponding Jacobian matrix, and obtain a target joint torque based on a sum of multiple second products.

13. The control method according to claim 12, wherein: The plurality of tactile units are distributed in a cylindrical shape on the robotic arm, and a first distance between any two adjacent tactile units is equal. The fourth transformation matrix for converting the coordinate system of the joints of the assisting robot to the coordinate system of the tactile units includes: Based on the first distance corresponding to the tactile unit, a fourth transformation matrix for converting a coordinate system where the joint of the assistance robot is located to a coordinate system where the tactile unit is located is determined.

14. The control method according to any one of claims 1 to 11, wherein: The control method further includes: In the process of the assisting robot following the target object, when an obstacle is identified within a preset angle range, detecting a target size of the obstacle; When the target size is outside a preset size range, the assisting robot is controlled to avoid the obstacle.

15. The control method according to claim 14, wherein: The controlling the assisting robot to avoid the obstacle comprises: detecting a second distance between the assisting robot and the obstacle, and constructing a virtual space in which the assisting robot is located when the second distance is less than or equal to a preset distance threshold; Determining an obstacle position of the obstacle and a set end position of the assisting robot in the virtual space; determining a first force vector of the obstacle according to the obstacle position and the obstacle mass of the obstacle, and determining a second force vector of the end position according to the end position and the preset mass of the end position; The first force vector and the second force vector are synthesized to obtain a target force vector, and the supporting robot is controlled to avoid the obstacle according to the target force vector.

16. The control method according to any one of claims 1 to 15, wherein: The tactile sensor is provided with a plurality of tactile units, and when the tactile sensor detects that the mechanical arm is subjected to an external force, the supporting robot is compliantly controlled, including: When the tactile sensor detects that an external force is applied to the robotic arm, obtaining current external force data of the tactile sensor, and determining the number of activations of the tactile unit according to the external force data; When the external force data indicates that the external force applied to the tactile sensor is greater than or equal to an external force threshold, and the activation quantity is greater than or equal to a preset quantity threshold, the supporting robot is compliantly controlled.

17. A support robot, comprising a control module, a robotic arm, and a target sensor for object perception, wherein the robotic arm is provided with a tactile sensor: The control module is configured to, in response to recognizing that the target object has an intention to require assistance, control the assistance robot to move toward the target object, control the robotic arm to move to a target joint position when the assistance robot moves to a target movement position, and perform compliant control on the assistance robot when the tactile sensor detects that an external force is applied to the robotic arm during or after the robotic arm moves to the target joint position; in, The target movement position and the target joint position are both determined according to the relative position between the target object and the target sensor.

18. The assistance robot according to claim 17, wherein: The supporting robot includes a main trunk, and the robotic arm includes a first sub-arm and a second sub-arm. One end of the first sub-arm is movably connected to the main trunk, and the other end of the first sub-arm is movably connected to the second sub-arm. The tactile sensors are both provided on the first sub-arm and the second sub-arm.

19. The assistance robot according to claim 18, wherein: The two ends of the first sub-arm are respectively connected to a first joint component and a second joint component, the other end of the first joint component is connected to a third joint component, and the third joint component is connected to the main trunk; One end of the second sub-arm is connected to the second joint component, the other end of the second sub-arm is connected to the fourth joint component, the fourth joint component is connected to the fifth joint component, the fifth joint component is connected to the sixth joint component, and the sixth joint component is connected to the end effector.

20. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the control method according to any one of claims 1 to 16 when executing the computer program.

21. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the control method according to any one of claims 1 to 16.

22. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the control method according to any one of claims 1 to 16 is implemented.

Images