A robot compliance control method, device, equipment and storage medium

Abstract

The embodiments of the present application relate to the technical field of compliant motion control of collaborative robots, and more particularly to a method, apparatus, device, and storage medium for compliant robot control. The method comprises the following steps: first, constructing a feature space of a controller; then, aggregating the measured external forces to obtain feedback forces, and mapping the feedback forces to the feature space; next, solving the feature space based on force feedback to obtain a reference trajectory with compliant characteristics; then, outputting the desired motion of the robot based on visual negative feedback; and finally, mapping the desired motion to the robot joint space. The compliant control method provided in the present application can integrate pressure data with tactile data and apply them to the compliant control of the robot, thereby compensating for the visual blind spots of the visual sensor and achieving active compliant control of the entire robot body.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of compliant motion control of collaborative robots, and in particular to a robot compliant control method, apparatus, device, and storage medium. Background Art

[0002] Compliant control of a robot's underlying structure is an effective way to interact with the outside world. Through compliant control, the robot can avoid violent collisions with objects, thereby preventing damage to surrounding objects, especially people. This paves the way for robots to operate in manned environments. Currently, most robots utilize visual servo systems and incorporate force sensors at the end of the manipulator for end-of-line force feedback. This enables the robot hand to proactively avoid obstacles and maintain contact force within a safe threshold as it approaches a target object, enabling safe manipulation of surrounding objects, especially people.

[0003] Currently, visual servoing suffers from technical drawbacks: visual sensors have blind spots. During operation, visual sensors (especially those mounted on the robot's hand) often focus on the relative position and trajectory between the manipulator and obstacles, often failing to obtain image data for other parts of the robot. Furthermore, the torque sensor at the end of the robot can only measure the interaction force between the manipulator and the arm. Therefore, the robot's overall compliant control remains insufficient in complex and unknown scenarios. Summary of the Invention

[0004] The embodiments of the present application provide a robot compliance control method, apparatus, device and storage medium to compensate for the visual blind spots of visual sensors and achieve active compliance control of the entire robot.

[0005] To solve the above technical problems, in the first aspect, an embodiment of the present application provides a robot compliance control method, including: constructing a feature space of a controller; collecting the measured external forces to obtain feedback forces, and mapping the feedback forces to the feature space; solving the feature space based on force feedback to obtain a reference trajectory with compliance characteristics; outputting the expected motion amount of the robot based on visual negative feedback; and mapping the expected motion amount to the robot joint space.

[0006] In some exemplary embodiments, constructing a feature space of a controller includes: determining the dimension and basis vector group of the feature space based on the task space and the image feature set; and constructing a mapping function from the visual space to the feature space, a mapping function from the Cartesian space to the feature space, and a mapping function from the robot joint space to the feature space, respectively.

[0007] In some exemplary embodiments, the mapping function from the visual space to the feature space is:

[0008]

[0009] The mapping function from Cartesian space to feature space is:

[0010]

[0011] The mapping function from the robot joint space to the feature space is:

[0012]

[0013] Among them, V represents visual space, S represents feature space, D represents Cartesian space, and J represents robot joint space.

[0014] In some exemplary embodiments, after constructing the feature space of the controller, the method further includes: constructing a robot arm dynamics model in the feature space;

[0015] The feature space is shown as follows:

[0016] S∈R k

[0017] Among them, S represents the feature space and k represents the dimension of the feature space.

[0018] In some exemplary embodiments, the measured external forces are aggregated to obtain feedback forces, and the feedback forces are mapped to the feature space, including: performing spatiotemporal alignment on the external forces measured by the electronic skin, and converting the coordinates of the force-bearing points to the robot base coordinates; based on the coordinates of the force-bearing points, the contact area is determined, and the pressure of each electronic skin unit is solved; the pressure obtained by the electronic skin is integrated to determine the total pressure value, and the contact force center is determined; based on the contact force center, the Jacobian matrix corresponding to each contact force and the feature space is determined to obtain the contact force measured by the electronic skin; the contact force reference value and the contact force measured by the electronic skin are mapped to the feature space, and the contact force reference value and the contact force measured by the electronic skin are subtracted to obtain the external force interpolation.

[0019] In some exemplary embodiments, a reference trajectory with a compliant characteristic is obtained by solving the feature space based on force feedback; the method includes: mapping the reference trajectory in the Cartesian space given by the upper-level controller to the feature space; using the force feedback signal and the reference trajectory signal as input to solve a new reference trajectory with a compliant characteristic; wherein the calculation formula for solving the new reference trajectory with a compliant characteristic is:

[0020]

[0021] In some exemplary embodiments, the robot joint space is expressed as follows:

[0022] J∈R n

[0023] Where J represents the robot joint space and n is the number of joints.

[0024] In the second aspect, an embodiment of the present application also provides a robot compliance control device, including a space construction module, a force management module, an admittance control module and a visual servo module; the space construction module is used to construct the feature space of the controller; the force management module is used to collect the measured external forces, obtain feedback forces, and map the feedback forces to the feature space; the admittance control module is used to solve the feature space based on force feedback to obtain a reference trajectory with compliance characteristics; the visual servo module is used to output the expected motion amount of the robot according to visual negative feedback, and map the expected motion amount to the robot joint space.

[0025] In addition, the present application also provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned robot compliant control method.

[0026] In addition, the present application also provides a computer-readable storage medium storing a computer program, characterized in that when the computer program is executed by a processor, the above-mentioned robot compliance control method is implemented.

[0027] The technical solution provided by the embodiments of the present application has at least the following advantages:

[0028] The embodiments of the present application provide a robot compliance control method, apparatus, device and storage medium, the method comprising: first, constructing a feature space of a controller; then, collecting the measured external forces to obtain feedback forces, and mapping the feedback forces to the feature space; next, solving the feature space based on force feedback to obtain a reference trajectory with compliance characteristics; then, based on visual negative feedback, outputting the expected amount of motion of the robot; finally, mapping the expected amount of motion to the robot joint space. The robot compliance control method provided in the present application integrates the pressure data observed by the tactile electronic skin with the camera tactile data and applies them to the compliance control of the robot, so that the robot has the compliance characteristics of the whole body. When the robot's visual blind spot touches the outside world, the robot can adjust its movement in time to reduce the touch force to within the safety threshold, thereby achieving safe and reliable operation in complex environments.

[0029] In addition, the robot compliance control method provided by this application is based on the feature space underlying motion control method. By applying the tactile data observed by the electronic skin to the robot motion control, it effectively solves the challenges brought by heterogeneous data fusion. This method expands the source of feedback observation data for robot motion control and improves the robustness of the observation data source in robot motion control. In addition, the present invention applies the tactile data observed by the electronic skin to the robot's motion control link, not just to the robot's perception link. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] One or more embodiments are exemplarily described by the pictures in the corresponding drawings. These exemplifications do not constitute limitations on the embodiments. Unless otherwise stated, the pictures in the drawings do not constitute proportional limitations.

[0031] Figure 1 A flowchart of a robot compliance control method provided in one embodiment of the present application;

[0032] Figure 2 A schematic structural diagram of a robot compliance control device provided in one embodiment of the present application;

[0033] Figure 3 A schematic diagram of a robot compliance control method provided in one embodiment of the present application applied to robot motion control;

[0034] Figure 4 A schematic diagram of the force management module flow provided in one embodiment of the present application;

[0035] Figure 5 A schematic structural diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0036] As can be seen from the background technology, in the current existing robot compliance control methods, due to the visual blind spots of the visual sensor, the overall compliance control of the robot is obviously insufficient for complex and unknown scenes.

[0037] With the development of sensing and communication technologies, electronic skin with tactile senses has enabled robots to have full-body perception capabilities. To address the above technical issues, the present invention provides a robot compliance control method, comprising: first, constructing a feature space of a controller; then, aggregating the measured external forces to obtain feedback forces, and mapping the feedback forces to the feature space; next, solving the feature space based on force feedback to obtain a reference trajectory with compliance characteristics; then, outputting the robot's desired motion based on visual negative feedback; and finally, mapping the desired motion to the robot's joint space.

[0038] This application proposes a low-level compliance control method that integrates electronic skin touch and camera vision. By complementing the advantages of electronic skin with vision, it compensates for the defects of the visual blind spots of visual sensors and realizes active compliance control of the robot's entire body.

[0039] The following detailed description of the various embodiments of the present application is provided in conjunction with the accompanying drawings. However, those skilled in the art will appreciate that many technical details are provided in the various embodiments of the present application to facilitate a better understanding of the present application. However, even without these technical details and the various variations and modifications based on the following embodiments, the technical solutions claimed in the present application can still be implemented.

[0040] See Figure 1 , an embodiment of the present application provides a robot compliance control method, comprising the following steps:

[0041] Step S1: Construct the feature space of the controller.

[0042] Step S2: The measured external forces are collected to obtain feedback forces, and the feedback forces are mapped to the feature space.

[0043] Step S3: Solve in the feature space based on force feedback to obtain a reference trajectory with compliant characteristics.

[0044] Step S4: Based on the visual negative feedback, output the expected movement amount of the robot.

[0045] Step S5: Map the expected motion amount to the robot joint space.

[0046] It should be noted that the robot compliance control method provided by this application applies the tactile data observed by the electronic skin to the robot's motion control link, rather than just to the robot's perception link. The purpose of this application is to propose a bottom-level compliance control method that integrates electronic skin touch and camera vision. This method makes full use of the data observed by the electronic skin with tactile function and the visual camera to perform touch perception on the robot's entire body, and completes the fusion of vision and touch in the feature space. This method effectively solves problems such as spatiotemporal inconsistency in heterogeneous data fusion, and enables the robot to have whole-body compliance. Therefore, the robot can operate safely in a human environment and prevent violent collisions with surrounding objects and people.

[0047] Specifically, constructing the feature space of the controller in step S1 is mainly to provide a multi-dimensional space to achieve the spatiotemporal unification of vision and touch.

[0048] In some embodiments, after step S1 constructs the feature space of the controller, the method further includes step S11 of constructing a robot arm dynamics model in the feature space.

[0049] The feature space is shown as follows:

[0050] S∈R k

[0051] Among them, S represents the feature space and k represents the dimension of the feature space.

[0052] After constructing the controller's feature space, the dynamic model of the robotic arm is constructed in the feature space. Then, the measured external forces are aggregated. Specifically, the forces measured by the electronic skin are aggregated. Then, the force management module in the motion controller collects the feedback forces (using Next, the admittance control module solves the reference trajectory (denoted by x) with compliant characteristics in the feature space (denoted by S) based on force feedback. d′ Then, the feature points extracted by the visual camera are geometrically mapped to the feature space S; the desired motion of the robot (denoted by Δx) is solved based on the visual negative feedback; finally, the desired motion Δx is mapped to the robot joint space (denoted by J).

[0053] In some exemplary embodiments, the robot joint space is expressed as follows:

[0054] J∈R n

[0055] Where J represents the robot joint space and n is the number of joints.

[0056] In some embodiments, constructing the feature space of the controller in step S1 includes the following steps:

[0057] Step S101: Determine the dimension and basis vector group of the feature space based on the task space and the image feature set.

[0058] Step S102: construct a mapping function from visual space to feature space, a mapping function from Cartesian space to feature space, and a mapping function from robot joint space to feature space respectively.

[0059] Specifically, in some embodiments, the mapping function from visual space to feature space is:

[0060]

[0061] The mapping function from Cartesian space to feature space is:

[0062]

[0063] The mapping function from the robot joint space to the feature space is:

[0064]

[0065] Among them, V represents visual space, S represents feature space, D represents Cartesian space, and J represents robot joint space.

[0066] In some embodiments, in step S2, the measured external forces are aggregated to obtain feedback forces, and the feedback forces are mapped to the feature space, including the following steps:

[0067] Step S201: Perform spatiotemporal registration on the external force measured by the electronic skin, and convert the coordinates of the force point into the robot base coordinates.

[0068] Step S202: Based on the coordinates of the force-bearing points, the contact area is determined, and the pressure of each electronic skin unit is solved; the pressure obtained by the electronic skin is integrated to determine the total pressure value, and the contact force center is determined.

[0069] Step S203: Based on the contact force center, determine the Jacobian matrix corresponding to each contact force and the feature space to obtain the contact force measured by the electronic skin.

[0070] Step S204 : Mapping the contact force reference value and the contact force measured by the electronic skin to the feature space, and performing a subtraction between the contact force reference value and the contact force measured by the electronic skin to obtain an external force interpolation value.

[0071] Specifically, the force management module of the motion controller converts the contact force reference value (denoted by F ri ) and the contact force measured by the electronic skin (expressed by F i Represented) is mapped to the controller feature space, and the difference between the two is taken as the force input of the admittance control module in the motion controller. The calculation formula is:

[0072]

[0073] In some exemplary embodiments, step S3 solves the feature space based on force feedback to obtain a reference trajectory with compliant characteristics, including the following steps:

[0074] Step S301: Map the reference trajectory in the Cartesian space given by the upper controller to the feature space.

[0075] Step S302: Take the force feedback signal and the reference trajectory signal as input to solve a new reference trajectory with compliance characteristics (using x d′ express).

[0076] The calculation formula for solving the new reference trajectory with compliant characteristics is:

[0077]

[0078] like Figure 2 As shown, an embodiment of the present application also provides a robot compliance control device, including a space construction module 101, a force management module 102, an admittance control module 103 and a visual servo module 104; the space construction module 101 is used to construct a feature space of the controller; the force management module 102 is used to collect the measured external forces, obtain feedback forces, and map the feedback forces to the feature space; the admittance control module 103 is used to solve in the feature space based on force feedback to obtain a reference trajectory with compliance characteristics; the visual servo module 104 is used to output the expected motion amount of the robot according to visual negative feedback, and map the expected motion amount to the robot joint space.

[0079] See Figure 3 and Figure 4 Taking a robotic arm as an example, the force management module of the robotic arm motion controller converts the data measured by the electronic skin into the robot base coordinates. Based on the coordinates of the electronic skin units, it determines the area of contact between the robot and the outside world and calculates the pressure of each electronic skin unit. The pressure obtained by the electronic skin is integrated to determine the total pressure and the center of contact force. The Jacobian matrix corresponding to each contact force in the feature space is then determined. The external force is then mapped into the feature space and subtracted from the reference force to be imported into the admittance control module in the feature space. The admittance control module of the motion controller adjusts the desired motion trajectory input by the system based on the set system admittance characteristics to produce a compliant motion trajectory. After extracting features from the target point and actuator positions, the visual camera maps the eigenvalues into the feature space for negative feedback of the trajectory, resulting in the desired motion of the robotic arm. Finally, the motion controller outputs the required motion control variable for the robotic arm and maps it to the joint space.

[0080] Compared with the existing technology, the robot compliance method provided in this application is a heterogeneous data fusion method applied to the underlying control of the robot. This method realizes the fusion of vision and force perception in the feature space, improves the utilization efficiency of the robot's electronic skin data, and realizes the active compliance characteristics of the robot's entire body.

[0081] refer to Figure 5 Another embodiment of the present application provides an electronic device, comprising: at least one processor 110; and a memory 111 communicatively connected to the at least one processor; wherein the memory 111 stores instructions that can be executed by the at least one processor 110, and the instructions are executed by the at least one processor 110 so that the at least one processor 110 can execute any of the above method embodiments.

[0082] The memory 111 and the processor 110 are connected using a bus. The bus may include any number of interconnected buses and bridges. The bus connects various circuits of one or more processors 110 and the memory 111. The bus may also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits. These are all well known in the art and are therefore not described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver may be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor 110 is transmitted over a wireless medium via an antenna. Furthermore, the antenna receives data and transmits the data to the processor 110.

[0083] The processor 110 is responsible for managing the bus and general processing, and may also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. The memory 111 may be used to store data used by the processor 110 when performing operations.

[0084] Another embodiment of the present application relates to a computer-readable storage medium storing a computer program, which implements the above method embodiment when executed by a processor.

[0085] Based on the above technical solutions, the embodiments of the present application provide a robot compliance control method, device, equipment and storage medium, the method comprising: first, constructing a feature space of the controller; then, collecting the measured external forces to obtain feedback forces, and mapping the feedback forces to the feature space; next, solving the feature space based on force feedback to obtain a reference trajectory with compliance characteristics; then, based on visual negative feedback, outputting the expected amount of motion of the robot; finally, mapping the expected amount of motion to the robot joint space. The robot compliance control method provided in the present application integrates the pressure data observed by the tactile electronic skin with the camera tactile data and applies them to the compliance control of the robot, so that the robot has the compliance characteristics of the whole body. When the robot's visual blind spot touches the outside world, the robot can adjust its movement in time to reduce the touch force to within the safety threshold, thereby achieving safe and reliable operation in complex environments.

[0086] In addition, the robot compliance control method provided by this application is based on the feature space underlying motion control method. By applying the tactile data observed by the electronic skin to the robot motion control, it effectively solves the challenges brought by heterogeneous data fusion. This method expands the source of feedback observation data for robot motion control and improves the robustness of the observation data source in robot motion control. In addition, the present invention applies the tactile data observed by the electronic skin to the robot's motion control link, not just to the robot's perception link.

[0087] That is, those skilled in the art will understand that all or part of the steps in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a program, which is stored in a storage medium and includes a number of instructions for causing a device (which may be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps in the above-mentioned methods of each embodiment of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.

[0088] Those skilled in the art will appreciate that the above-described embodiments are specific examples for implementing the present application, and that in actual applications, various changes in form and detail may be made thereto without departing from the spirit and scope of the present application. Any person skilled in the art may make changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be subject to the scope defined in the claims.

Claims

1. A robot compliance control method, characterized in that: include: Construct the feature space of the controller; Aggregating the measured external forces to obtain feedback forces, and mapping the feedback forces to the feature space; Solving the feature space based on force feedback to obtain a reference trajectory with compliant characteristics; Based on visual negative feedback, output the robot's expected movement amount; Mapping the desired motion to the robot joint space; The step of aggregating the measured external forces to obtain feedback forces, and mapping the feedback forces to the feature space includes: Perform spatiotemporal registration of the external force measured by the electronic skin, and transform the coordinates of the force point into the robot base coordinates; Based on the coordinates of the force-bearing points, the contact area is determined, and the pressure of each electronic skin unit is solved; the pressure obtained by the electronic skin is integrated to determine the total pressure value, and the contact force center is determined; Based on the contact force center, determining the Jacobian matrix corresponding to each contact force and the feature space to obtain the contact force measured by the electronic skin; The contact force reference value and the contact force measured by the electronic skin are mapped to the feature space, and the contact force reference value and the contact force measured by the electronic skin are subtracted to obtain an external force interpolation value.

2. The robot compliance control method according to claim 1, characterized in that: The constructing of the feature space of the controller includes: Determining the dimension and basis vector group of the feature space based on the task space and the image feature set; A mapping function from the visual space to the feature space, a mapping function from the Cartesian space to the feature space, and a mapping function from the robot joint space to the feature space are constructed respectively.

3. The robot compliance control method according to claim 2, characterized in that: The mapping function from the visual space to the feature space is: The mapping function from the Cartesian space to the feature space is: The mapping function from the robot joint space to the feature space is: Among them, V represents visual space, S represents feature space, D represents Cartesian space, and J represents robot joint space.

4. The robot compliance control method according to claim 1, characterized in that: After constructing the characteristic space of the controller, the method further includes: constructing a dynamic model of the robot arm in the characteristic space; The feature space is shown as follows: S∈R k Among them, S represents the feature space and k represents the dimension of the feature space.

5. The robot compliance control method according to claim 1, characterized in that: Solving the problem in the feature space based on force feedback to obtain a reference trajectory with compliant characteristics includes: Mapping the reference trajectory in the Cartesian space given by the upper controller to the feature space; Taking the force feedback signal and the reference trajectory signal as input, a new reference trajectory with compliant characteristics is solved; The calculation formula for solving the new reference trajectory with compliant characteristics is:

6. The robot compliance control method according to claim 1, characterized in that: The robot joint space is shown as follows: J∈R n Where J represents the robot joint space and n is the number of joints.

7. A robot compliance control device, characterized in that: Including space construction module, force management module, admittance control module and visual servo module; The space construction module is used to construct the feature space of the controller; The force management module is used to collect the measured external forces to obtain feedback forces, and map the feedback forces to the feature space; The admittance control module is used to solve the feature space based on force feedback to obtain a reference trajectory with a compliant characteristic; The visual servo module is used to output the desired motion amount of the robot according to the visual negative feedback, and map the desired motion amount to the robot joint space; The step of aggregating the measured external forces to obtain feedback forces, and mapping the feedback forces to the feature space includes: Perform spatiotemporal registration of the external force measured by the electronic skin, and transform the coordinates of the force point into the robot base coordinates; Based on the coordinates of the force-bearing points, the contact area is determined, and the pressure of each electronic skin unit is solved; the pressure obtained by the electronic skin is integrated to determine the total pressure value, and the contact force center is determined; Based on the contact force center, determining the Jacobian matrix corresponding to each contact force and the feature space to obtain the contact force measured by the electronic skin; The contact force reference value and the contact force measured by the electronic skin are mapped to the feature space, and the contact force reference value and the contact force measured by the electronic skin are subtracted to obtain an external force interpolation value.

8. An electronic device, characterized in that: include: at least one processor; as well as, A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor so as to enable the at least one processor to execute the robot compliant control method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the robot compliance control method according to any one of claims 1 to 6 is implemented.

Images