Robot-based hinged object control method, system and equipment and medium

Abstract

The invention provides a robot-based hinged object control method, system and device and a medium, and the method comprises the steps: firstly obtaining the initial hinged attribute of an object according to the hinged type of a target hinged object, so as to preliminarily determine the motion characteristics of the object when the hinged motion is triggered; and then based on the initial hinging attribute, the robot is controlled to carry out motion track control on the target hinging type object, so that an initial track execution result of the robot in the control process is obtained. And finally, through a preset ASSG algorithm and an initial trajectory execution result, performing attribute correction on the initial hinge attribute of the object, and based on the obtained hinge correction attribute, controlling the robot to repeatedly perform motion trajectory control on the object. The method aims at fully knowing physical attributes and hinging motion characteristics of a target hinging type object through iterative optimization based on a single real-time track result, so that a stable robot hinging control method is generated, and the control effect of a robot on the hinging type object is effectively improved.

Description

Technical Field

[0001] The present application relates to the field of reinforcement learning technology, and in particular to a robot-based articulated object control method, system, device and medium. Background Art

[0002] In current related technologies, the automated control of rigid objects by robots has been widely used. However, due to the complex physical structure of articulated objects and the limited action space dimension, the current automated control of articulated objects by robots cannot adapt to many different types of articulated objects, and the control effect is poor.

[0003] Therefore, how to improve the control effect of the robot on articulated objects has become a technical problem that technical personnel in this field urgently need to solve. Summary of the invention

[0004] Based on the above problems, in order to improve the control effect of the robot on articulated objects, the embodiments of the present application provide a robot-based articulated object control method, system, device and medium.

[0005] The embodiments of the present application disclose the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides a robot-based articulated object control method, comprising:

[0007] According to the articulation type of the target articulated object, an initial articulation property of the target articulated object is acquired; the initial articulation property is used to characterize the motion characteristics of the target articulated object when the articulation motion is triggered;

[0008] Based on the initial articulation properties, controlling the robot to perform motion trajectory control on the target articulated object to obtain an initial trajectory execution result;

[0009] Based on the preset ASSG algorithm and the initial trajectory execution result, the initial articulation attributes are corrected to obtain articulation correction attributes, and the robot is controlled by the articulation correction attributes to repeatedly control the motion trajectory of the target articulated object; the preset ASSG algorithm is an online reinforcement learning algorithm based on a single trajectory execution result.

[0010] In a possible implementation, the obtaining of the initial articulation properties of the target articulated object includes:

[0011] Acquire an RGB image and a depth image of the target articulated object;

[0012] Performing image segmentation processing on the RGB image to determine a first area position of a hinge motion trigger point and a second area position of the object surface;

[0013] Based on the first area position and the second area position, aligning the RGB image with the depth image to determine the articulation type of the target articulated object;

[0014] The initial articulation attribute is determined according to the articulation type, the first region position, the second region position, and the depth image.

[0015] In a possible implementation, the articulation type includes: rotational articulation and translational articulation;

[0016] The determining the initial articulation attribute according to the articulation type, the first region position, the second region position and the depth image comprises:

[0017] When the articulation type is the rotary articulation, determining a first articulation attribute and a second articulation attribute according to the first area position, the second area position and the spatial geometric information in the depth image, and determining the first articulation attribute and the second articulation attribute as the initial articulation attribute;

[0018] When the articulation type is the translational articulation, determining the second articulation attribute according to the first area position, the second area position and the spatial geometric information in the depth image, and determining the second articulation attribute as the initial articulation attribute;

[0019] The first articulation attribute is the spatial position information of the center point of the edge of the surface area of ​​the object, and the second articulation attribute is the articulation movement direction of the object.

[0020] In a possible implementation, the initial trajectory execution result includes: trajectory execution completion degree and average execution tension when performing the motion trajectory control;

[0021] The performing of attribute correction on the initial joint attribute based on the preset ASSG algorithm and the initial trajectory execution result to obtain the joint correction attribute includes:

[0022] Determining a motion trajectory execution distance based on the trajectory execution completion degree;

[0023] The motion trajectory execution distance, the average execution tension and the expected trajectory execution distance for the target articulated object are introduced into the preset ASSG algorithm to determine an execution feedback reward; the execution feedback reward is used to characterize the motion trajectory control effect of the machine on the target articulated object;

[0024] The initial articulation attribute is modified based on the execution feedback reward to obtain the modified articulation attribute.

[0025] In a possible implementation, a force sensor is provided at the end of a mechanical arm of the robot, and the method further includes:

[0026] When the robot controls the motion trajectory of the target articulated object, the execution pulling force is acquired in real time through the force sensor;

[0027] When the execution pulling force is reset to zero or exceeds a preset safety threshold, the trajectory control operation of the robot is terminated.

[0028] In a possible implementation, the controlling the robot to perform motion trajectory control on the target articulated object based on the initial articulation attribute includes:

[0029] generating a motion execution trajectory according to the first region position and the initial articulation attribute;

[0030] Based on the motion execution trajectory, the robot is controlled to perform motion trajectory control on the target articulated object.

[0031] In a second aspect, an embodiment of the present application provides a robot-based articulated object control system, comprising:

[0032] An attribute acquisition module, used for acquiring initial articulation attributes of the target articulated object according to the articulation type of the target articulated object; the initial articulation attributes are used for characterizing the motion characteristics of the target articulated object when the articulation motion is triggered;

[0033] A first trajectory control module, configured to control the robot to perform motion trajectory control on the target articulated object based on the initial articulation attribute, so as to obtain an initial trajectory execution result;

[0034] The second trajectory control module is used to correct the articulation properties based on a preset ASSG algorithm and the initial trajectory execution result to obtain the articulation correction properties, and control the robot to repeatedly control the motion trajectory of the target articulated object through the articulation correction properties; the preset ASSG algorithm is an online reinforcement learning algorithm based on a single trajectory execution result.

[0035] In a possible implementation, the attribute acquisition module is specifically used to:

[0036] Acquire an RGB image and a depth image of the target articulated object;

[0037] Performing image segmentation processing on the RGB image to determine a first area position of a hinge motion trigger point and a second area position of the object surface;

[0038] Based on the first area position and the second area position, aligning the RGB image with the depth image to determine the articulation type of the target articulated object;

[0039] The initial articulation property is determined according to the articulation type, the first region position and the second region position.

[0040] In a third aspect, an embodiment of the present application provides an electronic device, the device comprising: a processor, a memory, and a system bus;

[0041] The processor and the memory are connected via the system bus;

[0042] The memory is used to store one or more programs, and the one or more programs include instructions. When the instructions are executed by the processor, the processor executes any possible robot-based articulated object control method in the first aspect.

[0043] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any possible robot-based articulated object control method in the first aspect.

[0044] Compared with the prior art, the present application has the following beneficial effects: the embodiment of the present application provides a robot-based articulated object control method, system, device and medium. In the method, the initial articulation properties of the object are first obtained according to the articulation type of the target articulated object, so as to preliminarily determine the motion characteristics of the object when the articulated motion is triggered. Further, based on the initial articulation properties, the robot is controlled to control the motion trajectory of the target articulated object, so as to obtain the initial trajectory execution result of the robot in this control process. Finally, the initial articulation properties of the object are corrected by the preset ASSG algorithm and the initial trajectory execution result of the robot, and the robot is controlled to repeatedly control the motion trajectory of the object based on the obtained articulation correction properties. The correction process of the articulation properties is repeated in this way, and each correction of the articulation properties will be corrected based on the actual trajectory execution result of the robot this time, so as to fully understand the physical properties and articulation motion characteristics of the target articulated object, and then generate a stable robot articulation control method, which effectively improves the control effect of the robot on articulated objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0046] Figure 1 A schematic flow chart of a robot-based articulated object control method provided in an embodiment of the present application;

[0047] Figure 2 A schematic diagram of the structure of a vehicle temperature abnormality warning system provided in an embodiment of the present application;

[0048] Figure 3 A schematic diagram of a robot-based articulated object control method provided in an embodiment of the present application;

[0049] Figure 4 A schematic diagram of the architecture of an ASSG algorithm provided in an embodiment of the present application;

[0050] Figure 5 A schematic diagram of a flow chart of a method for modifying joint properties provided in an embodiment of the present application;

[0051] Figure 6 A schematic diagram of the relationship between execution resistance and optimization times provided in an embodiment of the present application;

[0052] Figure 7 A schematic diagram of the structure of a robot-based articulated object control system provided in an embodiment of the present application;

[0053] Figure 8 A schematic structural diagram of a robot-based articulated object control electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of this application more clear, the following is a further detailed description of this application in combination with specific embodiments and with reference to the accompanying drawings. It should be noted that the embodiments described in the embodiments of this application are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0055] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "include" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right", etc. are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0056] As described above, in current related technologies, automated control of rigid objects by robots has been widely used. However, due to the complex physical structure of articulated objects and the limited action space dimension, the current automated control of articulated objects by robots cannot adapt to many different types of articulated objects, and the control effect is poor.

[0057] Therefore, how to improve the control effect of the robot on articulated objects has become a technical problem that technical personnel in this field urgently need to solve.

[0058] In order to solve the above problems, the embodiments of the present application provide a robot-based articulated object control method, system, device and medium. In the method, the initial articulation properties of the object are first obtained according to the articulation type of the target articulated object, so as to preliminarily determine the motion characteristics of the object when the articulated motion is triggered. Further, based on the initial articulation properties, the robot is controlled to control the motion trajectory of the target articulated object, so as to obtain the initial trajectory execution result of the robot in this control process. Finally, the initial articulation properties of the object are corrected by the preset ASSG algorithm and the initial trajectory execution result of the robot, and the robot is controlled to repeatedly control the motion trajectory of the object based on the obtained articulation correction properties. The correction process of the articulation properties is repeated in this way, and each correction of the articulation properties will be corrected based on the actual trajectory execution result of the robot this time, so as to fully understand the physical properties and articulation motion characteristics of the target articulated object, and then generate a stable robot articulation control method, which effectively improves the control effect of the robot on articulated objects.

[0059] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0060] See also Figure 1 , which is a flow chart of a robot-based articulated object control method provided in an embodiment of the present application, and specifically includes the following steps:

[0061] S101: Acquire initial articulation properties of the target articulated object according to the articulation type of the target articulated object; the initial articulation properties are used to characterize the motion characteristics of the target articulated object when the articulation motion is triggered.

[0062] As can be seen from the description of the background technology in the previous text, current articulated objects often have complex physical structures. When a robot performs automated operations such as pulling and closing on an articulated object (such as a door, refrigerator, drawer, etc.), the automated operation will be constrained by the complex physical structure of the articulated object, resulting in the robot being unable to plan an effective articulated action execution path. Therefore, in order to address this problem, in the initial stage of controlling an articulated object, the embodiment of the present application needs to obtain the initial articulation properties of the target articulated object according to the articulation type of the target articulated object, so as to preliminarily understand the articulation characteristics of the target articulated object.

[0063] Among them, the initial articulation properties are used to characterize the motion characteristics of the object when the articulation motion is triggered. Through the articulation properties of the object, the motion trajectory of the object when the articulation motion is triggered can be calculated, thereby driving the robot to execute the motion trajectory to achieve automatic control of the target articulated object.

[0064] The motion characteristics of an articulated object can be understood as the motion path of the object when the articulated motion is triggered, the articulation trigger point when the object triggers the articulation motion, and the surface area of ​​the object when the articulation motion is triggered. For example, taking a door as an example, the form of the door triggering the articulation motion is usually a quarter circle, and its motion path when starting the motion is a quarter circle with the plane width of the door as the circumference radius. Similarly, the door handle corresponds to the articulation trigger point, and the plane where the door handle is located corresponds to the surface area of ​​the articulation motion.

[0065] In this step, the articulation type and initial articulation properties of the target articulated object need to be obtained through image analysis of the object. This process will be introduced below in conjunction with the drawings of specific embodiments.

[0066] See also Figure 2 and Figure 3 , Figure 2 A schematic diagram of a process for obtaining initial articulation properties provided in an embodiment of the present application. Figure 3 A schematic diagram of a robot-based articulated object control method provided in an embodiment of the present application. Figure 2 As shown, the method specifically comprises the following steps:

[0067] S1011: Acquire an RGB image and a depth image of the target articulated object.

[0068] First, it is necessary to use a camera set outside the robot device to capture RGB images and depth images of the target articulated object.

[0069] The core difference between depth images and RGB images lies in the type of information they carry and their purpose. RGB images record visual features such as color, texture, and lighting on the surface of articulated objects through three color channels: red, green, and blue. They are mainly used for semantic understanding (such as identifying door handles and segmenting object regions). Depth images, on the other hand, use a single-channel value to represent the distance information from each pixel to the camera. They can reflect the three-dimensional spatial structure of articulated objects, such as surface geometry, coordinates of the contact point of the articulation, and so on.

[0070] Therefore, by acquiring the articulation properties of the target articulated object based on its depth image and RGB image, we can combine the object's color semantic information and three-dimensional geometric information to improve the accuracy and robustness of the robot's perception of articulated objects, thereby ensuring the data accuracy of the initial articulation properties.

[0071] S1012: Perform image segmentation processing on the RGB image to determine a first area position of the articulation motion trigger point and a second area position of the object surface.

[0072] Subsequently, the RGB image is segmented to determine the regional position information (first regional position) of the articulated motion trigger point of the target articulated object in the RGB image, and the regional position information (second regional position) of the surface of the object to which the articulated motion trigger point belongs.

[0073] For details, please refer to Figure 3 In the example, Figure 3 In the example of a door as a target hinged object, the RGB image of the door is segmented by SAM2 image segmentation technology to determine the position of the door handle and the area on the surface of the object to which the door handle belongs.

[0074] In a possible implementation, the position of the articulation motion trigger point can also be determined by analyzing the depth image. Figure 3 The AnyGrasp method used in the method can determine the point closest to the door handle in space as the grasping point (and the hinge motion trigger point). In actual application scenarios, the image analysis of RGB and the image analysis of the depth image can be combined to improve the accuracy of obtaining the position of the first area, which will not be described in detail in this embodiment.

[0075] S1013: Based on the first area position and the second area position, align the RGB image with the depth image to determine the articulation type of the target articulated object.

[0076] Based on the determination of the trigger point of the articulated motion and the surface of the object to which it belongs, the RGB image and the depth image are compared and analyzed, and the spatial geometric information represented in the depth image is used to determine the articulation type of the target articulated object, so as to obtain specific articulation properties according to different articulation types in the future.

[0077] Specifically, the RGB pixel coordinates corresponding to the trigger point position (first area position) and the surface area position (second area position) obtained after image segmentation can be mapped to the coordinate system of the depth image using pre-calibrated parameters (such as camera intrinsic parameters), thereby generating three-dimensional spatial coordinates for the target articulated object and determining the articulation type based on the obtained three-dimensional spatial coordinates.

[0078] S1014: Determine the initial articulation attribute according to the articulation type, the first area position, the second area position, and the depth image.

[0079] It is understandable that, since articulated objects often have complex physical structures, the initial articulation properties used to characterize the articulation motion characteristics of articulated objects of different articulation types may vary greatly. Therefore, in order to clarify the articulation characteristics of objects of different articulation types, the embodiments of the present application need to determine the initial articulation properties of the object by combining the articulation type of the object, the position of the articulation motion trigger point (first area position), the area position of the surface of the object to which it belongs (second area position), and the spatial geometric information represented in the depth image.

[0080] The articulation type of the object is divided into rotational articulation and translational articulation. When the articulation type is rotational articulation, it is necessary to determine the first articulation attribute and the second articulation attribute according to the first area position, the second area position and the spatial geometric information in the depth image, and determine the first articulation attribute and the second articulation attribute as the initial articulation attribute.

[0081] The first articulation attribute is used to characterize the spatial position information of the edge center point of the surface area of ​​the object, and the second articulation attribute is used to characterize the articulation movement direction of the object.

[0082] It is understandable that when the hinge type of an object is a rotary hinge, it is necessary to clarify the rotation axis of the object when the rotary hinge motion is triggered, as well as the rotation direction of the object. Exemplarily, for rotary hinged objects such as cabinet doors and microwave ovens, for the surface containing the hinge motion trigger point (handle), the center point of the edge of the surface area is taken as the starting point p of the rotation axis, and the spatial position information of the starting point p is the first hinge attribute. Accordingly, the direction of the edge end of the surface area is further obtained, and this direction can be used as the direction of the rotation axis (such as the direction of the door rotating around the edge of the door frame), that is, the movement direction q of the object when the hinge motion is triggered, to obtain the second hinge attribute.

[0083] On the other hand, when the articulation type is the translational articulation, a second articulation attribute is determined according to the first region position, the second region position and spatial geometric information in the depth image, and the second articulation attribute is determined as the initial articulation attribute.

[0084] When the hinge type of an object is a translational hinge similar to a drawer, because the hinge axis of a translational hinged object does not have a significant impact on the physical structure like a rotational hinged object, it is only necessary to obtain the translation direction of the object when the hinge motion is triggered. Taking a drawer as an example, based on the first area position of the drawer hinge motion trigger point and the second area position of the plane to which the trigger point belongs, the normal vector of the plane where the second area position is located is obtained, and the translation direction q of the drawer when the hinge motion is triggered, that is, the second hinge attribute, can be determined.

[0085] S102: Based on the initial articulation properties, control the robot to perform motion trajectory control on the target articulated object to obtain an initial trajectory execution result.

[0086] After obtaining the initial articulation properties, a motion execution trajectory that the robot needs to refer to when controlling the movement of the target articulated object is generated based on the first area position of the articulation motion trigger point and the articulation motion characteristics represented by the initial articulation properties, so that the robot can control the target articulated object to perform articulated motion and obtain the initial trajectory execution result after the articulation motion is completed.

[0087] As can be seen from the foregoing, the initial articulation properties in the embodiments of the present application are obtained by image segmentation processing of RGB images, and alignment and analysis of depth images. This method relies on data captured by an external camera, which may often have a certain degree of error, so the motion trajectory generated under this error may also cause the robot to have poor control over the object. Therefore, in order to improve the robot's control effect on articulated objects, after controlling the robot to control the motion trajectory of the object based on the initial articulation properties, it is necessary to obtain the initial trajectory execution result of the robot. The initial trajectory execution result can characterize the control effect of the robot when executing trajectory automation control, so as to facilitate the subsequent optimization of the articulation properties according to the real-time trajectory execution results, so that the trajectory execution results can fit the expected execution trajectory as closely as possible, so as to achieve the purpose of optimizing the robot's control effect.

[0088] Among them, the initial trajectory execution result includes the trajectory execution completion degree, and the average execution tension of the robot to control the object to perform articulated motion during a single motion trajectory. A force sensor is set at the end of the robot's mechanical arm in the embodiment of the present application. The force sensor can obtain the execution tension of the robot when pulling the articulated motion trigger point to control the articulated motion in real time, and monitor the numerical value of the execution tension in real time. When the execution tension is set to zero, it indicates that the robot's mechanical arm gripper may fall off the object. At this time, it is necessary to terminate the trajectory control operation and use the trajectory execution data as the trajectory execution result. Similarly, when the execution tension exceeds the preset safety threshold, it indicates that there is a large difference between the robot's action execution trajectory and the actual articulated motion trajectory of the object. At this time, it is also necessary to terminate the trajectory control operation and obtain the corresponding trajectory execution result.

[0089] Correspondingly, the execution completion of the trajectory can be obtained by obtaining the current execution trajectory distance when the execution tension received by the force sensor is set to zero, and calculating it with the preset expected trajectory execution distance, so as to obtain the trajectory execution completion of the robot in this control articulated motion process. In addition, the average execution tension can be determined by the average value of the average force monitored by the sensor of the robot during this control process, which is not limited in this embodiment.

[0090] S103: Based on a preset ASSG algorithm and the initial trajectory execution result, the initial articulation attributes are corrected to obtain articulation correction attributes, and the robot is controlled by the articulation correction attributes to repeatedly control the motion trajectory of the target articulated object; the preset ASSG algorithm is an online reinforcement learning algorithm based on a single trajectory execution result.

[0091] This step is the core link of this application. Its purpose is to analyze the robot's trajectory execution results through the pre-designed ASSG (Accelerated Single-Step Gradient, single-step online reinforcement learning) algorithm, and to correct the initial articulation properties of the object. Through repeated iterations and online training, the robot's understanding of the articulation characteristics of the target articulated object is optimized, so that the robot has a more accurate understanding of the articulation characteristics of the target object, and then generates a motion control trajectory that is more in line with the actual articulation characteristics of the object, thereby improving the control effect.

[0092] The core of the preset ASSG algorithm is to perform online optimization and reinforcement learning on the articulation properties of the object based on the trajectory execution results generated by the robot each time it performs articulation motion control, aiming to gradually adjust the articulation properties through the robot's execution feedback and optimize the neural network strategy used for reinforcement learning in its algorithm. The preset ASSG algorithm actor-critic architecture in the embodiment of this application can be found in its specific architecture. Figure 4 The schematic diagram of the architecture of an ASSG algorithm is shown in FIG. In the figure, (s, a, r) ​​is used to represent the (state, action, reward) tuple in the Markov process, traj is used to represent the action execution trajectory sent to the robot, R represents the execution feedback reward generated by the robot for the trajectory execution result after the trajectory execution result is generated, and the execution feedback reward can characterize the control effect of the machine on the motion trajectory of the target articulated object, and Replay Buffer represents the experience replay pool, which is used to pack the state s, the network-generated action a, and the execution feedback reward R into the storage pool, so that when the network is trained, the training data can be randomly taken from the replay pool. MLP represents a multi-layer perceptron, Q is used to represent the Q value used in the training of the critic network, (p, q) represents the articulation correction attribute generated by the network, which is used to generate the action execution trajectory of the robot, and Ornstein-Uhlenbeck noise represents the noise introduced during the training process.

[0093] Depend on Figure 4 It can be seen that the ASSG algorithm corrects the articulation properties of the target articulated object in real time based on the execution results of each motion trajectory of the robot, thereby optimizing the control trajectory of the robot. Next, step S103 will be described in detail in conjunction with the accompanying drawings of a specific process embodiment.

[0094] See also Figure 5 , which is a flow chart of a method for modifying joint properties provided in an embodiment of the present application, and specifically includes the following steps:

[0095] S1031: Determine the motion trajectory execution distance based on the trajectory execution completion degree.

[0096] S1032: Importing the motion trajectory execution distance, the average execution tension, and the expected trajectory execution distance for the target articulated object into the preset ASSG algorithm to determine an execution feedback reward; the execution feedback reward is used to characterize the motion trajectory control effect of the machine on the target articulated object;

[0097] S1033: performing articulation attribute correction on the initial articulation attribute based on the execution feedback reward to obtain the articulation correction attribute.

[0098] As can be seen from the foregoing, the actual articulation property correction process in the embodiment of the present application needs to be determined based on the trajectory execution result of the robot when controlling the articulated motion. In this process, it is necessary to calculate the execution feedback reward based on the motion trajectory execution distance, average execution tension, and pre-set expected trajectory execution distance in the trajectory execution result. The execution feedback reward is used to characterize the motion trajectory control effect of the current robot on the target articulated object. By correcting the initial articulation property of the object based on the execution feedback reward, the robot's trajectory execution path can be optimized, thereby achieving the effect of improving the control effect.

[0099] Specifically, the method for calculating the execution feedback reward based on the preset ASSG algorithm can be found in the following formula:

[0100]

[0101] Where R represents the execution feedback reward, l represents the motion trajectory execution distance in the trajectory execution result, L represents the expected execution distance for a specific target articulated object, α represents the unified metric hyperparameter, and F represents the average execution force.

[0102] In addition, the process of repairing the initial joint properties is shown in the following formula:

[0103] p * =p+p a

[0104]

[0105] In the formula, p * represents the joint correction attribute corresponding to the first joint attribute, q * represents the articulation correction attribute corresponding to the second articulation attribute, p represents the first articulation attribute in the initial articulation attribute, q represents the second articulation attribute in the initial articulation attribute, and p a and q a Respectively represent the correction values ​​corresponding to each hinge attribute.

[0106] In this way, based on the robot's trajectory execution results each time, the articulation properties are optimized in real time to ensure that the robot's actual motion execution trajectory is more in line with the expected motion execution trajectory. As the number of optimizations increases, the resistance encountered by the robot during execution will gradually decrease. The specific effect can be seen in Figure 6 A schematic diagram showing the relationship between execution resistance and optimization times is shown.

[0107] The embodiment of the present application provides a robot-based control method for articulated objects. In the method, the initial articulation properties of the object are first obtained according to the articulation type of the target articulated object, so as to preliminarily determine the motion characteristics of the object when the articulated motion is triggered. Furthermore, based on the initial articulation properties, the robot is controlled to control the motion trajectory of the target articulated object, so as to obtain the initial trajectory execution result of the robot in this control process. Finally, the initial articulation properties of the object are corrected by the preset ASSG algorithm and the initial trajectory execution result of the robot, and the robot is controlled to repeatedly control the motion trajectory of the object based on the obtained articulation correction properties. The correction process of the articulation properties is repeated in this way, and each correction of the articulation properties will be corrected based on the actual trajectory execution result of the robot this time, so as to fully understand the physical properties and articulation motion characteristics of the target articulated object, and then generate a stable robot articulation control method, which effectively improves the control effect of the robot on articulated objects.

[0108] A robot-based articulated object control system provided in an embodiment of the present application is introduced below. The robot-based articulated object control system described below and the robot-based articulated object control method described above can refer to each other.

[0109] See also Figure 7 , which is a schematic diagram of the structure of a robot-based articulated object control system provided in an embodiment of the present application, and specifically includes the following modules:

[0110] The attribute acquisition module 100 is used to acquire the initial articulation attribute of the target articulated object according to the articulation type of the target articulated object; the initial articulation attribute is used to characterize the motion characteristics of the target articulated object when the articulation motion is triggered;

[0111] A first trajectory control module 200, configured to control the robot to perform motion trajectory control on the target articulated object based on the initial articulation attribute, so as to obtain an initial trajectory execution result;

[0112] The second trajectory control module 300 is used to correct the articulation properties based on a preset ASSG algorithm and the initial trajectory execution result to obtain the articulation correction properties, and control the robot to repeatedly control the motion trajectory of the target articulated object through the articulation correction properties; the preset ASSG algorithm is an online reinforcement learning algorithm based on a single trajectory execution result.

[0113] In a possible implementation, the attribute acquisition module 100 is specifically used to:

[0114] Acquire an RGB image and a depth image of the target articulated object;

[0115] Performing image segmentation processing on the RGB image to determine a first area position of a hinge motion trigger point and a second area position of the object surface;

[0116] Based on the first area position and the second area position, aligning the RGB image with the depth image to determine the articulation type of the target articulated object;

[0117] The initial articulation property is determined according to the articulation type, the first region position and the second region position.

[0118] See also Figure 8 , which is a schematic diagram of the structure of an electronic device for controlling articulated objects based on a robot provided in an embodiment of the present application, including:

[0119] A memory 11, used for storing computer programs;

[0120] The processor 12 is used to implement the steps of a robot-based articulated object control method described in any of the above method embodiments when executing the computer program.

[0121] In this embodiment, the device may be a vehicle-mounted computer, a PC (Personal Computer), or a terminal device such as a smart phone, a tablet computer, a PDA, or a portable computer.

[0122] The device may include a memory 11 , a processor 12 , and a bus 13 .

[0123] Among them, the memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the device, such as a hard disk of the device. In other embodiments, the memory 11 can also be an external storage device of the device, such as a plug-in hard disk equipped on the device, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Further, the memory 11 can also include both an internal storage unit of the device and an external storage device. The memory 11 can not only be used to store application software installed in the device and various types of data, such as program codes for executing a robot-based articulated object control method, but also can be used to temporarily store data that has been output or is to be output. The processor 12 can be a central processing unit (CPU) in some embodiments.

[0124] In some embodiments, the processor 12 can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 11, such as executing the program code of a robot-based articulated object control method.

[0125] The bus 13 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0126] Furthermore, the device may also include a network interface 14, which may optionally include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the device and other electronic devices.

[0127] Optionally, the device may further include a user interface 15, which may include a display (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 15 may also include a standard wired interface and a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device, etc. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the device and to display a visual user interface.

[0128] Figure 8 Only the device with components 11-15 is shown, and it can be understood by those skilled in the art that Figure 8 The structure shown does not constitute a limitation of the device, and may include fewer or more components than shown, or combine certain components, or arrange the components differently.

[0129] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, an embodiment of the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the robot-based articulated object control method as described in any of the above embodiments.

[0130] The computer-readable media of the embodiments of the present application include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0131] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the robot-based articulated object control method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0132] It should be noted that each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for methods, systems, electronic devices and media, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments. The methods, systems, electronic devices and media described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components indicated as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0133] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A robot-based articulated object control method, characterized in that: include: According to the articulation type of the target articulated object, an initial articulation property of the target articulated object is acquired; the initial articulation property is used to characterize the motion characteristics of the target articulated object when the articulation motion is triggered; Based on the initial articulation properties, controlling the robot to perform motion trajectory control on the target articulated object to obtain an initial trajectory execution result; Based on the preset ASSG algorithm and the initial trajectory execution result, the initial articulation attributes are corrected to obtain articulation correction attributes, and the robot is controlled by the articulation correction attributes to repeatedly control the motion trajectory of the target articulated object; the preset ASSG algorithm is an online reinforcement learning algorithm based on a single trajectory execution result.

2. The method according to claim 1, characterized in that The step of obtaining the initial articulation properties of the target articulated object includes: Acquire an RGB image and a depth image of the target articulated object; Performing image segmentation processing on the RGB image to determine a first area position of a hinge motion trigger point and a second area position of the object surface; Based on the first area position and the second area position, aligning the RGB image with the depth image to determine the articulation type of the target articulated object; The initial articulation attribute is determined according to the articulation type, the first region position, the second region position, and the depth image.

3. The method according to claim 2, characterized in that The articulation types include: rotary articulation and translational articulation; The determining the initial articulation attribute according to the articulation type, the first region position, the second region position and the depth image comprises: When the articulation type is the rotary articulation, determining a first articulation attribute and a second articulation attribute according to the first area position, the second area position and the spatial geometric information in the depth image, and determining the first articulation attribute and the second articulation attribute as the initial articulation attribute; When the articulation type is the translational articulation, determining the second articulation attribute according to the first area position, the second area position and the spatial geometric information in the depth image, and determining the second articulation attribute as the initial articulation attribute; The first articulation attribute is the spatial position information of the center point of the edge of the surface area of ​​the object, and the second articulation attribute is the articulation movement direction of the object.

4. The method according to claim 1, characterized in that: The initial trajectory execution result includes: trajectory execution completion degree and average execution tension when performing the motion trajectory control; The performing of attribute correction on the initial joint attribute based on the preset ASSG algorithm and the initial trajectory execution result to obtain the joint correction attribute includes: Determining a motion trajectory execution distance based on the trajectory execution completion degree; The motion trajectory execution distance, the average execution tension and the expected trajectory execution distance for the target articulated object are introduced into the preset ASSG algorithm to determine an execution feedback reward; the execution feedback reward is used to characterize the motion trajectory control effect of the machine on the target articulated object; The initial articulation attribute is modified based on the execution feedback reward to obtain the modified articulation attribute.

5. The method according to claim 1, characterized in that A force sensor is provided at the end of the robot's mechanical arm, and the method further comprises: When the robot controls the motion trajectory of the target articulated object, the execution pulling force is acquired in real time through the force sensor; When the execution pulling force is reset to zero or exceeds a preset safety threshold, the trajectory control operation of the robot is terminated.

6. The method according to claim 3, characterized in that The controlling the robot to control the motion trajectory of the target articulated object based on the initial articulation attribute includes: generating a motion execution trajectory according to the first region position and the initial articulation attribute; Based on the motion execution trajectory, the robot is controlled to perform motion trajectory control on the target articulated object.

7. A robot-based control system for articulated objects, characterized in that: include: An attribute acquisition module, used for acquiring initial articulation attributes of the target articulated object according to the articulation type of the target articulated object; the initial articulation attributes are used for characterizing the motion characteristics of the target articulated object when the articulation motion is triggered; A first trajectory control module, configured to control the robot to perform motion trajectory control on the target articulated object based on the initial articulation attribute, so as to obtain an initial trajectory execution result; The second trajectory control module is used to correct the articulation properties based on a preset ASSG algorithm and the initial trajectory execution result to obtain the articulation correction properties, and control the robot to repeatedly control the motion trajectory of the target articulated object through the articulation correction properties; the preset ASSG algorithm is an online reinforcement learning algorithm based on a single trajectory execution result.

8. The system according to claim 7, characterized in that The attribute acquisition module is specifically used for: Acquire an RGB image and a depth image of the target articulated object; Performing image segmentation processing on the RGB image to determine a first area position of a hinge motion trigger point and a second area position of the object surface; Based on the first area position and the second area position, aligning the RGB image with the depth image to determine the articulation type of the target articulated object; The initial articulation property is determined according to the articulation type, the first region position and the second region position.

9. An electronic device, characterized in that: The device comprises: a processor, a memory and a system bus; The processor and the memory are connected via the system bus; The memory is used to store one or more programs, and the one or more programs include instructions, which, when executed by the processor, enable the processor to execute the robot-based articulated object control method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the robot-based articulated object control method described in any one of claims 1 to 6 is implemented.

Images