A robot grasping device for moving objects based on visual perception

Abstract

This invention belongs to the field of robot control and discloses a visual perception-based robot grasping device for moving objects. It includes: an environmental image and component image acquisition module, a component information acquisition module, a central control module, an environmental map generation module, a 3D scene map generation module, an annotation module, an information acquisition module, a visual perception module, a recognition and posture determination module, a positioning information module, a motion prediction module, a grasping point determination module, a path planning module, a movement control module, and a grasping module. This invention's visual perception-based robot grasping device for moving objects utilizes multiple technologies such as visual perception and motion prediction to achieve the grasping of moving objects, while also improving grasping efficiency and success rate. The method of this invention has a wide range of applications and can be used in high-end manufacturing and aerospace fields. It is highly adaptable to complex environments and operates stably.

Description

Technical Field

[0001] The invention belongs to the field of robot control, and in particular relates to a robot grasping device for moving objects based on visual perception. Background Art

[0002] At present, as robots are widely used in various fields such as industry, agriculture, and service industry, robot grasping technology, as one of the main research directions of robot control technology, has also received more and more attention and research. In particular, vision-based robot dexterous grasping has always been the focus of robot intelligent control. Whether in unstructured or unknown dynamic environments, visual sensors can provide real-time information to robots, thereby increasing the flexibility and robustness of robot grasping systems. In recent years, with the rapid development of mobile robot technology, vision-based mobile robot systems composed of mobile platforms, serial manipulators, and vision sensors have been increasingly used in the field of intelligent manufacturing due to their own mobility, high maneuverability of multiple serial manipulators, and real-time perception of visual systems. Operations such as object handling and assembly have improved the degree of mechanization and intelligence in the manufacturing field.

[0003] However, existing robot grasping devices still have defects such as insufficient accuracy in grasping dynamic objects, low grasping efficiency, and inability to predict the trajectory of moving objects in real time.

[0004] Through the above analysis, the problems and defects of the existing technology are as follows: the existing robot grasping device still has defects such as insufficient accuracy in grasping dynamic objects, low grasping efficiency, and inability to predict the trajectory of moving objects in real time. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention provides a robot grasping device for moving objects based on visual perception.

[0006] The present invention is implemented as follows: a moving object robot grasping device based on visual perception, the moving object robot grasping device based on visual perception comprises:

[0007] A visual perception module is connected to the central control module and is used to obtain image data in the current environment in real time using a visual perception device provided on the robot;

[0008] An identification and posture determination module, connected to the central control module, is used to identify and determine the moving object to be grasped from the image data in the current environment and obtain corresponding motion data;

[0009] A motion prediction module, connected to the central control module, for predicting motion trajectory data of the moving object to be grasped based on the determined current position data of the moving object to be grasped and the generated three-dimensional scene map in combination with the motion posture of the moving object to be grasped;

[0010] A grasping point determination module, connected to the central control module, is used to predict the range of the grasping point of the robot grasping the moving object based on the predicted motion trajectory data of the moving object to be grasped and the relevant information of the robot;

[0011] A path planning module, connected to the central control module, is used to mark the motion trajectory data of the moving object to be grasped and the current position data of the robot in the three-dimensional scene map, and to plan the path of the robot to the grasping point range;

[0012] The grasping module is connected to the central control module and is used to control the robot to grasp the moving object.

[0013] Furthermore, the moving object robot grasping device based on visual perception also includes:

[0014] The environment image and component image acquisition module is connected to the central control module and is used to obtain image data of the moving object and the environment where the robot is located, as well as image data of all components and objects in the environment where the moving object and the robot are located;

[0015] The component information acquisition module is connected to the central control module and is used to obtain the size information of all components and objects in the environment where the moving object and the robot are located;

[0016] The central control module is connected with the environment image and component image acquisition module, the component information acquisition module, the environment map generation module, the three-dimensional scene map generation module, the annotation module, the information acquisition module, the visual perception module, the recognition and posture determination module, the positioning information module, the motion prediction module, the grasping point determination module, the path planning module, the mobile control module and the grasping module, and is used to control the normal operation of each module by using a single-chip microcomputer or a controller;

[0017] An environment map generation module, connected to the central control module, is used to construct an environment map where the moving object and the robot are located based on the image data of the moving object and the environment where the robot is located;

[0018] A three-dimensional scene map generation module is connected to the central control module and is used to construct a three-dimensional scene map of the environment where the moving object and the robot are located based on the environment map where the moving object and the robot are located, combined with the image data of all components and objects in the environment where the moving object and the robot are located, and the size information of all components and objects;

[0019] A marking module, connected to the central control module, is used to determine the position information of all components and objects by combining the image data of the moving object and all components and objects in the environment where the robot is located and the size information of all components and objects, and mark the position data of all components and objects in the three-dimensional scene map;

[0020] An information acquisition module, connected to the central control module, is used to obtain relevant information about the moving object to be grasped and the robot;

[0021] A positioning information module is connected to the central control module and is used to determine the current position data of the moving object to be grasped based on the identified moving object to be grasped and the relevant information of the moving object to be grasped and the robot;

[0022] The mobile control module is connected to the central control module and is used to control the robot to move to the predicted grasping point based on the path planning results.

[0023] Furthermore, the information collection module includes:

[0024] A moving object information acquisition unit, used to obtain relevant information of the moving object to be captured;

[0025] The robot information collection unit is used to obtain relevant information of the robot.

[0026] Furthermore, the relevant information of the robot includes: current position information, posture information, and movement speed of the robot.

[0027] Furthermore, the recognition and posture determination module includes:

[0028] an identification unit, configured to identify and determine the moving object to be captured from the image data in the current environment in combination with the collected relevant information of the moving object to be captured;

[0029] The motion information extraction unit is used to determine the motion information of the moving object to be captured based on the identified moving object to be captured, combined with the collected relevant information of the moving object to be captured and the continuous image data of the current environment.

[0030] Furthermore, the motion information of the moving object to be grasped includes: a motion posture and a motion speed of the moving object to be grasped.

[0031] Furthermore, the grasping module controls the robot to grasp the moving object, including:

[0032] First, a real-time visual perception image is obtained to determine whether the moving object is within the grasping range of the robot. If not, the robot is controlled to move to the next state or current position of the moving object.

[0033] Secondly, obtaining relevant information of the moving object, and determining grasping parameters based on the relevant information of the moving object and the positional relationship between the moving object and the robot;

[0034] Finally, the robot is controlled to adjust its posture and grasp the moving object based on the grasping parameters.

[0035] Another object of the present invention is to provide a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the moving object robot grasping device based on visual perception.

[0036] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the visual perception-based moving object robot grasping device.

[0037] Another object of the present invention is to provide an information data processing terminal, which is used to implement the moving object robot grasping device based on visual perception.

[0038] In combination with the above technical solutions and the technical problems solved, please analyze the advantages and positive effects of the technical solutions to be protected by the present invention from the following aspects:

[0039] First, in view of the technical problems existing in the above-mentioned prior art and the difficulty of solving the problems, the technical solutions to be protected by the present invention and the results and data during the research and development process are closely combined to analyze in detail and deeply how the technical solutions of the present invention solve the technical problems, and some creative technical effects brought about after solving the problems. The specific description is as follows:

[0040] The robot grasping device for moving objects based on visual perception of the present invention is based on multiple technologies such as visual perception and motion prediction, thereby realizing the grasping of moving objects and improving the efficiency and success rate of grasping.

[0041] The present invention recognizes and perceives moving objects based on visual perception, and combines three-dimensional scene maps with motion prediction and grasping point prediction to overcome the defects of the prior art such as insufficient accuracy in grasping dynamic objects, low grasping efficiency, and inability to predict the trajectory of moving objects in real time.

[0042] Second, considering the technical solution as a whole or from the perspective of the product, the technical effects and advantages of the technical solution to be protected by the present invention are described in detail as follows:

[0043] The method of the present invention has a wide range of applications and can be used in high-end manufacturing and aerospace fields. It has strong adaptability to complex environments and a stable operating process. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is an interactive schematic diagram of a robot grasping device for moving objects based on visual perception provided by an embodiment of the present invention;

[0045] Figure 2 It is a schematic diagram of the structure of the information collection module provided by an embodiment of the present invention;

[0046] Figure 3 is a schematic diagram of the structure of the recognition and posture determination module provided by an embodiment of the present invention;

[0047] In the figure: 1. Information acquisition module; 11. Moving object information acquisition unit; 12. Robot information acquisition unit; 2. Recognition and posture determination module; 21. Recognition unit; 22. Motion information extraction unit. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0049] 1. Explanatory Examples In order to enable those skilled in the art to fully understand how to implement the present invention, this section provides an illustrative example that expands and describes the technical solution of the claims.

[0050] like Figure 1 As shown, the moving object robot grasping device based on visual perception provided by the embodiment of the present invention includes:

[0051] The environment image and component image acquisition module is connected to the central control module and is used to obtain image data of the moving object and the environment where the robot is located, as well as image data of all components and objects in the environment where the moving object and the robot are located;

[0052] The component information acquisition module is connected to the central control module and is used to obtain the size information of all components and objects in the environment where the moving object and the robot are located;

[0053] The central control module is connected with the environment image and component image acquisition module, the component information acquisition module, the environment map generation module, the three-dimensional scene map generation module, the annotation module, the information acquisition module, the visual perception module, the recognition and posture determination module, the positioning information module, the motion prediction module, the grasping point determination module, the path planning module, the mobile control module and the grasping module, and is used to control the normal operation of each module by using a single-chip microcomputer or a controller;

[0054] An environment map generation module, connected to the central control module, is used to construct an environment map where the moving object and the robot are located based on the image data of the moving object and the environment where the robot is located;

[0055] A three-dimensional scene map generation module is connected to the central control module and is used to construct a three-dimensional scene map of the environment where the moving object and the robot are located based on the environment map where the moving object and the robot are located, combined with the image data of all components and objects in the environment where the moving object and the robot are located, and the size information of all components and objects;

[0056] A marking module, connected to the central control module, is used to determine the position information of all components and objects by combining the image data of the moving object and all components and objects in the environment where the robot is located and the size information of all components and objects, and mark the position data of all components and objects in the three-dimensional scene map;

[0057] An information acquisition module, connected to the central control module, is used to obtain relevant information about the moving object to be grasped and the robot;

[0058] A visual perception module is connected to the central control module and is used to obtain image data in the current environment in real time using a visual perception device provided on the robot;

[0059] An identification and posture determination module, connected to the central control module, is used to identify and determine the moving object to be grasped from the image data in the current environment and obtain corresponding motion data;

[0060] A positioning information module is connected to the central control module and is used to determine the current position data of the moving object to be grasped based on the identified moving object to be grasped and the relevant information of the moving object to be grasped and the robot;

[0061] A motion prediction module, connected to the central control module, for predicting motion trajectory data of the moving object to be grasped based on the determined current position data of the moving object to be grasped and the generated three-dimensional scene map in combination with the motion posture of the moving object to be grasped;

[0062] A grasping point determination module, connected to the central control module, is used to predict the range of the grasping point of the robot grasping the moving object based on the predicted motion trajectory data of the moving object to be grasped and the relevant information of the robot;

[0063] A path planning module, connected to the central control module, is used to mark the motion trajectory data of the moving object to be grasped and the current position data of the robot in the three-dimensional scene map, and to plan the path of the robot to the grasping point range;

[0064] A mobile control module, connected to the central control module, is used to control the robot to move to the predicted grasping point based on the path planning result;

[0065] The grasping module is connected to the central control module and is used to control the robot to grasp the moving object.

[0066] like Figure 2 As shown, the information collection module 1 provided in the embodiment of the present invention includes:

[0067] A moving object information acquisition unit 11 is used to obtain relevant information of the moving object to be captured;

[0068] The robot information acquisition unit 12 is used to obtain relevant information of the robot.

[0069] The relevant information of the robot provided by the embodiment of the present invention includes: the current position information, posture information, and movement speed of the robot.

[0070] like Figure 3 As shown, the recognition and posture determination module 2 provided in the embodiment of the present invention includes:

[0071] An identification unit 21 is used to identify and determine the moving object to be captured from the image data in the current environment in combination with the collected relevant information of the moving object to be captured;

[0072] The motion information extraction unit 22 is used to determine the motion information of the moving object to be captured based on the identified moving object to be captured, combined with the collected relevant information of the moving object to be captured and the continuous image data of the current environment.

[0073] The motion information of the moving object to be grasped provided by the embodiment of the present invention includes: the motion posture and the motion speed of the moving object to be grasped.

[0074] The grasping module provided in the embodiment of the present invention controls the robot to grasp the moving object, including:

[0075] First, a real-time visual perception image is obtained to determine whether the moving object is within the grasping range of the robot. If not, the robot is controlled to move to the next state or current position of the moving object.

[0076] Secondly, obtaining relevant information of the moving object, and determining grasping parameters based on the relevant information of the moving object and the positional relationship between the moving object and the robot;

[0077] Finally, the robot is controlled to adjust its posture and grasp the moving object based on the grasping parameters.

[0078] 2. Application Examples: In order to prove the creativity and technical value of the technical solution of the present invention, this section provides application examples of the technical solution of the claims on specific products or related technologies.

[0079] The present invention applies the moving object robot grasping device based on visual perception to a computer device, which includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor executes the moving object robot grasping device based on visual perception.

[0080] The present invention applies the moving object robot grasping device based on visual perception to a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor executes the moving object robot grasping device based on visual perception.

[0081] The present invention applies the moving object robot grasping device based on visual perception to an information data processing terminal.

[0082] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0083] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A robot grasping device for moving objects based on visual perception, characterized in that: The moving object robot grasping device based on visual perception comprises: A visual perception module is connected to the central control module and is used to obtain image data in the current environment in real time using a visual perception device provided on the robot; An identification and posture determination module, connected to the central control module, is used to identify and determine the moving object to be grasped from the image data in the current environment and obtain corresponding motion data; A motion prediction module, connected to the central control module, for predicting motion trajectory data of the moving object to be grasped based on the determined current position data of the moving object to be grasped and the generated three-dimensional scene map in combination with the motion posture of the moving object to be grasped; A grasping point determination module, connected to the central control module, is used to predict the range of the grasping point of the robot grasping the moving object based on the predicted motion trajectory data of the moving object to be grasped and the relevant information of the robot; A path planning module, connected to the central control module, is used to mark the motion trajectory data of the moving object to be grasped and the current position data of the robot in the three-dimensional scene map, and to plan the path of the robot to the grasping point range; The grasping module is connected to the central control module and is used to control the robot to grasp the moving object; The moving object robot grasping device based on visual perception also includes: The environment image and component image acquisition module is connected to the central control module and is used to obtain image data of the moving object and the environment where the robot is located, as well as image data of all components and objects in the environment where the moving object and the robot are located; The component information acquisition module is connected to the central control module and is used to obtain the size information of all components and objects in the environment where the moving object and the robot are located; The central control module is connected with the environment image and component image acquisition module, the component information acquisition module, the environment map generation module, the three-dimensional scene map generation module, the annotation module, the information acquisition module, the visual perception module, the recognition and posture determination module, the positioning information module, the motion prediction module, the grasping point determination module, the path planning module, the mobile control module and the grasping module, and is used to control the normal operation of each module by using a single-chip microcomputer or a controller; An environment map generation module, connected to the central control module, is used to construct an environment map where the moving object and the robot are located based on the image data of the moving object and the environment where the robot is located; A three-dimensional scene map generation module is connected to the central control module and is used to construct a three-dimensional scene map of the environment where the moving object and the robot are located based on the environment map where the moving object and the robot are located, combined with the image data of all components and objects in the environment where the moving object and the robot are located, and the size information of all components and objects; A marking module, connected to the central control module, is used to determine the position information of all components and objects by combining the image data of the moving object and all components and objects in the environment where the robot is located and the size information of all components and objects, and mark the position data of all components and objects in the three-dimensional scene map; An information acquisition module, connected to the central control module, is used to obtain relevant information about the moving object to be grasped and the robot; A positioning information module is connected to the central control module and is used to determine the current position data of the moving object to be grasped based on the identified moving object to be grasped and the relevant information of the moving object to be grasped and the robot; The mobile control module is connected to the central control module and is used to control the robot to move to the predicted grasping point based on the path planning results.

2. The moving object robot grasping device based on visual perception as claimed in claim 1, characterized in that: The information collection module includes: A moving object information acquisition unit, used to obtain relevant information of the moving object to be captured; The robot information collection unit is used to obtain relevant information of the robot.

3. The moving object robot grasping device based on visual perception as claimed in claim 2, characterized in that: The relevant information of the robot includes: current position information, posture information, and movement speed of the robot.

4. The moving object robot grasping device based on visual perception as claimed in claim 1, characterized in that: The recognition and posture determination module includes: an identification unit, configured to identify and determine the moving object to be captured from the image data in the current environment in combination with the collected relevant information of the moving object to be captured; The motion information extraction unit is used to determine the motion information of the moving object to be captured based on the identified moving object to be captured, combined with the collected relevant information of the moving object to be captured and the continuous image data of the current environment.

5. The moving object robot grasping device based on visual perception as claimed in claim 4 is characterized in that: The motion information of the moving object to be grasped includes: the motion posture and the motion speed of the moving object to be grasped.

6. The moving object robot grasping device based on visual perception as claimed in claim 1, characterized in that: The grasping module controls the robot to grasp the moving object, including: First, a real-time visual perception image is obtained to determine whether the moving object is within the grasping range of the robot. If not, the robot is controlled to move to the next state or current position of the moving object. Secondly, obtaining relevant information of the moving object, and determining grasping parameters based on the relevant information of the moving object and the positional relationship between the moving object and the robot; Finally, the robot is controlled to adjust its posture and grasp the moving object based on the grasping parameters.

7. A computer device, characterized in that: The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the moving object robot grasping device based on visual perception as claimed in any one of claims 1-6.

8. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the moving object robot grasping device based on visual perception as claimed in any one of claims 1 to 6.

9. An information data processing terminal, characterized in that: The information data processing terminal is used to implement the moving object robot grasping device based on visual perception as described in any one of claims 1-6.

Images