Robot vision recognition system and robot having the same

Abstract

The present invention discloses a robot vision recognition system and a robot having the system, including: an acquisition module, which is composed of at least four cameras on a horizontal plane, and is used to jointly acquire image information of an executed unit; an identification module, which takes the extension line of the axis point of the execution unit as the front as the scale scale of the acquired multiple image information, and identifies the executed unit frame by frame within the identification area of ​​the image information; a processing module, which is used to perform image processing and analysis on the image information in turn and establish a three-dimensional model, and establish dynamic three-dimensional simulation dynamic data of the three-dimensional model based on the results of the image processing and analysis; a data processing module. The invention optimizes the collected image to make the calculation of the given robot arm movement parameters more accurate, thereby achieving the technical purpose of reducing the gap between the axis of the grasping and the axis of the object.

Description

Technical Field

[0001] The present invention relates to the technical field of visual systems, and in particular to a recognition system for robot vision and a robot having the system. Background Art

[0002] The robot vision recognition system is mainly used to identify objects or help the robot avoid situations while moving. It recognizes objects based on the collected image information. Given an algorithm, the robot takes corresponding actions to complete industrial production or avoid obstacles in front.

[0003] As disclosed in the publication (announcement) number: CN109693263A, the publication (announcement) date: 2019-04-30, a robot vision recognition system is disclosed, characterized in that it includes a visual sensor, an image acquisition module and an image processing module connected in sequence; the image acquisition module is provided with a lighting module; and also includes a database processing system connected to the image processing module, the database processing system includes a cloud storage server, an image integration module and a data comparison module, and the image integration module is respectively connected to the cloud storage server and the data comparison module. Beneficial effects achieved by the present invention: the database processing system in the present invention includes a cloud storage server, an image integration module and a data comparison module, the image integration module is respectively connected to the cloud storage server and the data comparison module, and can realize the integration and storage of received images; 2. The image scanning module can scan the object and then identify it through the database processing system.

[0004] Or as disclosed in the publication (announcement) number: CN109968319A, the publication (announcement) date: 2019-07-05, a robot visual recognition system and method for plugging in cables is disclosed, including a distribution terminal detection flow branch body, which is used to carry and fix the distribution terminal under test; the distribution terminal detection flow branch body includes several detection stations; a plug-in robot, which is used to plug and unplug the tested distribution terminal on the detection station; and a distribution terminal detection platform, which is used to detect the tested distribution terminal after plugging in the detection station. The present invention adopts a detection station to carry and fix the tested distribution terminal, plugs in the tested distribution terminal through a plug-in robot, and the detection equipment detects the distribution terminal after plugging in, and transmits the detection results to the offline device, so as to facilitate the offline device to handle qualified products and unqualified products.

[0005] In the prior art including the above two patents, it can be seen that robots in the early stage are mainly used in industrial production activities to perform tasks such as grasping, handling, welding, etc., especially in the grasping work link of objects being conveyed on the assembly line. Since the target is always in a moving state, and the acquisition unit wants to realize the acquisition of dynamic object images without image confusion, it is necessary to process the collected dynamic images and feed them back to the processor after processing. The processor executes the computer program based on the image. However, the inaccuracy of the contour recognition of moving objects under image processing makes the manipulator able to grasp the moving object, but there is a certain gap between the grasping axis and the object axis. How to narrow this gap and reduce the gap between the grasping axis and the object axis is expected to be well solved. Summary of the invention

[0006] The object of the present invention is to provide a robot vision recognition system and a robot having the system, so as to solve the above-mentioned problems.

[0007] In order to achieve the above object, the present invention provides the following technical solution: a robot vision recognition system, comprising:

[0008] A collection module, consisting of at least four cameras on a horizontal plane, for jointly collecting image information of the executed unit;

[0009] An identification module, taking the extension line of the axis point of the execution unit as the front as the scale of the collected multiple image information, and identifying the executed unit within the identification area of ​​the image information frame by frame;

[0010] A processing module, used to perform image processing analysis and establish a three-dimensional model on the image information in sequence, and establish dynamic three-dimensional simulation dynamic data of the three-dimensional model based on the results of the image processing analysis;

[0011] The data processing module calculates the dynamic compensation parameters of the executed unit relative to the executing unit according to the dynamic three-dimensional simulation dynamic data.

[0012] Preferably, the acquisition module includes an indication guide unit arranged on each of the cameras, the indication guide unit emits a plurality of indication light spots located on the same straight line to indicate that the acquisition positions of the plurality of cameras match therewith, the angle between the direction of the indication light spots emitted by the indication guide unit and the ground is a preset inclination angle, and the indication guide unit emits the indication light spots onto the belt surface of the transmission belt conveying the executed unit at the preset inclination angle after the acquisition module is adjusted to a horizontal level.

[0013] Preferably, the acquisition module includes an XYZ-axis electric moving mechanism, and the camera is installed on a pan-tilt platform of the XYZ-axis electric moving mechanism. The movement of the camera on the pan-tilt platform enables the acquisition position of the camera to move on the XYZ axis, so that the acquisition position of the camera remains coaxial with the indication light spot.

[0014] Preferably, the acquisition module further includes an image processing unit for segmenting the pixel units of the acquisition picture of the camera, identifying and extracting multiple pixel units of the acquisition picture (where N ≥ 3) by an image mask method, and naming them P i , where i∈[1,N];

[0015] And P i Take the coordinate point as the center and cut out N pictures with a pixel size of 200*200;

[0016] The pixel unit is an executed unit.

[0017] As a preference, the pixel unit P i The surrounding small image is identified, Gaussian blur is performed on the surrounding area, the blurred area is picked up and filled, and color range recognition is performed to generate n i +1 coordinate point as the center of the contour image around the pixel unit.

[0018] Preferably, the coordinate points are divided into core points and boundary points, and the coordinate points divided into boundary points in the image are output respectively. A polygonal box is drawn on the pixel unit according to the coordinate points divided into boundary points in each image, and the internal area of ​​the polygonal box is the detected pixel unit.

[0019] Preferably, the three-dimensional model is provided to the data processing module to be executed as unit coordinate parameters, wherein: the Z-axis direction vector is [a, b, c] T , denoted as vector The moving direction vector in the dynamic three-dimensional simulation dynamic data is recorded as The coordinate system transformation formula is: Where R ZYX is the transformation matrix, and its expression is:

[0020]

[0021] Among them, α is the adjusted Euler angle corresponding to the Z axis, β is the adjusted Euler angle corresponding to the Y axis, γ is the adjusted Euler angle corresponding to the X axis, cosα is abbreviated as cα, and other functions are similar.

[0022] Preferably, the executed unit dynamic compensation parameter is a vector calculated by the transformation matrix equation: vector For the known Z-axis vector of the original tool coordinate system, substitute the vector parameters into Get the xyz axis adjustment Euler angle parameters of the execution unit.

[0023] A robot comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the dynamic compensation parameters of the executed unit output by the data processing module in the steps of the robot vision recognition system described in the above scheme are implemented to drive the executing unit to complete the predetermined workstation operation of the executed unit.

[0024] Preferably, the execution unit is a robotic arm with a robotic claw.

[0025] In the above technical scheme, a robot vision recognition system and a robot having the system provided by the present invention have the following beneficial effects: based on the processing of the captured image, the pixel units of the image capture are identified and segmented to obtain more accurate contour parameters, so as to facilitate the calculation of the given three-dimensional coordinate parameters of the executed unit to be more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0027] Figure 1 A schematic diagram of an implementation provided for an embodiment of the present invention;

[0028] Figure 2 A schematic diagram of a module structure provided in an embodiment of the present invention;

[0029] Figure 3 A schematic diagram of the unit structure under the acquisition module provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] Example 1

[0032] like Figure 1-2As shown, a robot vision recognition system and a robot having the system, comprising:

[0033] A collection module, consisting of at least four cameras on a horizontal plane, for jointly collecting image information of the executed unit;

[0034] The identification module uses the extended line of the axis point of the execution unit as the front as the scale of the collected multiple image information, and identifies the executed unit within the identification area of ​​the image information frame by frame;

[0035] A processing module, used to perform image processing analysis on the image information and establish a three-dimensional model in sequence, and to establish dynamic three-dimensional simulation dynamic data of the three-dimensional model based on the results of the image processing analysis;

[0036] The data processing module calculates the dynamic compensation parameters of the executed unit relative to the executing unit according to the dynamic three-dimensional simulation dynamic data.

[0037] Furthermore, the robot includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the dynamic compensation parameters of the executed unit output by the data processing module in the steps of the visual recognition system are realized to drive the executing unit to complete the predetermined station operation of the executed unit. The executing unit is specifically a robotic arm with a mechanical claw.

[0038] Furthermore, the three-dimensional model in the above embodiment is provided to the data processing module to be executed as unit coordinate parameters, where: the Z-axis direction vector is [a, b, c] T , denoted as vector The moving direction vector in the dynamic 3D simulation dynamic data is recorded as The coordinate system transformation formula is: Where R ZYX is the transformation matrix, and its expression is:

[0039]

[0040] Among them, α is the adjusted Euler angle corresponding to the Z axis, β is the adjusted Euler angle corresponding to the Y axis, γ is the adjusted Euler angle corresponding to the X axis, cosα is abbreviated as cα, cosβ is abbreviated as cβ, cosγ is abbreviated as cγ, and so on for other trigonometric functions.

[0041] Furthermore, the dynamic compensation parameter of the executed unit is the vector calculated by the transformation matrix equation vector For the known Z-axis vector of the original tool coordinate system, substitute the vector parameters into Get the xyz axis adjustment Euler angle parameters of the execution unit.

[0042] Example 2

[0043] like Figure 3 As shown, the acquisition module in the above-mentioned visual recognition system performs the recognition, acquisition, pixel processing, and contour recognition of the executed unit, as follows:

[0044] In summary, the acquisition module includes an indication guide unit arranged on each camera, the indication guide unit emits a plurality of indication light spots located on the same straight line to indicate that the acquisition positions of the plurality of cameras match therewith, the angle between the direction of the indication light spots emitted by the indication guide unit and the ground is a preset tilt angle, and the indication guide unit emits the indication light spots to the belt surface of the transmission belt of the executed unit according to the preset tilt angle after the acquisition module is adjusted horizontally; the preset tilt angle is within an angle range of greater than or equal to 0 degrees and less than or equal to 90 degrees;

[0045] The camera is installed on the pan-tilt of the XYZ axis electric moving mechanism. The camera's acquisition position is moved on the XYZ axis by moving the camera on the pan-tilt, so that the camera's acquisition position and the indicator light spot remain coaxial.

[0046] The image processing unit is used to segment the pixel units collected by the camera's acquisition picture, and identify and extract multiple pixel units of the acquisition picture (where N ≥ 3) through the image mask method, and name them P i , where i∈[1,N];

[0047] Image processing unit P i Take the coordinate point as the center and cut out N pictures with a pixel size of 200*200;

[0048] The pixel unit is the executed unit.

[0049] Furthermore, in the above embodiment, the image processing unit is also used for pixel units P i The surrounding small image is identified, Gaussian blur is performed on the surrounding area, the blurred area is picked up and filled, and color range recognition is performed to generate n i +1 coordinate point as the center of the surrounding pixel contour image.

[0050] Furthermore, in the above technical solution, the coordinate points are divided into core points and boundary points, and the coordinate points divided into boundary points in the image are output respectively. A polygonal box is drawn on the pixel unit according to the coordinate points divided into boundary points in each image, and the area inside the polygonal box is the detected pixel unit.

[0051] In summary, based on the processing of the captured image, the pixel units of the image are identified and segmented to obtain more accurate contour parameters, so as to make the calculation of the given three-dimensional coordinate parameters of the executed unit more accurate.

[0052] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0053] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0054] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0055] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0056] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

[0057] The embodiments of the present application also provide a specific implementation of an electronic device capable of implementing all steps of the method in the above embodiments, and the electronic device specifically includes the following contents:

[0058] Processor, memory, communications interface and bus;

[0059] Wherein, the processor, memory, and communication interface communicate with each other via the bus;

[0060] The processor is used to call the computer program in the memory, and when the processor executes the computer program, all the steps in the method in the above embodiment are implemented.

[0061] The embodiments of the present application also provide a computer-readable storage medium capable of implementing all the steps of the method in the above embodiments, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, all the steps of the method in the above embodiments are implemented.

[0062] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the hardware + program embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. Although the embodiment of this specification provides the method operation steps described in the embodiment or flow chart, more or less operation steps can be included based on conventional or non-creative means. The order of steps listed in the embodiment is only one way of executing the order of many steps, and does not represent the only execution order. When the device or terminal product in practice is executed, it can be executed in sequence or in parallel according to the method shown in the embodiment or the accompanying drawings (for example, a parallel processor or a multi-threaded processing environment, or even a distributed data processing environment). The term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, product or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such a process, method, product or device. In the absence of more restrictions, it is not excluded that there are other identical or equivalent elements in the process, method, product or device including the elements. For the convenience of description, the above device is described by dividing it into various modules according to its functions. Of course, when implementing the embodiments of this specification, the functions of each module can be implemented in the same one or more software and / or hardware, or the module implementing the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The present invention is described with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the process and / or box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0063] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, the embodiments of this specification may be in the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the embodiments of this specification may be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts refer to the partial description of the method embodiment. In the description of this specification, the description of the reference term "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of this specification.

[0064] In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradicting each other. The above is only an embodiment of the embodiment of this specification and is not intended to limit the embodiment of this specification. For those skilled in the art, the embodiment of this specification may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiment of this specification shall be included in the scope of the claims of the embodiment of this specification.

Claims

1. A robot vision recognition system, characterized in that: include: A collection module, composed of at least four cameras on a horizontal plane, for jointly collecting image information of the executed unit; The acquisition module includes an indication guide unit arranged on each of the cameras, the indication guide unit emits a plurality of indication light spots located on the same straight line to indicate that the acquisition positions of the plurality of cameras match the same straight line, the angle between the direction of the indication light spots emitted by the indication guide unit and the ground is a preset tilt angle, and the indication guide unit emits the indication light spots to the belt surface of the transmission belt conveying the executed unit according to the preset tilt angle after the acquisition module is adjusted to be horizontal; the preset tilt angle is within an angle range of greater than or equal to 0 degrees and less than or equal to 90 degrees; the acquisition module includes an XYZ-axis electric moving mechanism, the camera is mounted on a pan-tilt platform of the XYZ-axis electric moving mechanism, and the acquisition position of the camera is moved on the XYZ axis by the movement of the camera on the pan-tilt platform, so that the acquisition position of the camera remains coaxial with the indication light spot; An identification module, taking the extension line of the axis point of the execution unit as the front as the scale of the collected multiple image information, and identifying the executed unit within the identification area of ​​the image information frame by frame; A processing module, used to perform image processing analysis and establish a three-dimensional model on the image information in sequence, and establish dynamic three-dimensional simulation dynamic data of the three-dimensional model based on the results of the image processing analysis; A data processing module calculates a dynamic compensation parameter of an executed unit relative to an executing unit according to the dynamic three-dimensional simulation dynamic data; The three-dimensional model is provided to the data processing module to be executed as unit coordinate parameters, where the Z-axis direction vector is [a, b, c] T , denoted as vector , the moving direction vector in the dynamic three-dimensional simulation dynamic data is recorded as , the coordinate system transformation formula is: , where R ZYX is the transformation matrix, and its expression is: ； Among them, α is the adjusted Euler angle corresponding to the Z axis, β is the adjusted Euler angle corresponding to the Y axis, γ is the adjusted Euler angle corresponding to the X axis, cosα is abbreviated as cα, cosβ is abbreviated as cβ, cosγ is abbreviated as cγ, and other trigonometric functions are similar; The executed unit dynamic compensation parameter is a vector calculated by the transformation matrix equation ,vector For the known Z-axis vector of the original tool coordinate system, substitute the vector parameters into , get the xyz axis adjustment Euler angle parameters of the execution unit.

2. A robot comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is executed by the processor, the dynamic compensation parameters of the executed unit output by the data processing module in the steps of the robot vision recognition system according to claim 1 are implemented to drive the executing unit to complete the predetermined workstation operation on the executed unit.

3. A robot according to claim 2, characterized in that: The execution unit is specifically a robotic arm with a robotic claw.

Images