Industrial robot visual processing control system

Abstract

The invention belongs to the technical field of visual tracking algorithms, in particular to an industrial robot visual processing control system, and relates to a visual tracking system.The visual tracking system comprises image capturing, target detecting, target tracking, position information obtaining and a visual tracking algorithm, and the visual tracking algorithm starts to be started after target detecting; the visual tracking algorithm comprises information extraction, a visual odometer and visual obstacle avoidance, the industrial robot visual processing control system can track an object, and when the robot loses own coordinates due to shielding, power failure and the like in the moving process, the robot can track the object; the current position of the robot can be located from a known map through a relocation algorithm, then the target position can be estimated again, information such as pixel distribution, brightness and color can be converted into digital signals through conversion of image signals and data signals, the digital signals can be conveniently processed by a processing system, and the automatic recognition function is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of visual tracking algorithms, in particular to a visual processing control system for an industrial robot. Background Art

[0002] Machine vision is to use machines to replace human eyes for measurement and judgment. Machine vision system refers to the process of converting the captured target into image signals through machine vision products (i.e. image capture devices, divided into CMOS and CCD), and transmitting them to a dedicated image processing system. According to pixel distribution, brightness, color and other information, they are converted into digital signals. The image processing system performs various operations on these signals to extract the characteristics of the target, and then controls the equipment action on site according to the results of the judgment. Since the 1960s, people have begun to study machine vision systems. At the beginning, the visual system could only recognize objects like building blocks on the plane. By the 1970s, it could recognize certain processing parts, as well as indoor tables, telephones and other items. Although the research work at that time progressed rapidly, it could not be used in practice. This is because the amount of information in the visual system is extremely large, and the hardware system for processing this information is very large and takes a long time. The three-dimensional objects in the objective world are converted into two-dimensional plane images through sensors (such as cameras), and then the images of the objects are output through image processing. Usually, robots need two types of information to judge the position and shape of objects, namely distance information and light and dark information. Of course, as an object visual information, there is also color information, but it is not as important as the first two types of information in identifying the position and shape of the object. The robot vision system is highly dependent on light and often requires good lighting conditions to make the image formed by the object clearest, enhance the detection information, and overcome problems such as shadows, low contrast, and mirror reflection. The characteristics of the machine vision system are to improve the flexibility and automation of production. In some dangerous working environments that are not suitable for manual operations or occasions where artificial vision is difficult to meet the requirements, machine vision is often used to replace artificial vision. At the same time, in the process of mass industrial production, the efficiency and accuracy of using artificial vision to check product quality are low. The use of machine vision detection methods can greatly improve production efficiency and the degree of automation of production. In addition, machine vision is easy to realize information integration and is the basic technology for realizing computer integrated manufacturing.

[0003] It is precisely because machine vision systems can quickly acquire large amounts of information, are easy to process automatically, and are easy to integrate design information and processing control information, that in modern automated production processes, people widely use machine vision systems in areas such as working condition monitoring, finished product inspection, and quality control.

[0004] At present, most robot tracking algorithms simply track the target. If the target is lost, the robot will not be able to work properly. Robots will be widely used in our lives in the future. For this reason, we propose an industrial robot vision processing control system. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides an industrial robot vision processing control system to solve the above problems.

[0006] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: a core part visual tracking system, the visual tracking system includes capturing images, target detection, target tracking, obtaining position information and a visual tracking algorithm, the visual tracking algorithm starts after target detection, and the visual tracking algorithm includes information extraction, visual odometer and visual obstacle avoidance.

[0007] Preferably, the image capture includes an image acquisition device, a signal conversion and image processing system. The image acquisition device is a robot that uses some camera equipment to sample images. The signal conversion and image processing system are a series of operations performed by the robot on the image after the image is captured. The image acquisition device can be used to identify objects in front of the robot. The signal conversion is to convert the image into a signal that the robot can recognize by itself, and finally it is handed over to the image processing system for processing.

[0008] Preferably, the target detection includes digital-to-image conversion and detection processing, wherein the digital-to-image conversion is the change of the signal performed by the robot, and the detection processing is a complex processing work on the captured image, which is captured and processed once and finally stored in the internal data. Through the conversion of image signals and data signals, the pixel distribution and brightness, color and other information can be converted into digital signals, which is convenient for the processing system to process them and realize the automatic recognition function.

[0009] Preferably, the target tracking includes sensor tracking and tracking signal processing. The sensor tracking is that the robot tracks the target using sensors. The tracking signal processing is the signal that the sensor transmits back to the robot after tracking the target. The robot will perform a series of processing on the returned signal. Tracking through sensors can not only accurately locate, but also save a lot of resources.

[0010] Preferably, the acquisition of position information includes GPS positioning, position information processing and return system. The GPS positioning is a common positioning system, which locates the tracked object and the robot itself. The position information processing and return system processes some position information of the tracked object after positioning. Accurate positioning can be performed through the GPS positioning system, and then returned to the interior of the robot for position information processing, and finally the position information of the located object is located.

[0011] Preferably, the information extraction includes visual sensors, light sensors and binocular vision sensors. The information extraction is operated by the robot using some sensors. The visual sensors and binocular vision sensors are installed in the robot's eyes. The light sensor senses the object in front of the robot based on light and then feeds back to the robot. The visual sensors, light sensors and binocular vision sensors can be used for the robot to observe the object to be tracked, and then feed back to the robot through pixel distribution, brightness, color and other information.

[0012] Preferably, the visual odometer includes visual mapping, repositioning, closed-loop detection and object recognition. The visual odometer will be started when the robot is working and record the entire process. The visual mapping is a depth map established by the robot. The repositioning is started again after the target disappears to reposition the target. The closed-loop detection is enabled only when the visual odometer deviates. The object recognition is to identify the object through an image capture device. The visual odometer works from the beginning and records information from the power-on position to the positioning position. When the robot loses its own coordinates due to occlusion, power failure, etc. during movement, the repositioning algorithm will locate the current position of the robot from the known map and then re-estimate the target position.

[0013] Preferably, the visual obstacle avoidance includes an obstacle avoidance sensor, data transmission and obstacle avoidance. The obstacle avoidance sensor is installed at the bottom of the robot. The data transmission is feedback provided by the robot when it encounters an obstacle. The obstacle avoidance is the avoidance of objects. This is a function of the robot itself and will not be affected by external interference.

[0014] Preferably, the signal conversion and image processing systems are performed inside the robot after feedback and will not be disturbed by external factors. The internal operations will be protected by the robot itself and will not be detected by external equipment, which is both safe and secure.

[0015] Compared with the prior art, the present invention provides an industrial robot visual processing control system, which has the following beneficial effects:

[0016] The present invention can realize object tracking through an industrial robot visual processing control system. The visual odometer works from the beginning to record information from the power-on position to the positioning position. When the robot loses its own coordinates due to occlusion, power failure and other reasons during movement, the repositioning algorithm will locate the current position of the robot from the known map, and then re-estimate the target position. Signal conversion is to convert the image into a signal that the robot can recognize by itself, and finally hand it over to the image processing system for processing. Through the conversion of image signals and data signals, pixel distribution and brightness, color and other information can be converted into digital signals, which is convenient for the processing system to process them and realize automatic recognition function. The internal operation will be protected by the robot itself and will not be detected by external equipment, which is both safe and secure. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is the overall system flow chart of the present invention;

[0018] Figure 2 It is a flow chart of the visual tracking algorithm of the present invention;

[0019] Figure 3 This is a schematic diagram of the image capturing process of the present invention;

[0020] Figure 4 This is a schematic diagram of the target detection process of the present invention;

[0021] Figure 5 This is a schematic diagram of the target tracking process of the present invention;

[0022] Figure 6 This is a schematic diagram of the process of obtaining location information in the present invention.

[0023] In the figure: 1. Visual tracking system; 2. Capture image; 3. Target detection; 4. Target tracking; 5. Obtain position information; 6. Visual tracking algorithm; 21. Image acquisition device; 22. Signal conversion; 23. Image processing system; 31. Digital-to-image conversion; 32. Detection processing; 41. Sensor tracking; 42. Tracking signal processing; 51. GPS positioning; 52. Position information processing; 53. Return system; 61. Information extraction; 62. Visual odometer; 63. Visual obstacle avoidance; 611. Visual sensor; 612. Optical sensor; 613. Binocular vision sensor; 621. Visual mapping; 622. Repositioning; 623. Closed-loop detection; 624. Identify objects; 631. Obstacle avoidance sensor; 632. Data transmission; 633. Avoid obstacles. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] See also Figure 1-6 In this implementation scheme: the core part is a visual tracking system 1, the visual tracking system 1 includes capturing images 2, target detection 3, target tracking 4, obtaining position information 5 and a visual tracking algorithm 6, the visual tracking algorithm 6 starts after the target detection 3, the visual tracking algorithm 6 includes information extraction 61, a visual odometer 62 and a visual obstacle avoidance 63.

[0026] The image capturing device 2 includes an image capturing device 21, a signal conversion 22 and an image processing system 23. The image capturing device 21 is a robot using some camera equipment to sample images, and the signal conversion 22 and the image processing system 23 are a series of operations performed by the robot on the image after the image is captured.

[0027] The target detection 3 includes digital-to-image conversion 31 and detection processing 32. The digital-to-image conversion 31 is the change of the signal performed by the robot, and the detection processing 32 is a complex processing work on the captured image, which is processed once and finally stored in the internal data.

[0028] The target tracking 4 includes sensor tracking 41 and tracking signal processing 42. The sensor tracking 41 is that the robot tracks the target using sensors, and the tracking signal processing 42 is the signal sent back to the robot by the sensor after tracking the target. The robot will perform a series of processing on the returned signal.

[0029] The acquisition of position information 5 includes GPS positioning 51, position information processing 52 and return system 53. The GPS positioning 51 is a common positioning system for positioning the tracked object and the robot itself. The position information processing 52 and return system 53 process some position information of the tracked object after positioning.

[0030] The information extraction 61 includes a visual sensor 611, a light sensor 612 and a binocular vision sensor 613. The information extraction 61 is operated by the robot using some sensors. The visual sensor 611 and the binocular vision sensor 613 are both installed in the robot's eyes. The light sensor 612 senses the object in front of the robot based on light and then feeds back to the robot.

[0031] The visual odometer 62 includes visual mapping 621, repositioning 622, closed-loop detection 623 and object recognition 624. The visual odometer 62 is activated when the robot is working and records the entire process. The visual mapping 621 is a depth map established by the robot. The repositioning 622 is activated after the target disappears to reposition the target. The closed-loop detection 623 is enabled only when the visual odometer 62 deviates. The object recognition 624 is to recognize the object through the image capture device 21.

[0032] The visual obstacle avoidance 63 includes an obstacle avoidance sensor 631, data transmission 632 and obstacle avoidance 633. The obstacle avoidance sensor 631 is installed at the bottom of the robot. The data transmission 632 is the feedback provided by the robot when it encounters an obstacle. The obstacle avoidance 633 is the avoidance of objects. The signal conversion 22 and the image processing system 23 are both performed inside the robot after feedback and will not be interfered by external factors.

[0033] As a preferred technical solution of the present invention, the present invention captures the target by capturing an image 2. After capturing the target, the target will be detected, and then the visual tracking algorithm 6 will be started to track the target. Then the position information of the target can be obtained, and finally the visual tracking system 1 inside the robot is returned. When the robot loses its own coordinates due to occlusion, power failure, etc. during movement, there is no need to worry. Repositioning 622 will locate the current position of the robot from a known map, and then repositioning 622 will be performed, and there is no need to worry about the position being lost.

[0034] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An industrial robot visual processing control system, the core part of which is a visual tracking system (1), characterized in that: The visual tracking system (1) includes capturing images (2), target detection (3), target tracking (4), acquiring position information (5) and a visual tracking algorithm (6). The visual tracking algorithm (6) is started after the target detection (3). The visual tracking algorithm (6) includes information extraction (61), a visual odometer (62) and visual obstacle avoidance (63).

2. The industrial robot visual processing control system according to claim 1, characterized in that: The image capturing device (2) includes an image capturing device (21), a signal conversion device (22) and an image processing system (23). The image capturing device (21) is a robot using some camera equipment to sample images, and the signal conversion device (22) and the image processing system (23) are a series of operations performed by the robot on the image after the image is captured.

3. The industrial robot visual processing control system according to claim 1, characterized in that: The target detection (3) includes digital-to-image conversion (31) and detection processing (32). The digital-to-image conversion (31) is a signal change performed by the robot, and the detection processing (32) is a complex processing operation on the captured image, which is processed once and finally stored in the internal data.

4. The industrial robot visual processing control system according to claim 1, characterized in that: The target tracking (4) includes sensor tracking (41) and tracking signal processing (42). The sensor tracking (41) is that the robot tracks the target using the sensor. The tracking signal processing (42) is the signal that the sensor transmits back to the robot after tracking the target. The robot will perform a series of processing on the returned signal.

5. The industrial robot visual processing control system according to claim 1, characterized in that: The acquisition of position information (5) includes GPS positioning (51), position information processing (52) and a return system (53). The GPS positioning (51) is a common positioning system for positioning the tracked object and the robot itself. The position information processing (52) and the return system (53) are processes for some position information of the tracked object after positioning.

6. The industrial robot visual processing control system according to claim 1, characterized in that: The information extraction (61) includes a visual sensor (611), a light sensor (612) and a binocular vision sensor (613). The information extraction (61) is operated by the robot using some sensors. The visual sensor (611) and the binocular vision sensor (613) are both installed in the robot's eyes. The light sensor (612) senses the object at the front end of the robot based on light and then feeds back to the robot.

7. The industrial robot visual processing control system according to claim 1, characterized in that: The visual odometer (62) includes visual mapping (621), repositioning (622), closed-loop detection (623) and object recognition (624). The visual odometer (62) is activated when the robot is working and records the entire process. The visual mapping (621) is a depth map established by the robot. The repositioning (622) is activated after the target disappears to reposition the target. The closed-loop detection (623) is activated only when the visual odometer (62) deviates. The object recognition (624) recognizes the object through the image capture device (21).

8. The industrial robot visual processing control system according to claim 1, characterized in that: The visual obstacle avoidance (63) comprises an obstacle avoidance sensor (631), data transmission (632) and obstacle avoidance (633), wherein the obstacle avoidance sensor (631) is installed at the bottom of the robot, the data transmission (632) is feedback provided by the robot when encountering an obstacle, and the obstacle avoidance (633) is avoiding the object.

9. The industrial robot visual processing control system according to claim 2, characterized in that: The signal conversion (22) and image processing system (23) are both performed inside the robot after feedback and will not be disturbed by external factors.

Images