Manipulator control system based on visual perception
By using a vision-sensing robotic arm control system, multi-angle imaging and spectral analysis technologies are employed to reduce the impact of reflections, thus solving the problem of inaccurate detection on metal surfaces by vision inspection systems and achieving precise processing and quality inspection stability of reflective images.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-14
- Publication Date
- 2026-04-14
AI Technical Summary
Traditional visual perception inspection systems struggle to accurately capture details when inspecting metal surfaces due to glare issues, leading to inaccurate detection.
The system employs a vision-based robotic arm control system, which includes modules for image acquisition, vision perception and analysis, control center, robotic arm control, reflection detection, and anti-reflection processing. It reduces or eliminates reflection through multi-angle imaging and spectral analysis, delineates and scores reflection areas, and improves image quality and detection accuracy.
It effectively reduces or eliminates image distortion, improves the accuracy of visual perception systems in analyzing and processing reflective images, and ensures the stability and precision of quality inspection of metal mechanical parts.
Smart Images
Figure CN121848367A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of mechanical processing and production technology, and in particular to a vision-based robotic arm control system. Background Technology
[0002] In the process of machining and production, it is necessary to conduct quality inspections on the products after production to ensure high quality control requirements. For example, in automobile manufacturing, the inspection of machined parts, such as engine cylinder heads or other metal products, is required.
[0003] In the process of large-scale production of automotive parts, mass automated production is usually carried out in factories. The precision requirements of automotive parts are very high, and traditional manual inspection cannot meet the standards of the inspection values. Therefore, large factories usually equip themselves with vision-based inspection systems, which are combined with robotic arms to perform precision inspection on automotive parts and determine whether the automotive parts meet the production standards.
[0004] Considering that metal machinery is typically used in the manufacturing of automotive parts, such as engine cylinder heads or other metal products, which usually have smooth and highly reflective surfaces, and these surfaces may have a metallic luster due to machining or surface treatment, and since the perception and detection system usually uses lenses instead of human eyes for detection, reflection problems occur during detection, causing the perception and detection system to be unable to accurately capture details on the surface, such as dents, cracks or other defects, making it difficult for the machine vision system to correctly identify or measure them. Summary of the Invention
[0005] To overcome the above shortcomings, this invention provides a vision-based robotic arm control system, which aims to improve upon the existing technology where the perception and detection system typically uses a lens instead of the human eye for detection, resulting in reflection problems during detection and causing the perception and detection system to fail to accurately capture details on the surface.
[0006] To achieve the above objectives, the present invention adopts the following technical solution: a vision-perception-based robotic arm control system, comprising an image acquisition module that acquires images of the environment; a vision perception and analysis module connected to the image acquisition module that perceives and analyzes the acquired image data; a control center module connected to the vision perception and analysis module that receives conclusions from the vision perception and analysis module; a robotic arm control module connected to the control center module that receives control commands from the control center module to operate and move the robotic arm; a finished product judgment module connected to the robotic arm control module that ultimately judges the detected mechanical parts; a reflective detection module connected to the robotic arm control module that detects reflective areas in the image; an anti-reflective processing module connected to the reflective detection module that processes reflective areas in the image using image processing technology; a multi-angle imaging module connected to the anti-reflective processing module that acquires images of the target from multiple angles; and the multi-angle imaging module connected to the finished product judgment module.
[0007] As a further description of the above technical solution:
[0008] The visual perception and analysis module includes a feature extraction and target tracking module, which extracts features from the image and tracks the target. The feature extraction and target tracking module is connected to a recognition and classification module, which can further process the data in the feature extraction and target tracking module. The recognition and classification module is connected to an analysis result and anomaly detection module.
[0009] As a further description of the above technical solution:
[0010] The control center module includes a decision generation module, which receives information from the visual perception and analysis module and generates corresponding plans and decisions based on the information. The decision generation module is connected to an anomaly recording module, which records anomaly information received by the control center module. The anomaly recording module is also connected to a control execution module, which executes the control commands generated by the decision generation module.
[0011] As a further description of the above technical solution:
[0012] The robotic arm control module includes a control flipping module, which controls the flipping action of the robotic arm. The control flipping module is connected to a pick-and-move module, which controls the robotic arm to pick up and move mechanical parts.
[0013] As a further description of the above technical solution:
[0014] The reflection detection module includes a spectral analysis algorithm module, which performs spectral analysis on the image. The spectral analysis algorithm module is connected to a reflection suppression technology module, which uses reflection suppression technology to reduce or eliminate reflections in the image.
[0015] As a further description of the above technical solution:
[0016] The anti-reflective processing module includes a reflective area segmentation module, which identifies and segments reflective areas in an image. The reflective area segmentation module is connected to an area positioning module, which determines the location of the reflective areas.
[0017] As a further description of the above technical solution:
[0018] The module for dividing reflective areas includes a regional grading and scoring module. The regional grading and scoring module, based on the reflective scoring definition module, grades and scores the reflective intensity of different areas in the image. The regional grading and scoring module is connected to the reflective scoring definition module, which stores the standards for reflective scoring.
[0019] As a further description of the above technical solution:
[0020] The finished product judgment module includes a finished product storage module, which stores mechanical parts judged as finished products. A defective product storage module is connected to the finished product storage module, which stores mechanical parts judged as defective products.
[0021] As a further description of the above technical solution:
[0022] The region positioning module is connected to a dynamic range adjustment module, which adjusts the dynamic range of the image. The dynamic range adjustment module is also connected to a real-time performance optimization module, which performs real-time performance optimization on the image.
[0023] As a further description of the above technical solution:
[0024] The reflectivity scoring definition module is connected to a storage optimization module, which stores reflectivity information and optimizes based on the results to improve the accuracy of the next reflectivity scoring definition.
[0025] The present invention has the following beneficial effects:
[0026] 1. In this invention, by setting an anti-reflective processing module, image distortion caused by reflection is reduced or eliminated, thereby improving the usability and quality of the image, improving the accuracy and reliability of the visual perception system in analyzing and processing reflective images, and further ensuring the standard and quality of the production of metal mechanical parts.
[0027] 2. In this invention, by setting up a module for dividing reflective areas, the reflective part of the image is divided into different areas, and these areas are scored or processed to achieve more accurate and targeted reflective processing, so that the system can process reflective images better and more accurately, and further improve the stability of quality inspection of metal mechanical parts. Attached Figure Description
[0028] Figure 1 This is a flowchart of a vision-based robotic arm control system proposed in this invention.
[0029] Figure 2 This is a flowchart of the visual perception and analysis module of a vision-based robotic arm control system proposed in this invention.
[0030] Figure 3 This is a flowchart of the control center module of a vision-based robotic arm control system proposed in this invention;
[0031] Figure 4 This is a flowchart of the robot control module of a vision-based robot control system proposed in this invention;
[0032] Figure 5 This is a flowchart of a reflection detection module for a vision-based robotic arm control system proposed in this invention.
[0033] Figure 6 This is a flowchart of an anti-reflective processing module for a vision-based robotic arm control system proposed in this invention.
[0034] Figure 7 The flowchart of the module for dividing reflective areas in a vision-based robotic arm control system proposed in this invention is shown.
[0035] Figure 8 This is a flowchart of the finished product judgment module of a vision-based robotic arm control system proposed in this invention.
[0036] Legend:
[0037] 1. Image Acquisition Module; 2. Visual Perception and Analysis Module; 201. Feature Extraction and Target Tracking Module; 202. Recognition and Classification Module; 203. Analysis Results and Anomaly Detection Module; 3. Control Center Module; 301. Decision Generation Module; 302. Anomaly Recording Module; 303. Control Execution Module; 4. Robot Arm Control Module; 401. Control Flipping Module; 402. Picking and Moving Module; 5. Reflection Detection Module; 501. Spectral Analysis Algorithm Module; 502. Reflection Suppression Technology Module; 6. Anti-reflection Processing Module; 601. Reflection Area Division Module; 60101. Area Grading and Scoring Module; 60102. Reflection Suppression Technology Module; 60103. Storage Optimization Module; 602. Area Positioning Module; 603. Dynamic Range Adjustment Module; 604. Real-time Performance Optimization Module; 7. Multi-angle Imaging Module; 8. Finished Product Judgment Module; 801. Finished Product Storage Module; 802. Defective Product Storage Module. Detailed Implementation
[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0039] Reference Figures 1-8This invention provides an embodiment of a vision-based robotic arm control system, comprising an image acquisition module that acquires images of the environment and transmits the acquired image information to a vision perception and analysis module. The vision perception and analysis module is connected to the image acquisition module and performs perception and analysis on the acquired image data, including target detection, object recognition, and position calculation. A control center module is connected to the vision perception and analysis module. The control center module receives conclusions from the vision perception and analysis module and issues control commands to subsequently operate the robotic arm control module. A robotic arm control module is connected to the control center module, and the robotic arm control module receives commands from the control center module. The system controls the robotic arm with commands to operate and move it. A finished product judgment module is connected to the robotic arm control module. This module ultimately judges the inspected mechanical parts to determine if they meet production requirements. A reflective detection module is also connected to the robotic arm control module. This module detects reflective areas in the image. An anti-reflective processing module is connected to the anti-reflective processing module. This module uses image processing technology to process reflective areas in the image, improving image quality and the accuracy of visual perception. A multi-angle imaging module is connected to the anti-reflective processing module. This module acquires images of the target from multiple angles, increasing the system's understanding and recognition capabilities of the target image. The multi-angle imaging module is connected to the finished product judgment module.
[0040] The visual perception and analysis module includes a feature extraction and target tracking module. This module extracts features from the image and tracks the target. The extracted features include edges, textures, and colors. Target tracking involves tracking and identifying a specific target across consecutive frames. The feature extraction and target tracking module is connected to a recognition and classification module. This module further processes the data from the feature extraction and target tracking module, matching and classifying targets in the image with predefined categories. The recognition and classification module is also connected to an analysis result and anomaly detection module. After classifying the image information, the analysis result and anomaly detection module compares the image information with previously set standard information to detect anomalies.
[0041] The control center module includes a decision generation module, which receives information from the visual perception and analysis module and generates corresponding plans and decisions based on the information. The decision generation module is connected to an anomaly recording module, which records anomaly information received by the control center module. This allows the system to make better judgments based on experience when the same problem occurs again. The anomaly recording module is also connected to a control execution module, which executes the control commands generated by the decision generation module. The robotic arm control module includes a control flipping module, which controls the flipping action of the robotic arm. When other surfaces of the mechanical parts need to be inspected, the robotic arm flips the mechanical parts. The control flipping module is also connected to a pick-and-move module, which controls the robotic arm to pick up and move the mechanical parts, facilitating the separate collection and storage of finished or defective mechanical parts.
[0042] The reflection detection module includes a spectral analysis algorithm module, which improves the identification of reflective areas by performing spectral analysis on the image and considering the different spectral characteristics of different materials when reflecting light. The spectral analysis algorithm module is connected to a reflection suppression technology module, which reduces or eliminates reflections in the image, improving image visibility. The anti-reflection processing module includes a reflection area segmentation module, which identifies and segments reflective areas in the image for targeted processing. This module is connected to a region positioning module, which determines the location of reflective areas, accurately locating the reflections in the image for improved accuracy in subsequent reflection processing. The region positioning module is connected to a dynamic range adjustment module, which adjusts the dynamic range of the image to adapt to areas of different brightness, ensuring consistent image quality. Finally, the dynamic range adjustment module is connected to a real-time performance optimization module, which optimizes the image's performance in real time to ensure the system maintains high efficiency during image processing.
[0043] The reflective area segmentation module includes a region grading and scoring module, which grades and scores the reflective intensity of different regions in the image based on the reflective scoring definition module. The region grading and scoring module is connected to the reflective scoring definition module, which stores the reflective scoring standards and assigns corresponding scores to different levels of reflectivity. The reflective scoring definition module is connected to a storage optimization module, which stores reflective information and optimizes based on the results to improve the accuracy of the next reflective scoring definition. The finished product judgment module includes a finished product storage module, which stores mechanical parts judged as finished products. The finished product storage module is connected to a defective product storage module, which stores mechanical parts judged as defective products.
[0044] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A vision-based robotic arm control system, characterized in that: The system includes an image acquisition module that acquires images from the environment; a visual perception and analysis module connected to the image acquisition module that perceives and analyzes the acquired image data; a control center module connected to the visual perception and analysis module that receives conclusions from the visual perception and analysis module; a robotic arm control module connected to the control center module that receives control commands from the control center module to operate and move the robotic arm; a finished product judgment module connected to the robotic arm control module that ultimately judges the detected mechanical parts; a reflectivity detection module connected to the robotic arm control module that detects reflective areas in the image; an anti-reflectivity processing module connected to the reflectivity detection module that processes reflectivity in the image using image processing technology; and a multi-angle imaging module connected to the anti-reflectivity processing module that acquires images of the target from multiple angles and is connected to the finished product judgment module.
2. The vision-based robotic arm control system according to claim 1, characterized in that: The visual perception and analysis module includes a feature extraction and target tracking module, which extracts features from the image and tracks the target. The feature extraction and target tracking module is connected to a recognition and classification module, which can further process the data in the feature extraction and target tracking module. The recognition and classification module is connected to an analysis result and anomaly detection module.
3. The vision-based robotic arm control system according to claim 1, characterized in that: The control center module includes a decision generation module, which receives information from the visual perception and analysis module and generates corresponding plans and decisions based on the information. The decision generation module is connected to an anomaly recording module, which records anomaly information received by the control center module. The anomaly recording module is also connected to a control execution module, which executes the control commands generated by the decision generation module.
4. The vision-based robotic arm control system according to claim 1, characterized in that: The robotic arm control module includes a control flipping module, which controls the flipping action of the robotic arm. The control flipping module is connected to a pick-and-move module, which controls the robotic arm to pick up and move mechanical parts.
5. A vision-based robotic arm control system according to claim 1, characterized in that: The reflection detection module includes a spectral analysis algorithm module, which performs spectral analysis on the image. The spectral analysis algorithm module is connected to a reflection suppression technology module, which uses reflection suppression technology to reduce or eliminate reflections in the image.
6. The vision-based robotic arm control system according to claim 1, characterized in that: The anti-reflective processing module includes a reflective area segmentation module, which identifies and segments reflective areas in an image. The reflective area segmentation module is connected to an area positioning module, which determines the location of the reflective areas.
7. A vision-based robotic arm control system according to claim 6, characterized in that: The module for dividing reflective areas includes a regional grading and scoring module. The regional grading and scoring module, based on the reflective scoring definition module, grades and scores the reflective intensity of different areas in the image. The regional grading and scoring module is connected to the reflective scoring definition module, which stores the standards for reflective scoring.
8. A vision-based robotic arm control system according to claim 1, characterized in that: The finished product judgment module includes a finished product storage module, which stores mechanical parts judged as finished products. A defective product storage module is connected to the finished product storage module, which stores mechanical parts judged as defective products.
9. A vision-based robotic arm control system according to claim 6, characterized in that: The region positioning module is connected to a dynamic range adjustment module, which adjusts the dynamic range of the image. The dynamic range adjustment module is also connected to a real-time performance optimization module, which performs real-time performance optimization on the image.
10. A vision-based robotic arm control system according to claim 7, characterized in that: The reflectivity scoring definition module is connected to a storage optimization module, which stores reflectivity information and optimizes based on the results to improve the accuracy of the next reflectivity scoring definition.