Dynamic product surface defect detection system and method based on mechanical arm

Abstract

According to the dynamic product surface defect detection system and method based on the mechanical arm, a high-precision camera and the flexible mechanical arm are integrated, and automatic detection is achieved. The method comprises the steps of camera configuration, connection of a mechanical arm and an actuator, defect detection algorithm selection, detection process setting and detection starting. The system works cooperatively through the core module, provides an efficient and reliable automatic detection solution, is suitable for complex industrial environments and diversified detection requirements, is particularly suitable for large and three-dimensional target surface defect detection, realizes all-directional defect detection, and can realize the comprehensive detection of defects through flexible combination of various advanced image processing technologies. And the detection accuracy and efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of dynamic defect detection, and particularly relates to a dynamic product surface defect detection system and method based on a robotic arm. Background Art

[0002] With the rapid development of intelligent manufacturing and industrial automation, robotic arm-assisted dynamic defect detection systems have emerged as a key technology for industrial quality control. By combining high-precision cameras with flexible robotic arms, these systems make full use of the wide working range and precise motion control capabilities of robotic arms to achieve multi-dimensional and large-scale dynamic defect detection of products. Intelligent image recognition technology is one of the core advantages of such systems. It integrates high-resolution cameras and advanced image processing algorithms to clearly capture minute defects on the surface of workpieces. The introduction of deep learning and artificial intelligence enables robotic arms to analyze images and identify common defect types, such as cracks, depressions, and color differences, significantly improving the accuracy and efficiency of defect detection. In addition, the flexibility of the robotic arm operating system allows it to adjust production parameters in a timely manner with the help of real-time data feedback, avoiding defect problems in mass production, thereby enhancing product consistency and quality.

[0003] Robotic arm-assisted dynamic defect detection systems have demonstrated significant advantages in actual industrial applications. Such systems not only improve detection accuracy and efficiency but also provide multi-angle and multi-level information by integrating multi-sensor systems, such as vision sensors, laser scanners, and ultrasonic sensors, to comprehensively evaluate surface and internal defects of workpieces. This method of integrating sensor data enables robotic arms to achieve more comprehensive and accurate defect detection, effectively enhancing the detection level of complex workpieces. Research shows that the detection accuracy of the integrated sensor system is 30% higher than that of a single sensor system. In addition, although significant progress has been made in the application of robotic arms in the defect detection of complex workpieces, there is still room for further development. In the future, more advanced artificial intelligence algorithms and sensor technologies can be explored to improve detection accuracy and adaptability. By combining big data and cloud computing technologies, more intelligent production monitoring and fault prediction can be achieved, promoting the collaborative work of robotic arms and other automation devices to realize a more efficient production process and quality control.

[0004] In practical industrial applications, a robotic arm-assisted dynamic defect detection system needs to have a high degree of adaptability and robustness. The system should not only be able to detect surface defects of products but also adapt to different working environments and product types. In addition, with the development of deep learning technology, deep learning-based defect detection algorithms have attracted attention due to their excellent feature extraction capabilities and high accuracy. These algorithms can automatically optimize the detection process and improve the accuracy and efficiency of detection by learning a large amount of data. However, how to effectively integrate these algorithms into a robotic arm-assisted dynamic defect detection system remains a technical challenge. The dynamic product surface defect detection system based on a robotic arm proposed in this invention realizes all-round surface defect detection of large and three-dimensional targets by integrating a high-precision camera and a flexible robotic arm, combined with advanced image processing and deep learning technologies, improving the accuracy and efficiency of detection. Summary of the Invention

[0005] To overcome the deficiencies of the prior art, the present invention aims to propose a dynamic product surface defect detection system and method based on a robotic arm. By integrating a high-precision camera and a flexible robotic arm, this system realizes multi-dimensional and large-range dynamic defect detection of products, significantly improving the detection efficiency and reducing the time cost. This system is particularly suitable for surface defect detection of large and three-dimensional targets, can comprehensively cover all surfaces of the target, realize all-round defect detection, and at the same time has a high degree of flexibility and adaptability to adapt to various complex industrial environments and diverse detection requirements.

[0006] According to the first aspect of the present invention, there is provided a dynamic product surface defect detection system based on a robotic arm, the system comprising:

[0007] A camera configuration module for setting camera parameters to capture clear and precise images;

[0008] A robotic arm configuration module for setting the movement path and speed of the robotic arm;

[0009] A defect detection algorithm parameter configuration module for allowing users to adjust and optimize the parameters of the defect detection algorithm;

[0010] A dynamic defect detection process configuration module for setting the detection process, detection sequence, and key detection areas;

[0011] A dynamic defect detection execution module for coordinating the above modules to perform the actual defect detection task.

[0012] Further, the camera configuration module includes:

[0013] A camera model selection unit for selecting a camera suitable for the detection task;

[0014] A resolution setting unit for setting the resolution of the camera;

[0015] An image display mode setting unit for setting the display mode of the image;

[0016] A camera internal parameter calibration unit for precisely calibrating the internal parameters of the camera.

[0017] Furthermore, the robotic arm configuration module includes:

[0018] A robotic arm model selection unit for selecting the corresponding robotic arm model;

[0019] An actuator model selection unit for selecting the corresponding actuator model;

[0020] A robotic arm connection unit for establishing the connection between the robotic arm and the actuator;

[0021] A robotic arm debugging unit for observing and ensuring that the robotic arm moves precisely along a predetermined path.

[0022] Furthermore, the defect detection algorithm parameter configuration module includes:

[0023] A detection algorithm selection unit for selecting a suitable detection algorithm;

[0024] An algorithm parameter setting unit for setting algorithm parameters according to the characteristics of the selected algorithm.

[0025] Furthermore, the dynamic defect detection process configuration module includes:

[0026] A detection process setting unit for constructing a detection process including the robotic arm moving to a fixed point, hovering, and moving along a fixed route;

[0027] A detection task automatic completion unit for automatically completing the detection task according to the preset parameters and process, and recording the detection results.

[0028] Furthermore, the dynamic defect detection execution module includes:

[0029] A real-time feedback unit for providing visual and auditory feedback during the detection process to ensure that the user can monitor the detection status in real time;

[0030] A detection task automatic completion unit for automatically completing the detection task according to the preset parameters and process.

[0031] Furthermore, the camera includes a high-resolution camera for capturing images of the various surfaces of a three-dimensional target and / or a long-focus camera for capturing long-distance images of a large-range target.

[0032] According to another aspect of the present invention, a method for dynamic product surface defect detection may include the following steps:

[0033] Provide the dynamic product surface defect detection system based on the robotic arm as described above;

[0034] Camera configuration step, including selecting the camera model, resolution, and image display mode, and performing camera internal parameter calibration;

[0035] Robotic arm and actuator connection step, including selecting the robotic arm and actuator models and connecting them;

[0036] Detection parameter configuration step, including selecting the detection algorithm and setting the algorithm parameters;

[0037] Detection process setting step, flexibly setting different detection processes according to specific dynamic detection requirements;

[0038] Start detection step, after ensuring that both the working camera and the robotic arm observation camera are turned on, execute the preset detection process and complete the dynamic detection task.

[0039] Further, for the all-round surface defect detection of three-dimensional objects, the method may further include:

[0040] Adjust the movement path of the robotic arm to ensure that the camera can capture each surface of the three-dimensional object from different angles; analyze the images captured from multiple angles to identify and locate the defects on the surface of the three-dimensional object.

[0041] Further, for the comprehensive defect detection of large-scale objects, the method may further include:

[0042] Adjust the movement path of the robotic arm to ensure that the camera can cover all areas of the large-scale object; analyze the images captured from different areas to identify and locate the defects of the large-scale object.

[0043] Compared with the prior art, the present invention has the following advantages:

[0044] First, through the wide working range and precise motion control ability of the robotic arm, the present invention realizes the defect detection of the all-round surface of large and three-dimensional objects, improving the comprehensiveness and accuracy of detection.

[0045] Second, the system design of the present invention allows users to flexibly configure the detection process and parameters according to different detection requirements, improving the adaptability of the system and the operation convenience of users.

[0046] Third, the system of the present invention supports a variety of defect detection algorithms, and can select the most suitable algorithm according to specific detection tasks and target characteristics, improving the flexibility and accuracy of detection.

[0047] Fourth, the system design of the present invention takes into account the complexity of the industrial site. By integrating high-precision cameras and flexible robotic arms, it can operate stably in various complex industrial environments, improving the robustness of the system.

[0048] Fifth, the system design of the present invention also considers the simplicity of operation. Through an intuitive user interface and a concise operation process, it reduces the learning cost and operation difficulty of users.

[0049] Sixth, the system design of the present invention also has high scalability and can easily integrate new detection technologies and algorithms to adapt to the development of future technologies and changes in market demands. Description of the Drawings

[0050] Figure 1 It is a composition module and operation flow chart of a dynamic product surface defect detection system based on a robotic arm;

[0051] Figure 2 It is a configuration flow chart of a defect detection algorithm for a dynamic product surface defect detection system based on a robotic arm. Detailed Embodiments

[0052] The present invention will be described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several changes and improvements can still be made. These all belong to the protection scope of the present invention.

[0053] Reference Figure 1 and Figure 2 , a dynamic product surface defect detection system based on a robotic arm of the present invention will be described:

[0054] System Environment: The hardware configuration adopted by the system is a CPU of 13th generation INTEL Core i5 13400F, a GPU of NVIDIA RTX4060Ti, a six-axis collaborative robotic arm of Innfos Cluon model, a camera of ORBBEC Gemini Pro model, the software environment is CUDA 12.2, and the development environment is Ubuntu 22.04.

[0055] A dynamic product surface defect detection system based on a robotic arm includes a camera configuration module, a robotic arm configuration module, a defect detection algorithm parameter configuration module, a dynamic defect detection process configuration module, and a dynamic defect detection execution module. Among them, the camera configuration module is used to set camera parameters to capture clear and accurate images, including: a camera model selection unit for selecting a camera suitable for the detection task; a resolution setting unit for setting the resolution of the camera; an image display mode setting unit for setting the display mode of the image; and a camera internal parameter calibration unit for accurately calibrating the internal parameters of the camera. The robotic arm configuration module is used to set the movement path and speed of the robotic arm. The robotic arm model selection unit is used to select the corresponding robotic arm model, including: an actuator model selection unit for selecting the corresponding actuator model; a robotic arm connection unit for establishing the connection between the robotic arm and the actuator; and a robotic arm debugging unit for observing and ensuring that the robotic arm moves precisely along the predetermined path. The defect detection algorithm parameter configuration module is used to allow users to adjust and optimize the parameters of the defect detection algorithm, including: a detection algorithm selection unit for selecting a suitable detection algorithm; and an algorithm parameter setting unit for setting algorithm parameters according to the characteristics of the selected algorithm. The dynamic defect detection process configuration module is used to set the detection process, detection sequence, and key detection areas, including: a detection process setting unit for constructing a detection process including the robotic arm moving to a fixed point, hovering, and moving along a fixed route; and a detection task automatic completion unit for automatically completing the detection task according to the preset parameters and processes and recording the detection results. The dynamic defect detection execution module is responsible for coordinating the above modules to execute the actual defect detection task, including: a real-time feedback unit for providing visual and auditory feedback during the detection process to ensure that users can monitor the detection status in real time; and a detection task automatic completion unit for automatically completing the detection task according to the preset parameters and processes.

[0056] In the camera configuration module setting area of the system interface, the user first selects a camera model suitable for the detection task, including various industrial cameras such as ORBBEC Gemini Pro and Gemini2 models. The user can select an appropriate resolution according to the size and shape of the target object, such as 1920x1080 or 1280x720. The image display mode can be set to color, grayscale, or infrared mode to meet different detection requirements. If there are pre-calibrated camera internal parameters, the user can directly apply these parameters and start the camera. If there are no ready-made internal parameters, the user clicks the internal parameter calibration button, and the system guides the user to the calibration interface to perform accurate camera internal parameter calibration to ensure that the camera can capture high-quality images in the best state.

[0057] At the robotic arm connection interface, the user selects a robotic arm of the Innfos Cluon model and connects the corresponding actuator. After successful connection, the user can view the field of view captured by the camera on the outside of the robotic arm in real time by clicking the camera button on the left. The user clicks the robotic arm debugging button to visually observe the operating state of the robotic arm and ensure that it moves precisely along the predetermined path.

[0058] The user selects a suitable algorithm from among the various detection algorithms supported by the system, such as FasterR-CNN, YOLOV8, etc. based on deep learning, and carefully sets the algorithm parameters according to the characteristics of the selected algorithm to optimize the detection effect. After turning on the camera, the system automatically starts the detection process. The user can observe the detection effect in real time in the detection algorithm display area and make parameter adjustments as needed.

[0059] According to specific dynamic detection requirements, the user can flexibly set different detection processes, including key operations such as the robotic arm moving to a fixed point and hovering. The user determines the logic and sequence of the entire detection process according to the characteristics of the detection task and the requirements of the detection target. After setting is completed, the user can export the complete detection process for reuse or fine-tuning in subsequent detection tasks.

[0060] Before starting the detection, the user ensures that both the working camera and the robotic arm observation camera are turned on to observe the detection results and the movement process of the robotic arm in real time. In the execution work area, the user executes the previously set detection process to perform multi-dimensional dynamic detection tasks. The system automatically completes the detection task according to the preset parameters and processes to ensure the accuracy and efficiency of the detection.

[0061] Actual scenario test; In an actual scenario, place the targets of the object categories included in the dataset on the experimental table, and use the robotic arm-assisted dynamic defect detection system as the vision drive to detect the detection accuracy of the detection model in the actual scenario. By comparing the actual detection results with the manual detection results, verify the accuracy and reliability of the system.

[0062] The system of the present invention is particularly suitable for the all-round surface defect detection of three-dimensional targets. The specific implementation is as follows:

[0063] During the three-dimensional defect detection process, the system first captures images of each surface of the three-dimensional target through a high-resolution camera. The camera configuration module allows the user to select a camera model suitable for the three-dimensional detection task and set appropriate resolution and image display modes. The robotic arm configuration module is responsible for positioning the camera at different positions and angles of the three-dimensional target to achieve all-round shooting.

[0064] In the defect detection algorithm parameter configuration module, the user can select the Faster R-CNN algorithm based on deep learning and combine the attention mechanism and multi-scale feature fusion technology to improve the accuracy of defect detection. The model trained on the NEU steel surface defect dataset can be used as a reference. Through steps such as data preprocessing, model initialization, Region Proposal Network (RPN) configuration, feature extraction, multi-scale feature fusion, loss function definition, optimizer configuration, model training, model validation, model fine-tuning, and model testing, the efficient detection of three-dimensional target surface defects can be finally achieved.

[0065] For the comprehensive defect detection of large-range targets, the system uses a long-focus camera to capture long-distance images, and the robotic arm moves the camera to different areas of the large-range target to achieve comprehensive coverage.

[0066] During the large-range defect detection process, the system adjusts the movement path of the robotic arm to ensure that the camera can cover all areas of the large-range target. The images captured from different areas will be analyzed to identify and locate the defects of the large-range target.

[0067] The following takes the steel surface defect detection as an example to introduce the steps of using and configuring the detection algorithm of the dynamic defect detection system. The model trained by the present invention using Faster R-CNN on the NEU (Northeastern University) steel surface defect dataset is used as a typical defect detection model. As Figure 2 shown, the specific implementation method is as follows:

[0068] The data preprocessing steps include normalizing the images in the NEU steel surface defect dataset, as well as annotating the bounding boxes and class labels of the defect regions.

[0069] The model initialization steps include selecting a pre-trained deep learning model as the base network of Faster R-CNN and loading the weights pre-trained on a large dataset.

[0070] The Region Proposal Network (RPN) configuration steps include setting the sizes and ratios of the RPN anchors (anchor) and determining the threshold of the RPN network to generate candidate defect regions.

[0071] The feature extraction steps use the base network to extract multi-scale features of the images and combine the attention mechanism to enhance the model's ability to identify defect features.

[0072] The multi-scale feature fusion steps fuse feature maps at different levels to utilize the low-level detailed information and high-level semantic information.

[0073] The loss function definition step defines the loss function of Faster R-CNN, including the class loss, the bounding box regression loss, and possibly the attention loss.

[0074] The optimizer configuration step selects a suitable optimizer, such as SGD, Adam, and sets the learning rate and other optimization parameters.

[0075] The model training step trains the Faster R-CNN model using the NEU steel surface defect dataset, and updates the model weights through backpropagation and gradient descent methods.

[0076] The model validation step evaluates the model performance on the validation set and adjusts the hyperparameters to optimize the precision and recall of the model.

[0077] The model fine-tuning step fine-tunes the model according to the performance feedback on the validation set to improve the detection accuracy of specific types of steel surface defects.

[0078] The model testing step evaluates the performance of the final model using the test set to ensure the effectiveness and generalization ability of the model in practical applications.

[0079] The post-processing step screens and optimizes the defect detection results output by the model to remove false detections and duplicate detections, and improves the accuracy and reliability of the detection results.

[0080] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Without conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A dynamic product surface defect detection system based on a robotic arm, characterized in that, The system includes: A camera configuration module for setting camera parameters to capture clear and precise images; A robotic arm configuration module for setting the movement path and speed of the robotic arm; A defect detection algorithm parameter configuration module for allowing users to adjust and optimize the parameters of the defect detection algorithm; A dynamic defect detection process configuration module for setting the detection process, detection sequence, and key detection areas; A dynamic defect detection execution module for coordinating the above modules to execute the actual defect detection task.

2. The system according to claim 1, wherein The camera configuration module includes: A camera model selection unit for selecting a camera suitable for the detection task; A resolution setting unit for setting the resolution of the camera; An image display mode setting unit for setting the display mode of the image; A camera internal parameter calibration unit for precisely calibrating the internal parameters of the camera.

3. The system according to claim 1, characterized in that, The robotic arm configuration module includes: A robotic arm model selection unit for selecting the corresponding robotic arm model; An actuator model selection unit for selecting the corresponding actuator model; A robotic arm connection unit for establishing the connection between the robotic arm and the actuator; A robotic arm debugging unit for observing and ensuring that the robotic arm moves precisely along the predetermined path.

4. The system according to claim 1, characterized in that, The defect detection algorithm parameter configuration module includes: A detection algorithm selection unit for selecting a suitable detection algorithm; An algorithm parameter setting unit for setting algorithm parameters according to the characteristics of the selected algorithm.

5. The system according to claim 1, wherein The dynamic defect detection process configuration module includes: A detection process setting unit for constructing a detection process including the robotic arm moving to a fixed point, hovering, and moving along a fixed route; A detection task automatic completion unit for automatically completing the detection task according to the preset parameters and process, and recording the detection results.

6. The system according to claim 1, characterized in that, The dynamic defect detection execution module includes: A real-time feedback unit for providing visual and auditory feedback during the detection process to ensure that users can monitor the detection status in real time; A detection task automatic completion unit for automatically completing the detection task according to the preset parameters and process.

7. The system according to claim 1, characterized in that, The camera includes a high-resolution camera for capturing images of the various surfaces of a three-dimensional target and / or a long-focus camera for capturing long-distance images of a large-range target.

8. A dynamic product surface defect detection method, characterized in that, It includes the following steps: Providing a robotic arm-based dynamic product surface defect detection system as described in claim 1; A camera configuration step, including selecting the camera model, resolution, and image display mode, and performing camera internal parameter calibration; A robotic arm and actuator connection step, including selecting the robotic arm and actuator models and establishing the connection; A detection parameter configuration step, including selecting the detection algorithm and setting the algorithm parameters; A detection process setting step, flexibly setting different detection processes according to specific dynamic detection requirements; A start detection step, after ensuring that both the working camera and the robotic arm observation camera are turned on, executing the preset detection process and completing the dynamic detection task.

9. The method according to claim 8, characterized in that, For the all-round surface defect detection of a three-dimensional target, it further includes: Adjusting the movement path of the robotic arm to ensure that the camera can capture the various surfaces of the three-dimensional target from different angles; analyzing the images captured from multiple angles to identify and locate the defects on the surface of the three-dimensional target.

10. The method according to claim 8, wherein For the comprehensive defect detection of a large-range target, it further includes: Adjust the movement path of the robotic arm to ensure that the camera can cover all areas of a large-scale target; analyze the images captured from different areas to identify and locate the defects of the large-scale target.

Images