Industrial robot detection operation method

Abstract

The invention belongs to the technical field of industrial robots, and discloses an industrial robot detection operation method, which comprises the following steps of: S1, transmitting a workpiece to a detection area, and acquiring a multi-angle and multi-light-source image of the workpiece; s2, preprocessing the collected original image of the workpiece; s3, performing target detection on the preprocessed image by using a target detection algorithm; and S4, according to a detection result, a mechanical arm is applied to carry out post-processing on the workpiece, and the mechanical arm is controlled to transfer the workpiece to a designated position. The defect that workpieces are detected manually on a production line is overcome, and the manpower requirement is reduced.

Description

Technical Field

[0001] The invention belongs to the technical field of industrial robots, and in particular relates to an industrial robot detection operation method. Background Art

[0002] In today's fast-developing industrial age, traditional production lines have long relied on workers to operate workpiece inspection. Workers must devote a lot of time and energy to carefully inspect each workpiece. They need to rely on their own experience and visual judgment to determine whether the workpiece meets quality standards. However, this manual inspection method has many drawbacks that are difficult to overcome.

[0003] First of all, manual inspection is an extremely time-consuming and labor-intensive task. Workers are prone to fatigue during the long inspection process, which may reduce the accuracy and efficiency of the inspection. Moreover, as the scale of production continues to expand, the workload of manual inspection will also increase exponentially, which undoubtedly brings huge labor cost pressure to enterprises.

[0004] Secondly, it is difficult to fully quantify the standards for manual inspection. Different workers may have different inspection results for the same workpiece due to differences in experience, skill level, and subjective judgment. This inconsistency not only affects the quality stability of the product, but may also lead to a decrease in customer trust in the company.

[0005] At the same time, with the rapid development of science and technology, compared with traditional manual inspection, robot automated inspection technology is gradually becoming the new favorite of modern industry with its unique advantages. This technology can perform inspections with amazing high precision. With its advanced sensors and sophisticated algorithms, robots can accurately identify extremely small defects on workpieces to ensure that product quality meets the highest standards.

[0006] Furthermore, robot automated testing technology is extremely efficient. They can work tirelessly and continuously, greatly shortening the testing time and increasing the overall speed of production. At the same time, robots can perform testing strictly in accordance with pre-set standards without being affected by subjective factors, ensuring the consistency and reliability of the test results.

[0007] In view of the above reasons, it is urgent to invent an industrial robot inspection method to meet the urgent needs of modern industry for high precision and high efficiency. It can not only reduce costs and improve competitiveness for enterprises, but also promote the entire industrial field to develop in a more intelligent and automated direction. Summary of the invention

[0008] Based on this, the present invention provides an industrial robot detection operation method to overcome the drawbacks of relying on manual inspection of workpieces on a production line and reduce manpower requirements.

[0009] The industrial robot detection operation method provided by the present invention comprises the following steps:

[0010] S1 transmits the workpiece to the inspection area and obtains multi-angle and multi-light source images of the workpiece;

[0011] S2 preprocesses the collected original image of the workpiece;

[0012] S3 uses the target detection algorithm to perform target detection on the preprocessed image;

[0013] S4 uses a robotic arm to post-process the workpiece according to the detection results, and controls the robotic arm to transfer the workpiece to a specified position.

[0014] Furthermore, after the workpiece is post-processed by the robot arm in step S4, the operation method further includes:

[0015] S5 performs manual re-inspection on the post-processed workpieces, and manually reviews the inspection results to further confirm the workpieces that may have been mis-inspected.

[0016] Furthermore, in step S1, the workpiece is transferred to the inspection area through the human-computer interaction interface.

[0017] Furthermore, in step S1, multi-angle and multi-light source images of the workpiece are acquired by automatic optical inspection equipment.

[0018] Furthermore, step S2 pre-processes the collected original image of the workpiece; including:

[0019] S21 denoising and enhancement: receiving an original image of a workpiece, removing noise from the original image and enhancing the contrast of the image;

[0020] S22 Image Standardization: Normalize the denoised and enhanced images to make the image features consistent under different light sources and environments;

[0021] S23 Histogram Equalization: Adjust the normalized image histogram distribution to enhance image contrast.

[0022] Furthermore, in step S3, the target detection algorithm is specifically:

[0023] S301 acquires a large number of workpiece images and uses labels to annotate data for various workpiece types;

[0024] S302 uses a data enhancement method and a regional data enhancement method to perform contrast enhancement and / or color adjustment on the workpiece or the key part of the workpiece, and performs blur processing on the background or non-key area; the enhanced key area and the background are synthesized together to generate a final enhanced image;

[0025] S303 inputs the data-enhanced image into the model for training and generates a weight file;

[0026] S304 uses the trained model weights to process the image data in real time, detects and classifies the workpieces in the image, and outputs the position information of the workpieces after identifying the workpieces.

[0027] Furthermore, the method for generating the weight file is:

[0028] Real-time analysis of the target density during the detection process, that is, the number of prediction boxes in each feature layer, increases the weight when a large number of small targets are detected in the low-level feature map, and decreases the weight otherwise.

[0029] Furthermore, in step S3, target detection is performed on the preprocessed image using a target detection algorithm; including:

[0030] S31 feature extraction: inputting the preprocessed image into the trained model, wherein the trained model extracts the structure and texture features of the workpiece from the preprocessed image;

[0031] S32 Detection and classification: the trained model detects and classifies various parts of the workpiece, and identifies the type of the workpiece and its position in the image;

[0032] S33 outputs the detection result: the trained model outputs the detection result to the human-computer interaction interface, and the detection result includes the specific part, type and position information of the workpiece.

[0033] Further, step S4 uses a robot arm to post-process the workpiece according to the detection result; including:

[0034] S41 Positioning: Based on the detected workpiece position information, the robot arm obtains the three-dimensional coordinates of the workpiece and locates the workpiece position;

[0035] S42 Grasping Path Planning: Use the dynamic programming algorithm to find a collision-free path in the high-dimensional space as the grasping path, and then apply the B-spline curve to smooth the planned grasping path;

[0036] S43 transfer path planning: after the robot arm successfully grasps the workpiece, the robot generates a path from the grasping position to the placement area as a transfer path, and then applies a B-spline curve to smooth the planned transfer path;

[0037] S44 post-processing operation: the robot arm transfers the detected workpieces to a designated location for classified stacking according to the workpiece type, state, or priority;

[0038] S45 Operation confirmation and recording: The robot automatically records the status of each operation and retains the workpiece type and location information.

[0039] Furthermore, in step S42 and step S43, the method of applying the B-spline curve to perform path smoothing is:

[0040] The B-spline curve is directly embedded in the iterative process of dynamic programming, a smooth path is gradually generated, and a local smooth curve is calculated in each dynamic programming step;

[0041] Dynamically adjust speed and / or acceleration parameter values ​​based on changes in path curvature.

[0042] The beneficial effects of the present invention are:

[0043] (1) The target detection algorithm of the present invention adopts a data enhancement method during model training, so that the model has strong adaptability to workpiece features in different light and angles, and increases the robustness of detection. Thus, the robot system can efficiently process high-resolution images with low latency.

[0044] (2) In the grasping path planning and transfer path planning, the present invention applies B-spline curves to smooth the planned grasping paths and transfer paths, and directly embeds the B-spline curves into the iterative process of dynamic programming, so as to gradually generate smooth paths instead of smoothing them after the paths are planned. The local smooth curves are calculated in each dynamic programming step, thereby avoiding frequent adjustments to the overall path.

[0045] (3) In addition, the present invention applies motion control on a smooth path, adds speed and acceleration control, and dynamically adjusts the speed / acceleration parameters according to the change of path curvature, thereby ensuring path smoothness and avoiding jitter. Adaptive acceleration control can enable the robot arm to adjust the acceleration value on path segments with different curvatures, for example, reducing speed and acceleration when making sharp turns, and appropriately increasing speed on straight sections. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A flow chart of an industrial robot detection operation method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. The preferred embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive.

[0048] The industrial robot detection operation method provided by the present invention has the following principle: the industrial robot operation system mainly includes five key components, namely, operation interface, product transmission, camera shooting, intelligent recognition and robot grasping. The operation interface controls the opening and closing of the entire robot operation system, and can observe the real-time data captured by the camera and the results of the target detection model recognition. Product transmission, transporting the product to a position where the robot is convenient for taking photos and detection. Camera shooting, the camera takes pictures of the product from multiple angles. Intelligent recognition, the target detection model recognizes the product and feeds back to the robot. Robot grasping, the robot grasps the product and operates.

[0049] It should be noted that the product of the present invention also refers to a workpiece.

[0050] like Figure 1 As shown, the industrial robot detection operation method provided by the embodiment of the present invention includes the following steps:

[0051] S1 transfers the workpiece to the inspection area and obtains multi-angle and multi-light source images of the workpiece.

[0052] (1) Operate through the human-computer interaction interface to transfer the workpiece to the inspection area.

[0053] On the operation interface of the workshop, the robot system is started through the operation control panel, and the conveyor gradually transfers the workpiece to the inspection area, and the robot system enters the automatic inspection and grasping state. Automatic inspection includes the inspection of sensors, cameras and robotic arms; if a fault is found, a warning will be issued on the operation interface to prevent the operation from starting.

[0054] The operation interface includes function buttons such as start, stop, and emergency stop, and can display the robot system status information (such as motor temperature, working speed, etc.) in real time, and can also display the real-time picture of the detection area, so that workers can fully control the current operation status. After starting the robot system, the operation interface will load information such as equipment operation status, operation progress, and detection statistics.

[0055] (2) Obtain multi-angle and multi-light source images of the workpiece through automatic optical inspection equipment.

[0056] AOI (Automated Optical Inspection) is the full name in Chinese. The basic principle is to use imaging technology to compare whether the object to be tested is too different from the standard image to determine whether the object to be tested meets the standard.

[0057] AOI equipment integrates a variety of light sources and can fully capture workpieces at different angles to ensure high resolution and clarity of the image, providing high-quality data for the subsequent accurate detection of target detection models. The multi-angle imaging characteristics of AOI equipment are particularly suitable for the detection of workpieces with complex structures, and can effectively reduce the impact of occlusion and reflection on detection accuracy.

[0058] The present invention uses the multi-light source and multi-angle shooting technology of the AOI equipment to obtain the detection image. Even if the workpieces are randomly placed on the conveyor belt, it can ensure that the defect features are captured, effectively improving the fault tolerance of the detection.

[0059] S2 preprocesses the collected original image of the workpiece.

[0060] Specifically include:

[0061] S21 Denoising and Enhancement: Receives the original image of the workpiece from the camera and enhances the detection effect through a series of preprocessing operations. Typical preprocessing includes contrast enhancement, denoising, and image resolution adjustment to obtain a clearer and more detailed image before inputting the target detection model.

[0062] S22 Image Standardization: Normalize the denoised and enhanced images to make the image features consistent under different light sources and environments; ensure that the images collected under different ambient lighting conditions remain consistent as inputs to the target detection model, thereby improving the generalization ability of the target detection model.

[0063] S23 Histogram Equalization: Adjust the normalized image histogram distribution to enhance image contrast.

[0064] Before inputting the image into the target detection model, histogram equalization is applied to enhance the contrast of the image, making the sample features clearer under complex lighting conditions, thereby improving the detection accuracy.

[0065] The preprocessed images are input into the object detection model for training or inference.

[0066] S3 uses the target detection algorithm to perform target detection on the preprocessed image.

[0067] The target detection algorithm of this embodiment is a deep learning algorithm based on the YOLOv8 model. Of course, in other embodiments, training can also be performed based on other models.

[0068] The target detection algorithm of this embodiment is specifically:

[0069] S301 acquires a large number of workpiece images and uses labels to annotate data for various workpiece types.

[0070] After preprocessing, the images are labeled using the label img tool. The labeled categories include normal products and products so that the model can learn the characteristics of different categories. During the collection process, samples of different shapes, sizes and lighting conditions are included as comprehensively as possible to improve the generalization ability of the model.

[0071] S302 uses a data enhancement method to perform contrast enhancement and / or color adjustment on the workpiece or the key parts of the workpiece, and blur the background or non-key areas; synthesize the enhanced key areas and background together to generate a final enhanced image; expand the diversity of training samples and improve the generalization ability of the model.

[0072] In this embodiment, the data enhancement method uses Mosaic, which is to stitch multiple different images into a new sample, thereby enhancing the model's ability to adapt to the diversity of sample combinations and distributions. At the same time, other commonly used enhancement methods such as rotation, scaling, random cropping, horizontal flipping, etc. are also applied to different images to improve the richness of the data.

[0073] Table 1 Regional data enhancement

[0074]

[0075] S303 inputs the data-enhanced image into the YOLOv8 model for training and generates a weight file.

[0076] During the experiment, in order to train the target detection model, the following experimental parameters were set:

[0077] Learning rate: The initial learning rate is 0.01, which is gradually decayed during training through the learning rate scheduler to ensure that the model is more stable when approaching the optimal solution.

[0078] Iteration rounds: The training iteration rounds are set to 1000 epochs to ensure that the model can fully learn the data in enough rounds.

[0079] Training equipment: The experiment was conducted on a high-performance server, which was configured with four NVIDIA GeForceRTX 3090Ti GPUs to support large-scale parallel training acceleration. The training was conducted using the Pytorch framework, and the image processing part was supported by the OpenCV library. All necessary dependent libraries were installed in the experimental environment to ensure efficient training.

[0080] After training is completed, the weight file generated by the target detection model is deployed to the robot target detection equipment in the workshop to ensure that the target detection model can run stably in the production environment.

[0081] Among them, the method for generating the weight file is:

[0082] Real-time analysis of the target density during the detection process, that is, the number of prediction boxes in each feature layer, increases the weight when a large number of small targets are detected in the low-level feature map, and decreases the weight otherwise.

[0083] Table 2 Adaptive weight adjustment based on small target density

[0084]

[0085] S304 uses the trained model weights to process the image data in real time, detects and classifies the workpieces in the image, and outputs the position information of the workpieces after identifying the workpieces.

[0086] Use the target detection algorithm to perform target detection on the preprocessed image, including:

[0087] S31 Feature extraction: The preprocessed image is input into the trained model, and the trained model (ie, the target detection model) extracts features such as the structure and texture of the workpiece from the preprocessed image.

[0088] Many prediction boxes are predicted on each image, and the weight of each layer is adaptively adjusted based on the density of small objects. The more prediction boxes there are, the higher the weight of this image needs to be, otherwise it will be inaccurate.

[0089] In this embodiment, the feature map has three layers, and the size of each layer is twice that of the previous layer, which is convenient for detecting targets of different sizes.

[0090] S32 Detection and classification: The trained model detects and classifies various parts of the workpiece, and identifies the type of the workpiece and its position in the image.

[0091] S33 outputs the detection results: the trained model outputs the detection results to the human-computer interaction interface, and the detection results include the specific part, type and location information of the workpiece for the operator to view in real time.

[0092] S4 uses a robotic arm to post-process the workpiece according to the detection results, and controls the robotic arm to transfer the workpiece to a specified position.

[0093] Specifically include:

[0094] S41 Positioning: Based on the detected workpiece position information, the robot arm obtains the three-dimensional coordinates of the workpiece (including height, angle and other information) so that the robot arm can quickly locate the workpiece.

[0095] In addition, a calibration action can be added after positioning. The robotic arm will calibrate the positioning accuracy in real time based on factors such as ambient light and angle to ensure accurate grasping.

[0096] S42 Grasping Path Planning: Use a dynamic programming algorithm to find a collision-free path in high-dimensional space as the grasping path, and then apply B-spline curves to smooth the planned grasping path to reduce sharp corners of the path, thereby smoothing the turns in the path.

[0097] Generate a smooth path through multiple control points to ensure the smooth and smooth movement of the robot arm, reduce vibration or sudden movement, which can extend the life of the robot arm and improve the accuracy of operation. When approaching the target, the speed of the robot arm is automatically reduced to avoid the impact of impact on the product.

[0098] Table 3 Path planning algorithm based on B-spline smoothing

[0099]

[0100]

[0101] S43 Transfer path planning: After the robot arm successfully grasps the workpiece, the robot generates a path from the grasping position to the placement area as the transfer path, and then applies the B-spline curve to smooth the planned transfer path to avoid abrupt movements and ensure the stability of the transfer process.

[0102] It should be noted that in step S42 and step S43, the specific method of applying the B-spline curve to perform path smoothing is:

[0103] The B-spline curve is directly embedded in the iterative process of dynamic programming, a smooth path is gradually generated, and a local smooth curve is calculated in each dynamic programming step;

[0104] Dynamically adjust speed and / or acceleration parameter values ​​based on changes in path curvature.

[0105] S44 post-processing operation: According to the type, status, or priority of the workpiece, the robot arm transfers the detected workpiece to a designated location for classified stacking.

[0106] For workpieces that require further processing or special treatment, the robot system will automatically dispatch the robot arm to move them to other designated workstations. For example, after grabbing a watch strap, it needs to be cleaned, polished or painted, and the robot arm will place it in different processing stations according to the process requirements.

[0107] S45 Operation confirmation and recording: The robot automatically records the status of each operation and retains the workpiece type and location information.

[0108] The robotic arm automatically completes the entire process of workpiece identification, recording and operation, providing supporting data for subsequent quality management and improvement.

[0109] S5 performs manual re-inspection on the post-processed workpieces, and manually reviews the inspection results to further confirm the workpieces that may have been mis-inspected.

[0110] The present invention adopts a deep learning algorithm based on the YOLOv8 model as the target detection algorithm. In the feature pyramid network (FPN) of YOLOv8, the weight of the low-level feature map is adaptively enhanced to increase the attention paid to small targets. The detection accuracy of the model under high-resolution images is improved to ensure that tiny features can be accurately identified. Through this algorithm, the robot system can efficiently process high-resolution images with low latency.

[0111] The data enhancement method is used during model training to make the model more adaptable to the features of workpieces in different light and angles, thereby increasing the robustness of detection.

[0112] Through data enhancement and diversified training, the target detection model has the adaptability to workpieces of different sizes and shapes, and can accurately detect a variety of workpieces and defect shapes.

[0113] The present invention combines the target detection method of artificial intelligence technology with robotics technology, which improves the accuracy, speed and robustness of target detection, can better complete the detection tasks of objects with diverse sizes, blurred edges and concentrated positions, and improves the efficiency and reliability of industrial production.

[0114] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0115] The above embodiments only express the preferred implementation modes of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.

Claims

1. An industrial robot detection operation method, characterized in that: The following steps are involved: S1 transmits the workpiece to the inspection area and obtains multi-angle and multi-light source images of the workpiece; S2 preprocesses the collected original image of the workpiece; S3 uses the target detection algorithm to perform target detection on the preprocessed image; S4 uses a robotic arm to post-process the workpiece according to the detection results, and controls the robotic arm to transfer the workpiece to a specified position.

2. The industrial robot detection operation method according to claim 1, characterized in that: After the robot arm is used to post-process the workpiece in step S4, the operation method further includes: S5 performs manual re-inspection on the post-processed workpieces, and manually reviews the inspection results to further confirm the workpieces that may have been mis-inspected.

3. The industrial robot detection operation method according to claim 1, characterized in that: In step S1, the workpiece is transferred to the inspection area through the human-computer interaction interface.

4. The industrial robot detection operation method according to claim 1, characterized in that: In step S1, multi-angle and multi-light source images of the workpiece are acquired by automatic optical inspection equipment.

5. The industrial robot detection operation method according to claim 1, characterized in that: Step S2 preprocesses the collected original image of the workpiece; including: S21 denoising and enhancement: receiving an original image of a workpiece, removing noise from the original image and enhancing the contrast of the image; S22 Image Standardization: Normalize the denoised and enhanced images to make the image features consistent under different light sources and environments; S23 Histogram Equalization: Adjust the normalized image histogram distribution to enhance image contrast.

6. The industrial robot detection operation method according to claim 1, characterized in that: In step S3, the target detection algorithm is specifically: S301 acquires a large number of workpiece images and uses labels to annotate data for various workpiece types; S302 uses a data enhancement method to perform contrast enhancement and / or color adjustment on the workpiece or a key part of the workpiece, and performs blur processing on the background or non-key areas; The enhanced key area and the background are synthesized together to generate a final enhanced image; S303 inputs the data-enhanced image into the model for training and generates a weight file; S304 uses the trained model weights to process the image data in real time, detects and classifies the workpieces in the image, and outputs the position information of the workpieces after identifying the workpieces.

7. The industrial robot detection operation method according to claim 6, characterized in that: The method for generating the weight file is: Real-time analysis of the target density during the detection process, that is, the number of prediction boxes in each feature layer, increases the weight when a large number of small targets are detected in the low-level feature map, and decreases the weight otherwise.

8. The industrial robot detection operation method according to claim 3, characterized in that: In step S3, target detection is performed on the preprocessed image using a target detection algorithm; including: S31 feature extraction: inputting the preprocessed image into the trained model, wherein the trained model extracts the structure and texture features of the workpiece from the preprocessed image; S32 Detection and classification: the trained model detects and classifies various parts of the workpiece, and identifies the type of the workpiece and its position in the image; S33 outputs the detection result: the trained model outputs the detection result to the human-computer interaction interface, and the detection result includes the specific part, type and position information of the workpiece.

9. The industrial robot detection operation method according to claim 1, characterized in that: Step S4: post-processing the workpiece using a robot arm according to the detection result; including: S41 Positioning: Based on the detected workpiece position information, the robot arm obtains the three-dimensional coordinates of the workpiece and locates the workpiece position; S42 Grasping Path Planning: Use the dynamic programming algorithm to find a collision-free path in the high-dimensional space as the grasping path, and then apply the B-spline curve to smooth the planned grasping path; S43 transfer path planning: after the robot arm successfully grasps the workpiece, the robot generates a path from the grasping position to the placement area as a transfer path, and then applies a B-spline curve to smooth the planned transfer path; S44 post-processing operation: the robot arm transfers the detected workpieces to a designated location for classified stacking according to the workpiece type, state, or priority; S45 Operation confirmation and recording: The robot automatically records the status of each operation and retains the workpiece type and location information.

10. The industrial robot detection operation method according to claim 9, characterized in that: In step S42 and step S43, the method of applying the B-spline curve to perform path smoothing is: The B-spline curve is directly embedded in the iterative process of dynamic programming, a smooth path is gradually generated, and a local smooth curve is calculated in each dynamic programming step; Dynamically adjust speed and / or acceleration parameter values ​​based on changes in path curvature.

Images