Robot wood board feeding method based on visual positioning and binocular recognition distance measurement

Abstract

The invention discloses a robot wood board feeding method based on visual positioning and binocular recognition distance measurement. The method comprises the following steps: carrying out image acquisition on a wood board through two camera modules mounted on a mechanical arm; an improved YOLO V8 model is used for processing the image, and accurate positioning and size measurement of the wood board are achieved; the method comprises the following steps: acquiring a depth map and a point cloud map by using two camera modules, de-weighting depth values in the depth map and obtaining coordinates, matching a central point of a bounding box with the depth map, obtaining a distance between a robot and a target object through geometric distance measurement, and calculating the position and posture of a wood board in a three-dimensional space according to a measurement result; and the mechanical arm adjusts the grabbing position and angle according to a preset program and a calculation result, and automatic feeding of the wood boards is completed. The wood board feeding device can improve the wood board feeding efficiency and precision, reduces manual operation, is suitable for the fields of furniture manufacturing, building construction and the like, and has important practical value.

Description

Technical Field

[0001] The invention relates to the field of furniture manufacturing and robot technology, and in particular to a robot wood board feeding method based on visual positioning and binocular recognition and ranging. Background Art

[0002] In the fields of furniture manufacturing and construction, wood boards are commonly used as a material. The loading process usually requires manual handling and positioning, which is not only inefficient, but also labor-intensive and prone to errors. With the development of robotics, more and more automated equipment is being used in production lines to improve production efficiency and product quality. Visual positioning technology is a technology that uses a computer vision system to identify, locate and track target objects. Through the camera module installed on the robotic arm, the image information of the target object can be obtained, and then analyzed through the image processing algorithm to achieve accurate positioning of the target object. Binocular recognition and ranging technology is a technology that uses two cameras to shoot the same scene from different angles and obtains depth information by calculating the parallax between the two camera images. This technology can be used to measure the distance and size of objects and provide robots with more accurate spatial information. At present, some wood board loading methods based on visual positioning and binocular recognition and ranging technology have been proposed. For example, a wood board loading method based on monocular vision uses a single camera installed on the robotic arm to collect images of the wood board, and then uses an image processing algorithm to locate and measure the size of the wood board. However, this method can only obtain information on a two-dimensional plane, and cannot obtain the position and posture of the wooden board in three-dimensional space, thus limiting its application in complex scenes. In order to solve the above problems, the present invention proposes a robot wooden board feeding method based on visual positioning and binocular recognition ranging. The method processes the image by improving the YOLO V8 model to achieve accurate positioning and size measurement of the wooden board; two camera modules are used to obtain a depth map and a point cloud map, and the depth values ​​in the depth map are deduplicated and the coordinates are obtained. The center point of the bounding box is matched with the depth map, and the distance between the robot and the target object is obtained by geometric ranging. According to the measurement results, the position and posture of the wooden board in three-dimensional space are calculated; the robotic arm adjusts the grasping position and angle according to the preset program and calculation results to complete the automatic feeding of the wooden board. The present invention can improve the efficiency and accuracy of wooden board feeding, reduce manual operations, and is suitable for furniture manufacturing, construction and other fields, and has important practical value. Summary of the invention

[0003] To solve the above problems, the present invention proposes a robot wood board loading method based on visual positioning and binocular recognition and ranging. According to the objects detected by the network model, the binocular recognition and ranging algorithm and calibration algorithm are used to establish the robot coordinates and binocular camera coordinate system, and guide the robot arm to control the wood board loading. Effectively improve the efficiency of wood board loading.

[0004] To achieve the above purpose, the present invention provides the following technical solution: a robot wood board feeding method based on visual positioning and binocular recognition and ranging, comprising the following steps: Step 1: calibrate the binocular camera to obtain internal and external parameters. The specific steps are as follows: According to the required resolution, the binocular camera parameters are set, and the binocular camera is called to shoot chessboard images from different angles, and instantiated segmentation is performed to refine the left and right images; the left and right image data sets are calibrated using the computer vision library to obtain the left and right distortion coefficient vectors, and the rotation matrix and translation vector of the left and right cameras are obtained through algorithms and transposition operations, and the parameters are reprojected and verified. In actual applications, fine-tuning is performed according to needs to achieve the best results. The steps are as follows: Step 1.1, determine the chessboard image size and the required resolution of the binocular camera; Step 1.2, call the binocular camera to shoot chessboard images at different angles and perform instantiation segmentation; Step 1.3, generate the intrinsic parameter matrix and distortion coefficient through the computer vision library and the instantiated segmented left and right camera images; Step 1.4, call the computer vision library to generate the rotation matrix and translation vector of the binocular camera; Step 1.5, reproject the new checkerboard image using the obtained internal and external parameters; Step 1.6, fine-tune the internal and external parameters according to specific needs to achieve the best effect; Step 2: Build a data set, filter, divide and calibrate the data set. The specific steps are as follows: Step 2.1, define the wooden board as the target of the data set and collect the target image through the camera; Step 2.2, preprocess the collected data to remove abnormal images and missing images; Step 2.3, performing image enhancement operation on the image to obtain a clear and diversified image; Step 2.4, annotating the image in step 2.3 to obtain an annotated image; Step 2.5, divide the annotated images in step 2.4 into training set, validation set and test set according to the required proportions; Step 3: Improve the YOLO V8 network model and add the EMA attention mechanism. The specific steps are as follows: Step 3.1, add EMA related calculation logic into the backbone network; Step 3.2, add the parameters and initialization code related to the EMA attention mechanism in the initialization file; Step 3.4, modify the YOLO V8 configuration file and customize the yaml file to modify the model architecture and parameters; Step 3.5, monitor the performance of the model and make adjustments as needed; Step 4: Train and deploy the wood board detection model as follows: Step 4.1, use the training set data in step 2.5 to train the improved YOLO V8 network to obtain the detection model; Step 4.2, monitor the loss function value and accuracy during training, and adjust the learning rate and other hyperparameters to obtain the best performance; Step 4.3, evaluate the detection model and adjust the model structure and parameters according to the evaluation results to improve the generalization ability and accuracy of the model; Step 4.4, convert the trained model into ONNX format, and perform initial inference in combination with the camera initial algorithm; Step 4.5, stereo matching the target calculated centroid point inferred in step 4.4 with the depth map and establishing a mapping relationship between the robot arm coordinates and the binocular camera coordinate system to perform visual guided grasping and loading; Step 4.5, stereo matching is performed between the target calculated centroid point inferred in step 4.4 and the depth map, and a mapping relationship between the robot arm coordinates and the binocular camera coordinate system is established to perform visually guided grasping and loading.

[0005] Furthermore, in step 1, the internal and external parameters of the binocular camera include: internal parameter matrix, focal length, principal point, radial distortion, rotation matrix, translation vector, average reprojection error, reprojection error, and reprojection point.

[0006] Furthermore, in step 2.1, the wood board dataset sample includes image data, annotation type, dataset format and data enhancement.

[0007] Furthermore, in step 3, improving YOLO V8 includes introducing an EMA attention mechanism, optimizing a loss function, enhancing a data method, and introducing a new network architecture.

[0008] Furthermore, in step 4.4, the depth map includes the depth information of the target object in the three-dimensional space, and a corresponding relationship between the two-dimensional image and the three-dimensional space point is established.

[0009] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) The present invention can more accurately identify and locate the position and size of the wooden board by introducing the improved YOLO V8 network model and binocular camera system. This high-precision identification and positioning capability significantly improves the accuracy and efficiency of wooden board loading and reduces the error rate and time cost of manual operation; (2) The present invention greatly reduces the reliance on manual operation and reduces the labor intensity through the automated visual guidance grasping system, while improving the consistency and reliability of the operation; (3) The system of the present invention can adapt to wooden boards of different sizes, shapes and textures, as well as different working environments and conditions. This strong adaptability enables the method to be widely used in various wooden board loading scenarios without being restricted by specific conditions; (4) Since the present invention adopts an automated visual guidance grasping system, the operator does not need to directly contact the wooden board or enter a dangerous working environment, thereby greatly reducing the safety risks in the working process; (5) The system design of the present invention takes into account ease of use and maintainability, allowing users to easily set up, adjust and upgrade the system. At the same time, with the continuous development of technology, the system can also easily integrate new functions and algorithms to maintain its advanced nature and competitiveness. (6) The application of the present invention patent has promoted the automation and intelligence level of the wood processing industry, and helped to improve the production efficiency and product quality of the entire industry. At the same time, it also provides a technical solution that can be used as a reference for other similar industries, and promotes the technological progress and development of related industries.

[0010] Figure 1 The figure is a flow chart of a method for robot wood board loading based on visual positioning and binocular recognition and ranging.

[0011] Specific implementation method: The following will be combined with the drawings in the embodiments of the patent of the present invention to clearly and completely describe the technical solutions in the embodiments of the patent of the present invention. Obviously, the described embodiments are only part of the embodiments of the patent of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the patent of the present invention.

[0012] See also Figure 1 The present invention provides a robot wood board feeding method based on visual positioning and binocular recognition ranging, which is completed by a computer system and includes the following steps: Step 1: calibrate the binocular camera to obtain internal and external parameters; Step 1.1, determine the chessboard image size and the required resolution of the binocular camera; Step 1.2: Use the binocular camera to capture a series of checkerboard images from different angles. Ensure that the images at each angle have sufficient coverage and clarity. Step 1.3, instantiate the segmentation step 1.2 to capture the image, and distinguish the left eye image and the right eye image based on the binocular camera; Step 1.4, use the left eye image data set to calibrate the left eye camera, and obtain the left eye intrinsic parameter matrix K1 and the left eye distortion coefficient vector D1. Similarly, use the right eye image data set to calibrate the right eye camera, and obtain the right eye intrinsic parameter matrix K2 and the right eye distortion coefficient vector D2; Step 1.5, take the parameters K1, K2, D1, and D2 obtained by left and right eye measurements as input, and then use the chessboard images with one-to-one correspondence between the left and right eyes to call the computer vision library to output the left and right eye rotation matrix R and translation vector T; Step 1.6: According to the verification results, if the error is large or not in line with expectations, the calibration process can be adjusted and optimized, and the results with large errors can be eliminated. The saved results include the intrinsic parameter matrix, distortion coefficient vector, rotation matrix and translation vector. Step 2: Build a data set, filter, divide and calibrate the data set: Step 2.1, define the wooden board as the target of the data set, and collect target images frame by frame through the camera (the target images are not less than 500); Step 2.2, preprocess the collected data to remove abnormal images and missing images; Step 2.3, performing image enhancement operations including linear transformation, neighborhood processing, rotation, and color transformation on the image in step 2.2 to obtain a clear and diversified image; Step 2.4, annotating the clear image in step 2.3 to obtain annotated data; Step 2.5, divide the annotated images in step 2.4 into training set, validation set and test set according to the required proportions; Step 3: Improve the YOLO V8 network model and add the EMA attention mechanism; Step 3.1, add EMA related calculation logic into the backbone network; Step 3.2, add the parameters and initialization code related to the EMA attention mechanism in the initialization file; Step 3.4, modify the YOLO V8 configuration file, and customize the yaml file to modify the model architecture and parameter settings; Step 3.5, monitor the performance of the model and make adjustments as needed; Step 4: Train and deploy the wood board detection model; Step 4.1, use the training set data in step 2.5 to train the improved YOLO V8 network to obtain the detection model; Step 4.2, monitor the loss function value and accuracy during training, and adjust the learning rate and other hyperparameters to obtain the best performance; Step 4.3, perform model evaluation on the detection model, including mean absolute error and mean square error, and adjust the model structure and parameters according to the evaluation results to improve the generalization ability and accuracy of the model; Step 4.4, convert the detection model in step 4.3 into ONNX format and deploy it in the production environment for actual reasoning tasks; Step 4.5, stereo matching the target calculated centroid point inferred in step 4.4 with the depth map to obtain the depth value; Step 4.6, establish the mapping relationship between the robot arm coordinates and the binocular camera coordinate system; Step 4.7, applying the mapping relationship established in step 4.6, converting the detected object coordinates and the depth value obtained in step 4.4 into a world coordinate system; Step 4.8, use the coordinates of step 4.7 to establish communication between the visual detection algorithm and the robotic arm through the Modbus protocol to perform visual-guided grasping and loading.

[0013] It should be noted that the above content only illustrates the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications all fall within the protection scope of the claims of the present invention.

Claims

1. A robot wood board feeding method based on visual positioning and binocular recognition ranging, characterized in that: The steps include: Step 1: calibrate the binocular camera to obtain internal and external parameters. The specific steps are as follows: Set the binocular camera parameters according to the required resolution, call the binocular camera to shoot chessboard images from different angles, and perform instantiation segmentation to refine the left and right images; use the computer vision library to calibrate the left and right image data sets to obtain the left and right distortion coefficient vectors, and obtain the rotation matrix and translation vector of the left and right cameras through algorithms and transposition operations, and reproject them to verify the parameters. In actual applications, fine-tune according to needs to achieve the best results. The steps are as follows: Step 1.1, determine the chessboard image size and the required resolution of the binocular camera; Step 1.2, call the binocular camera to shoot chessboard images at different angles and perform instantiation segmentation; Step 1.3, generate the intrinsic parameter matrix and distortion coefficient through the computer vision library and the instantiated segmented left and right camera images; Step 1.4, call the computer vision library to generate the rotation matrix and translation vector of the binocular camera; Step 1.5, reproject the new checkerboard image using the obtained internal and external parameters; Step 1.6, fine-tune the internal and external parameters according to specific needs to achieve the best effect; Step 2: Build a data set, filter, divide and calibrate the data set. The specific steps are as follows: Step 2.1, define the wooden board as the target of the data set and collect the target image through the camera; Step 2.2, preprocess the collected data to remove abnormal images and missing images; Step 2.3, performing image enhancement operation on the image to obtain a clear and diversified image; Step 2.4, annotating the image in step 2.3 to obtain an annotated image; Step 2.5, divide the annotated images in step 2.4 into training set, validation set and test set according to the required proportions; Step 3: Improve the YOLO V8 network model and add the EMA attention mechanism. The specific steps are as follows: Step 3.1, add EMA related calculation logic into the backbone network; Step 3.2, add the parameters and initialization code related to the EMA attention mechanism in the initialization file; Step 3.4, modify the YOLO V8 configuration file and customize the yaml file to modify the model architecture and parameters; Step 3.5, monitor the performance of the model and make adjustments as needed; Step 4: Train and deploy the wood board detection model as follows: Step 4.1: Use the training set data from step 2.5 to train the improved YOLO V8 network to obtain the detection model Step 4.2, monitor the loss function value and accuracy during training, and adjust the learning rate and other hyperparameters to obtain the best performance; Step 4.3, evaluate the detection model and adjust the model structure and parameters according to the evaluation results to improve the generalization ability and accuracy of the model; Step 4.4, convert the trained model into ONNX format, and perform initial inference in combination with the camera initial algorithm; Step 4.5, stereo matching is performed between the target calculated centroid point inferred in step 4.4 and the depth map, and a mapping relationship between the robot arm coordinates and the binocular camera coordinate system is established to perform visually guided grasping and loading.

2. The robot wood board feeding method based on visual positioning and binocular recognition ranging according to claim 1 is characterized in that: Use a binocular camera module to capture images of the wooden board from different angles, and process these images through a computer vision library to obtain depth information and point cloud images.

3. The robot wood board feeding method based on visual positioning and binocular recognition ranging according to claim 1 is characterized in that: Build a dataset, filter, divide and calibrate the dataset to ensure that there is enough diverse and high-quality data for training the model. Use the improved YOLO V8 model combined with the EMA attention mechanism to improve the accuracy and speed of wood board detection. Convert the trained model to ONNX format and combine it with the camera real-time algorithm for real-time inference.

4. The robot wood board feeding method based on visual positioning and binocular recognition and ranging according to claim 1 is characterized in that: The binocular camera calibration technology is used to obtain the intrinsic and extrinsic parameters, including the distortion coefficient vector, rotation matrix and translation vector, and reprojection verification is performed to ensure the accuracy of the parameters.

5. The robot wood board feeding method based on visual positioning and binocular recognition and ranging according to claim 1 is characterized in that: According to the calculated depth value and coordinates, the distance between the robot and the board is determined by geometric ranging, and the position and posture of the board in three-dimensional space are calculated. The target center of mass is calculated and stereo matching is performed with the depth map to establish the mapping relationship between the robot arm coordinates and the binocular camera coordinate system. The robot arm automatically adjusts the grasping position and angle according to the preset program and calculation results to achieve accurate loading of the board.

Images