System and method for visually correcting tightening position of equipment

Abstract

The invention discloses a visual correction equipment tightening position system and method, and the system comprises the following components: a first camera which is designed and arranged at the inclined upper part of a workpiece, is located at the same side with a backlight source, is located above the backlight source, and is used for determining the position of a tightening hole in order to match an operation environment in an actual scene; the second camera is located on the other side of the workpiece, located on the same horizontal line with the backlight and used for shooting the highest height of the workpiece; the backlight source is positioned below the first camera, is positioned on the same horizontal line with the second camera, and is used for acquiring a better outer contour image of the workpiece, reducing the influence of environmental factors and simultaneously providing proper brightness for determining a hole site; the system is matched with the method, the technologies of perspective correction, template matching, camera calibration and the like are combined, the tightening position is automatically recognized and corrected through a visual system, camera millimeter-level recognition errors and robot millimeter-level positioning errors can be removed, and the nail tightening work efficiency and precision are improved.

Description

Technical Field

[0001] The invention relates to the field of automation technology, and in particular to a tightening position system of visual correction equipment and a method thereof. Background Art

[0002] In various assembly processes, workers inevitably make mistakes from grabbing to hand-held screwing and tightening parts. Robotic tightening assembly technology can greatly reduce human involvement and even achieve completely unmanned operation, which has significant advantages. These advantages include improving production efficiency, reducing labor costs and avoiding human errors, so it has been widely used in modern manufacturing.

[0003] However, existing robotic tightening systems face some challenges in practical applications. They usually first establish the relative coordinate relationship between the robot and the product, then photograph the tightening position features, identify the position features and tighten after tightening, and repeat this step, resulting in low efficiency. At the same time, due to the existence of multiple variables in the actual operating environment, such as the position error of the workpiece, the distortion of the visual system, etc., this method of fixing coordinates often leads to inaccurate positioning problems in practice. Specifically, when the workpiece is not accurately placed in the predetermined position, the robot may deviate, resulting in the inability to correctly complete the tightening operation. In addition, the distortion of the visual system, the height of the workpiece and changes in ambient light will also affect the positioning accuracy of the robot. Summary of the invention

[0004] The object to be tightened in the present invention is a machined product, and the repeatability of its positioning reference and the point to be tightened is good. A matching positioning workbench can be designed to ensure that the tightening point of the product only appears at the fixed point position. Figure 1 As shown, there are many types of products, and the points to be tightened for each product are distributed in a variety of ways. Therefore, the points to be tightened can be obtained at one time by shooting with a wide field of view camera, but the shooting accuracy of the wide field of view camera is low, and the absolute positioning accuracy of the robot is low, which makes it impossible for the robot to complete the tightening according to the recognized point coordinates.

[0005] The purpose of the present invention is to provide a method capable of automatic correction and precise positioning in view of the defects and shortcomings of the prior art. All fixed tightening points can be pre-taught to the robot program and the point sequence number can be compiled. The camera identifies several of the fixed tightening points that need to be tightened, automatically matches the point sequence number, and calls the corresponding teaching program of the robot. The advantage is that the millimeter-level recognition error of the camera and the millimeter-level positioning error of the robot can be eliminated, and the error is only the robot's repeated positioning error as low as 0.05mm.

[0006] To achieve the above purpose, the technical solution adopted by the present invention is: a visual correction equipment tightening position system, the innovative features of which include the following components: The first camera: In order to match the operating environment in the actual scene, it is designed to be placed obliquely above the workpiece, on the same side as the backlight source, and above the backlight source to determine the position of the tightening hole; The second camera is located on the other side of the workpiece, at the same level as the backlight source, and is used to capture the side image of the workpiece; Backlight: Located below the first camera and on the same horizontal line as the second camera, it is used to help obtain the image of the outer contour of the workpiece, reduce the impact of environmental factors, and provide appropriate brightness for determining the hole position.

[0007] A method for visually correcting the tightening position of equipment, the innovation of which includes the following steps: (1) Camera calibration: The effective parameters of the internal and external parameters are obtained through calibration, multiple images of the chessboard are taken, and the internal and external parameters of the camera are calculated through OpenCV; (2) Image acquisition: Acquire images of the tightening hole position and the workpiece by using a first camera and a second camera respectively; (3) Perspective correction: In image processing, the perspective distortion caused by the shooting angle is corrected by using the perspective transformation algorithm and developed using OpenCv; (4) Template matching: Template matching is to find the area in the image that best matches the pre-set template image; a matching method based on the Normalized Cross Correlation (NCC); (5) Coordinate error matching: Calculate the difference between the actual coordinates and the theoretical coordinates, use the Euclidean distance to determine the two points with the smallest error, and match the hole numbers; (6) Workpiece height measurement: Calculate the maximum height of the workpiece and send it to the tightening equipment to avoid collision with the workpiece when tightening the nails.

[0008] Furthermore, the perspective transformation algorithm in step 3 is used in image processing to correct image distortion caused by shooting angle; In image processing, perspective transformation is an important technique used to correct perspective distortion caused by shooting angles. In particular, when shooting a plane with multiple holes from above, the holes near the plane appear larger than the holes far away, resulting in image distortion. In order to match the hole positions, a perspective transformation algorithm is used for correction, and OpenCV is used for development.

[0009] The basic principle is to use four corresponding points (four points in the source image and four points in the target image) to calculate a 3×3 perspective transformation matrix H, and then apply the matrix to transform the image.

[0010] Suppose the four points in the source image are (x1, y1), (x2, y2), (x3, y3), (x4, y4), and the four corresponding points in the target image are (x1′, y1′), (x2′, y2′), (x3′, y3′), (x4′, y4′). The perspective transformation matrix H can be expressed as:

[0011] The formula for calculating the perspective transformation is:

[0012] Where (x,y) is a point in the source image, (x′,y′) is a point in the target image, is the normalization factor. Through normalization, we can get the coordinates of the target point. In order to further improve the accuracy of perspective correction, an error correction factor ϵ is introduced to take into account the small errors that may exist in the actual shooting process. The corrected perspective transformation formula can be expressed as:

[0013] Among them, ϵ is a correction factor obtained by fitting experimental data, which can further reduce the small errors in perspective transformation and improve the accuracy of the corrected image. In this way, image perspective correction can be performed more accurately, which is suitable for industrial inspection and image analysis tasks with high precision requirements.

[0014] The implementation steps are as follows Read the image; get four points: get four points on the target area image, these four points are the four vertices of the area that needs to be corrected; calculate the perspective transformation matrix: use the cv2.getPerspectiveTransform function to calculate the perspective transformation matrix; apply perspective transformation: use the cv2.warpPerspective function to apply the perspective transformation matrix to transform the source image into the corrected image.

[0015] Furthermore, the calculation formula of the matching method in step 4 is:

[0016] Among them, R(x, y) represents the degree of match between the template image T at the position (x, y) and the image I. The larger the value, the higher the match. w and h are the width and height of the template image respectively.

[0017] Furthermore, the Euclidean distance calculation formula in step 5 is as follows: .

[0018] Furthermore, the template matching algorithm can be replaced by the YOLOv5 algorithm. The network structure of the YOLOv5 algorithm mainly includes three parts: Backbone, Neck and Head: (1) Backbone: A convolutional neural network that aggregates and forms image features at different image granularities. YOLOv5 uses CSPDarknet53 as its backbone, which improves the model’s reasoning speed by improving the network structure while maintaining high accuracy. (2) Neck: A series of network layers that mix and combine image features and pass them to the prediction layer. YOLOv5 uses PANet (Path Aggregation Network) as the Neck, which achieves the fusion and utilization of multi-scale features through top-down path enhancement and bottom-up feature fusion. (3) Head: Predict image features, generate bounding boxes and predict categories. YOLOv5 adopts the Anchor-Free method of YOLOv3, calculates the loss between the predicted box and the GT box, optimizes the network parameters, and achieves accurate detection of the target.

[0019] Furthermore, the YOLOv5 algorithm uses Mosaic data enhancement at the input end. It generates a new training sample by randomly scaling, cropping, arranging and splicing four pictures, so that the model can learn more target features during the training process.

[0020] Furthermore, in the YOLOv5 algorithm, the algorithm embeds the adaptive anchor box calculation into the code, so that the model can automatically adjust the size and proportion of the anchor box during training to adapt to the characteristics of different data sets.

[0021] The beneficial effects of the present invention are: 1. High precision: (1) Perspective correction: The perspective correction algorithm solves the image distortion problem caused by the camera position and angle, ensuring high accuracy in hole position identification. A correction factor is introduced to improve the accuracy of the correction process, ensuring that images taken at different angles and positions can be processed in the same coordinate system.

[0022] (2) Template matching: The accuracy of hole position recognition is improved through the template matching algorithm based on the correlation coefficient. The template matching algorithm can accurately find the target hole position based on the template in a complex background, reduce the occurrence of misjudgment, and improve the accuracy of subsequent hole position matching.

[0023] (3) Point matching: All fixed tightening points can be pre-taught to the robot program and the point numbers can be compiled. The camera identifies the points that need to be tightened among all the fixed tightening points, automatically matches the point numbers, and calls the corresponding teaching program of the robot. The advantage is that the millimeter-level recognition error of the camera and the millimeter-level positioning error of the robot can be eliminated, and the error is only as low as 0.05mm of the robot's repeated positioning error.

[0024] 2. Security: Workpiece height measurement: By installing a camera on the side of the workpiece, the maximum height of the workpiece can be monitored in real time to avoid collision between the equipment and the workpiece.

[0025] 3. Efficiency: (1) Automated processing: The entire system realizes fully automated processing, from image acquisition, perspective correction, template matching to the final tightening position correction, all are completed automatically, improving work efficiency.

[0026] (2) Real-time: The use of efficient image processing algorithms ensures that the system can perform hole position correction and workpiece height measurement at the tightening position in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 To provide a matching positioning workbench; Figure 2 It is the principle flow chart of the present invention; Figure 3 It is a system composition diagram of the present invention; Figure 4 Schematic diagram of the network structure of the YOLOv5 algorithm in the present invention; Figure 5 This is a schematic diagram of the backlight source acquiring a workpiece image in the present invention.

[0028] Description of reference numerals: 1. First camera, 2. Second camera, 3. Backlight, 4. Workpiece. DETAILED DESCRIPTION

[0029] The present invention will be further described below in conjunction with the accompanying drawings.

[0030] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be understood that the specific implementation methods described herein are only used to explain the present invention and are not used to limit the present invention.

[0031] Example 1 See also Figure 2 , a principle flow chart of a tightening position system for visual correction equipment; See also Figure 3, a visual correction device tightening position system, comprising the following components: The first camera 1 is designed to be arranged diagonally above the workpiece 4, on the same side as the backlight source, and above the backlight source, and is used to determine the position of the tightening holes. In order to match the operating environment in the actual scene, the first camera 1 is designed to be arranged diagonally above the workpiece, and on the same side as the backlight source to reduce the impact of the backlight on recognition. The camera 1 arranged diagonally above will cause the captured image to be distorted, the holes will appear larger when they are closer and smaller when they are farther away, affecting recognition, and the image needs to be adjusted according to the algorithm.

[0032] The second camera 2 is designed to be located on the other side of the workpiece, at the same horizontal line as the backlight source, and is used to capture the side image of the workpiece. Through the backlight source, the aperture of the second camera 2 is adjusted so that the workpiece appears as a black area in the picture, which can better focus on the workpiece to be measured and reduce the influence of other factors.

[0033] Backlight source 3: located below the first camera and on the same horizontal line as the second camera, it is used to obtain the image of the outer contour of the workpiece to reduce the impact of environmental factors and provide appropriate brightness for hole position matching.

[0034] Example 2 A method for visually correcting the tightening position of equipment, comprising the following steps: (1) Camera calibration: The effective parameters of the internal and external parameters are obtained through calibration, multiple images of the chessboard are taken, and the internal and external parameters of the camera are calculated through OpenCV; The main steps of camera calibration are as follows: Use a camera to photograph the calibration plate from different angles and positions.

[0035] Acquire images at multiple angles and distances to ensure adequate viewing angle coverage and that the calibration plate is clearly visible in every image.

[0036] Extract Corner Points For each captured calibration board image, identify and extract the corners or feature points on the calibration board in the image by using cv2.findChessboardCorners in OpenCV.

[0037] Calculate internal parameters and distortion parameters Use the cv2.calibrateCamera function in OpenCV to calculate the camera's intrinsic parameters and distortion parameters. The main formula is as follows:

[0038] mtx is the camera internal parameter matrix, where f x and f y is the focal length, C x and Cy are the principal point coordinates.

[0039] dist is the distortion coefficient, which describes the distortion characteristics of the lens and contains five parameters: dist=[k1,k2,p1,p2,k3] Where k1, k2, k3 are radial distortion coefficients, and p1, p2 are tangential distortion coefficients. The parameters are used by the camera to perform distortion correction on the captured image. The distortion correction formula is:

[0040] (2) Image acquisition: Acquire images of the tightening hole position and the workpiece by using a first camera and a second camera respectively; (3) Perspective correction: In image processing, perspective transformation is an important technique used to correct perspective distortion caused by shooting angles. In particular, when shooting a plane with multiple holes from above, the holes near the plane appear larger than the holes far away, resulting in image distortion. In order to match the hole positions, a perspective transformation algorithm is used for correction, and OpenCV is used for development.

[0041] The basic principle is to use four corresponding points (four points in the source image and four points in the target image) to calculate a 3×3 perspective transformation matrix H, and then apply the matrix to transform the image.

[0042] Suppose the four points in the source image are (x1, y1), (x2, y2), (x3, y3), (x4, y4), and the four corresponding points in the target image are (x1′, y1′), (x2′, y2′), (x3′, y3′), (x4′, y4′). The perspective transformation matrix H can be expressed as:

[0043] The formula for calculating the perspective transformation is:

[0044] Where (x,y) is a point in the source image, (x′,y′) is a point in the target image, is the normalization factor. Through normalization, we can get the coordinates of the target point. In order to further improve the accuracy of perspective correction, an error correction factor ϵ is introduced to take into account the small errors that may exist in the actual shooting process. The corrected perspective transformation formula can be expressed as:

[0045] Among them, ϵ is a correction factor obtained by fitting experimental data, which can further reduce the small errors in perspective transformation and improve the accuracy of the corrected image. In this way, image perspective correction can be performed more accurately, which is suitable for industrial inspection and image analysis tasks with high precision requirements.

[0046] The implementation steps are as follows S1: read image; S2: Get four points: Get four points on the target area image, which are the four vertices of the area to be corrected; S3: Calculate the perspective transformation matrix: Use the cv2.getPerspectiveTransform function to calculate the perspective transformation matrix; S4: Apply perspective transformation: Use cv2.warpPerspective function to apply the perspective transformation matrix to transform the source image into the rectified image.

[0047] (4) Template matching: In the present invention, by combining hole position matching with template matching, the template matching method is used to find the position of the hole in the image, and the hole position features are further used for precise positioning and detection.

[0048] Template matching calculates the matching degree between the template image and the target image at different locations by using a matching method based on the Normalized Cross Correlation (NCC). The formula is as follows:

[0049] in: R(x, y) represents the degree of match between the template image T and the target image I at position (x, y).

[0050] T(i, j) represents the pixel value of the template image at position (i, j).

[0051] I(x+i, y+j) represents the pixel value of the target image at position (x+i, y+j).

[0052] Represents the average pixel value of the template image.

[0053] Represents the average pixel value of the target image in the area covered by the template.

[0054] w and h are the width and height of the template image respectively.

[0055] When matching holes, it is necessary to extract the features of the holes. The Canny edge detection algorithm is used to extract the edges of the holes: edges=Canny(I,threshold1,threshold2) After extracting the edge of the hole, you can use the contour detection algorithm to find the contour of the hole: contours=findContours(edges,mode,method) When combining hole matching with template matching, follow these steps: S1: Template matching: First, use the template matching method to find the rough matching position of the template image in the target image. The position with the highest matching degree is the candidate position of the template image.

[0056] S2: Hole feature extraction: The features of the hole are extracted around the matched candidate positions.

[0057] S3: Feature matching: By comparing the hole position features in the template image with the hole position features in the target image, the hole position is further accurately located and the matching degree of the hole position features is calculated. The specific calculation method is as follows: 1. Template matching: Use the NCC method to calculate the matching matrix R(x, y) and find the position with the highest matching degree (x0, y0) 2. Hole feature extraction: In the target image I, with (x0, y0) as the center, extract the edge and contour of the hole.

[0058] 3. Exact match: Compare the hole features in the template image T with the hole position features extracted from the target image I, calculate the matching degree, and accurately locate the hole position. The specific calculation steps are as follows: Calculate the average pixel value of the template image :

[0059] Select the location (x, y) and its corresponding coverage area in the target image: The pixel value of the coverage area I(x+i,y+j) Average pixel value of the covered area :

[0060] Calculate the numerator part:

[0061] Calculate the denominator:

[0062] Calculate the matching degree R(x,y):

[0063] (5) Coordinate error matching: Calculate the difference between the actual coordinates and the theoretical coordinates, use the Euclidean distance to determine the two points with the smallest error, and match the hole numbers.

[0064] In the process of robot tightening assembly, in order to accurately match the points that need to be tightened and call the corresponding teaching program, the difference between the actual coordinates and the theoretical coordinates is calculated, and the Euclidean distance is used to determine the two points with the smallest error.

[0065] S1: Pre-teach fixed tightening points and compile point numbers First, all fixed tightening points need to be entered into the robot system in advance through teaching. Each tightening point has a unique number (point sequence number) and its theoretical coordinate position is recorded. This step ensures that the robot can call the corresponding teaching program when performing the tightening task.

[0066] S2: Identify the actual tightening point At the beginning of the assembly operation, the camera scans the workpiece and identifies the points that need to be tightened among all the fixed tightening points. The camera captures the actual coordinate positions of these points.

[0067] S3: Calculate coordinate difference In order to ensure the accuracy of the tightening operation, the difference between the identified actual coordinates and the pre-taught theoretical coordinates is calculated. Set the actual coordinates to (x1, y1, z1) and the theoretical coordinates to (x2, y2, z3), and calculate the difference using the Euclidean distance formula:

[0068] in, Represents the Euclidean distance between the first actual point and the second theoretical point.

[0069] S4: The point with the smallest matching error By calculating the Euclidean distance between all actual points and theoretical points, find the two points with the smallest error and match them. The specific steps are as follows: For each actual tightening point 1, calculate the Euclidean distance between it and all theoretical points 2 .

[0070] Select the theoretical point 1 with the smallest distance, that is, find the smallest .

[0071] Match the actual point 1 with the corresponding theoretical point 2.

[0072] S5: Automatically match the point number and call the teaching program Once the matching is completed, the system will automatically determine the theoretical point number corresponding to the actual tightening point. Using this point number, the system can call the pre-taught robot program to ensure that the robot performs the tightening operation accurately.

[0073] (6) Workpiece height measurement, the image diagram obtained is shown in Figure 5 , Figure 5 It represents the image of the workpiece under the backlight, where the black area represents the workpiece. Figure 5 , the light source direction is from inside to outside, and the impact of subsequent algorithm processing on the image is reduced by lighting first, thereby improving the accuracy of detection: S1: Image reading and preprocessing The image is read using OpenCV and preprocessed to reduce noise. The preprocessing includes converting the image to grayscale and applying Gaussian blur.

[0074] 1. Grayscale conversion:

[0075] Among them I R ,I G ,I B are the pixel values ​​of the red, green, and blue channels of the image, respectively.

[0076] Gaussian Blur:

[0077] in is the standard deviation of the Gaussian kernel.

[0078] S2: Edge Detection The gradient of the image on the x-axis and y-axis is calculated, and then the Canny edge detection algorithm is used to extract the edges in the image.

[0079] 1. Calculate image gradient:

[0080] Gradient magnitude and direction:

[0081] 2.Canny edge detection:

[0082] S3: Straight line detection The region of interest (ROI) is extracted from the image, grayscale conversion and edge detection are performed on the ROI area, straight lines in the image are detected using Hough transform, and the detected straight lines are fitted.

[0083] 1. Hough Transform:

[0084] in, Represents the distance from the straight line to the origin of the coordinate system. Represents the angle between the line and the x-axis.

[0085] 2. Fitting straight line: Assuming the detected line segment endpoints are (x1, y1) and (x2, y2), the equation of the line is:

[0086] Among them, the slope m and intercept b are:

[0087] S4: Calculate vertical distance Fit the straight line and calculate the vertical distance between the top and bottom of the workpiece to obtain the height of the workpiece. By detecting the relative distance between the two straight lines, the highest point position is obtained and sent to the device to avoid collision.

[0088] 1. Equations of the top and bottom lines: Top straight line:

[0089] Bottom line:

[0090] 2. Calculate the vertical distance: Assume that at a certain x coordinate, the y coordinates of the top and bottom edges are y top and bottom , then the height H of the workpiece is:

[0091] 3. Highest point position detection: By calculating the highest point position of the workpiece over the entire length, detecting the maximum vertical distance between the two straight lines, the highest point position is obtained:

[0092] S5: Combined with workpiece height measurement, hole position matching can be performed more accurately: 1. Calculated workpiece height H maxIt helps to determine the position height of the workpiece, thereby improving the accuracy of hole matching.

[0093] 2. Position adjustment in hole matching: According to the detected workpiece height, the search range in the template matching algorithm is adjusted to ensure that the height change of the workpiece is taken into account during the matching process, and to avoid collision between the device and the workpiece:

[0094] in, Is a function that adjusts the matching result according to the workpiece height.

[0095] In this embodiment, the template matching algorithm can be replaced by the YOLOv5 algorithm. The YOLOv5 algorithm is a target detection algorithm based on deep learning. Its core idea is to regard target detection as a regression problem and directly predict the position and category of the target through a convolutional neural network. Compared with traditional target detection algorithms, YOLOv5 has higher detection speed and accuracy, and is particularly suitable for real-time target detection tasks; the network structure of the YOLOv5 algorithm mainly includes three parts: Backbone, Neck and Head: (1) Backbone: A convolutional neural network that aggregates and forms image features at different image granularities. YOLOv5 uses CSPDarknet53 as its backbone, which improves the model’s reasoning speed by improving the network structure while maintaining high accuracy. (2) Neck: A series of network layers that mix and combine image features and pass them to the prediction layer. YOLOv5 uses PANet (Path Aggregation Network) as the Neck, which achieves the fusion and utilization of multi-scale features through top-down path enhancement and bottom-up feature fusion. (3) Head: Predict image features, generate bounding boxes and predict categories. YOLOv5 adopts the Anchor-Free method of YOLOv3, calculates the loss between the predicted box and the GT box, optimizes the network parameters, and achieves accurate detection of the target.

[0096] In this embodiment, the YOLOv5 algorithm uses Mosaic data enhancement at the input end to generate a new training sample by randomly scaling, cropping, arranging and splicing four pictures, so that the model can learn more target features during the training process.

[0097] In this embodiment, in the YOLOv5 algorithm, the algorithm embeds adaptive anchor box calculation into the code, so that the model can automatically adjust the size and proportion of the anchor box during the training process to adapt to the characteristics of different data sets.

[0098] The YOLOv5 algorithm obtains a data set by annotating the holes, divides it into a training set, a validation set, and a test set, puts it into the YOLOv5 network for training, and finally deploys it to the workstation, connects the camera for real-time target recognition, and selects the hole positions for positioning.

[0099] The core of the present invention lies in the image processing technology and its specific method in the visual correction robot tightening position method; including but not limited to perspective correction algorithm, template matching algorithm, camera calibration method, coordinate error matching algorithm and workpiece height measurement technology.

[0100] In summary, the present invention has the following advantages: (1) Perspective correction: The perspective correction algorithm is a key technology to ensure that the image accurately reflects the actual scene and can solve the image distortion problem caused by the camera position and angle.

[0101] (2) Template matching: The template matching algorithm can accurately find the target hole position in a complex background, thereby improving the accuracy of hole position recognition.

[0102] (3) Camera calibration: By obtaining the internal and external parameters of the camera, we ensure that the captured images can truly reflect the positional relationships in the actual scene.

[0103] (4) Coordinate error matching: The coordinate error matching algorithm automatically corrects the error in the image by calculating the difference between the actual coordinates and the theoretical coordinates. The camera identifies several of the fixed tightening points that need to be tightened, automatically matches the point numbers, and calls the corresponding teaching program of the robot. This is the core technical point of the present invention.

[0104] (5) Workpiece height measurement: Workpiece height measurement is a key technology to ensure a safe distance between the equipment and the workpiece. It avoids collision between the equipment and the workpiece by monitoring the maximum height of the workpiece in real time.

[0105] The above description is only used to illustrate the technical solution of the present invention rather than to limit it. Other modifications or equivalent substitutions made to the technical solution of the present invention by ordinary technicians in this field should be included in the scope of the claims of the present invention as long as they do not depart from the spirit and scope of the technical solution of the present invention.

Claims

1. A visual correction equipment tightening position system, characterized in that: Includes the following components: The first camera: In order to match the operating environment in the actual scene, it is designed to be placed obliquely above the workpiece, on the same side as the backlight source, and above the backlight source to determine the position of the tightening hole; The second camera is located on the other side of the workpiece, at the same level as the backlight source, and is used to capture the side image of the workpiece; Backlight: Located below the first camera and on the same horizontal line as the second camera, it is used to help obtain the outer contour image of the workpiece, reduce the impact of environmental factors, and provide appropriate brightness for determining the hole position.

2. A method for visually correcting the tightening position of equipment, characterized in that: The following steps are involved: (1) Camera calibration: The effective parameters of the internal and external parameters are obtained through calibration, multiple images of the chessboard are taken, and the internal and external parameters of the camera are calculated through OpenCV; (2) Image acquisition: Acquire images of the tightening hole area and the workpiece by using a first camera and a second camera respectively; (3) Perspective correction: In image processing, the perspective distortion caused by the shooting angle is corrected by using the perspective transformation algorithm and developed using OpenCv; (4) Template matching: Template matching is to find the area in the image that best matches the pre-set template image; a matching method based on the Normalized Cross Correlation (NCC); (5) Coordinate error matching: Calculate the difference between the actual coordinates and the theoretical coordinates, use the Euclidean distance to determine the two points with the smallest error, and match the serial numbers of the hole points; (6) Workpiece height measurement: Calculate the maximum height of the workpiece and send it to the tightening equipment to avoid collision with the workpiece when tightening the nails.

3. The method for tightening the position of the visual correction device according to claim 2, characterized in that: The perspective transformation algorithm in step 3 is used in image processing to correct image distortion caused by shooting angle; The perspective transformation algorithm is represented by a 3x3 matrix M, which transforms a point (x, y) in the original image into a point (x', y') in the new image. The specific formula is as follows: Where M is the 3x3 perspective transformation matrix: The transformed coordinates (x′, y′) are calculated using the following formula: Implementation steps Read and display the image: First read the image to be corrected and display it in the window; Click to get four points: Click four points on the image, which are the four vertices of the area to be corrected; Calculate the perspective transformation matrix: Use the cv2.getPerspectiveTransform function to calculate the perspective transformation matrix; Apply perspective transform: Use cv2.warpPerspective function to apply the perspective transformation matrix to transform the source image into the rectified image.

4. The method for tightening the position of the visual correction device according to claim 2, characterized in that: The calculation formula of the matching method in step 4 is: Among them, R(x, y) represents the degree of match between the template image T at the position (x, y) and the image I. The larger the value, the higher the match. w and h are the width and height of the template image respectively.

5. The method for tightening the position of the visual correction device according to claim 2, characterized in that: The Euclidean distance calculation formula in step 5 is as follows: 。 6. The method for tightening the position of the visual correction equipment according to claim 2 or 3, characterized in that: The template matching algorithm can be replaced by the YOLOv5 algorithm. The network structure of the YOLOv5 algorithm mainly includes three parts: Backbone, Neck and Head: (1) Backbone: A convolutional neural network that aggregates and forms image features at different image granularities. YOLOv5 uses CSPDarknet53 as its backbone, which improves the model’s reasoning speed by improving the network structure while maintaining high accuracy. (2) Neck: A series of network layers that mix and combine image features and pass them to the prediction layer. YOLOv5 uses PANet (Path Aggregation Network) as the Neck, which achieves the fusion and utilization of multi-scale features through top-down path enhancement and bottom-up feature fusion. (3) Head: Predict image features, generate bounding boxes and predict categories. YOLOv5 adopts the Anchor-Free method of YOLOv3, calculates the loss between the predicted box and the GT box, optimizes the network parameters, and achieves accurate detection of the target.

7. The method for tightening the position of the vision correction device according to claim 6, characterized in that: The YOLOv5 algorithm uses Mosaic data enhancement at the input end. It generates a new training sample by randomly scaling, cropping, arranging and splicing four pictures, so that the model can learn more target features during the training process.

8. The method for tightening the position of the vision correction device according to claim 6, characterized in that: In the YOLOv5 algorithm, the algorithm embeds the adaptive anchor box calculation into the code, so that the model can automatically adjust the size and proportion of the anchor box during training to adapt to the characteristics of different data sets.

Images