Vision correction device tightening position system and method

Abstract

The application discloses a visual correction equipment tightening position system and a method thereof, which comprises the following components: a first camera, which is arranged obliquely above a workpiece in order to match an operation environment in an actual scene, is located on the same side with a backlight source and is above the backlight source, and is used for determining a tightening hole position; a second camera, which is located on the other side of the workpiece and is on the same horizontal line with the backlight source, is used for shooting the highest height of the workpiece; the backlight source is located below the first camera and is on the same horizontal line with the second camera, is used for obtaining a better workpiece outer contour image, reducing the influence of environmental factors, and can provide suitable brightness for determining the hole position; and a system cooperation method, which combines perspective correction, template matching, camera calibration and other technologies, automatically identifies and corrects the tightening position through a visual system, can remove millimeter-level identification errors of the camera and millimeter-level positioning errors of a robot, and improves the work efficiency and precision of the screw tightening.

Description

Technical Field

[0001] This invention relates to the field of automation technology, and more specifically to a tightening position system and method for a vision correction device. Background Technology

[0002] In various assembly processes, workers inevitably make mistakes when grasping, screwing in, and tightening parts. Robotic tightening and assembly technology, however, can significantly reduce human intervention, even achieving fully automated operation, offering substantial advantages. These advantages include increased production efficiency, reduced labor costs, and avoidance of human error, leading to its widespread application in modern manufacturing.

[0003] However, existing robotic tightening systems face several challenges in practical applications. They typically involve first establishing the relative coordinates between the robot and the product, then capturing images of the tightening position features, identifying these features, and finally tightening the product – a repetitive cycle that leads to inefficiency. Furthermore, the fixed coordinate method often results in inaccurate positioning due to various variables in the actual operating environment, such as workpiece position errors and vision system distortion. Specifically, when the workpiece is not accurately placed in the predetermined position, the robot may deviate, preventing it from completing the tightening operation correctly. In addition, vision system distortion, workpiece height, and changes in ambient light can also affect the robot's positioning accuracy. Summary of the Invention

[0004] The object to be tightened in this invention is a machined product with good repeatability of its positioning reference and tightening point. A matching positioning worktable can be designed to ensure that the tightening point of the product only appears at a fixed position, as shown in the attached figure. Figure 1 As shown, there are many types of products, and the tightening points for each product are distributed in various ways. Therefore, a wide field-of-view camera can capture the tightening points in one shot. However, the wide field-of-view camera has low shooting accuracy, and the robot's absolute positioning accuracy is also low, making it impossible for the robot to complete the tightening according to the identified point coordinates.

[0005] The purpose of this invention is to address the shortcomings and deficiencies of existing technologies by providing a method for automatic correction and precise positioning. All fixed tightening points can be pre-programmed into the robot and assigned point numbers. The camera identifies several points requiring tightening, automatically matches the point numbers, and invokes the corresponding teaching program for the robot. The advantage lies in eliminating millimeter-level recognition errors of the camera and millimeter-level positioning errors of the robot, with the error reduced to a mere 0.05mm repeatability error.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is: a tightening position system for a vision correction device, the innovation of which includes the following components:

[0007] First camera: In order to match the actual operating environment, it is designed to be placed diagonally above the workpiece, on the same side as the backlight, and above the backlight, to determine the position of the tightening hole.

[0008] Second camera: Located on the other side of the workpiece, at the same horizontal level as the backlight, used to capture images of the side of the workpiece;

[0009] Backlight: Located below the first camera and at the same horizontal level as the second camera, it helps to acquire images of the workpiece's outer contour, reduces the influence of environmental factors, and provides appropriate brightness for determining hole positions.

[0010] An innovative method for tightening the position of a vision correction device includes the following steps:

[0011] (1) Camera calibration:

[0012] The effective parameters of the intrinsic and extrinsic parameters are obtained through calibration. Multiple images of the checkerboard pattern are taken, and the intrinsic and extrinsic parameters of the camera are calculated using OpenCV.

[0013] (2) Image acquisition:

[0014] Images of the tightening hole location and the workpiece are acquired using a first camera and a second camera, respectively.

[0015] (3) Perspective correction:

[0016] In image processing, perspective distortion caused by shooting angle is corrected by using a perspective transformation algorithm and developed using OpenCV.

[0017] (4) Template matching:

[0018] Template matching is a matching method that finds the region in an image that best matches a pre-defined template image; it is based on the correlation coefficient (Normalized Cross Correlation, NCC).

[0019] (5) Coordinate error matching:

[0020] Calculate the difference between the actual coordinates and the theoretical coordinates, use Euclidean distance to determine the two points with the smallest error, and match the hole sequence number;

[0021] (6) Workpiece height measurement:

[0022] Calculate the maximum height of the workpiece and send it to the tightening equipment to avoid collision with the workpiece during screw tightening.

[0023] Furthermore, the perspective transformation algorithm in step 3 is used in image processing to correct image distortion caused by the shooting angle;

[0024] In image processing, perspective transformation is an important technique used to correct perspective distortion caused by shooting angle. This is especially true when shooting a plane with multiple holes from an oblique angle above, where holes closer to the viewer appear larger than those further away, leading to image distortion. To achieve hole alignment, a perspective transformation algorithm is used for correction, developed using OpenCV.

[0025] The basic principle is to use four corresponding points (four points in the source image and four points in the target image) to calculate a 3×3 perspective transformation matrix H, and then apply this matrix to transform the image.

[0026] Let the four points in the source image be (x1, y1), (x2, y2), (x3, y3), and (x4, y4), and the four corresponding points in the target image be (x1′, y1′), (x2′, y2′), (x3′, y3′), and (x4′, y4′). The perspective transformation matrix H can be represented as:

[0027]

[0028] The formula for calculating perspective transformation is:

[0029]

[0030] Where (x, y) are points in the source image, and (x′, y′) are points in the target image. This is the normalization factor. Through normalization, we can obtain the coordinates of the target point. To further improve the accuracy of perspective correction, an error correction factor ϵ is introduced, taking into account the minute errors that may exist during actual shooting. The corrected perspective transformation formula can be expressed as:

[0031]

[0032] Here, ϵ is a correction factor obtained by fitting experimental data, which can further reduce small errors in perspective transformation and improve the accuracy of the corrected image. This method allows for more precise image perspective correction, suitable for high-precision industrial inspection and image analysis tasks.

[0033] The implementation steps are as follows:

[0034] Read the image; obtain four points: obtain four points on the target region image, which are the four vertices of the region to be corrected; calculate the perspective transformation matrix: use the cv2.getPerspectiveTransform function to calculate the perspective transformation matrix; apply the perspective transformation: use the cv2.warpPerspective function to apply the perspective transformation matrix, transforming the source image into the corrected image.

[0035] Furthermore, the calculation formula for the matching method in step 4 is as follows:

[0036]

[0037] Where R(x,y) represents the degree of matching between the template image T and the image I at position (x,y), and the larger the value, the higher the degree of matching. w and h are the width and height of the template image, respectively.

[0038] Furthermore, the Euclidean distance calculation formula in step 5 is as follows:

[0039] .

[0040] Furthermore, the template matching algorithm can be replaced by the YOLOv5 algorithm, whose network structure mainly includes three parts: Backbone, Neck, and Head.

[0041] (1) Backbone: A convolutional neural network that aggregates and forms image features at different fine-grained levels; YOLOv5 uses CSPDarknet53 as the backbone. While maintaining high accuracy, this network improves the inference speed of the model by improving the network structure.

[0042] (2) Neck: A series of network layers that mix and combine image features and pass the image features to the prediction layer; YOLOv5 uses PANet (Path Aggregation Network) as the Neck, and achieves the fusion and utilization of multi-scale features through top-down path enhancement and bottom-up feature fusion;

[0043] (3) Head: Predicts image features, generates bounding boxes and predicts categories; YOLOv5 adopts the Anchor-Free method of YOLOv3, optimizes network parameters by calculating the loss between the predicted box and the ground truth box, and achieves accurate detection of the target.

[0044] Furthermore, the YOLOv5 algorithm employs Mosaic data augmentation at the input end. By randomly scaling, cropping, arranging, and stitching four images, a new training sample is generated, enabling the model to learn more target features during training.

[0045] Furthermore, in the YOLOv5 algorithm, adaptive anchor box calculation is embedded into the code, enabling the model to automatically adjust the size and proportion of the anchor boxes during training to adapt to the characteristics of different datasets.

[0046] The beneficial effects of this invention are:

[0047] 1. High precision:

[0048] (1) Perspective Correction: The perspective correction algorithm solves the image distortion problem caused by camera position and angle, ensuring high accuracy in hole position recognition. A correction factor is introduced to improve the accuracy of the correction process and ensure that images taken at different angles and positions can be processed in the same coordinate system.

[0049] (2) Template matching: The accuracy of hole location identification is improved by using a template matching algorithm based on correlation coefficients. The template matching algorithm can accurately find the target hole location in complex backgrounds based on the template, reducing the occurrence of misjudgments and improving the accuracy of subsequent hole location matching.

[0050] (3) Point matching: All fixed tightening points can be pre-taught to the robot program and assigned point numbers. The camera identifies several points that need to be tightened among all the fixed tightening points, automatically matches the point numbers, and calls the corresponding teaching program of the robot. The advantage is that it can eliminate the millimeter-level recognition error of the camera and the millimeter-level positioning error of the robot. The error is only the robot's repeatability error as low as 0.05mm.

[0051] 2. Security:

[0052] Workpiece height measurement: By installing a camera on the side of the workpiece, the maximum height of the workpiece can be monitored in real time to avoid collisions between the equipment and the workpiece.

[0053] 3. High efficiency:

[0054] (1) Automated processing: The entire system achieves fully automated processing, from image acquisition, perspective correction, template matching to final tightening position correction, all of which are completed automatically, improving work efficiency.

[0055] (2) Real-time performance: The system adopts efficient image processing algorithms to ensure that the hole position correction of the tightening position and the workpiece height measurement can be performed in real time. Attached Figure Description

[0056] Figure 1 For use with a matching positioning worktable;

[0057] Figure 2 This is a flowchart illustrating the principle of the present invention;

[0058] Figure 3 This is a system composition diagram of the present invention;

[0059] Figure 4 This is a schematic diagram of the network structure of the YOLOv5 algorithm in this invention;

[0060] Figure 5This is a schematic diagram of acquiring workpiece images using a backlight in this invention.

[0061] Explanation of reference numerals in the attached figures:

[0062] 1. First camera, 2. Second camera, 3. Backlight, 4. Workpiece. Detailed Implementation

[0063] The invention will now be further described with reference to the accompanying drawings.

[0064] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0065] Example 1

[0066] See Figure 2 A schematic diagram of the tightening position system of a vision correction device;

[0067] See Figure 3 A tightening position system for a vision correction device includes the following components:

[0068] First camera 1: Designed and positioned diagonally above workpiece 4, on the same side as the backlight and above the backlight, to determine the position of the tightening hole; In order to match the actual operating environment, the first camera 1 is designed and positioned diagonally above the workpiece, and placed on the same side as the backlight to reduce the impact of backlight on recognition; The diagonally positioned camera 1 will cause some distortion in the captured image, making the hole appear larger when closer and smaller when farther away, which will affect recognition, and the image needs to be adjusted according to the algorithm.

[0069] Second camera 2: Designed and positioned on the other side of the workpiece, at the same horizontal level as the backlight, for capturing images of the workpiece's side. By adjusting the aperture of the second camera 2 through the backlight, the workpiece is displayed as a black area in the image, allowing for better focus on the workpiece to be measured and reducing the influence of other factors.

[0070] Backlight 3: Located below the first camera and on the same horizontal line as the second camera, it is used to acquire the outer contour image of the workpiece, reduce the influence of environmental factors, and provide appropriate brightness for hole matching.

[0071] Example 2

[0072] A method for tightening the position of a vision correction device includes the following steps:

[0073] (1) Camera calibration:

[0074] The effective parameters of the intrinsic and extrinsic parameters are obtained through calibration. Multiple images of the checkerboard pattern are taken, and the intrinsic and extrinsic parameters of the camera are calculated using OpenCV.

[0075] The main steps of camera calibration are as follows:

[0076] Use a camera to photograph the calibration board from different angles and positions.

[0077] Acquire images from multiple angles and distances to ensure sufficient field of view coverage and that the calibration board is clearly visible in each image.

[0078] Extracting corner points

[0079] For each captured image of the calibration board, corner points or feature points on the calibration board are identified and extracted from the image. This is done using cv2.findChessboardCorners in OpenCV.

[0080] Calculate intrinsic parameters and distortion parameters

[0081] The `cv2.calibrateCamera` function in OpenCV is used to calculate the camera's intrinsic and distortion parameters. The main formulas are as follows:

[0082]

[0083] mtx is the camera intrinsic parameter matrix, where f x and f y It's the focal length, C x and C y These are the coordinates of the principal point.

[0084] dist is the distortion coefficient, which describes the distortion characteristics of a lens and includes five parameters:

[0085] dist=[k1,k2,p1,p2,k3]

[0086] Where k1, k2, and k3 are radial distortion coefficients, and p1 and p2 are tangential distortion coefficients. These parameters are used by the camera to correct distortion in the captured image. The formula for distortion correction is:

[0087]

[0088] (2) Image acquisition:

[0089] Images of the tightening hole location and the workpiece are acquired using a first camera and a second camera, respectively.

[0090] (3) Perspective correction:

[0091] In image processing, perspective transformation is an important technique used to correct perspective distortion caused by shooting angle. This is especially true when shooting a plane with multiple holes from an oblique angle above, where holes closer to the viewer appear larger than those further away, leading to image distortion. To achieve hole alignment, a perspective transformation algorithm is used for correction, developed using OpenCV.

[0092] The basic principle is to use four corresponding points (four points in the source image and four points in the target image) to calculate a 3×3 perspective transformation matrix H, and then apply this matrix to transform the image.

[0093] Let the four points in the source image be (x1, y1), (x2, y2), (x3, y3), and (x4, y4), and the four corresponding points in the target image be (x1′, y1′), (x2′, y2′), (x3′, y3′), and (x4′, y4′). The perspective transformation matrix H can be represented as:

[0094]

[0095] The formula for calculating perspective transformation is:

[0096]

[0097] Where (x, y) are points in the source image, and (x′, y′) are points in the target image. This is the normalization factor. Through normalization, we can obtain the coordinates of the target point. To further improve the accuracy of perspective correction, an error correction factor ϵ is introduced, taking into account the minute errors that may exist during actual shooting. The corrected perspective transformation formula can be expressed as:

[0098]

[0099] Here, ϵ is a correction factor obtained by fitting experimental data, which can further reduce small errors in perspective transformation and improve the accuracy of the corrected image. This method allows for more precise image perspective correction, suitable for high-precision industrial inspection and image analysis tasks.

[0100] The implementation steps are as follows:

[0101] S1: Read the image;

[0102] S2: Obtain four points: Obtain four points on the target region image. These four points are the four vertices of the region that needs to be corrected.

[0103] S3: Calculate the perspective transformation matrix: Use the cv2.getPerspectiveTransform function to calculate the perspective transformation matrix;

[0104] S4: Apply perspective transformation: Use the cv2.warpPerspective function to apply the perspective transformation matrix to transform the source image into the corrected image.

[0105] (4) Template matching:

[0106] In this invention, by combining hole location matching and template matching, the location of the hole is found in the image using the template matching method, and the hole location features are further used for precise positioning and detection.

[0107] Template matching calculates the matching degree between the template image and the target image at different locations using a matching method based on the normalized cross correlation (NCC) coefficient. The formula is as follows:

[0108]

[0109] in:

[0110] R(x, y) represents the degree of matching between the template image T and the target image I at position (x, y).

[0111] T(i, j) represents the pixel value of the template image at position (i, j).

[0112] I(x+i, y+j) represents the pixel value of the target image at position (x+i, y+j).

[0113] This represents the average pixel value of the template image.

[0114] This represents the average pixel value of the target image within the template coverage area.

[0115] w and h are the width and height of the template image, respectively.

[0116] When performing hole location matching, it is necessary to extract the features of the holes. The Canny edge detection algorithm is used to extract the edges of the holes.

[0117] edges=Canny(I,threshold1,threshold2)

[0118] After extracting the edges of the hole, a contour detection algorithm can be used to find the contour of the hole:

[0119] contours=findContours(edges,mode,method)

[0120] When combining hole position matching and template matching, follow these steps:

[0121] S1: Template Matching

[0122] First, template matching is used to find a rough matching position in the target image that corresponds to the template image. The position with the highest matching degree is then considered the candidate position in the template image.

[0123] S2: Pore location feature extraction:

[0124] Features of the holes are extracted around the matched candidate locations.

[0125] S3: Feature matching:

[0126] By comparing the hole location features in the template image and the hole location features in the target image, the position of the hole is further accurately located, and the matching degree of the hole location features is calculated. The specific calculation method is as follows:

[0127] 1. Template matching:

[0128] The matching degree matrix R(x, y) is calculated using the NCC method, and the position with the highest matching degree (x0, y0) is found.

[0129] 2. Hole location feature extraction:

[0130] Extract the edges and contours of the hole centered at (x0, y0) in the target image I.

[0131] 3. Exact match:

[0132] The features of the holes in the template image T are compared with the hole location features extracted from the target image I. The matching degree is calculated, and the position of the holes is accurately located. The specific calculation steps are as follows:

[0133] Calculate the average pixel value of the template image :

[0134]

[0135] Select a location (x, y) and its corresponding coverage area in the target image:

[0136] The pixel values ​​of the covered area are I(x+i,y+j).

[0137] Average pixel value of the covered area :

[0138]

[0139] Calculate the numerator:

[0140]

[0141] Calculate the denominator:

[0142]

[0143] Calculate the matching degree R(x,y):

[0144]

[0145] (5) Coordinate error matching:

[0146] Calculate the difference between the actual coordinates and the theoretical coordinates, use Euclidean distance to determine the two points with the smallest error, and match the hole sequence number.

[0147] During the robot's tightening and assembly process, in order to accurately match the points that need to be tightened and call the corresponding teaching program, the difference between the actual coordinates and the theoretical coordinates is calculated, and the Euclidean distance is used to determine the two points with the smallest error.

[0148] S1: Pre-teach and fix the tightening points and assign point numbers.

[0149] First, all fixed tightening points need to be pre-programmed into the robot system via teaching. Each tightening point has a unique number (point sequence number) and its theoretical coordinates are recorded. This step ensures that the robot can call the corresponding teaching program when performing tightening tasks.

[0150] S2: Identify the actual tightening points

[0151] At the start of the assembly process, the camera scans the workpiece and identifies the points that need tightening among all the fixed tightening points. The camera then captures the actual coordinates of these points.

[0152] S3: Calculate coordinate difference

[0153] To ensure the accuracy of the tightening operation, the difference between the identified actual coordinates and the pre-taught theoretical coordinates is calculated. The actual coordinates are set as (x1, y1, z1) and the theoretical coordinates as (x2, y2, z3). The difference is calculated using the Euclidean distance formula:

[0154]

[0155] in, This represents the Euclidean distance between the first actual point and the second theoretical point.

[0156] S4: The point with the smallest matching error

[0157] By calculating the Euclidean distance between all actual points and theoretical points, the two points with the smallest error are found and matched. The specific steps are as follows:

[0158] For each actual tightening point 1, calculate its Euclidean distance to all theoretical points 2. .

[0159] Choose the theoretical point 1 with the smallest distance, that is, find the smallest... .

[0160] Match the actual point 1 with the corresponding theoretical point 2.

[0161] S5: Automatically match point numbers and call the teaching program.

[0162] Once the matching is complete, the system will automatically determine the theoretical position number corresponding to the actual tightening point. Using this position number, the system can call the pre-taught robot program to ensure that the robot accurately performs the tightening operation.

[0163] (6) Workpiece height measurement, see the schematic diagram of the obtained image. Figure 5 , Figure 5 This represents the image of the workpiece captured by the camera under backlighting, where the black areas represent the workpiece. Figure 5 The light source is directed from the inside out. This initial lighting method reduces the impact of subsequent algorithmic processing on the image, thus improving detection accuracy.

[0164] S1: Image Reading and Preprocessing

[0165] The image is read using OpenCV and preprocessed to reduce noise. Preprocessing includes converting the image to grayscale and applying Gaussian blur.

[0166] 1. Grayscale conversion:

[0167]

[0168] Where I R I G I B These are the pixel values ​​of the red, green, and blue channels of the image, respectively.

[0169] Gaussian blur:

[0170]

[0171] in It is the standard deviation of the Gaussian kernel.

[0172] S2: Edge Detection

[0173] Calculate the gradients of the image on the x and y axes, and then use the Canny edge detection algorithm to extract the edges in the image.

[0174] 1. Calculate the image gradient:

[0175]

[0176] Gradient magnitude and direction:

[0177]

[0178] 2. Canny Edge Detection:

[0179]

[0180] S3: Line Detection

[0181] Extract the region of interest (ROI) from the image, perform grayscale conversion and edge detection on the ROI, use Hough transform to detect straight lines in the image, and fit the detected straight lines.

[0182] 1. Hough Transform:

[0183]

[0184] in, This represents the distance from the line to the origin of the coordinate system. This represents the angle between the line and the x-axis.

[0185] 2. Fitting a straight line:

[0186] Assuming the detected endpoints of the line segment are (x1, y1) and (x2, y2), the equation of the line is:

[0187]

[0188] Wherein, the slope m and the intercept b are respectively:

[0189]

[0190] S4: Calculate vertical distance

[0191] The height of the workpiece is obtained by fitting a straight line and calculating the vertical distance between the top and bottom edges. The highest point position is then determined by detecting the relative distance between the two lines and sent to the device to prevent collisions.

[0192] 1. Equations of the top and bottom lines:

[0193] Top edge line:

[0194]

[0195] Base line:

[0196]

[0197] 2. Calculate the vertical distance:

[0198] Suppose that at a certain x-coordinate, the y-coordinates of the top and bottom edges are y and y' respectively. top and y bottom Then the height H of the workpiece is:

[0199]

[0200] 3. Highest point position detection:

[0201] The position of the highest point is obtained by calculating the position of the highest point of the workpiece along its entire length and detecting the maximum perpendicular distance between two straight lines.

[0202]

[0203] S5: Combining workpiece height measurement allows for more accurate hole matching.

[0204] 1. Calculated workpiece height H max It helps determine the position and height of the workpiece, thereby improving the accuracy of hole matching.

[0205] 2. Position adjustment during hole matching:

[0206] Based on the detected workpiece height, the search range in the template matching algorithm is adjusted to ensure that changes in workpiece height are taken into account during the matching process, while also avoiding collisions between the equipment and the workpiece.

[0207]

[0208] in, It is a function that adjusts the matching result based on the workpiece height.

[0209] In this embodiment, the template matching algorithm can be replaced by the YOLOv5 algorithm. YOLOv5 is a deep learning-based object detection algorithm. Its core idea is to treat object detection as a regression problem, directly predicting the object's location and category through a convolutional neural network. Compared to traditional object detection algorithms, YOLOv5 has higher detection speed and accuracy, making it particularly suitable for real-time object detection tasks. The network structure of the YOLOv5 algorithm mainly includes three parts: Backbone, Neck, and Head.

[0210] (1) Backbone: A convolutional neural network that aggregates and forms image features at different fine-grained levels; YOLOv5 uses CSPDarknet53 as the backbone. While maintaining high accuracy, this network improves the inference speed of the model by improving the network structure.

[0211] (2) Neck: A series of network layers that mix and combine image features and pass the image features to the prediction layer; YOLOv5 uses PANet (Path Aggregation Network) as the Neck, and achieves the fusion and utilization of multi-scale features through top-down path enhancement and bottom-up feature fusion;

[0212] (3) Head: Predicts image features, generates bounding boxes and predicts categories; YOLOv5 adopts the Anchor-Free method of YOLOv3, optimizes network parameters by calculating the loss between the predicted box and the ground truth box, and achieves accurate detection of the target.

[0213] In this embodiment, the YOLOv5 algorithm uses Mosaic data augmentation at the input end. By randomly scaling, cropping, arranging, and stitching four images, a new training sample is generated, enabling the model to learn more target features during training.

[0214] In this embodiment, the YOLOv5 algorithm embeds adaptive anchor box calculation into the code, enabling the model to automatically adjust the size and proportion of the anchor boxes during training to adapt to the characteristics of different datasets.

[0215] The YOLOv5 algorithm obtains a dataset by labeling holes, divides it into training, validation, and test sets, feeds it into the YOLOv5 network for training, and finally deploys it to the workstation, connects to a camera for real-time target recognition, and selects the hole locations for positioning.

[0216] The core of this invention lies in the image processing technology and specific methods in the visual correction robot tightening position method; including but not limited to perspective correction algorithm, template matching algorithm, camera calibration method, coordinate error matching algorithm, and workpiece height measurement technology.

[0217] In summary, the present invention has the following advantages:

[0218] (1) Perspective correction: Perspective correction algorithm is a key technology to ensure that the image accurately reflects the actual scene and can solve the image distortion problem caused by camera position and angle.

[0219] (2) Template matching: The template matching algorithm can accurately find the target hole in a complex background, which improves the accuracy of hole identification.

[0220] (3) Camera calibration: By obtaining the camera's internal and external parameters, ensure that the captured images can truly reflect the positional relationships in the actual scene.

[0221] (4) Coordinate error matching: The coordinate error matching algorithm automatically corrects the error in the image by calculating the difference between the actual coordinates and the theoretical coordinates. The camera identifies several points that need to be tightened among all the fixed tightening points, automatically matches the point number, and calls the corresponding teaching program of the robot. This is the core technology of the present invention.

[0222] (5) Workpiece height measurement: Workpiece height measurement is a key technology to ensure a safe distance between the equipment and the workpiece. By monitoring the maximum height of the workpiece in real time, collisions between the equipment and the workpiece can be avoided.

[0223] The above description is only used to illustrate the technical solution of the present invention and is not intended to limit it. Any other modifications or equivalent substitutions made by those skilled in the art to the technical solution of the present invention, as long as they do not depart from the spirit and scope of the technical solution of the present invention, should be covered within the scope of the claims of the present invention.

Claims

1. A tightening position system for a vision correction device, characterized in that, Includes the following components: First camera: In order to match the actual operating environment, it is designed to be placed diagonally above the workpiece, on the same side as the backlight, and above the backlight, to determine the position of the tightening hole. Second camera: Located on the other side of the workpiece, at the same horizontal level as the backlight, used to capture images of the side of the workpiece; Backlight: Located below the first camera and at the same horizontal level as the second camera, it helps to acquire images of the workpiece's outer contour, reduces the influence of environmental factors, and provides appropriate brightness for determining hole positions; It also includes a method for tightening the position of a vision correction device, comprising the following steps: (1) Camera calibration: Effective parameters of the internal and external parameters were obtained through calibration. Multiple images of the chessboard pattern were taken, and then... OpenCV calculates camera intrinsic and extrinsic parameters; (2) Image acquisition: Images of the tightening hole area and the workpiece are acquired using a first camera and a second camera, respectively. (3) Perspective correction: In image processing, perspective distortion caused by shooting angle is corrected by using a perspective transformation algorithm, which is developed using OpenCV. (4) Template matching: Template matching is a matching method that finds the region in an image that best matches a pre-defined template image; it is based on Normalized Cross Correlation (NCC). (5) Coordinate error matching: Calculate the difference between the actual coordinates and the theoretical coordinates, use Euclidean distance to determine the two points with the smallest error, and match the hole point numbers; (6) Workpiece height measurement: Calculate the maximum height of the workpiece and send it to the tightening equipment to avoid collision with the workpiece during screw tightening; The perspective transformation algorithm in step 3 is used in image processing to correct image distortion caused by the shooting angle. The perspective transformation algorithm uses a 3x3 matrix H to transform a point (x, y) in the original image into a point (x′, y′) in the new image. The specific formula is as follows: Where H is a 3x3 perspective transformation matrix: To further improve the accuracy of perspective correction, an error correction factor ∈ is introduced; The transformed coordinates (x′, y′) are calculated using the following formula: The steps for implementing step 3 are as follows: Read and display the image: First, read the image to be corrected and display it in the window; Click to get four points: Click on four points on the image; these four points are the four vertices of the area to be corrected. Calculate the perspective transformation matrix: Use the cv2.getPerspectiveTransform function to calculate the perspective transformation matrix; Applying perspective transformation: Use the cv2.warpPerspective function to apply the perspective transformation matrix to transform the source image into the corrected image; The calculation formula for the matching method in step 4 is as follows: Where R(x,y) represents the degree of matching between the template image T and the image I at position (x,y), and the larger the value, the higher the degree of matching. w and h are the width and height of the template image, respectively. The Euclidean distance calculation formula in step 5 is as follows:

2. The method for tightening the position of the vision correction device according to claim 1, characterized in that: The template matching algorithm can be replaced by the YOLOv5 algorithm, whose network structure mainly includes three parts: Backbone, Neck, and Head. (1) Backbone: A convolutional neural network that aggregates and forms image features at different fine-grained levels; YOLOv5 uses CSPDarknet53 as the backbone. This network improves the inference speed of the model by improving the network structure while maintaining high accuracy. (2) Neck: A series of network layers that mix and combine image features and pass the image features to the prediction layer; YOLOv5 uses PANet (Path Aggregation Network) as the Neck, and achieves the fusion and utilization of multi-scale features through top-down path enhancement and bottom-up feature fusion; (3) Head: Predicts image features, generates bounding boxes and predicts categories; YOLOv5 adopts the Anchor-Free method of YOLOv3, optimizes network parameters by calculating the loss between the predicted box and the ground truth box, and achieves accurate target detection.

3. The method for tightening the position of the vision correction device according to claim 2, characterized in that: The YOLOv5 algorithm uses Mosaic data augmentation on the input side. It generates a new training sample by randomly scaling, cropping, arranging, and stitching four images, enabling the model to learn more target features during training.

4. The method for tightening the position of the vision correction device according to claim 2, characterized in that: In the YOLOv5 algorithm, adaptive anchor box calculation is embedded in the code, enabling the model to automatically adjust the size and proportion of the anchor boxes during training to adapt to the characteristics of different datasets.

Images