Method for realizing bolt image distortion correction based on QR code bolt positioning

Abstract

The invention relates to the technical field of image processing, in particular to a QR code-based bolt positioning and bolt image distortion correction method, which is characterized in that a bolt is accurately positioned by using a QR code, and distortion correction is carried out through a bolt image associated with the QR code, so that the shielding problem of square gasket correction is avoided; the method not only can realize accurate positioning of the bolt, but also provides accurate and reliable basic data for bolt loosening damage identification based on the image processing technology, and solves the problem that in the prior art, when shadow shielding exists in the gasket area, the gray gradient of the shielding area is similar, and the reliability is high. The areas may be wrongly identified as a connected domain in the image processing process; therefore, accurate identification and extraction of a gasket area are hindered, the stability of bolt image distortion correction is obviously reduced, and the accuracy of a bolt loosening damage identification result is influenced.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for bolt positioning based on QR code and achieving bolt image distortion correction. Background Art

[0002] Bolted connections have been widely used in many fields such as civil engineering, aerospace, and mechanical manufacturing due to their advantages such as easy installation and disassembly, economy, and durability. However, bolted connections may be affected by external environmental factors, such as temperature changes, vibrations caused by earthquakes, and repeated loads caused by strong winds. These factors may cause bolts to loosen, which in turn may cause serious problems with structural stability and pose a potential threat to safety. Therefore, it is crucial to ensure the stability and reliability of bolted connections. With the rapid development of the computer field, bolt loosening damage identification technology based on image processing technology has become one of the main methods for bolt loosening damage monitoring. In the bolt loosening damage identification method based on image processing technology, the positioning of bolts in bolt connection nodes and the distortion correction stability of bolt images are directly related to the efficiency and reliability of damage identification results.

[0003] In the existing technology, most methods are aimed at bolt recognition, while there are relatively few studies on the specific positioning of bolts in three-dimensional space. At present, the perspective correction technology of bolt images mainly adopts the following methods: First, the four corner points of the bolt connection node plate and the homography matrix principle are used to correct the perspective distortion of the image; second, the center points of the four corner bolts in the bolt group and the homography matrix principle are used to correct the perspective distortion of the image; third, the four corner points of the square gasket of a single bolt in the single-row bolt connection flange node are combined with the homography matrix principle to correct the image distortion; In the above method, the correction technology based on the four corner points of the square gasket is very sensitive to the situation where the gasket edge line in the bolt image is obscured by shadows; when there is shadow obscuration in the gasket area, due to the similar grayscale gradients of the obscured area, these areas may be mistakenly identified as a connected domain during the image processing process; this not only hinders the accurate identification and extraction of the gasket area, but also significantly reduces the stability of the bolt image distortion correction, thereby affecting the accuracy of the bolt loosening damage identification results. Summary of the invention

[0004] The purpose of the present invention is to provide a method for bolt positioning based on QR code and realizing bolt image distortion correction, aiming to solve the technical problem in the prior art that when there is shadow occlusion in the gasket area, these areas may be mistakenly identified as a connected domain during image processing due to the similar grayscale gradient of the occluded area; this not only hinders the accurate identification and extraction of the gasket area, but also significantly reduces the stability of the bolt image distortion correction, thereby affecting the accuracy of the bolt loosening damage identification results.

[0005] To achieve the above purpose, the present invention adopts a method for bolt positioning based on QR code and realizing bolt image distortion correction. The following steps are involved: Step 1: At the bolt connection that needs to be inspected, stick a square QR code tightly on the screw of the corresponding bolt and enter the information into the QR code; Step 2: Use a camera to photograph the bolt connection to obtain a bolt image; Step 3: Use an adaptive filter to remove noise from the bolt image and convert it into a grayscale image. Based on the Otsu method, the global optimal binary segmentation threshold is obtained to perform image binarization. Step 4: Delete small areas, invert, and fill gaps to filter out the area that only contains the QR code; Step 5: Based on the processed grayscale image, the Canny and Hough methods are used to detect the edge line of the bolt and determine the coordinates of the four corner points; Step 6: Calculate the homography matrix based on the coordinates of the four corner points to perform image perspective correction.

[0006] Among them, in step 1, when pasting the QR code, it is necessary to ensure that the diagonal length of the QR code is smaller than the screw diameter.

[0007] Among them, the bolt image is denoised by an adaptive filter and converted into a grayscale image to simplify the image, reduce the amount of data, make the image better highlight features such as texture, contour and edge line, and make it easier to segment and identify the image.

[0008] Among them, the global optimal binary segmentation threshold is obtained based on the Otsu method to perform image binarization to remove the grayscale information in the image. The grayscale information in the image is removed, and only black and white are retained to make the image clearer, improve the efficiency and speed of image processing, and also reduce noise and interference, and highlight the shape and edge lines of the target.

[0009] Among them, in step five, the edge lines of the denoised bolt image are identified by the Canny edge detection operator, and the Hough line detection operator detects the straight lines in the edge lines identified by the Canny edge detection operator. These straight lines are the edge lines of the QR code.

[0010] Among them, in step 5, the threshold of the Canny edge detection operator is 0.75, the RhoResolution in the Hough transform is 0.7, and the ThetaResolution is 0.65.

[0011] Among them, in step 5, the specific method of determining the coordinates of the four corner points is: Since the captured image may be distorted after image processing, multiple line segments may appear in the four directions of up, down, left, and right, making it impossible to accurately extract the equation of the line; Therefore, it is necessary to extract the endpoint coordinates of the edge lines detected in each direction of the detected QR code; The extracted edge coordinate points are classified according to the top, bottom, left and right directions of the QR code; Since the endpoint coordinates have the same X-axis coordinates, the least squares method cannot be used for fitting. Therefore, it is necessary to determine whether the endpoints can be fitted into a straight line in each direction using the least squares method for the upper, lower, left, and right classified endpoints: If possible, use the straight line equation to find the intersection points of the four straight lines (upper, lower, left, and upper right) to determine the coordinates of the four corner points; If not, rotate the coordinates of the classified points 45 degrees counterclockwise, and then use the least squares method to fit them into four straight lines, calculate the intersection of the four straight lines, and finally rotate the intersection 45 degrees clockwise to determine the coordinates of the corner points.

[0012] The present invention discloses a method for bolt positioning based on QR code and realizing bolt image distortion correction. At the part of the bolt connection that needs to be detected, a square QR code is tightly pasted on the screw rod of the corresponding bolt, and information is entered into the QR code; the bolt connection is photographed by a camera to obtain a bolt image; the bolt image is denoised by an adaptive filter and converted into a grayscale image, and the image is binarized based on the global optimal binary segmentation threshold obtained by the Otsu method; small areas are deleted, reversed, and gaps are filled to screen out the area part containing only the QR code; according to the processed grayscale image, the Canny and Hough methods are used to detect the edge line of the bolt and determine the coordinates of the four corner points; the homography matrix is ​​calculated according to the coordinates of the four corner points to perform image perspective correction; The present invention uses QR codes to accurately locate bolts, and performs distortion correction through the bolt image associated with the QR code, thereby avoiding the occlusion problem caused by square gasket correction; not only can the accurate positioning of the bolts be achieved, but also accurate and reliable basic data is provided for bolt loosening damage identification based on image processing technology, and the problem in the prior art that when there is shadow occlusion in the gasket area, these areas may be mistakenly identified as a connected domain during image processing due to the similar grayscale gradient of the occluded area; this not only hinders the accurate identification and extraction of the gasket area, but also significantly reduces the stability of the bolt image distortion correction, thereby affecting the technical problem of the accuracy of the bolt loosening damage identification result. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0014] Figure 1 It is a flowchart of the steps of the method for locating bolts based on QR codes and correcting bolt image distortion of the present invention.

[0015] Figure 2 It is a top view of the QR code pasting position of the method for positioning bolts based on QR codes and correcting bolt image distortion of the present invention.

[0016] Figure 3 It is a side view of the QR code pasting position of the method for positioning bolts based on QR codes and correcting bolt image distortion of the present invention. DETAILED DESCRIPTION

[0017] Embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, but should not be construed as limiting the present invention.

[0018] See also Figure 1~Figure 3 ,in Figure 1 is a flowchart of the steps of the method for locating bolts based on QR codes and correcting bolt image distortion of the present invention, Figure 2 It is a top view of the QR code pasting position of the method for positioning bolts based on QR codes and correcting bolt image distortion of the present invention, Figure 3 It is a side view of the QR code pasting position of the method for positioning bolts based on QR codes and correcting bolt image distortion of the present invention.

[0019] The present invention provides a method for bolt positioning based on QR code and realizing bolt image distortion correction, comprising the following steps: S101. At the bolt connection location that needs to be inspected, a square QR code is tightly attached to the screw of the corresponding bolt, and the information is entered into the QR code; According to this specific embodiment, for the existing installed bolt connection, a square QR code is tightly pasted on the screw rod of the corresponding bolt according to the bolt connection structure proposed by the present invention, and when pasting the QR code, it is necessary to ensure that the diagonal length of the QR code is less than the screw rod diameter.

[0020] S102, photographing the bolt connection with a camera to obtain a bolt image; S103, performing adaptive filter denoising on the bolt image, converting the image into a grayscale image, and performing image binarization based on the global optimal binary segmentation threshold obtained by the Otsu method; According to this specific implementation method, the bolt image is subjected to adaptive filter denoising and converted into a grayscale image to simplify the image, reduce the amount of data, and make the image better highlight features such as texture, contour and edge lines, making it easier to segment and identify the image. The global optimal binary segmentation threshold is obtained based on the Otsu method to perform image binarization to remove grayscale information in the image. The grayscale information in the image is removed, and only black and white colors are retained to make the image clearer, improve the efficiency and speed of image processing, and at the same time reduce noise and interference, and highlight the shape and edge lines of the target.

[0021] S104, deleting small areas, reversing, and filling gaps to filter out the area that only contains the QR code; S105, based on the processed grayscale image, using Canny and Hough methods to detect the edge line of the bolt and determine the coordinates of the four corner points; In this specific implementation, the edge lines of the bolt image after denoising are identified by the Canny edge detection operator, and the straight lines among the edge lines identified by the Canny edge detection operator are detected by the Hough line detection operator, and these straight lines are the edge lines of the QR code; the threshold of the Canny edge detection operator is 0.75, and the RhoResolution in the Hough transform is 0.7, and the ThetaResolution is 0.65; The specific method for determining the coordinates of the four corner points is: Since the captured image may be distorted after image processing, multiple line segments may appear in the four directions of up, down, left, and right, making it impossible to accurately extract the equation of the line; Therefore, it is necessary to extract the endpoint coordinates of the edge lines detected in each direction of the detected QR code; The extracted edge coordinate points are classified according to the top, bottom, left and right directions of the QR code; Since the endpoint coordinates have the same X-axis coordinates, the least squares method cannot be used for fitting. Therefore, it is necessary to determine whether the endpoints can be fitted into a straight line in each direction using the least squares method for the upper, lower, left, and right classified endpoints: If possible, use the straight line equation to find the intersection points of the four straight lines (upper, lower, left, and upper right) to determine the coordinates of the four corner points; If not, rotate the coordinates of the classified points 45 degrees counterclockwise, and then use the least squares method to fit them into four straight lines, calculate the intersection of the four straight lines, and finally rotate the intersection 45 degrees clockwise to determine the coordinates of the corner points.

[0022] S106, calculating the homography matrix according to the coordinates of the four corner points, and performing image perspective correction.

[0023] A method for bolt positioning based on QR code and achieving bolt image distortion correction is used in the embodiment. At the part of the bolt connection that needs to be detected, a square QR code is tightly pasted on the screw rod of the corresponding bolt, and the information is entered into the QR code; the bolt connection is photographed by a camera to obtain a bolt image; the bolt image is denoised by an adaptive filter and converted into a grayscale image, and the image is binarized based on the global optimal binary segmentation threshold obtained by the Otsu method; small areas are deleted, reversed, and gaps are filled to filter out the area part containing only the QR code; the bolt edge line is detected by using the Canny and Hough methods based on the processed grayscale image, and the coordinates of the four corner points are determined; the homography matrix is ​​calculated based on the coordinates of the four corner points to perform image perspective correction; The present invention uses QR codes to accurately locate bolts, and performs distortion correction through the bolt image associated with the QR code, thereby avoiding the occlusion problem caused by square gasket correction; not only can the accurate positioning of the bolts be achieved, but also accurate and reliable basic data is provided for bolt loosening damage identification based on image processing technology, and the problem in the prior art that when there is shadow occlusion in the gasket area, these areas may be mistakenly identified as a connected domain during image processing due to the similar grayscale gradient of the occluded area; this not only hinders the accurate identification and extraction of the gasket area, but also significantly reduces the stability of the bolt image distortion correction, thereby affecting the technical problem of the accuracy of the bolt loosening damage identification result.

[0024] What is disclosed above is only a preferred embodiment of the present invention, and it certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A method for bolt positioning based on QR code and bolt image distortion correction. It is characterized in that The following steps are involved: Step 1: At the bolt connection that needs to be inspected, stick a square QR code tightly on the screw of the corresponding bolt and enter the information into the QR code; Step 2: Use a camera to photograph the bolt connection to obtain a bolt image; Step 3: Use an adaptive filter to remove noise from the bolt image and convert it into a grayscale image. Based on the Otsu method, the global optimal binary segmentation threshold is obtained to perform image binarization. Step 4: Delete small areas, invert, and fill gaps to filter out the area that only contains the QR code; Step 5: Based on the processed grayscale image, the Canny and Hough methods are used to detect the edge line of the bolt and determine the coordinates of the four corner points; Step 6: Calculate the homography matrix based on the coordinates of the four corner points to perform image perspective correction.

2. The method for bolt positioning and bolt image distortion correction based on QR code according to claim 1, characterized in that: In step 1, when attaching the QR code, you need to ensure that the diagonal length of the QR code is smaller than the screw diameter.

3. The method for bolt positioning and bolt image distortion correction based on QR code according to claim 2, characterized in that: The bolt image is denoised by an adaptive filter and converted into a grayscale image to simplify the image and reduce the amount of data.

4. The method for bolt positioning and bolt image distortion correction based on QR code according to claim 3, characterized in that: Based on the Otsu method, the global optimal binary segmentation threshold is obtained for image binarization to remove the grayscale information in the image. The grayscale information in the image is removed, and only black and white are retained to make the image clearer, improve the efficiency and speed of image processing, and also reduce noise and interference, and highlight the shape and edge lines of the target.

5. The method for bolt positioning and bolt image distortion correction based on QR code according to claim 4, characterized in that: In step five, the edge lines of the denoised bolt image are identified by the Canny edge detection operator, and the Hough line detection operator detects the straight lines in the edge lines identified by the Canny edge detection operator. These straight lines are the edge lines of the QR code.

6. The method for bolt positioning and bolt image distortion correction based on QR code according to claim 5, characterized in that: In step 5, the threshold of the Canny edge detection operator is 0.75, the RhoResolution in the Hough transform is 0.7, and the ThetaResolution is 0.

65.

7. The method for bolt positioning and bolt image distortion correction based on QR code according to claim 6, characterized in that: In step 5, the specific method of determining the coordinates of the four corner points is: Since the captured image may be distorted after image processing, multiple line segments may appear in the four directions of up, down, left, and right, making it impossible to accurately extract the equation of the line; Therefore, it is necessary to extract the endpoint coordinates of the edge lines detected in each direction of the detected QR code; The extracted edge coordinate points are classified according to the top, bottom, left and right directions of the QR code; Since the endpoint coordinates have the same X-axis coordinates, the least squares method cannot be used for fitting. Therefore, it is necessary to determine whether the endpoints can be fitted into a straight line in each direction using the least squares method for the upper, lower, left, and right classified endpoints: If possible, use the straight line equation to find the intersection points of the four straight lines (upper, lower, left, and upper right) to determine the coordinates of the four corner points; If not, rotate the coordinates of the classified points 45 degrees counterclockwise, and then use the least squares method to fit them into four straight lines, calculate the intersection points of the four straight lines, and finally rotate the intersection points 45 degrees clockwise to determine the coordinates of the corner points.

Images