Method and device for accurately identifying loose nuts / caps in steel structures based on rapid photographic inspection

Abstract

The present invention discloses a method and device for accurately identifying loose nuts / caps in steel structures based on rapid photographic inspection. The method and device collect images of node plates and then perform perspective correction on the collected image information, thereby converting images collected at different angles to the same viewing angle, eliminating visual errors caused by shooting angles and facilitating subsequent analysis. The method and device first identify the characterizing orientation lines of the unilateral contour lines corresponding to each nut / cap and then use overlay analysis to obtain the change in the loosening angle of the nut / cap, thereby improving the accuracy of nut / cap identification. The present invention is not easily affected by the environment and can eliminate identification errors caused by subjective judgment. It not only significantly saves time and labor costs, but more importantly, significantly improves the accuracy and efficiency of nut / cap inspection.

Description

Technical Field

[0001] The present invention relates to a bridge detection technology, and in particular to a method and device for accurately identifying loose nuts / caps of steel structures based on rapid photographic inspection. Background Art

[0002] Steel structures, with their advantages of lightness, high strength, and rapid assembly, are increasingly being used in the construction of large bridges, buildings, and other structures. Long-span steel bridges often utilize factory-fabricated steel beam segments, which are then connected on-site using high-strength nuts and caps to form the overall bridge structure. High-strength nut and cap connections generally offer advantages such as ease of operation, rapid construction, and reliable performance. However, among the tens of thousands of on-site connection points on a steel bridge (which can total up to ten million high-strength nuts and caps), quality issues inevitably occur due to factors such as manufacturing errors, fitting accuracy, and process limitations. Furthermore, bridges operate outdoors and are subjected to the impact fatigue of vehicle loads over extended periods. These high-strength nuts and caps in non-ideal connection points can gradually loosen, reducing structural safety margins and even leading to failure of bridge components or the entire structure. Statistics show that the incidence of defects in nut and cap connection points in steel bridges reaches 33.3%, of which 58.1% are due to loose or failed nuts and caps. Therefore, the importance of detecting loose nuts / caps at connection nodes in the safety of large bridge structures is self-evident.

[0003] In actual structural engineering, to detect whether a nut has loosened, people usually draw a clear straight line between each nut and the connecting member after the nut is fully tightened. If the nut has loosened, the change can be observed with the naked eye. This method is simple in principle, but in large bridges and house structures, there are hundreds or even thousands of nuts. Using the "scribing method" to check each nut is inevitably time-consuming and labor-intensive. Moreover, since structural components are exposed to the wind and sun for a long time, the drawn lines will fade and discolor after years of exposure, making it impossible to determine the specific looseness of each nut.

[0004] Traditional methods for detecting loose nuts / caps also include periodic inspections such as hammering and torque wrenching. However, these methods still require significant manpower and resources when the number of nuts / caps is large, and they also pose safety risks and are highly subjective, making them difficult to meet the current goal of detecting loose nuts / caps economically, quickly, accurately, and efficiently. Currently, there are also methods that use contact sensors for periodic inspections, but these sensors come in various types, including those based on acoustic waves, circuit impedance, and magnetic field technology. While these methods offer sensitive and rapid detection, they generally suffer from disadvantages such as difficult installation, high cost and maintenance, and susceptibility to environmental influences.

[0005] Therefore, in the face of the current problem that the detection of high-strength nuts / nuts on steel structure bridges is easily affected by the environment and requires a lot of manpower and material resources, there is an urgent need for a method and device for accurately identifying loose steel structure nuts / nuts based on rapid photographic inspection. Summary of the Invention

[0006] The purpose of the present invention is to provide a method and device for accurately identifying loose steel structure nuts / nuts based on rapid photographic inspection, so as to solve the problems that the current high-strength nut / nut detection of steel structure bridges is easily affected by the environment, has safety hazards, is highly subjective, and requires a lot of manpower and material resources.

[0007] To solve the above technical problems, the present invention provides a method for accurately identifying loose nuts / caps in steel structures based on rapid photographic inspection, comprising:

[0008] Collect image information of the gusset plate at the position to be inspected and name each nut / nut on the gusset plate;

[0009] Performing perspective correction on the collected image information to obtain a corrected image;

[0010] Identify the orientation line representing any single-side contour line of each regular hexagonal nut / cap in the rectified image;

[0011] The standard parts of regular hexagonal nuts / nuts are overlapped with the characterization orientation lines corresponding to each nut / nut to obtain the orientation status of each regular hexagonal nut / nut (the design center position information of each nut / nut is obtained based on the prior knowledge of the node plate nut / nut design). Then, a quantitative overlay analysis is performed based on the changes in the orientation status information of each regular hexagonal nut / nut during each inspection to obtain the quantitative information of the loosening angle of each nut / nut.

[0012] Furthermore, perspective correction is performed on the collected image information to obtain a corrected image including:

[0013] Obtain the position information of at least four reference points on the node plate;

[0014] According to the position information of at least four reference points, the image information collected during the inspection is uniformly perspective corrected using homography transformation to obtain a corrected image.

[0015] Furthermore, identifying the characterizing orientation line of any single-side contour line of each regular hexagonal nut / cap in the rectified image includes:

[0016] Use the Canny operator to perform edge detection on the gusset plate nut / cap image to obtain a pixel map;

[0017] Perform Hough line detection on the pixel image. Based on the mapping between the known side length of the regular hexagonal nut / cap and the camera angle image, select the corresponding edge characterization line of the nut / cap.

[0018] For the pixel points of the clearest edge line of the selected regular hexagonal nut / mother image, use the least squares straight line fitting algorithm to obtain the fitting orientation line of one edge line of the nut / mother regular hexagon;

[0019] For the edge orientation lines of a regular hexagon of a nut / nut, the fitting standard deviation of each fitting line is calculated respectively, and the line with the smallest fitting standard deviation is selected as the characterizing orientation line of the unilateral contour line of the nut.

[0020] Furthermore, based on the image data collected during each inspection, the position status information of each nut / nut of the node plate is analyzed and obtained. The position status of the nut / nut hexagon at the same design center position during the previous and subsequent inspections is quantitatively analyzed to obtain the quantitative information of the loosening angle of each nut / nut.

[0021] Furthermore, in view of the problem that the orientation recognition analysis of the nut / mother edge line far away from the imaging center position has a deviation problem that increases with distance, the method also includes: using a trained neural network model to predict the deviation of each nut / nut in the collected image information, and compensating for the deviation of each nut / nut based on the prediction result.

[0022] Furthermore, the training method of the neural network model includes:

[0023] The centroid of the nut / cap closest to the imaging center is taken as the imaging origin, and the distance between the centroids of the other nuts / caps and the imaging origin is recorded as r i The angle between the line connecting a nut / nut and the imaging origin and the horizontal axis is denoted as θ i The detection deviation when each nut / nut does not rotate is recorded as β i ;

[0024] With r i ,θ i As input parameters, β i As the output parameter, a neural network model is established, and then the known r i ,θ i and β i The data is used as a training set to train the neural network model.

[0025] Furthermore, the method also includes: when a nut / nut has been inspected and identified multiple times and its contour orientation has not changed, further identifying whether there are pressure rotation friction marks generated by the rotation of the nut / nut on the node plate, if so, it is determined that the nut / nut has rotated; otherwise, the nut / nut has not rotated.

[0026] Furthermore, determining whether there are nut / nut pressure rotation friction marks on the gusset plate includes:

[0027] Preprocess the image information to obtain a binary image;

[0028] The binary image is segmented using the image segmentation method based on region growing to obtain the region segmentation information;

[0029] Whether there is nut / nut rotation on the gusset plate is determined based on whether the region segmentation information includes the nut / nut pressure rotation friction trace area.

[0030] In addition, the present application also provides a device for accurately identifying loose nuts / caps on steel structures based on rapid photographic inspection, comprising:

[0031] An image acquisition unit, used for acquiring image information of the node plate at the position to be detected;

[0032] A data processor is used to name each nut / nut on each node plate in the image information; then perform perspective correction on the collected image information to obtain a corrected image; then identify the characterization orientation line of the single-sided contour line corresponding to each nut / nut in the corrected image; based on the design center position naming information of each nut / nut on the node plate, overlap the regular hexagonal nut standard parts with the characterization orientation lines corresponding to each nut / nut to obtain the position information and orientation status information of each nut / nut, and based on the analysis of the photographic images of each inspection, perform quantitative overlay analysis on the characterization orientation status information of each corresponding nut / nut of the previous and subsequent inspection data to obtain quantitative information on the loosening angle of each nut / nut.

[0033] Furthermore, the image acquisition unit includes: utilizing an upper chord crossbar of an inspection vehicle installed on a large bridge and capable of moving longitudinally along the main beam, at least one intelligent robotic arm capable of moving along the crossbar is installed on the crossbar, and at least one image collector is installed on the intelligent robotic arm; through the longitudinal movement of the inspection vehicle, the movement of the robotic arm along the crossbar, and the telescoping, bending, and rotation of the intelligent robotic arm along the height direction of the main beam, image acquisition and inspection of the nut / nut connection status of each node plate of the steel structure main beam of the entire bridge are achieved; based on prior knowledge of the bridge structure design, the image collector is instructed to move to the position of each node plate to be inspected for image acquisition. The nut / nut group orientation status identification method of the present invention significantly relaxes the requirements for the position, direction, distance, and angle of the node plate image acquisition, which is conducive to the rapid acquisition of node plate image data of the entire bridge based on the above-mentioned equipment.

[0034] The beneficial effects of the present invention are as follows: The present invention utilizes an existing inspection vehicle that can move longitudinally along the main beams of large steel bridges, and mounts an intelligent robotic arm equipped with an image acquisition device on the upper crossbar of the vehicle. Through the longitudinal movement of the inspection vehicle, the movement of the robotic arm along the crossbar, and the telescoping, bending, and rotation of the intelligent robotic arm along the height of the main beam, an image acquisition inspection of the nut / cap connection status of each node plate of the entire steel structure main beam of the bridge is achieved. This significantly reduces the labor cost of traditional manual inspections and improves the comprehensiveness and scientific nature of quantitative analysis. Compared with using drones to acquire images, this method improves the safety, long-term nature, and clarity of image acquisition of various node plates of complex steel structures. The image captured is used to identify the looseness of nuts / caps on the node plate, making full use of the advantages of machine vision in terms of easy acquisition, quantification, storage, and analysis. By performing perspective correction on the captured image information, images captured from different perspectives can be converted to the same perspective, eliminating visual errors caused by shooting angles and facilitating subsequent analysis. By first identifying the characterizing azimuth line of the unilateral contour line corresponding to each nut / cap, and then obtaining the loosening angle change of the nut / cap based on the superposition difference analysis of the previous and next inspection image data, the accuracy of nut / cap identification can be improved. The present invention is not easily affected by the environment and can eliminate the recognition error caused by subjective judgment, not only significantly saving time and labor costs, but more importantly, significantly improving the accuracy and efficiency of nut / cap inspection. The present invention can be used for inspection of large bridges and building structures, avoiding safety hazards caused by the professional quality of inspectors. It can more scientifically judge the safety performance of bridges, accurately grasp the safety status of bridges using this technology, and realize bridge safety inspection and evaluation, which has positive significance for the standardization, processization, and intelligence of bridge inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The same reference numerals are used in these drawings to represent the same or similar parts. The exemplary embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0036] Figure 1 A flowchart of an identification method disclosed in an embodiment of this application;

[0037] Figure 2 This is a schematic diagram of the homography transformation principle disclosed in an implementation of this application;

[0038] Figure 3 This is a schematic diagram of the multi-stage nut / nut orientation stacking difference disclosed in an embodiment of the present application;

[0039] Figure 4 This is a schematic diagram of the definition of the nut / cap azimuth angle according to an embodiment of the present application;

[0040] Figure 5 This is a schematic diagram of the contour detection results disclosed in an embodiment of this application;

[0041] Figure 6 A schematic diagram of a neural network disclosed in an embodiment of this application;

[0042] Figure 7 This is a schematic diagram of the scratch determination steps disclosed in an implementation of this application;

[0043] Figure 8 This is a schematic diagram of the regional growth process disclosed in an implementation of this application. DETAILED DESCRIPTION

[0044] like Figure 1 A method for accurately identifying loose nuts / caps of steel structures based on rapid photographic inspection is shown, the method comprising:

[0045] S1: Collect the image information of the node plate at the position to be detected and name the nuts / nuts on the node plate; the naming rules of the nuts / nuts on each node plate of the whole bridge are: NL k (i, j)——N#th node, L#th node plate, kth detection image acquisition, nut / cap at the i-th row and j-th column position; by naming the nuts / caps, it is easy to determine the location information of each nut / cap.

[0046] S2: Perform perspective correction on the collected image information to obtain a corrected image;

[0047] S3: Identify the orientation line representing any single-side contour line of each bolt regular hexagonal nut / nut in the rectified image;

[0048] S4: Overlap the characterization orientation lines corresponding to the standard parts and each nut / nut to obtain the position and orientation status information of each nut / nut, and then perform quantitative overlay analysis based on the position and orientation status information of each nut / nut to obtain the quantitative information of the loosening angle of each nut / nut. Based on the quantitative information of the loosening angle, the rotation status of the nut / nut can be obtained.

[0049] The present invention collects node plate images and then uses image perspective correction, nut / nut contour recognition, and rotation angle overlay analysis to economically, quickly, accurately and efficiently detect the looseness of node plate nuts / nuts in bridge structures.

[0050] According to one embodiment of the present application, performing perspective correction on the collected image information to obtain a corrected image includes:

[0051] S21: Obtaining position information of at least four reference points on the node plate;

[0052] S22: Based on the position information of at least four reference points, the image information collected during each inspection is uniformly perspective corrected by using homography transformation to obtain a corrected image.

[0053] In actual image data acquisition, the camera or video camera used to capture nuts / capsule groups at different times is mounted at different locations, resulting in different tilted perspectives in each image. Therefore, a unified tilt correction must be performed on each image to obtain a corresponding frontal view for later analysis. Furthermore, by adding a perspective correction step to the captured image information, only one image is required for each visible gusset plate in each direction, significantly reducing the number of gusset plate images collected on-site.

[0054] The present invention combines the characteristics of the nut / cap component and uses homography transformation to perform unified perspective correction on each phase of the image. The steps and principles are as follows:

[0055] 1) Based on prior knowledge of rigid joints, it is known that the actual geometric dimensions and relative positions of the connecting plate and the nuts / nuts remain unchanged regardless of whether the nuts / nuts rotate under any angle of view. Therefore, the invariant coordinates of the four corner points of the connecting plate can be used to uniformly correct the orientation of images taken at different tilt angles.

[0056] In order to obtain the accurate position of the corner points, the Harris corner detection or line detection and intersection point detection methods are used to obtain the position information of the four corner points of the node plate;

[0057] 2) After obtaining the image corner position, use homography transformation to perform unified perspective correction. The principle diagram is as follows: Figure 2 ,

[0058] Homography transformation process and transformation matrix H n 、H k Can be expressed as:

[0059]

[0060]

[0061] Among them, x0 and y0 are the corresponding horizontal and vertical coordinate vectors of the four corner points of the connecting plate in the front view determined according to the actual design size of the connecting plate; k 、y k are the horizontal and vertical coordinate vectors of the four corner points of the connecting plate obtained after corner point detection in the k-th captured image; n 、y nare the horizontal and vertical coordinate vectors of the four corner points of the connecting plate obtained after corner point detection in the nth captured image. The homography transformation matrix can be obtained using the SVD decomposition method.

[0062] According to one embodiment of the present application, identifying the representative orientation line of the unilateral contour line corresponding to each nut / cap in the rectified image includes:

[0063] S31: performing edge detection on the gusset plate nut / nut image in the corrected image using a Canny operator to obtain a pixel map;

[0064] S32: Performing Hough line detection on the pixel image, and selecting a characterization line of the corresponding edge of the nut / nut based on a mapping judgment between the known side length of the regular hexagonal nut / nut and the camera angle image;

[0065] S33: For the pixel points of the clearest edge line of the selected nut / cap image, a fitting orientation line based on one of the edge lines of the selected nut / cap is performed using a least squares straight line fitting algorithm;

[0066] S34: For the regular hexagonal edge orientation lines of a nut / cap, the fitting standard deviation of each fitting line is calculated respectively, and the line with the smallest fitting standard deviation is selected as the representation orientation line of the single-side contour line of the nut.

[0067] Repeat the above steps to obtain the characterizing orientation line of the single-side contour line corresponding to each nut on the node plate.

[0068] Since each nut / nut is a regular hexagonal standard part with known dimensions, during the identification process, one side of the regular hexagonal standard part can be used to coincide with the characterization orientation line of each corresponding nut / nut obtained through image analysis, so that the position and orientation status information of each nut / nut on the node plate can be obtained.

[0069] According to one embodiment of the present application, quantitative overlay analysis is performed based on the position information and orientation state information of each nut / cap to obtain quantitative information on the loosening angle of each nut / cap, including:

[0070] The position information and orientation status information of each nut / nut are quantified, and then the quantified position and orientation status information of each nut / nut is subjected to quantitative overlay analysis with the position and orientation status information of the nut / nut in the reference image to obtain the quantitative information of the loose rotation angle of each nut / nut. For example, the quantified position and orientation image of each nut / nut of the node plate obtained by the Kth inspection is subjected to quantitative overlay analysis with the orientation of each nut / nut in the quantified position and orientation image of each nut / nut of the node plate obtained by the Nth inspection to obtain the quantitative information of the loose rotation angle of each nut / nut, as shown in the attached figure. Figure 3 As shown.K(或N) Take the nut / nut of (2,1) as an example to explain the definition of azimuth angle, as shown in the attached figure. Figure 4 As shown, in the K-th captured image, the angle between the nut / nut and the positive direction of the X-axis (the acute angle on the same side is uniformly taken) is defined as α K (2,1), similarly, in the image taken for the Nth time, the angle between the nut / nut and the positive direction of the X-axis becomes α N (2,1), so the rotation angle of the nut / nut during the two sampling intervals is:

[0071] Δ K-N (2,1)=|α K (2,1)-α N (2,1)|.

[0072] According to one embodiment of the present application, the method further includes: using a trained neural network model to predict the orientation detection deviation of each nut / cap in the collected image information, and compensating for the deviation of each nut / cap based on the prediction result. Theoretically, since each nut / cap does not rotate, after performing unified perspective correction and contour detection, the contours of each nut / cap in the two images should completely overlap. However, in the actual contour detection process, Figure 5 As shown, except for the nut / nut in the lower left corner, the outlines of the remaining nuts / nuts all experience varying degrees of deflection, with the degree of deflection increasing as the distance from the lower left corner increases. This phenomenon is caused by image distortion: when the image is captured, the center of the lens is close to the nut / nut in the lower left corner, and the further away from the lens center, the greater the distortion, ultimately leading to greater deviations in image outline detection. By compensating for the systematic deviations in image detection of nut / nut rotation angles, the present invention can eliminate the systematic errors caused by the image processing method, thereby making the measurement of nut / nut rotation angles more accurate.

[0073] According to one embodiment of the present application, a method for training a neural network model includes:

[0074] The nut / nut closest to the imaging center ( Figure 5 The centroid of the nut / nut in the lower left corner is used as the imaging origin, and the distance between the centroid of the other nuts / nuts and the imaging origin is recorded as r i The angle between the line connecting a nut / nut and the imaging origin and the horizontal axis is denoted as θ i The detection deviation when each nut / nut does not rotate is recorded as β i , these known data are used as training sets, as shown in the following table:

[0075]

[0076] 2) With r i ,θi As input parameters, β i As the output parameter, a 2×4×1 neural network model is established, such as Figure 6 As shown, the neural network is then trained using the data in the table above to obtain r i ,θ i , β i the intrinsic connection between the three;

[0077] 3) In this case, the deviation prediction values ​​of the neural network for the four nuts / caps in the training set were 0.107, 0.116, 0.096, and 0.095, respectively. The relative errors from the true values ​​were 6.14%, 8.66%, 7.87%, and 7.77%, respectively, showing good prediction results.

[0078] According to one embodiment of the present application, the method further includes: when a nut / nut has been inspected and identified multiple times and its contour orientation has not changed, further identifying whether there are pressure-rotation friction marks generated by the rotation of the nut / nut on the node plate, if so, the nut / nut has rotated; otherwise, the nut / nut has not rotated. For a certain nut / nut, after multiple image processing, it is found that its contour orientation has not changed. At this time, the nut / nut may have loosened by a 60° angle, or it may not have loosened; and under normal circumstances, since the nut / nut is tightly connected to the base plate, scratches will be left on the surface of the base plate when it rotates. Therefore, for such special conditions, the present application judges the rotation of the nut / nut by detecting whether there are scratches generated by the rotation of the nut / nut, which can prevent missed identification.

[0079] According to one embodiment of the present application, determining whether there are nut / cap scratches on the gusset plate includes:

[0080] The image information is preprocessed to obtain a binary image; the image information preprocessing includes grayscale processing and binary processing, such as Figure 7 As shown;

[0081] The binary image is segmented using the image segmentation method based on region growing to obtain the region segmentation information;

[0082] Whether there is nut / nut rotation on the gusset plate is determined based on whether the region segmentation information includes pressure rotation friction marks.

[0083] The basic idea of ​​region growing is to gather pixels with similar properties to construct segmented regions. Starting with a set of seed points, pixels in the region with similar properties (such as grayscale) to the seeds are appended to each seed in the growing region until the growth is completed. The steps are:

[0084] a. Randomly select a pixel in the image as a seed pixel and mark it;

[0085] b. Retrieve unmarked pixels near the seed, and if their difference is within the specified threshold (the threshold needs to be set according to different situations), merge them into the segmented region;

[0086] c. Repeat step b until the area stops expanding, and then randomly select a pixel in the non-selected area as the seed pixel again;

[0087] d. Repeat the above steps until each pixel in the image is assigned to a different region.

[0088] Image segmentation based on region growing of the scratch area can not only determine the looseness of the nut / cap, but also serve as a supplement to the loose angle overlay results when the nut / cap is loose, thereby increasing the accuracy of nut / cap loosening identification.

[0089] In addition, the present application also provides a device for accurately identifying loose nuts / caps on steel structures based on rapid photographic inspection, comprising:

[0090] An image acquisition unit, used for acquiring image information of the node plate at the position to be detected;

[0091] A data processor is used to name each nut / nut on each node plate in the image information; then perform perspective correction on the collected image information to obtain a corrected image; then identify the characterization orientation line of the single-sided contour line corresponding to each nut / nut in the corrected image; based on the design center position naming information of each nut / nut on the node plate, overlap the nut / nut standard parts with the characterization orientation line corresponding to each nut / nut to obtain the position information and orientation status information of each nut / nut, and based on the analysis of the photographic images of each inspection, perform quantitative overlay analysis on the characterization orientation status information of each corresponding nut / nut of the previous and subsequent inspection data to obtain quantitative information on the loosening angle of each nut / nut.

[0092] According to one embodiment of the present application, the image acquisition unit includes: utilizing an upper chord crossbar mounted on a large bridge and movable along the longitudinal direction of the main beam of an inspection vehicle, mounting at least one intelligent robotic arm capable of moving along the crossbar on the crossbar, and mounting at least one image acquisition device on the intelligent robotic arm; through the longitudinal movement of the inspection vehicle, the movement of the robotic arm along the crossbar, and the telescoping, bending, and rotation of the intelligent robotic arm along the height direction of the main beam, image acquisition and inspection of the bolt connection status of each node plate of the steel structure main beam of the entire bridge are achieved; based on prior knowledge of the bridge structure design, the image acquisition device is instructed to move to the position of each node plate to be inspected for image acquisition. The bolt group orientation status identification method of the present invention significantly relaxes the requirements for the position, direction, distance, and angle of the node plate image acquisition, which is conducive to the rapid acquisition of image data of the node plates of the entire bridge based on the above-mentioned equipment.

[0093] By utilizing a bridge inspection vehicle, at least one assembled robotic arm is installed on the upper crossbar; and based on the given structural position of the bridge node plate, one or more cameras or cameras are installed at appropriate positions of the robotic arm; during the image acquisition process, the robotic arm is extended and rotated to collect digital images of the node plates at different positions along the beam height; the robotic arm is moved along the crossbar of the vehicle to collect digital images of the node plates at different positions along the transverse direction of the bridge; and the vehicle is moved along the longitudinal direction of the bridge to collect digital images of the node plates at different positions along the longitudinal direction of the bridge.

[0094] The above-mentioned robotic arm can be an intelligent robotic arm, which is installed on the beam through a semi-encircling walking robotic arm connecting seat; the intelligent robotic arm is equipped with a control program that instructs the camera to reach the position and collect images based on prior knowledge of the bridge structure; by combining engineering prior knowledge for image collection, the image collection efficiency is improved.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for accurately identifying nuts / caps on steel structures based on rapid photographic inspection, the characteristics of which include: Collect image information of the gusset plate at the position to be inspected and name each nut / nut on the gusset plate; Performing perspective correction on the collected image information to obtain a corrected image; Identifying a representative orientation line of any single-sided contour line corresponding to each nut / screw in the corrected image; Overlap the characterization orientation lines corresponding to the standard part and each nut / nut to obtain the position information and orientation state information of each nut / nut, then quantify the position information and orientation state information of each nut / nut, and then perform quantitative overlay difference analysis on the quantized position and orientation state information of each nut / nut with the position and orientation state information of the nut / nut in the reference image to obtain the quantitative information of the loosening angle of each nut / nut; The trained neural network model is used to predict the deviation of each nut / cap in the collected image information, and the deviation of each nut / cap is compensated according to the prediction result. The training method of the neural network model includes: The centroid of the nut / cap closest to the imaging center is taken as the imaging origin, and the distance between the centroids of the other nuts / caps and the imaging origin is recorded as r i The angle between the line connecting a nut / nut and the imaging origin and the horizontal axis is denoted as θ i The detection deviation when each nut / nut does not rotate is recorded as β i ; With r i ,θ i As input parameters, β i As the output parameter, a neural network model is established, and then the known r i ,θ i and β i The data is used as a training set to train the neural network model; When a nut / cap has been inspected and identified multiple times and its contour orientation has not changed, it is further identified whether there are pressure rotation friction marks caused by the rotation of the nut / cap on the node plate. If so, the nut / cap has rotated; otherwise, the nut / cap has not rotated.

2. The method for accurately identifying nuts / caps of steel structures based on rapid photographic inspection according to claim 1 is characterized in that: The performing perspective correction on the collected image information to obtain a corrected image includes: Obtain the position information of at least four reference points on the node plate; According to the position information of at least four reference points, the image information collected during each inspection is uniformly perspective corrected using homography transformation to obtain a corrected image.

3. The method for accurately identifying nuts / caps of steel structures based on rapid photographic inspection according to claim 2, characterized in that: Identifying the characterizing orientation lines of the unilateral contour lines corresponding to each nut / cap in the rectified image includes: Performing edge detection on the gusset plate nut / nut image in the corrected image using a Canny operator to obtain a pixel map; Perform Hough line detection on the pixel image, and select the characterizing line of the corresponding edge of the nut / nut based on the mapping judgment between the known side length of the regular hexagonal nut / nut and the camera angle image; For the pixel points of the clearest edge line of the selected nut / cap image, a least squares straight line fitting algorithm is used to fit an orientation line based on one of the edge lines of the selected nut / cap; For the orientation lines of the regular hexagonal edge of a nut / cap, the fitting standard deviation of each fitting line is calculated respectively, and the line with the smallest fitting standard deviation is selected as the characterizing orientation line of the unilateral contour line of the nut / cap.

4. The method for accurately identifying nuts / caps of steel structures based on rapid photographic inspection according to claim 1, characterized in that: The identifying whether there are pressure rotation friction marks on the gusset plate includes: Preprocessing the image information to obtain a binary image; The binary image is segmented using the image segmentation method based on region growing to obtain the region segmentation information; Whether nut / cap rotation exists on the gusset plate is identified according to whether the region segmentation information includes pressure-rotation friction traces.

5. A device for accurately identifying nuts / caps of steel structures based on rapid photographic inspection, characterized in that: include: An image acquisition unit, used for acquiring image information of the node plate at the position to be detected; A data processor is configured to name each nut / nut on each node plate in the image information; perform perspective correction on the collected image information to obtain a corrected image; identify the characterization orientation line of the unilateral contour line corresponding to each nut / nut in the corrected image; overlap the nut / nut standard parts with the characterization orientation line corresponding to each nut / nut based on the design center position naming information of each nut / nut on the node plate to obtain position information and orientation status information of each nut / nut; and perform quantitative overlay analysis on the characterization orientation status information of each corresponding nut / nut in the previous and subsequent inspection data based on the analysis of the photographic images of each inspection to obtain quantitative information on the loosening angle of each nut / nut; The data processor is further configured to predict the deviation of each nut / cap in the collected image information using a trained neural network model, and to compensate for the deviation of each nut / cap based on the prediction result; the training method of the neural network model includes: The centroid of the nut / cap closest to the imaging center is taken as the imaging origin, and the distance between the centroids of the other nuts / caps and the imaging origin is recorded as r i The angle between the line connecting a nut / nut and the imaging origin and the horizontal axis is denoted as θ i The detection deviation when each nut / nut does not rotate is recorded as β i ; With r i ,θ i As input parameters, β i As the output parameter, a neural network model is established, and then the known r i ,θ i and β i The data is used as a training set to train the neural network model; When a nut / cap has been inspected and identified multiple times and its contour orientation has not changed, it is further identified whether there are pressure rotation friction marks caused by the rotation of the nut / cap on the node plate. If so, the nut / cap has rotated; otherwise, the nut / cap has not rotated.

6. The device for accurately identifying nuts and caps of steel structures based on rapid photographic inspection according to claim 5, characterized in that: The image acquisition unit includes a crossbar installed on the bridge, at least one robotic arm installed on the crossbar, and at least one image collector installed on the robotic arm; the robotic arm instructs the image collector to rotate and collect images based on prior knowledge of the bridge structure design.

Images