A method and equipment for identifying loose bolts in pre-assembled steel structures

Abstract

This invention relates to the field of image processing, specifically to a method and device for identifying loose bolts in pre-assembled steel structures. The method includes: acquiring a bolt tightness index P1 from the bolt posture features of an inspection image F1 and the bolt posture features of a template image; when P1 is greater than a first preset threshold, reconstructing the bolt posture features of F1 and the template image, using a drone to re-inspect each bolt fixing area and acquire an inspection image F2, acquiring the bolt tightness index P2 from the reconstructed bolt posture features of inspection image F2 and the reconstructed bolt posture features of the template image; when P2 is less than or equal to the first preset threshold, acquiring the inspection environment interference features of inspection image F2, removing the inspection environment interference features from the reconstructed bolt posture features of inspection image F2, and then performing bolt loosening identification. This invention achieves the goal of reliably identifying loose bolts even under diverse and changing environmental interference.

Description

Technical Field

[0001] This invention relates to the field of image processing, and specifically to a method and device for identifying loose bolts in pre-assembled steel structures. Background Technology

[0002] Large public buildings, such as stadiums, airport terminals, and convention centers, widely employ long-span prefabricated steel structures. These structures are typically assembled on-site using numerous bolts. The reliability of these bolted connections directly impacts the safety of the overall structure. Regularly inspecting each bolt using drones and identifying any loose bolts allows for the early detection and resolution of potential bolted connection risks.

[0003] However, during the inspection of each bolt and identification of whether the bolt is loose, the natural environment of the steel structure is different at different inspections, and the environmental interference is not constant. Natural environmental factors or conditions such as dust, dirt, corrosion and light can interfere with the bolt loosening identification results, leading to misidentification problems. Summary of the Invention

[0004] To address the aforementioned problems, this invention provides a method and device for identifying loose bolts in pre-assembled steel structures.

[0005] The present invention provides a method and device for identifying loose bolts in pre-assembled steel structures, which adopts the following technical solution:

[0006] One embodiment of the present invention provides a method for identifying loose bolts in pre-assembled steel structures, the method comprising the following steps:

[0007] Template images are acquired for each bolt fixing area, and bolt posture features are extracted from the template images; each bolt fixing area is inspected using a drone, and inspection images are acquired, denoted as F1;

[0008] The similarity between the bolt posture features of the inspection image F1 and the bolt posture features of the template image is used as the bolt tightness index P1 for each bolt fixing area. When P1 is greater than the first preset threshold, each bolt fixing area is marked as a bolt not loosened area. Furthermore, the bolt posture features of F1 and the template image are reconstructed so that the similarity between the reconstructed bolt posture features of F1 and the reconstructed bolt posture features of the template image is greater than P1.

[0009] After a preset time interval, each bolt fixing area is inspected again using a drone and an inspection image is collected, denoted as F2. The similarity between the reconstructed bolt posture features of inspection image F2 and the reconstructed bolt posture features of the template image is recorded as the bolt tightness index P2 for each bolt fixing area. When P2 is less than or equal to a first preset threshold, the differences between the reconstructed bolt posture features of all previous inspections and the bolt posture features before all previous reconstructions are recorded as the inspection environment interference features of inspection image F2. After removing the inspection environment interference features from the reconstructed bolt posture features of inspection image F2, bolt loosening is identified.

[0010] Preferably, the similarity between the bolt posture features of the inspection image F1 and the bolt posture features of the template image is used as the bolt tightening index P1 for each bolt fixing area, and the specific steps include the following:

[0011] For each inspection image F1 within any bolt fixing area, calculate the similarity between the bolt posture features of each inspection image F1 and the bolt posture features of the viewpoint matching image of each inspection image F1. The average similarity between all inspection images F1 within any bolt fixing area and the viewpoint matching image is denoted as the bolt tightening index P1 of any bolt fixing area. The viewpoint matching image of each inspection image F1 refers to the template image that is at the same viewpoint as each inspection image F1.

[0012] Preferably, the specific steps for reconstructing the bolt posture features of F1 and the template image, such that the similarity between the reconstructed bolt posture features of F1 and the reconstructed bolt posture features of the template image is greater than P1, are as follows:

[0013] For any bolt-fixed area determined to be bolt-free, and for each inspection image F1 within that bolt-fixed area, a feature of interest is randomly initialized for each inspection image F1; the bolt posture feature of the view matching image of each inspection image F1 is denoted as q2; the view matching image of each inspection image F1 refers to the template image that is at the same viewpoint as each inspection image F1.

[0014] The interest feature of each inspection image F1 is multiplied with the bolt posture feature in the same dimension to obtain the first vector, and the interest feature of each inspection image F1 is multiplied with q2 in the same dimension to obtain the second vector. The cosine similarity between the first vector and the second vector is denoted as the first similarity. The difference between the mean of the first similarity of all inspection images F1 and the view matching image within the bolt fixing area and P1 is denoted as the target value of the interest feature. The interest feature that maximizes the target value and has a maximum value greater than 0 is denoted as the preferred interest feature of each inspection image F1, and also as the preferred interest feature of the view matching image of each inspection image F1.

[0015] Preferably, the specific steps for reconstructing the bolt pose features of each inspection image F1 and the viewpoint matching image of each inspection image F1 based on the preferred interest features are as follows:

[0016] The preferred features of interest for each inspection image F1 are multiplied by the bolt posture features of each inspection image F1 in the same dimension to obtain the reconstructed bolt posture features for each inspection image F1.

[0017] The preferred features of interest for each inspection image are multiplied in the same dimension by the bolt posture features of the viewpoint matching image for each inspection image to obtain the reconstructed bolt posture features of the viewpoint matching image.

[0018] Preferably, the specific steps for recording the differences between the bolt posture features after each reconstruction and the bolt posture features before each reconstruction during the historical inspection process as the inspection environment interference features of the inspection image F2 are as follows:

[0019] For any bolt-fixed area determined to be not loose after each historical inspection, the difference between the reconstructed bolt posture features and the unreconstructed bolt posture features of each inspection image F1 within that bolt-fixed area is recorded as the environmental feature of each inspection image F1, and also serves as the environmental feature of the viewpoint matching image of each inspection image F1; the viewpoint matching image of each inspection image F1 refers to the template image that is at the same viewpoint as each inspection image F1.

[0020] After all historical inspections, the environmental features obtained from each template image are averaged and normalized to obtain the inspection environment interference features of each template image. These features are also used as the inspection environment interference features of inspection image F2, which is at the same viewpoint as each template image.

[0021] Preferably, the specific steps for removing the inspection environment interference features from the reconstructed bolt posture features of the inspection image F2 are as follows:

[0022] For any bolt fixing area where P2 is less than or equal to the first preset threshold, the preferred attention feature of the viewpoint matching image of each inspection image F2 within the bolt fixing area is also used as the preferred attention feature of each inspection image F2. Here, the viewpoint matching image of each inspection image F2 refers to the template image that is at the same viewpoint as each inspection image F2.

[0023] The reconstructed bolt attitude features of F2 are equal to the preferred interest features of F2 for each inspection image multiplied by the bolt attitude features in the same dimension.

[0024] The cosine similarity between the preferred features of each inspection image F2 and the inspection environment interference features of each inspection image F2 is denoted as the reconstruction error. For the value of each dimension in the inspection environment interference features, the N1 dimensions with the largest values ​​are selected and deleted from the reconstructed bolt posture features of F2 to obtain the target bolt posture features of each inspection image F2.

[0025] Preferably, the specific steps for identifying bolt loosening after removing the inspection environment interference features from the reconstructed bolt posture features of the inspection image F2 are as follows:

[0026] For any bolt-fixed area where P2 is less than or equal to the first preset threshold, the reconstructed bolt posture feature of the matching viewpoint image of each inspection image F2 within the bolt-fixed area is denoted as Q1, and the target bolt posture feature of each inspection image F2 is denoted as Q2. After deleting the missing dimension of Q2 from Q1, the cosine similarity between Q2 and Q1 is obtained and denoted as the second similarity of each inspection image F2. The mean of the second similarities of all inspection images F2 within the bolt-fixed area is used as the bolt tightening index P3 of the bolt-fixed area. When the bolt tightening index P3 is greater than the first preset threshold, it is determined that no bolt has become loose in the bolt-fixed area; when the bolt tightening index P3 is less than or equal to the first preset threshold, it is determined that the bolt has become loose in the bolt-fixed area.

[0027] Preferably, the specific steps for extracting bolt posture features from the template image are as follows:

[0028] The connected components of the bolt fixing region in each template image are segmented and referred to as the bolt fixing connected components. Within the bolt fixing connected components of each template image, key points in the template image are detected, and the connected components where all bolts are located within the bolt fixing connected components are segmented and referred to as the bolt connected components.

[0029] All keypoints within and outside the bolt-fixed connected domain are paired to form a keypoint pair set, denoted as S1. The pose features of keypoint pair set S1 are obtained. All keypoints within and outside the bolt-fixed connected domain are paired with all keypoints within the bolt-fixed connected domain to form a keypoint pair set, denoted as S2. The pose features of keypoint pair set S2 are obtained. The pose features of keypoint pairs S1 and S2 are concatenated to form the bolt pose features in each template image.

[0030] Preferably, the specific steps for reconstructing the bolt pose features of each inspection image F1 and the viewpoint matching image of each inspection image F1 based on the preferred attention features are as follows:

[0031] The preferred features of interest for each inspection image F1 are multiplied by the bolt posture features of each inspection image F1 in the same dimension to obtain the reconstructed bolt posture features for each inspection image F1.

[0032] The preferred features of interest for each inspection image are multiplied in the same dimension by the bolt posture features of the viewpoint matching image for each inspection image to obtain the reconstructed bolt posture features of the viewpoint matching image.

[0033] Preferably, the specific steps for obtaining the posture features are as follows:

[0034] The distance between each keypoint pair in the keypoint pair set and the inclination angle of the line containing the keypoint pair are denoted as the distance feature and distribution direction feature of each keypoint pair. The distance features and distribution direction features of all keypoint pairs in the keypoint pair set are plotted as histograms and then stitched together to obtain the pose feature of the keypoint pair set.

[0035] Another embodiment of the present invention provides a pre-assembled steel structure bolt loosening identification device. The device includes a drone, a camera mounted on the drone for acquiring template images and inspection images, and the drone also includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor reads the template images and inspection images acquired by the camera, and executes all the steps of the pre-assembled steel structure bolt loosening identification method described above when running the computer program.

[0036] The beneficial effects of the technical solution of the present invention are:

[0037] When P1 exceeds a first preset threshold, this invention marks each bolt-fixed area as a bolt-unloose area; and reconstructs the bolt posture features of F1 and the template image, ensuring that the similarity between the reconstructed bolt posture features of F1 and the reconstructed bolt posture features of the template image is greater than P1. This process, by reconstructing the bolt posture features, retains the features that reflect the bolt posture condition while filtering out features influenced by environmental interference. This allows the reconstructed bolt posture features to further characterize and reflect the bolt's unloose state, avoiding the problem that environmental interference noise in the bolt posture features cannot further describe the bolt's unloose state.

[0038] Furthermore, when P2 is less than or equal to a first preset threshold, the present invention records the differences between the bolt posture features after reconstruction in all historical inspection processes and the bolt posture features before reconstruction as the inspection environment interference features of inspection image F2. After removing the inspection environment interference features from the reconstructed bolt posture features of inspection image F2, bolt loosening identification is performed. The inspection environment interference features obtained in this process are based on all historical inspection processes and describe whether each dimension of the bolt posture features can be used to stably reflect the bolt posture under diverse and changing environmental interference. In this process, when the inspection is performed again (i.e., when inspection image F2 is acquired), the reconstruction of bolt posture features based on the previous inspection process (i.e., when inspection image F1 was acquired) is avoided. However, the reconstructed bolt posture features are not adapted to the current inspection environment, resulting in environmental noise interference in the reconstruction results. This further achieves the goal of reliably assessing bolt loosening under diverse and changing environmental interference. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart illustrating the steps of a method for identifying loose bolts in a pre-assembled steel structure according to an embodiment of the present invention.

[0041] Figure 2 This is a detailed flowchart of a method for identifying loose bolts in a pre-assembled steel structure, provided as an embodiment of the present invention. Detailed Implementation

[0042] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a method and device for identifying loose bolts in pre-assembled steel structures according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0043] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0044] The following description, in conjunction with the accompanying drawings, details the specific scheme of the method and equipment for identifying loose bolts in pre-assembled steel structures provided by this invention.

[0045] Example 1: Please refer to Figure 1 The diagram illustrates a flowchart of a method for identifying loose bolts in a pre-assembled steel structure according to an embodiment of the present invention. The method includes the following steps:

[0046] Step S101: Collect template images for each bolt fixing area and store the bolt posture features in the template images in a vector database.

[0047] The installed steel structure contains several bolt fixing areas, each containing one or more bolts. All the bolts in each bolt fixing area are used to tightly fix the different, separate steel structure components together.

[0048] In this embodiment, in each bolt-fixed area of ​​the newly installed steel structure, each bolt is tightly secured. A remotely controlled drone flies over each bolt-fixed area sequentially, and the camera on the drone captures video of each bolt-fixed area, covering various perspectives. In this embodiment, all frames in the video are recorded as template images for each bolt-fixed area. Each template image corresponds to a perspective, recording the attitude information of the bolts when they are not loose from a specific perspective. In this embodiment, the drone's camera captures 12 frames per second, and each frame is a grayscale image after grayscale conversion.

[0049] In this embodiment, an automatic inspection is performed using a drone at preset intervals. By comparing the images collected in each bolt fixing area during the subsequent inspection with the template image, it is determined whether there is any loose bolt in each bolt fixing area.

[0050] Before conducting UAV inspections, this embodiment first extracts the bolt attitude features from each template image and stores these features in a vector database. This allows for rapid comparison and judgment based on bolt attitude features when determining whether bolts are loose. All template images for each bolt fixing area are also stored in the vector database. Each template image and its corresponding bolt attitude feature represent a data entry in the database. In this embodiment, all data entries stored within each bolt fixing area have consecutive ID numbers.

[0051] The methods for storing and retrieving vector databases are well-known and will not be described in detail in this embodiment.

[0052] The bolt posture features in each template image are obtained by image processing of each template image. They describe the position and posture of all bolts relative to the steel structure within the bolt fixing area from a specific viewpoint. When the bolts become loose, the bolt posture features will also change.

[0053] As an example, the method for obtaining the bolt pose features in each template image is as follows:

[0054] (1) Segment the connected regions of the bolt fixing area in each template image, and denoted as the bolt fixing connected region.

[0055] Within the bolt-fixed connected component of each template image, key points in the template image are detected, and the connected components containing all bolts within the bolt-fixed connected component are segmented and denoted as bolt connected components. The difference between bolt-fixed connected components and bolt connected components is that the former includes both the image information of the bolts and the image information of the steel structure to which the bolts are fixed, while the latter only includes the image information of the bolts.

[0056] (2) For all key points within and outside the bolt-fixed connected domain, these key points are paired to obtain several key point pairs. These key point pairs constitute the key point pair set S1. The key point pair set S1 is processed as follows:

[0057] For each keypoint pair in S1, the distance between each keypoint pair and the inclination angle of the line containing the keypoint pair are denoted as the distance feature and distribution direction feature of each keypoint pair. The inclination angle refers to the counterclockwise inclination angle of the line relative to the horizontal line in the template image (this inclination angle is less than 180 degrees).

[0058] Plot the distance features of all keypoint pairs in S1 into a histogram, denoted as the distance histogram. The x-axis of each curve point in the distance histogram is the distance feature, and the y-axis is the distribution frequency of the distance feature.

[0059] Similarly, the distribution direction features of all keypoint pairs are plotted into a histogram, denoted as the distribution histogram. The x-axis of each curve point in the distribution histogram represents the distribution direction feature, and the y-axis represents the distribution frequency of the distribution direction feature.

[0060] The sequence of ordinates of all curve points in the distance histogram and the sequence of ordinates of all curve points in the distribution histogram are concatenated and treated as a vector, denoted as the pose feature of the keypoint pair set S1.

[0061] The attitude features of the keypoint pair set S1 describe the relative distribution of all keypoints in S1, representing the attitude information of the steel structure fixed by the bolts.

[0062] (3) Combine all key points inside and outside the bolt-fixed connected domain with all key points inside the bolt-connected domain to obtain several key point pairs. These key point pairs constitute the key point pair set S2, where one key point in each key point pair is inside and outside the bolt-fixed connected domain, and the other key point is inside the bolt-connected domain.

[0063] The attitude features of the key point pair set S2 are obtained according to the above method (2), which represent the attitude information of the bolt relative to the steel structure.

[0064] (4) In this embodiment, the pose features of key point pairs S1 and S2 are spliced ​​together as bolt pose features in each template image. The bolt pose features include both the pose of the steel structure and the pose of the bolt relative to the steel structure, ensuring that the bolt pose features can describe the pose information of the bolt from a specific viewpoint.

[0065] Step S102: Use a drone to inspect each bolt fixing area and collect inspection image F1. The similarity between F1 and the bolt posture features in the vector database is used as the bolt tightness index P1 for each bolt fixing area. When P1 is greater than the first preset threshold, each bolt fixing area is marked as a bolt not loosened area.

[0066] In this embodiment, a drone inspection is performed after a preset time interval, which is half a month.

[0067] During drone inspection, a fixed inspection path is set for the drone (i.e., the flight path used when acquiring template images in step S101). The drone flies along this path sequentially over each bolt fixing area, acquiring video of each bolt fixing area as it flies over it. All frames in this video contain information from different perspectives of each bolt fixing area. In this embodiment, several frames (e.g., 5 frames) are selected at equal intervals from the video as inspection images. In other embodiments, several frames can be randomly selected as inspection images; or all frames in the video can be used as inspection images.

[0068] At this point, multiple inspection images were collected for each bolt fixing area, and the different inspection images included steel structure information from different perspectives.

[0069] For any bolt-fixed area, all inspection images collected under that bolt-fixed area are represented as F1; obtain the bolt posture features in each inspection image F1 (similar to step S101). Read all template images of the bolt-fixed area from the vector database.

[0070] The cosine similarity between the bolt posture features of the inspection image F1 and the bolt posture features of the template image is denoted as the similarity between the inspection image F1 and the template image.

[0071] All inspection images F1 within the bolt fixing area are matched with all read template images KM to obtain all matching pairs. Each matching pair contains an inspection image F1 and a template image. The inspection image F1 and the template image in each matching pair obtained by KM matching have the highest similarity. Each matching pair contains an inspection image F1 and a template image from the same viewpoint. In this embodiment, the template image is recorded as the viewpoint matching image of the inspection image F1.

[0072] The average similarity between all inspection images F1 within any bolt-fixing area and their viewpoint matching images is obtained. The larger the average value, the more similar or identical the bolt posture in inspection image F1 and the posture in the template image are under the same viewpoint, and the lower the probability of the bolt in inspection image F1 being loose. In this embodiment, this average value is used as the bolt tightness index P1 for the bolt-fixing area. The larger the bolt tightness index P1, the more likely it is that loosening has not occurred.

[0073] When P1 is greater than the first preset threshold th1, it is determined that no bolt has become loose in the bolt fixing area. When P1 is less than or equal to the first preset threshold th1, it is determined that the bolt has become loose in the bolt fixing area.

[0074] This embodiment uses th1 equal to 0.8 as an example for description. In other embodiments, the preferred value range of th1 is [0.7, 0.95].

[0075] When the bolt fixing area is determined to be loose, it needs to be tightened again. After tightening, the template image of the bolt fixing area is re-acquired, all data of the bolt fixing area previously stored in the vector database are deleted, and then the new template image and its bolt posture features are re-stored in the vector database.

[0076] Step S103: When there is no bolt loosening in the bolt fixing area, reconstruct the bolt posture features in the vector database and the bolt posture features in F1, so that the similarity between F1 and the reconstructed bolt posture features in the vector database is greater than P1.

[0077] This step takes into account that during the inspection of the steel structure, natural environmental factors such as dust, dirt, corrosion, and lighting can interfere with the acquisition of bolt posture characteristics. In other words, environmental interference will exist in different inspections, and the environmental interference is not constant. These environmental interferences will affect the acquisition of bolt posture characteristics, resulting in the inclusion of interference noise from the natural environment in the bolt posture characteristics, which cannot further accurately describe the bolt posture.

[0078] Based on this, after each inspection is completed, when any bolt fixing area is determined to be free of bolt loosening, the bolt posture characteristics are reconstructed, specifically including:

[0079] (1) First of all, it should be noted that for any image (including inspection images and template images), each dimension of the bolt posture feature represents the distribution of key point pairs under a certain distance feature and distribution direction feature (e.g., distribution probability); interference from the natural environment will cause the distribution of key points to change, thereby affecting the distribution of key point pairs under some distance features and distribution direction features.

[0080] For any bolt-fixed area determined not to have experienced bolt loosening, and for each inspection image F1 of that bolt-fixed area, a focus feature is randomly initialized for each inspection image F1. Each focus feature is a vector with the same dimension as the bolt posture feature of inspection image F1. The value of each dimension of each focus feature is within the range [0, 1], and the value of each dimension of each focus feature is used to describe whether to focus on the distribution of key point pairs under a certain distance feature and distribution direction feature in each inspection image F1.

[0081] For any inspection image F1 of the bolt fixing area, the bolt posture feature of the inspection image F1 is denoted as q1, and the bolt posture feature of the view matching image of the inspection image F1 (read from the vector database) is denoted as q2.

[0082] Each dimension of the features of interest in the inspection image F1 is multiplied by each dimension of q1 to obtain a first vector, and each dimension of the features of interest in the inspection image F1 is multiplied by each dimension of q2 to obtain a second vector. The cosine similarity between the first vector and the second vector is denoted as the first similarity between the inspection image F1 and the viewpoint matching image.

[0083] This process focuses on recalculating and evaluating the similarity between bolt posture features in the inspection image F1 and the viewpoint-matched image by focusing on features.

[0084] Obtain the mean of the first similarity between all inspection images F1 of the bolt fixing area and the image matching its viewpoint. The difference between this mean and P1 is recorded as the target value of the feature of interest.

[0085] Using the simulated annealing algorithm, the features of interest for each inspection image F1 when the target value is maximized are obtained. When the maximum value of the target value is greater than 0 (that is, when the mean is greater than P), the features of interest for each inspection image F1 are recorded as preferred features of interest.

[0086] (2) In the above process, by optimizing the features of interest, it is possible to use the optimized features of interest (i.e., preferred features of interest) to describe the environmental interference of distance features and distribution direction features in different dimensions. This helps to screen out the distance features and distribution direction features that can reflect the bolt posture, and to screen out the distance features and distribution direction features of bolt posture that are mixed with environmental interference. The screened distance features and distribution direction features can further characterize and reflect the bolt's non-loose state, avoiding the problem that when some distance features and distribution direction features in the bolt posture features are mixed with environmental interference noise, they cannot further describe and reflect the bolt's non-loose state. Specifically, the smaller the value of each dimension of the preferred features of interest, the more necessary it is to screen out the distance features and distribution direction features mixed with environmental interference in that dimension. The larger the value of each dimension, the more necessary it is to retain the distance features and distribution direction features in that dimension that are not mixed with environmental interference.

[0087] Specifically, when the maximum value of the target value is less than or equal to 0 (that is, when the mean is less than or equal to P), it means that the influence of environmental interference on the bolt posture characteristics can be ignored. In this case, the value of each dimension in the feature of interest is set to 1, and it is recorded as the preferred feature of interest for each inspection image F1.

[0088] Each dimension of the preferred feature of each inspection image F1 is multiplied by each dimension of the bolt posture feature of each inspection image F1 to obtain the reconstructed bolt posture feature of each inspection image F1.

[0089] Each dimension of the preferred feature of interest for each inspection image is multiplied by each dimension of the bolt pose feature of the viewpoint-matched image for each inspection image, resulting in the reconstructed bolt pose feature of the viewpoint-matched image. The reconstructed bolt pose feature of the viewpoint-matched image is equivalent to reconstructing the bolt pose feature of the template image stored in the vector database.

[0090] It should be noted that the values ​​of each dimension in the bolt posture features before reconstruction (i.e., the bolt posture features obtained according to step S101) describe the distribution of keypoint pairs under a certain distance feature and distribution direction feature. The reconstructed bolt posture features are equivalent to adjusting the distribution of keypoint pairs under some distance features and distribution direction features in the bolt posture features before reconstruction. When comparing the inspection image F1 and the template image in the vector database, more attention is paid to the distance features and distribution direction features between keypoints that can reflect the bolt posture, avoiding interference from keypoints caused by environmental disturbances that are not conducive to identifying the bolt posture.

[0091] Each inspection image F1 and its corresponding viewpoint matching image correspond to the same preferred feature of interest;

[0092] (3) Thus, after each inspection, for any bolt fixing area where no bolt loosening has occurred, the above process reconstructs the bolt posture features of each inspection image F1 within the bolt fixing area; at the same time, each inspection image F1 within the bolt fixing area corresponds to a preferred feature for reconstructing the bolt posture features; at the same time, the view matching image of each inspection image F1 also corresponds to the preferred feature, and the bolt posture features of the view matching image are reconstructed based on the preferred feature (that is, the bolt posture features of some template images are reconstructed).

[0093] The reconstructed bolt pose features of the template image are stored in a vector database, along with the preferred features of interest for the template image. Note that if each template image already has reconstructed bolt pose features or preferred features of interest stored in the vector database, the existing reconstructed bolt pose features or preferred features of interest are deleted before the newly obtained reconstructed bolt pose features and preferred features of interest from the above process are stored in the vector database.

[0094] Step S104: Obtain environmental features based on the differences between the reconstructed bolt posture features and the bolt posture features before all previous reconstructions.

[0095] After each inspection, for any bolted area where no bolts have become loose, continue with the following steps:

[0096] The reconstructed bolt posture features of each inspection image F1 within the bolt fixing area are denoted as f3, and each inspection image F1 also has pre-reconstruction bolt posture features (obtained in step S101), denoted as f4. The difference between f3 and f4 is calculated to obtain the environmental features of each inspection image F1, where the value of each dimension of the environmental feature is equal to the absolute value of the difference between the values ​​of f3 and f4 in the same dimension. For each inspection image F1, the viewpoint matching image corresponds to the same environmental feature as the inspection image F1. The environmental features of this viewpoint matching image are also stored in the vector database.

[0097] Thus, for each template image F1 of any bolt fixing area, the data stored in the vector database for each template image includes: the template image itself, the reconstructed bolt posture features of the template image, and the environmental features of the template image each time the bolt fixing area is determined to be free of bolt loosening.

[0098] Specifically, the vector database contains the following two parts of template images:

[0099] (1) Template images of bolt-fixed areas that are determined not to have experienced bolt loosening are not the template images of the view matching images of inspection image F1;

[0100] (2) Template image of the bolt fixing area where bolt loosening is determined;

[0101] These two template images are not involved in the calculations of steps S103 and S104, meaning that these two template images do not have corresponding preferred features of interest, reconstructed bolt posture features, and environmental features. In this embodiment, the preferred features of interest corresponding to these two template images are set to vectors with all dimensions equal to 1, and the reconstructed bolt posture features are equal to the bolt posture features before reconstruction. In particular, these template images do not have corresponding environmental features.

[0102] Step S105: After a preset time interval, use a drone to inspect each bolt fixing area again and collect inspection image F2. The similarity between F2 and the reconstructed bolt posture features in the vector database is recorded as the bolt tightening index P2 for each bolt fixing area.

[0103] Steps S102 to S104 above pertain to the processing steps of the first inspection (equivalent to initializing the content stored in the vector database). This step and its subsequent steps pertain to the processing steps of subsequent inspections (equivalent to updating the content stored in the vector database), specifically as follows: Figure 2 As shown, it specifically includes:

[0104] After another preset time interval, the drone is used to inspect each bolt fixing area again and collect several inspection images of each bolt fixing area, which are represented as F2.

[0105] For any inspection image F2 of each bolt fixing area, for any template image in the vector database, the preferred interest feature of the template image is read from the vector database. Each dimension of the preferred interest feature is multiplied by each dimension of the bolt posture feature in the inspection image F2 (obtained according to step S101) to obtain f1. The reconstructed bolt posture feature of the template image is read from the database and represented as f2.

[0106] The cosine similarity between f1 and f2 is denoted as the reconstructed similarity between any inspection image F2 and any template image in each bolt fixing area.

[0107] The reconstructed similarity refers to the similarity of bolt posture features after enhancing the distance and distribution direction features between key points that can reflect bolt posture and reducing the interference of key points that are not conducive to identifying bolt posture caused by environmental interference. This can further accurately describe and compare the bolt posture in the inspection image F2 and the template image.

[0108] Furthermore, for each bolt-fixed area, there are all inspection images F2, and all template images for each bolt-fixed area read from the vector database.

[0109] All inspection images F2 are matched with all read template images using KM matching to obtain all matching pairs. Each matching pair contains an inspection image F2 and a template image. The inspection image F2 and the template image in each matching pair obtained by KM matching have the highest reconstruction similarity. Each matching pair contains an inspection image F2 and a template image from the same viewpoint. In this embodiment, the template image is denoted as the viewpoint matching image of inspection image F2.

[0110] The mean value of the reconstruction similarity between the F2 of all inspection images of each bolt fixing area and its view matching image is obtained. The larger the mean value, the more the interference of key points that are not conducive to identifying bolt posture caused by environmental interference is weakened. After that, the bolt posture in the bolt fixing area and the bolt posture in the template image are consistent in multiple views, indicating that there is no loosening.

[0111] This average value is recorded as the bolt tightening index P2 for each bolt fixing area.

[0112] When the bolt tightening index P2 is greater than the first preset threshold th1, it is determined that there is no bolt loosening within each bolt fixing area. Then, according to steps S103 and S104, the preferred interest features, reconstructed bolt posture features, and environmental features of each inspection image F2 and its view matching image are obtained and stored in the vector database (equivalent to updating the preferred interest features, reconstructed bolt posture features, and environmental features stored in the vector database). Then, the next inspection process is performed according to step S105.

[0113] When the bolt tightening index P2 is less than or equal to the first preset threshold th1, it indicates that after reducing the interference from key points that are unfavorable to bolt posture identification caused by environmental disturbances, there is still inconsistency between the bolt posture in the bolt fixing area and the bolt posture in the template image under some or all viewpoints. This inconsistency may be caused by bolt loosening under some or all viewpoints. Another possibility is that the reconstructed bolt posture features of F2 are obtained based on the preferred features of interest obtained in the previous inspection. For the current inspection process, the reconstructed bolt posture results of F2 are still subject to environmental noise interference.

[0114] Based on this, when the bolt tightening index P2 is less than or equal to the first preset threshold th1, the following processing is performed:

[0115] (1) Inspection processes that have been inspected in the past and have determined whether there is any loosening in each bolt fixing area are referred to as historical inspection processes;

[0116] Since multiple historical inspection processes may have been carried out previously, for each template image within each bolt fixing area, the vector database will store zero or more environmental features corresponding to each template image. These environmental features are calculated from several historical inspection processes.

[0117] (2) Calculate the mean of all environmental features of each template image to obtain the average environmental features. Normalize the dimension values ​​of all dimensions in the average environmental features using the softmax formula to obtain the inspection environment interference features of each template image.

[0118] (3) For any bolt fixing area where the bolt fastening index P2 is less than or equal to the first preset threshold th1, for each inspection image F2 of the bolt fixing area, and the view matching image of each inspection image F2, the inspection environment interference features corresponding to the view matching image are also recorded as the inspection environment interference features corresponding to each inspection image F2.

[0119] Specifically, when the template image has no corresponding environmental features (i.e., 0 corresponding environmental features), the inspection environment interference features of the template image are not calculated. At this time, the inspection image F2, which is at the same viewpoint as the template image, also no longer has corresponding environmental interference features.

[0120] Specifically, for bolt-fixed areas where bolt tightening index P2 is less than or equal to the first preset threshold th1, and for all inspection images F2 within the bolt-fixed area corresponding to the view matching images, if none of these view matching images store the corresponding environmental features in the vector database, it is determined that the bolt in the bolt-fixed area has become loose. In this case, the subsequent steps are not executed, but the next inspection process is carried out according to step S105.

[0121] Step S106: Remove the inspection environment interference features from the reconstructed bolt posture features in F2 and obtain the target bolt posture features.

[0122] For bolt-fixed areas where bolt fastening index P2 is less than or equal to the first preset threshold th1, this embodiment obtains the inspection environment interference characteristics of each inspection image F2 for each inspection image F2 according to step S105.

[0123] For the bolt posture features of each inspection image F2, the inspection environment interference features are obtained based on all historical inspection processes. They describe whether the distance and distribution direction features of each dimension can stably reflect the bolt posture under diverse and variable environmental interference. The smaller the value of each dimension in the inspection environment interference features, the more stable the distance and distribution direction features corresponding to each dimension can reflect the bolt posture under variable and uncertain environmental interference. The larger the value of each dimension, the more uncertain the distance and distribution direction features corresponding to each dimension are under variable and uncertain environmental interference, and the more likely they are to produce errors when used to describe the bolt posture.

[0124] Based on this, this embodiment removes the inspection environment interference features from the reconstructed bolt posture features of each inspection image F2 to obtain the target bolt posture features of each inspection image F2. The target bolt posture features can further reflect the bolt posture and avoid environmental interference.

[0125] As an optional example, the target bolt attitude features for each inspection image F2 are obtained by removing environmental interference features from the reconstructed bolt attitude features of each inspection image F2. The methods include:

[0126] For each inspection image F2, the viewpoint matching image is read from the vector database to obtain the inspection environment interference features of that viewpoint matching image, and used as the inspection environment interference features of each inspection image F2.

[0127] The preferred features of interest for the viewpoint matching image are read from the vector database and used as the preferred features of interest for each inspection image F2. Each dimension of the preferred features of interest for each inspection image F2 is multiplied by each dimension of the bolt posture features of each inspection image F2 to obtain the reconstructed bolt posture features of each inspection image F2.

[0128] For each dimension of the inspection environment interference feature of each inspection image F2, select the N0 dimensions with the largest values, and delete these N0 dimensions in the reconstructed bolt posture feature of F2 to obtain the target bolt posture feature.

[0129] This embodiment uses N0=10 as an example. In other embodiments, N0 can be set to one-third of the number of bolt posture feature dimensions (and rounded up).

[0130] In special cases, some inspection images F2 do not have corresponding inspection environment interference features. In this case, the inspection environment interference features are not removed, but the reconstructed bolt posture features of the inspection image F2 are directly used as the target bolt posture features.

[0131] As a preferred example, the method for removing inspection environment interference features from the reconstructed bolt posture features of each inspection image F2 to obtain the target bolt posture features of each inspection image F2 includes:

[0132] For each inspection image F2, the viewpoint matching image is read from the vector database to obtain the inspection environment interference features of that viewpoint matching image, and used as the inspection environment interference features of each inspection image F2.

[0133] The preferred features of interest for the viewpoint matching image are read from the vector database and used as the preferred features of interest for each inspection image F2. Each dimension of the preferred features of interest for each inspection image F2 is multiplied by each dimension of the bolt posture features of each inspection image F2 to obtain the reconstructed bolt posture features of each inspection image F2.

[0134] Furthermore, the cosine similarity between the preferred features of each inspection image F2 and the inspection environment interference features of each inspection image F2 is obtained and denoted as the reconstruction error.

[0135] The larger the reconstruction error, the more inconsistent the environmental interference from the distance and distribution direction features of different dimensions obtained from the previous inspection is with the stability of the bolt posture description based on these features. For example, dimensions that need to be retained (i.e., dimensions with larger values ​​in the preferred features) are easily affected by environmental interference and changes, leading to uncertainty (i.e., the inspection environmental interference features have larger values ​​in this dimension). Conversely, dimensions that can stably describe the bolt posture (i.e., dimensions with smaller values ​​in the inspection environmental interference features) need to be removed (i.e., the preferred features have smaller values ​​in this dimension). In this case, it indicates that after reconstructing the bolt posture features of F2 based on the preferred features obtained from the previous inspection, the reconstructed bolt posture features contain severe noise interference, requiring the removal of more inspection environmental interference features.

[0136] Conversely, the smaller the reconstruction error, the more consistent the environmental interference from the distance and distribution direction features of different dimensions obtained from the previous inspection is with the stable description of the bolt posture by the distance and distribution direction features of different dimensions. This indicates that after reconstructing the bolt posture features of F2 based on the preferred features of interest obtained from the previous inspection, the noise interference in the reconstructed bolt posture features is not serious. At this time, it is not necessary to remove too many inspection environmental interference features to ensure that the reconstructed bolt posture features have sufficient ability to describe the bolt posture.

[0137] Based on this, in this preferred example, for the value of each dimension in the inspection environment interference feature, the N1 dimensions with the largest values ​​are selected, and these N1 dimensions are deleted from the bolt posture features reconstructed in F2 to obtain the target bolt posture features.

[0138] N1 is positively correlated with the reconstruction error. The smaller the reconstruction error, the smaller N1, in which case it is not necessary to remove too many inspection environment interference features to ensure that the reconstructed bolt posture features have sufficient ability to describe the bolt posture; the larger the reconstruction error, the larger N1, in which case it is necessary to remove more inspection environment interference features to ensure that the reconstructed bolt posture features are not affected by the environment.

[0139] As an example, N1 is positively correlated with the reconstruction error, as shown in the following formula: Where g represents the reconstruction error, This indicates rounding up to the nearest integer.

[0140] Step S107: Identify bolt loosening based on the target bolt posture characteristics.

[0141] For any bolt-fixed region where the bolt tightening index P2 is less than or equal to the first preset threshold th1, and for all inspection images F2 within that bolt-fixed region, the reconstructed bolt posture features of the matched viewpoint image of each inspection image F2 are denoted as Q1, and the target bolt posture features of each inspection image F2 are denoted as Q2. The dimensions of Q2 and Q1 are different. The dimensions that were deleted in Q2 are obtained, and these dimensions are also deleted in Q1. Then, the cosine similarity between Q2 and Q1 is obtained and denoted as the second similarity of each inspection image F2. The second similarity of all inspection images F2 within that bolt-fixed region is... The mean similarity is recorded as the bolt tightness index P3 of the bolt fixing area. When the bolt tightness index P3 is greater than the first preset threshold th1, it is determined that there is no bolt loosening in the bolt fixing area. Then, according to the methods of steps S103 and S104, the preferred interest features, reconstructed bolt posture features and environmental features of each inspection image F2 and its view matching image are obtained and stored in the vector database (equivalent to updating the preferred interest features, reconstructed bolt posture features and environmental features stored in the vector database). Then, the next inspection process is carried out according to step S105.

[0142] When the bolt tightening index P3 is less than or equal to the first preset threshold th1, it is determined that the bolt in the bolt fixing area is loose, and then the next inspection process is carried out according to step S105.

[0143] This concludes the example.

[0144] Example 2: Step S101 of Example 1 includes: segmenting the connected regions of the bolt fixing area in each template image, referred to as the bolt fixing connected region; and includes: segmenting the connected regions where all bolts are located within the bolt fixing connected region.

[0145] As an example, this embodiment utilizes a semantic segmentation convolutional neural network (such as the DeepLabV3 network) for segmentation. The specific segmentation process is well-known and will not be described in detail in this embodiment.

[0146] Step S101 in Example 1 includes: plotting the distance features of all keypoint pairs in S1 into a histogram, denoted as the distance histogram. As an example, the specific process includes:

[0147] Let V be the maximum width and height of the smallest unconnected rectangle in the bolt-fixed connected region;

[0148] The ratio of the distance feature of each keypoint pair in S1 to V is denoted as the normalized distance feature of each keypoint pair. The purpose of using V as the denominator to calculate the ratio is to normalize the distance feature.

[0149] The normalized distance features of all key point pairs in S1 are divided into several first intervals. In this embodiment, the first intervals include: [0, 0.1), [0.1, 0.2), [0.2, 0.3), ..., [0.9, 0.1].

[0150] The frequency of occurrence of all normalized distance features in each first interval is counted. The center value of each first interval is used as the x-axis (representing different distance features), and the frequency of occurrence is used as the y-axis. Thus, the distance features of all key point pairs in S1 are plotted into a histogram, which is denoted as the distance histogram.

[0151] Step S101 in Example 1 includes: plotting the distribution direction features of all keypoint pairs into a histogram, denoted as the distribution histogram. As an example, the specific process includes:

[0152] The ratio of the distribution direction feature of each keypoint pair in S1 to 360 degrees is denoted as the normalized distribution direction feature of each keypoint pair. The purpose of using 360 degrees as the denominator to calculate the ratio is to normalize the distribution direction feature.

[0153] The normalized distribution direction features of all key point pairs in S1 are divided into several second intervals. In this embodiment, the second intervals include: [0, 0.1), [0.1, 0.2), [0.2, 0.3), ..., [0.9, 0.1].

[0154] The frequency of occurrence of all normalized distribution direction features in each second interval is counted. The center value of each second interval is used as the x-axis (representing different distribution direction features), and the frequency of occurrence is used as the y-axis. Thus, the distribution direction features of all key point pairs in S1 are plotted into a histogram, which is denoted as the distribution histogram.

[0155] Step S101 in Embodiment 1 includes: detecting key points in the template image. As an example, the specific process includes:

[0156] The Harris corner detection algorithm is used to obtain all corner points in the template image, and each corner point is recorded as a key point.

[0157] In other examples, the Canny edge detection algorithm is used to obtain all edge points in the template image, and each edge point is recorded as a keypoint. Still other examples utilize keypoint detection networks (such as the DKDNet network) to obtain keypoints.

[0158] In other examples, the key points obtained from all the above examples can also be used as the key points of this example.

[0159] Example 3: This example provides a pre-assembled steel structure bolt loosening identification device. The device includes a drone. The camera on the drone is used to collect template images and inspection images. The drone also includes a memory, a processor, and a computer program stored in the memory and run on the processor. The processor reads the template images and inspection images collected by the camera, and executes all the steps of all the above examples when running the computer program.

[0160] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for identifying loose bolts in pre-assembled steel structures, characterized in that, The method includes the following steps: Template images are acquired for each bolt fixing area, and bolt posture features are extracted from the template images; each bolt fixing area is inspected using a drone, and inspection images are acquired, denoted as F1; The similarity between the bolt posture features of the inspection image F1 and the bolt posture features of the template image is used as the bolt tightness index P1 for each bolt fixing area. When P1 is greater than the first preset threshold, each bolt fixing area is marked as a bolt not loosened area. Furthermore, the bolt posture features of F1 and the template image are reconstructed so that the similarity between the reconstructed bolt posture features of F1 and the reconstructed bolt posture features of the template image is greater than P1. The process of reconstructing the bolt posture features of F1 and the template image, such that the similarity between the reconstructed bolt posture features of F1 and the reconstructed bolt posture features of the template image is greater than P1, includes: for any bolt fixing area determined to be not loose, and for each inspection image F1 within that bolt fixing area, a focus feature is randomly initialized for each inspection image F1; the bolt posture feature of the view matching image of each inspection image F1 is denoted as q2; the view matching image of each inspection image F1 refers to the template image at the same viewpoint as each inspection image F1; the focus feature of each inspection image F1 is of the same dimension as the bolt posture feature. The first vector is obtained by multiplying the features of interest in each inspection image F1 with q2 in the same dimension. The second vector is obtained by multiplying the features of interest in each inspection image F1 with q2 in the same dimension. The cosine similarity between the first vector and the second vector is denoted as the first similarity. The difference between the mean of the first similarity between all inspection images F1 and the view matching image within the bolt fixing area and P1 is denoted as the target value of the features of interest. The features of interest that maximize the target value and have a maximum value greater than 0 are denoted as the preferred features of interest for each inspection image F1, and also as the preferred features of interest for the view matching image of each inspection image F1. Based on the preferred features of interest, the bolt posture features of each inspection image F1 and the view matching image of each inspection image F1 are reconstructed. After a preset time interval, each bolt fixing area is inspected again using a drone and an inspection image is collected, denoted as F2. The similarity between the reconstructed bolt posture features of inspection image F2 and the reconstructed bolt posture features of the template image is recorded as the bolt tightness index P2 for each bolt fixing area. When P2 is less than or equal to a first preset threshold, the differences between the reconstructed bolt posture features of all previous inspections and the bolt posture features before all previous reconstructions are recorded as the inspection environment interference features of inspection image F2. After removing the inspection environment interference features from the reconstructed bolt posture features of inspection image F2, bolt loosening is identified.

2. The method for identifying loose bolts in pre-assembled steel structures according to claim 1, characterized in that, The similarity between the bolt posture features of the inspection image F1 and the bolt posture features of the template image is used as the bolt tightening index P1 for each bolt fixing area. The specific steps include the following: For each inspection image F1 within any bolt fixing area, calculate the similarity between the bolt posture features of each inspection image F1 and the bolt posture features of the viewpoint matching image of each inspection image F1. The average similarity between all inspection images F1 within any bolt fixing area and the viewpoint matching image is denoted as the bolt tightening index P1 of any bolt fixing area. The viewpoint matching image of each inspection image F1 refers to the template image that is at the same viewpoint as each inspection image F1.

3. The method for identifying loose bolts in pre-assembled steel structures according to claim 1, characterized in that, The specific steps for defining the differences between the bolt posture features after each reconstruction and the bolt posture features before each reconstruction during the historical inspection process as the inspection environment interference features of inspection image F2 are as follows: For any bolt-fixed area determined to be not loose after each historical inspection, the difference between the reconstructed bolt posture features and the unreconstructed bolt posture features of each inspection image F1 within that bolt-fixed area is recorded as the environmental feature of each inspection image F1, and also serves as the environmental feature of the viewpoint matching image of each inspection image F1; the viewpoint matching image of each inspection image F1 refers to the template image that is at the same viewpoint as each inspection image F1. After all historical inspections, the environmental features obtained from each template image are averaged and normalized to obtain the inspection environment interference features of each template image. These features are also used as the inspection environment interference features of inspection image F2, which is at the same viewpoint as each template image.

4. The method for identifying loose bolts in pre-assembled steel structures according to claim 1, characterized in that, The specific steps for removing the inspection environment interference features from the reconstructed bolt posture features of the inspection image F2 are as follows: For any bolt fixing area where P2 is less than or equal to the first preset threshold, the preferred attention feature of the viewpoint matching image of each inspection image F2 within the bolt fixing area is also used as the preferred attention feature of each inspection image F2. Here, the viewpoint matching image of each inspection image F2 refers to the template image that is at the same viewpoint as each inspection image F2. The reconstructed bolt posture feature of F2 is equal to the preferred interest feature of each inspection image F2 multiplied by the bolt posture feature in the same dimension; The cosine similarity between the preferred features of each inspection image F2 and the inspection environment interference features of each inspection image F2 is denoted as the reconstruction error. For the value of each dimension in the inspection environment interference features, the N1 dimensions with the largest values ​​are selected and deleted from the reconstructed bolt posture features of F2 to obtain the target bolt posture features of each inspection image F2.

5. The method for identifying loose bolts in a pre-assembled steel structure according to claim 4, characterized in that, The specific steps for identifying bolt loosening after removing the inspection environment interference features from the reconstructed bolt posture features of the inspection image F2 are as follows: For any bolt-fixed area where P2 is less than or equal to the first preset threshold, the reconstructed bolt posture feature of the matching viewpoint image of each inspection image F2 within the bolt-fixed area is denoted as Q1, and the target bolt posture feature of each inspection image F2 is denoted as Q2. After deleting the missing dimension of Q2 from Q1, the cosine similarity between Q2 and Q1 is obtained and denoted as the second similarity of each inspection image F2. The mean of the second similarities of all inspection images F2 within the bolt-fixed area is used as the bolt tightening index P3 of the bolt-fixed area. When the bolt tightening index P3 is greater than the first preset threshold, it is determined that no bolt has become loose in the bolt-fixed area; when the bolt tightening index P3 is less than or equal to the first preset threshold, it is determined that the bolt has become loose in the bolt-fixed area.

6. The method for identifying loose bolts in pre-assembled steel structures according to claim 1, characterized in that, The specific steps for extracting bolt posture features from the template image are as follows: The connected components of the bolt fixing region in each template image are segmented and referred to as the bolt fixing connected components. Within the bolt fixing connected components of each template image, key points in the template image are detected, and the connected components where all bolts are located within the bolt fixing connected components are segmented and referred to as the bolt connected components. All keypoints within and outside the bolt-fixed connected domain are paired to form a keypoint pair set, denoted as S1. The pose features of keypoint pair set S1 are obtained. All keypoints within and outside the bolt-fixed connected domain are paired with all keypoints within the bolt-fixed connected domain to form a keypoint pair set, denoted as S2. The pose features of keypoint pair set S2 are obtained. The pose features of keypoint pairs S1 and S2 are concatenated to form the bolt pose features in each template image.

7. The method for identifying loose bolts in pre-assembled steel structures according to claim 1, characterized in that, The specific steps involved in reconstructing the bolt pose features of each inspection image F1 and the viewpoint matching image of each inspection image F1 based on the preferred attention features are as follows: The preferred features of interest for each inspection image F1 are multiplied by the bolt posture features of each inspection image F1 in the same dimension to obtain the reconstructed bolt posture features for each inspection image F1. The preferred features of interest for each inspection image are multiplied in the same dimension by the bolt posture features of the viewpoint matching image for each inspection image to obtain the reconstructed bolt posture features of the viewpoint matching image.

8. The method for identifying loose bolts in a pre-assembled steel structure according to claim 6, characterized in that, The specific steps for obtaining the pose features are as follows: The distance between each keypoint pair in the keypoint pair set and the inclination angle of the line containing the keypoint pair are denoted as the distance feature and distribution direction feature of each keypoint pair. The distance features and distribution direction features of all keypoint pairs in the keypoint pair set are plotted as histograms and then stitched together to obtain the pose feature of the keypoint pair set.

9. A pre-assembled steel structure bolt loosening identification device, the device comprising a drone, a camera mounted on the drone for acquiring template images and inspection images, the drone also comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor reads the template image and inspection image captured by the camera, and executes all the steps of the pre-assembled steel structure bolt loosening identification method according to any one of claims 1 to 8 when running the computer program.

Images