Method and equipment for identifying looseness of bolts of pre-assembled steel structure

Abstract

The invention relates to the field of image processing, in particular to a pre-assembled steel structure bolt looseness identification method and device, and the method comprises the steps: obtaining a bolt posture feature of an inspection image F1 and a bolt fastening index P1 of a bolt posture feature of a template image; when P1 is greater than a first preset threshold value, reconstructing bolt attitude features of F1 and the template image, utilizing the unmanned aerial vehicle to inspect each bolt fixing area again and acquire an inspection image F2, and obtaining a bolt fastening index P2 of the reconstructed bolt attitude features of the inspection image F2 and the reconstructed bolt attitude features of the template image; and when P2 is smaller than or equal to a first preset threshold value, acquiring inspection environment interference features of the inspection image F2, and performing bolt looseness identification after the inspection environment interference features are removed from the reconstructed bolt posture features of the inspection image F2. According to the invention, the purpose of reliably identifying the looseness of the bolt under the interference of various and changeable environments is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, in particular to a pre-assembled steel structure bolt loosening identification method and device. BACKGROUND

[0002] In large public buildings, such as stadiums, airport terminals, exhibition centers, etc., large-span pre-assembled steel structures are widely used. These structures are usually assembled into a whole by a large number of bolts on site. The reliability of bolt connection is directly related to the safety of the whole structure. Among them, by regularly inspecting each bolt by a drone and identifying whether the bolt is loose, the risk of bolt connection can be known and solved in advance.

[0003] However, in the process of inspecting each bolt and identifying whether the bolt is loose, the natural environment of the steel structure is different at different times of inspection, and the environmental interference is not fixed. Dust, dirt, corrosion and light, etc. Natural environmental effects or natural environmental conditions will interfere with the bolt loosening identification result, resulting in misidentification problems. SUMMARY

[0004] To solve the above problems, the present application provides a pre-assembled steel structure bolt loosening identification method and device.

[0005] The pre-assembled steel structure bolt loosening identification method and device of the present application adopts the following technical scheme: An embodiment of the present application provides a pre-assembled steel structure bolt loosening identification method, which comprises the following steps: Collecting a template image in each bolt fixing area and extracting bolt posture features in the template image; using a drone to inspect each bolt fixing area and collect an inspection image F1; The similarity of the bolt posture features of the inspection image F1 and the bolt posture features of the template image is taken as a bolt tightening index P1 of each bolt fixing area; when P1 is greater than a first preset threshold, each bolt fixing area is marked as a bolt non-loose area; and the bolt posture features of F1 and the template image are reconstructed, so that the similarity of the reconstructed bolt posture features of F1 and the reconstructed bolt posture features of the template image is greater than P1; After a preset time interval, the drone is used to inspect each bolt fixing area again and collect an inspection image F2, and the similarity of the reconstructed bolt posture features of the inspection image F2 and the reconstructed bolt posture features of the template image is taken as a bolt tightening index P2 of each bolt fixing area; when P2 is less than or equal to the first preset threshold, the difference between all the reconstructed bolt posture features in the historical inspection process and all the pre-reconstructed bolt posture features is taken as an inspection environment interference feature of the inspection image F2, and the bolt loosening identification is performed after removing the inspection environment interference feature from the reconstructed bolt posture features of the inspection image F2.

[0006] Preferably, the similarity of the bolt posture feature of the inspection image F1 and the bolt posture feature of the template image is taken as the bolt fastening indicator P1 of each bolt fixing area, and the specific steps include the following: For each inspection image F1 in any bolt fixing area, the similarity of the bolt posture feature of each inspection image F1 and the bolt posture feature of the perspective matching image of each inspection image F1 is calculated, and the average of the similarities of all inspection images F1 in any bolt fixing area and the perspective matching image is taken as the bolt fastening indicator P1 of any bolt fixing area; the perspective matching image of each inspection image F1 refers to the template image at the same perspective as each inspection image F1.

[0007] Preferably, the bolt posture features of the reconstruction F1 and the template image are reconstructed so that the similarity of the reconstructed bolt posture features of F1 and the template image is greater than P1, and the specific steps include the following: For any bolt fixing area determined to be not loose, and for each inspection image F1 in the bolt fixing area, a focus feature is randomly initialized for each inspection image F1; the bolt posture feature of the perspective matching image of each inspection image F1 is denoted as q2; the perspective matching image of each inspection image F1 refers to the template image at the same perspective as each inspection image F1. The focus feature of each inspection image F1 is multiplied with the bolt posture feature in the same dimension to obtain a first vector, and the focus feature of each inspection image F1 is multiplied with q2 in the same dimension to obtain a second vector, and the cosine similarity of the first vector and the second vector is denoted as a first similarity; the difference between the average of the first similarities of all inspection images F1 in the bolt fixing area and the perspective matching image and P1 is taken as the target value of the focus feature; the focus feature that maximizes the target value and has a maximum value greater than 0 is taken as the preferred focus feature of each inspection image F1, and also as the preferred focus feature of the perspective matching image of each inspection image F1; Preferably, the bolt posture features of each inspection image F1 and the perspective matching image of each inspection image F1 are reconstructed based on the preferred focus feature, and the specific steps include the following: The preferred focus feature of each inspection image F1 is multiplied with the bolt posture feature of each inspection image F1 in the same dimension to obtain the reconstructed bolt posture feature of each inspection image F1. The preferred focus feature of each inspection image is multiplied with the bolt posture feature of the perspective matching image of each inspection image in the same dimension to obtain the reconstructed bolt posture feature of the perspective matching image.

[0008] Preferably, the difference between the reconstructed bolt posture feature of each time in the historical inspection process and the bolt posture feature before reconstruction is recorded as the inspection environment interference feature of the inspection image F2, and the specific steps include the following: For any bolt fixing area determined as the bolt not loosened after each historical inspection, the difference between the reconstructed bolt posture feature of each inspection image F1 and the bolt posture feature before reconstruction in the bolt fixing area is recorded as the environment feature of each inspection image F1, which is also the environment feature of the perspective matching image of each inspection image F1; the perspective matching image of each inspection image F1 refers to the template image at the same perspective as each inspection image F1; The environment features of each template image obtained after all historical inspections are averaged and normalized to obtain the inspection environment interference feature of each template image, which is also the inspection environment interference feature of the inspection image F2 at the same perspective as each template image.

[0009] Preferably, the inspection environment interference feature is removed from the reconstructed bolt posture feature of the inspection image F2, and the specific steps include the following: For any bolt fixing area with P2 less than or equal to the first preset threshold, the preferred attention feature of the perspective matching image of each inspection image F2 in the bolt fixing area is also the preferred attention feature of each inspection image F2, wherein the perspective matching image of each inspection image F2 refers to the template image at the same perspective as each inspection image F2; The reconstructed bolt posture feature of F2 is equal to the preferred attention feature of each inspection image F2 multiplied by the bolt posture feature in the same dimension; The cosine similarity between the preferred attention feature of each inspection image F2 and the inspection environment interference feature of each inspection image F2 is recorded as the reconstruction error; for each dimension value in the inspection environment interference feature, select the N1 dimensions with the maximum value and delete them from the reconstructed bolt posture feature of F2 to obtain the target bolt posture feature of each inspection image F2.

[0010] Preferably, the bolt loosening identification is performed after the inspection environment interference feature is removed from the reconstructed bolt posture feature of the inspection image F2, and the specific steps include the following: For any one bolt fixing region whose P2 is less than or equal to the first preset threshold value, the reconstructed bolt posture feature of the matching view image of each inspection image F2 in the bolt fixing region is denoted as Q1, the target bolt posture feature of each inspection image F2 is denoted as Q2, the cosine similarity between Q2 and Q1 is obtained after deleting the dimension missing in Q2 in Q1, denoted as the second similarity of each inspection image F2, and the mean value of the second similarities of all the inspection images F2 in the bolt fixing region is taken as the bolt fastening index P3 of the bolt fixing region. When the bolt fastening index P3 is greater than the first preset threshold value, it is determined that the bolt fixing region does not have bolt loosening; when the bolt fastening index P3 is less than or equal to the first preset threshold value, it is determined that the bolt fixing region has bolt loosening.

[0011] Preferably, the specific steps of extracting the bolt posture feature in the template image include the following: The connected domain of the bolt fixing region in each template image is segmented, denoted as the bolt fixing connected domain; in the bolt fixing connected domain of each template image, the key points in the template image are detected, and the connected domain where all the bolts in the bolt fixing connected domain are located is segmented, denoted as the bolt connected domain; All the key points in the bolt fixing connected domain and outside the bolt connected domain are combined in pairs to obtain a key point pair set S1, and the posture feature of the key point pair set S1 is obtained; all the key points in the bolt fixing connected domain and outside the bolt connected domain are combined with all the key points in the bolt connected domain in pairs to obtain a key point pair set S2, and the posture feature of the key point pair set S2 is obtained; the posture features of the key point pairs S1 and S2 are spliced together as the bolt posture feature in each template image.

[0012] Preferably, the specific steps of reconstructing the bolt posture feature of each inspection image F1 and the view matching image of each inspection image F1 based on the preferred attention feature include the following: The preferred attention feature of each inspection image F1 is multiplied with the bolt posture feature of each inspection image F1 in the same dimension to obtain the reconstructed bolt posture feature of each inspection image F1; The preferred attention feature of each inspection image is multiplied with the bolt posture feature of the view matching image of each inspection image in the same dimension to obtain the reconstructed bolt posture feature of the view matching image.

[0013] Preferably, the specific steps of obtaining the posture feature include the following: The distance between each key point pair in the key point pair set and the inclination angle of the straight line on which the key point pair is located are denoted as the distance feature and the distribution direction feature of each key point pair.

[0014] Another embodiment of the present application provides a pre-assembled steel structure bolt loosening identification device, which comprises a UAV, a camera carried by the UAV for collecting a template image and an inspection image, a memory, a processor and a computer program stored in the memory and executable on the processor, the processor reads the template image and the inspection image collected by the camera, and executes all steps of the above-mentioned pre-assembled steel structure bolt loosening identification method when the computer program is executed.

[0015] The technical scheme of the present application has the following advantages: When P1 is greater than the first preset threshold, the present application marks each bolt fixing area as a bolt non-loosening area, and reconstructs the bolt posture feature of F1 and the template image, so that the similarity of the reconstructed bolt posture feature of F1 and the reconstructed bolt posture feature of the template image is greater than P1. In this process, the bolt posture feature is reconstructed, the features that can reflect the bolt posture in the bolt posture feature are retained, and the features mixed with environmental interference are screened out. The reconstructed bolt posture feature can further characterize and reflect the non-loosening state of the bolt, avoiding the problem that the bolt posture feature cannot further describe the non-loosening state of the bolt when environmental interference noise is mixed in the bolt posture feature.

[0016] Further, when P2 is less than or equal to the first preset threshold, the present application records the difference between all the second reconstructed bolt posture features and all the first reconstructed bolt posture features in the historical inspection process as the inspection environmental interference feature of the inspection image F2, and removes the inspection environmental interference feature from the reconstructed bolt posture feature of the inspection image F2 for bolt loosening identification. The inspection environmental interference feature obtained in this process is based on all the historical inspection processes, and describes whether each dimension of the bolt posture feature can be used to stably reflect the bolt posture under various and changeable environmental interference. In this process, when the current inspection is performed again (i.e., when the inspection image F2 is collected), the bolt posture feature is not reconstructed based on the last inspection process (i.e., when the inspection image F1 is collected), but the reconstructed bolt posture feature still has environmental noise interference due to the inadaptation of the current inspection environment, further achieving the purpose of reliable evaluation of bolt loosening under various and changeable environmental interference. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0018] Figure 1 A flow chart of a pre-assembled steel structure bolt loosening identification method provided by an embodiment of the present application; Figure 2 A detailed flow chart of a pre-assembled steel structure bolt loosening identification method provided by an embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to further illustrate the technical means and effects adopted by the present application to achieve the predetermined purpose, the following will combine the drawings and preferred embodiments to specifically describe the pre-assembled steel structure bolt loosening identification method and device according to the present application, the specific implementation, structure, features and effects thereof in detail. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0021] The following will specifically describe the specific scheme of the pre-assembled steel structure bolt loosening identification method and device provided by the present application in combination with the drawings.

[0022] Embodiment one: please refer to Figure 1 which shows a flow chart of steps of a pre-assembled steel structure bolt loosening identification method provided by an embodiment of the present application, and the method comprises the following steps: Step S101, collecting a template image in each bolt fixing area, and storing the bolt posture features in the template image in a vector database.

[0023] The completed steel structure contains several bolt fixing areas, each bolt fixing area includes one or more than one bolt, and all bolts in each bolt fixing area are used to tightly fix different mutually separated steel structure components.

[0024] In each bolt fixing area of the steel structure just installed in this embodiment, each bolt is tightly fixed, and a man-controlled unmanned aerial vehicle flies over each bolt fixing area in turn, and a camera on the unmanned aerial vehicle collects a video of each bolt fixing area, which covers various perspectives of each bolt fixing area. In this embodiment, all frame images in the video are recorded as template images of each bolt fixing area, each template image corresponds to a perspective, and the posture information of the bolt when it is not loosened under a specific perspective is recorded in the template image. In this embodiment, the camera of the unmanned aerial vehicle collects 12 images per second, and each image is a gray image after gray processing.

[0025] In this embodiment, subsequent automatic inspection is performed by using the unmanned aerial vehicle every preset time, and whether the bolt in each bolt fixing area is loosened is determined by comparing the images collected in each bolt fixing area during subsequent inspection and the template images.

[0026] Before the unmanned aerial vehicle performs inspection in this embodiment, the bolt posture features in each template image are extracted first, and the bolt posture features in each template image are stored in a vector database, so that subsequent comparison and determination of whether the bolt is loosened can be based on the bolt posture features. All template images of each bolt fixing area are also stored in the vector database, and each template image and the bolt posture features therein correspond to a piece of data in the vector database. In this embodiment, all pieces of data stored in each bolt fixing area have consecutive ID numbers.

[0027] The storage and reading method of the vector database is known, and this embodiment does not make specific description.

[0028] The bolt posture features in each template image are obtained by image processing of each template image, and describe the positions and postures of all bolts relative to the steel structure in the bolt fixing area under a specific perspective; when the bolt is loosened, the bolt posture features will also change.

[0029] As an example, the method for obtaining the bolt posture features in each template image is as follows: (1) The connected domain of the bolt fixing area in each template image is segmented, which is briefly denoted as the bolt fixing connected domain.

[0030] In the bolt fixing connected domain of each template image, the key points in the template image are detected, and the connected domains where all bolts in the bolt fixing connected domain are located are segmented, which are denoted as bolt connected domains. The difference between the bolt fixing connected domain and the bolt connected domain is that the former includes not only the image information of the bolt but also the image information of the steel structure fixed by the bolt, and the latter only includes the image information of the bolt.

[0031] (2) For all key points in the bolt fixation connected domain and outside the bolt connected domain, these key points are combined in pairs to obtain a plurality of key point pairs, which form a key point pair set S1. The key point pair set S1 is processed as follows: For each key point pair in S1, the distance between each key point pair and the inclination angle of the straight line on which the key point pair is located are recorded as the distance feature and the distribution direction feature of each key point pair. The inclination angle refers to the counterclockwise inclination angle of the straight line relative to the horizontal line in the template image (the inclination angle is less than 180 degrees).

[0032] The distance features of all key point pairs in S1 are plotted into a histogram, which is recorded as a distance histogram. The horizontal coordinate of each curve point in the distance histogram is the distance feature, and the vertical coordinate is the distribution frequency of the distance feature.

[0033] Similarly, the distribution direction features of all key point pairs are plotted into a histogram, which is recorded as a distribution histogram. The horizontal coordinate of each curve point in the distribution histogram is the distribution direction feature, and the vertical coordinate is the distribution frequency of the distribution direction feature.

[0034] The sequence of the vertical coordinates of all curve points in the distance histogram and the sequence of the vertical coordinates of all curve points in the distribution histogram are spliced together and regarded as a vector, which is recorded as the pose feature of the key point pair set S1.

[0035] The pose feature of the key point pair set S1 describes the relative distribution of all key points in S1, and represents the pose information of the steel structure fixed by the bolt.

[0036] (3) All key points in the bolt fixation connected domain and outside the bolt connected domain are combined with all key points in the bolt connected domain in pairs to obtain a plurality of key point pairs again, which form a key point pair set S2. In each key point pair, one key point is in the bolt fixation connected domain and outside the bolt connected domain, and the other key point is in the bolt connected domain.

[0037] The pose feature of the key point pair set S2 is obtained according to the above method (2), which represents the pose information of the bolt relative to the steel structure.

[0038] (4) In this embodiment, the pose features of the key point pairs S1 and S2 are spliced together as the bolt pose feature in each template image. The bolt pose feature contains both the pose of the steel structure and the pose of the bolt relative to the steel structure, which ensures that the bolt pose feature can describe the pose information of the bolt under a specific viewing angle.

[0039] Step S102, using the unmanned aerial vehicle to inspect each bolt fixing area and collect the inspection image F1, and the similarity between the bolt posture feature of the inspection image F1 and the vector database as the bolt fastening index P1 of each bolt fixing area; when P1 is greater than the first preset threshold, each bolt fixing area is marked as a bolt unloosening area.

[0040] The embodiment carries out unmanned aerial vehicle inspection once after a preset time interval, and the preset time interval is half a month.

[0041] When the unmanned aerial vehicle is inspected, a fixed inspection path (that is, the flight path used when the template image is collected in step S101) is set for the unmanned aerial vehicle, and the unmanned aerial vehicle flies over each bolt fixing area along the path. When flying over each bolt fixing area, a video of each bolt fixing area is collected, and all frames of the video contain information of different perspectives of each bolt fixing area. In the embodiment, a plurality of frames (for example, 5 frames) of images are selected as inspection images at equal intervals from the video. In other embodiments, a plurality of frames of images can be randomly selected as inspection images; or all frames of the video can be used as inspection images.

[0042] At this point, a plurality of inspection images are collected under each bolt fixing area, and different inspection images include steel structure information at different perspectives.

[0043] For any one bolt fixing area, all the inspection images collected under the bolt fixing area are represented as F1; the bolt posture feature in each inspection image F1 is obtained (the same as step S101). All the template images of the bolt fixing area are read from the vector database.

[0044] The cosine similarity between the bolt posture feature of the inspection image F1 and the bolt posture feature of the template image is recorded as the similarity between the inspection image F1 and the template image.

[0045] All the inspection images F1 in the bolt fixing area are matched with all the read template images KM, to obtain all the matching pairs, each matching pair containing an inspection image F1 and a template image. The inspection image F1 and the template image obtained by KM matching in each matching pair have the maximum similarity, and each matching pair contains an inspection image F1 and a template image at the same perspective. In the embodiment, the template image is recorded as the perspective matching image of the inspection image F1.

[0046] Obtaining the mean value of the similarity between all the inspection images F1 in any one bolt fixing area and the view angle matching images, the greater the mean value, the more similar or the same the posture of the bolt in the inspection image F1 and the posture in the template image under the same view angle, and the less likely the bolt in the inspection image F1 is loose. The mean value is taken as the bolt fastening index P1 of the bolt fixing area in this embodiment, and the greater the bolt fastening index P1, the more likely that no loosening occurs.

[0047] When P1 is greater than the first preset threshold th1, it is determined that the bolt fixing area does not have a bolt loosening situation. When P1 is less than or equal to the first preset threshold th1, it is determined that the bolt fixing area has a bolt loosening situation.

[0048] This embodiment is described by taking th1 equal to 0.8 as an example, and the preferred value range of th1 in other embodiments is [0.7, 0.95].

[0049] When the bolt fixing area is determined to have a loosening situation, re-fastening is needed, and after the fastening is completed, the template image of the bolt fixing area is re-acquired, all the data of the bolt fixing area previously stored in the vector database are deleted, and then the re-acquired template image and the bolt posture features thereof are re-stored in the vector database.

[0050] Step S103, when the bolt fixing area does not have a bolt loosening, the bolt posture features in the vector database and the bolt posture features in F1 are reconstructed, so that the similarity between F1 and the bolt posture features in the vector database and the reconstructed bolt posture features is greater than P1.

[0051] This step takes into account that when the steel frame structure is inspected, the natural environment effects or natural environment conditions such as dust, dirt, corrosion and light will interfere with the acquisition results of the bolt posture features, or in other words, there will be environmental interference in different inspections, and the environmental interference is not fixed. These environmental interferences will affect the acquisition results of the bolt posture features, resulting in that the bolt posture features contain the interference noise of the natural environment and cannot further accurately describe the posture of the bolt.

[0052] Based on this, whenever a patrol is completed, when any one bolt fixing area is determined not to have a bolt loosening, the bolt posture features are reconstructed, specifically including: (1) First of all, for any one image (including an inspection image and a template image), each dimension of the bolt posture features therein represents the distribution (for example, the distribution probability) of the key point pairs under a certain distance feature and distribution direction feature; the interference of the natural environment will cause the distribution of the key points to change, and then affect the distribution of the key point pairs under a part of the distance features and the distribution direction features.

[0053] For any one bolt fixing area determined as no bolt loosening occurs, and for each inspection image F1 of the bolt fixing area, a focus feature is randomly initialized for each inspection image F1, each focus feature being a vector with the same dimension as the bolt posture feature of the inspection image F1. The value of each dimension of each focus feature is in the interval [0, 1], and the value of each dimension of each focus feature is used to describe whether to focus on the distribution of the key point pair in each inspection image F1 under a certain distance feature and distribution direction feature.

[0054] For any one inspection image F1 of the bolt fixing area, the bolt posture feature of the inspection image F1 is denoted as q1, and the bolt posture feature of the perspective matching image (read from the vector database) of the inspection image F1 is denoted as q2. Each dimension of the focus feature of the inspection image F1 is multiplied by each dimension of q1 to obtain a first vector, and each dimension of the focus feature of the inspection image F1 is multiplied by each dimension of q2 to obtain a second vector. The cosine similarity of the first vector and the second vector is denoted as the first similarity of the inspection image F1 and the perspective matching image. The process re-computes and evaluates the similarity between the bolt posture feature of the inspection image F1 and the perspective matching image through the focus feature.

[0055] The average of the first similarities of all inspection images F1 of the bolt fixing area and their perspective matching images is obtained, and the difference between the average and P1 is denoted as the target value of the focus feature.

[0056] Using the simulated annealing algorithm, the focus feature of each inspection image F1 is obtained when the target value is maximum. When the maximum value of the target value is greater than 0 (i.e., the average is greater than P), the focus feature of each inspection image F1 is denoted as the optimal focus feature.

[0057] (2) In the above process, the focus feature is optimized, so that the optimized focus feature (i.e., the optimal focus feature) can be used to describe the environmental interference of the distance feature and the distribution direction feature in different dimensions, which helps to screen out the distance feature and the distribution direction feature that can reflect the bolt posture, and exclude the distance feature and the distribution direction feature of the bolt posture mixed with environmental interference. The screened distance feature and distribution direction feature can further characterize and reflect the un-loosening state of the bolt, avoiding the problem that when part of the distance feature and the distribution direction feature in the bolt posture feature is mixed with environmental interference noise, it cannot further describe the un-loosening state of the bolt. Specifically, the smaller the value of each dimension of the optimal focus feature, the more it needs to exclude the distance feature and the distribution direction feature in that dimension mixed with environmental interference. The larger the value of each dimension, the more it needs to retain the distance feature and the distribution direction feature in that dimension without environmental interference.

[0058] In particular, when the maximum value of the target value is less than or equal to 0 (i.e., the mean value is less than or equal to P), it indicates that the influence of environmental interference on the bolt posture feature can be ignored, at this time, the value of each dimension of the attention feature is set to 1, and the preferred attention feature of each inspection image F1 is recorded.

[0059] Each dimension of the preferred attention feature of each inspection image F1 is multiplied by each dimension of the bolt posture feature of each inspection image F1, to obtain the reconstructed bolt posture feature of each inspection image F1.

[0060] Each dimension of the preferred attention feature of each inspection image is multiplied by each dimension of the bolt posture feature of the perspective matching image, to obtain the reconstructed bolt posture feature of the perspective matching image. The reconstructed bolt posture feature of the perspective matching image is equivalent to reconstructing the bolt posture feature of the template image stored in the vector database.

[0061] It should be noted that the value of each dimension in the bolt posture feature before reconstruction (i.e., the bolt posture feature obtained according to step S101) describes the distribution of key point pairs under a certain distance feature and distribution direction feature. The reconstructed bolt posture feature is equivalent to adjusting the distribution of key point pairs under part of the distance feature and distribution direction feature in the bolt posture feature before reconstruction, and more attention is paid to the distance feature and distribution direction feature between key points that can reflect the bolt posture when comparing the inspection image F1 and the template image in the vector database, avoiding the interference of key points caused by environmental interference that is not conducive to identifying the bolt posture.

[0062] Each inspection image F1 and its corresponding perspective matching image correspond to the same preferred attention feature; (3) So far, each time an inspection is performed, for any bolt fixation area in which no bolt loosening occurs, the above process reconstructs the bolt posture feature of each inspection image F1 in the bolt fixation area; at the same time, each inspection image F1 in the bolt fixation area corresponds to a preferred attention feature used to reconstruct the bolt posture feature; at the same time, the perspective matching image of each inspection image F1 also corresponds to the preferred attention feature, and the bolt posture feature of the perspective matching image is reconstructed based on the preferred attention feature (i.e., the bolt posture feature of part of the template image is reconstructed).

[0063] The reconstructed bolt posture feature of the template image is stored in the vector database, and the preferred attention feature of the template image is also stored in the database. Note that in the vector database, if the reconstructed bolt posture feature or the preferred attention feature has been stored before for each template image, the existing reconstructed bolt posture feature or the preferred attention feature is deleted, and then the latest reconstructed bolt posture feature and the preferred attention feature obtained in the above process are stored in the vector database.

[0064] In step S104, the environment feature is obtained according to the difference between the reconstructed bolt posture feature and all the pre-reconstructed bolt posture features.

[0065] After each inspection, for any bolt fixation area in which no bolt loosening occurs, the following processing is continued: The reconstructed bolt posture feature of each inspection image F1 in the bolt fixation area is denoted as f3, and each inspection image F1 also has a pre-reconstructed bolt posture feature (obtained by step S101) corresponding thereto, denoted as f4. The difference between f3 and f4 is calculated to obtain the environment feature of each inspection image F1, wherein the value of each dimension of the environment feature is equal to the absolute value of the difference between the values of the same dimension of f3 and f4. For the perspective matching image of each inspection image F1, the perspective matching image and the inspection image F1 correspond to the same environment feature. The environment feature of the perspective matching image is also stored in the vector database.

[0066] At this point, for each template image F1 of any bolt fixation area, each piece of data corresponding to each template image stored in the vector database includes: the template image itself, the reconstructed bolt posture feature of the template image, and the environment feature of the template image when the bolt fixation area is determined to have no bolt loosening each time.

[0067] In particular, the vector database contains the following two parts of template images: (1) Template images of the bolt fixation area determined to have no bolt loosening, which are not perspective matching images of the inspection image F1; (2) Template images in the bolt fixation area determined to have bolt loosening; The two parts of template images do not participate in the calculation of steps S103 and S104, that is, the two parts of template images do not have corresponding preferred attention features, reconstructed bolt posture features, and environment features. In this embodiment, the preferred attention features corresponding to the two parts of template images are set to vectors with all dimensions equal to 1, and the reconstructed bolt posture features thereof are equal to the pre-reconstructed bolt posture features. In particular, this part of template images does not have corresponding environment features.

[0068] Step S105, after a preset time interval, the unmanned aerial vehicle is used to re-inspect each bolt fixing area and collect inspection images F2, and the similarity of the bolt posture features in the vector database and after reconstruction, denoted as the bolt tightening indicator P2 of each bolt fixing area.

[0069] The above steps S102 to S104 are related to the processing steps of the first inspection process (equivalent to initializing the content stored in the vector database). The present step and the subsequent steps of the present step are related to the processing steps of the subsequent inspection process (equivalent to updating the content stored in the vector database), which are specifically as follows Figure 2 As shown in the figure, specifically includes: After a preset time interval, the unmanned aerial vehicle is used to re-inspect each bolt fixing area and collect a plurality of inspection images of each bolt fixing area, denoted as F2.

[0070] For any one inspection image F2 of each bolt fixing area, for any one template image in the vector database, read the preferred attention features of the template image from the vector database, multiply each dimension of the preferred attention features with each dimension of the bolt posture features (obtained according to step S101) in the inspection image F2 to obtain f1; read the reconstructed bolt posture features of the template image from the database, denoted as f2.

[0071] The cosine similarity of f1 and f2 is denoted as the reconstruction similarity between any one inspection image F2 of each bolt fixing area and any one template image.

[0072] The reconstruction similarity represents that after enhancing the distance features and distribution direction features between the key points that can reflect the bolt posture and weakening the key points that are not conducive to identifying the bolt posture caused by environmental interference, the similarity of the bolt posture features is improved, which can further accurately describe and compare the posture of the bolt in the inspection image F2 and the template image.

[0073] Further, for all inspection images F2 of each bolt fixing area and all template images of each bolt fixing area read from the vector database.

[0074] All inspection images F2 and all read template images are subjected to KM matching to obtain all matching pairs, each matching pair contains an inspection image F2 and a template image, the inspection image F2 and the template image in each matching pair obtained by KM matching have the maximum reconstruction similarity, and the inspection image F2 and the template image in each matching pair are in the same perspective. The present embodiment denotes the template image as the perspective matching image of the inspection image F2.

[0075] Obtain the average of the reconstruction similarities of all the inspection images F2 of each bolt fixing region and the view angle matching images, the greater the average, the more the interference caused by the environment is weakened, which is not conducive to identifying the key points of the bolt posture, and after the interference, the bolt posture in the bolt fixing region and the bolt posture in the template image are consistent under multiple view angles, indicating that there is no loosening.

[0076] Record the average as the bolt fastening index P2 of each bolt fixing region.

[0077] When the bolt fastening index P2 is greater than the first preset threshold th1, it is determined that there is no bolt loosening in each bolt fixing region. Then, the preferred attention features, the reconstructed bolt posture features and the environment features of each inspection image F2 and the view angle matching images are obtained according to the method of steps S103 and S104, and are stored in the vector database (equivalent to updating the preferred attention features, the reconstructed bolt posture features and the environment features stored in the vector database), and then the next inspection process is performed according to step S105.

[0078] When the bolt fastening index P2 is less than or equal to the first preset threshold th1, it is indicated that after the interference caused by the environment is weakened, which is not conducive to identifying the key points of the bolt posture, the bolt posture in the bolt fixing region and the bolt posture in the template image are inconsistent under some or all view angles, and this inconsistency may be caused by bolt loosening under some or all view angles. There is another case, the reconstructed bolt posture features of F2 are obtained on the basis of the preferred attention features obtained in the last inspection, and for the current inspection process, the reconstruction result of the bolt posture of F2 still exists environmental noise interference.

[0079] Based on this, when the bolt fastening index P2 is less than or equal to the first preset threshold th1, the following processing is performed: (1) For the inspection process that has been inspected in history and has determined whether each bolt fixing region has loosening result, it is simply recorded as historical inspection process. Since multiple historical inspection processes may have been performed before, for each template image in each bolt fixing region, 0 or more environment features corresponding to each template image are stored in the vector database, and these environment features are calculated from a number of historical inspection processes.

[0080] (2) Obtain the average of all the environment features of each template image to obtain the average environment feature, and normalize all the dimension values in the average environment feature using the softmax formula to obtain the inspection environment interference feature of each template image.

[0081] (3) For any one bolt fixing area whose bolt fastening index P2 is less than or equal to the first preset threshold th1, for each inspection image F2 of the bolt fixing area, and for the view angle matching image of each inspection image F2, the inspection environment interference feature corresponding to the view angle matching image is also recorded as the inspection environment interference feature corresponding to each inspection image F2.

[0082] In particular, when the template image does not have a corresponding environment feature (i.e., has 0 corresponding environment features), the inspection environment interference feature of the template image is not calculated, and at this time, the inspection image F2 at the same view angle as the template image no longer corresponds to an environment interference feature.

[0083] In particular, for a bolt fixing area whose bolt fastening index P2 is less than or equal to the first preset threshold th1, and for the view angle matching images corresponding to all inspection images F2 in the bolt fixing area, if these view angle matching images do not have corresponding environment features stored in the vector database, it is determined that the bolt fixing area has a bolt loosening condition, and at this time, the subsequent steps are no longer performed, and the next inspection process is performed according to step S105.

[0084] Step S106: removing the inspection environment interference feature from the reconstructed bolt posture feature in F2 to obtain a target bolt posture feature.

[0085] For a bolt fixing area whose bolt fastening index P2 is less than or equal to the first preset threshold th1, for each inspection image F2 of the bolt fixing area, the inspection environment interference feature of each inspection image F2 is obtained according to step S105 in this embodiment.

[0086] For the bolt posture feature of each inspection image F2, the inspection environment interference feature is obtained based on all historical inspection processes, and describes whether each dimension distance feature and distribution direction feature can be used to stably reflect the posture of the bolt under the influence of a variety of and variable environment interference; the smaller the value of each dimension in the inspection environment interference feature, the more stable the distance feature and the distribution direction feature corresponding to each dimension can be used to reflect the posture of the bolt under the influence of a variety of and variable environment interference; the larger the value of each dimension, the more uncertain the distance feature and the distribution direction feature corresponding to each dimension are under the influence of a variety of and variable environment interference, and errors are easily generated when used to describe the posture of the bolt.

[0087] Based on this, the inspection environment interference feature is removed from the reconstructed bolt posture feature of each inspection image F2 to obtain a target bolt posture feature of each inspection image F2, and the target bolt posture feature can further reflect the posture of the bolt and avoid the interference of the environment.

[0088] As an optional example, the inspection environment interference feature is removed from the reconstructed bolt posture feature of each inspection image F2 to obtain a target bolt posture feature of each inspection image F2, and the method comprises the following steps: For the view angle matching image of each inspection image F2, the inspection environment interference feature of the view angle matching image is read from the vector database and taken as the inspection environment interference feature of each inspection image F2.

[0089] The preferred attention feature of the view angle matching image is read from the vector database and taken as the preferred attention feature of each inspection image F2, each dimension of the preferred attention feature of each inspection image F2 is multiplied by each dimension of the bolt posture feature of each inspection image F2, and the reconstructed bolt posture feature of each inspection image F2 is obtained.

[0090] For the value of each dimension in the inspection environment interference feature of each inspection image F2, the N0 dimensions with the maximum value are selected, and the N0 dimensions are deleted from the reconstructed bolt posture feature of F2 to obtain the target bolt posture feature.

[0091] In this embodiment, N0=10 is taken as an example for description. In other embodiments, N0 can be set to one third of the number of bolt posture feature dimensions (and rounded up).

[0092] In particular, part of the inspection images F2 do not have corresponding inspection environment interference features, at this time, the inspection environment interference feature is not removed, and the reconstructed bolt posture feature of the inspection image F2 is directly taken as the target bolt posture feature.

[0093] As a preferred example, the inspection environment interference feature is removed from the reconstructed bolt posture feature of each inspection image F2 to obtain the target bolt posture feature of each inspection image F2, and the method comprises the following steps: For the view angle matching image of each inspection image F2, the inspection environment interference feature of the view angle matching image is read from the vector database and taken as the inspection environment interference feature of each inspection image F2.

[0094] The preferred attention feature of the view angle matching image is read from the vector database and taken as the preferred attention feature of each inspection image F2, each dimension of the preferred attention feature of each inspection image F2 is multiplied by each dimension of the bolt posture feature of each inspection image F2, and the reconstructed bolt posture feature of each inspection image F2 is obtained.

[0095] Further, the cosine similarity between the preferred attention feature of each inspection image F2 and the inspection environment interference feature of each inspection image F2 is obtained, denoted as a reconstruction error; The greater the reconstruction error is, the more inconsistent the environmental interference doped with the distance features and the distribution direction features of different dimensions based on the last inspection is with the stable description of the bolt posture by the distance features and the distribution direction features of different dimensions. For example, the dimension that needs to be reserved (i.e., the dimension with a larger value in the preferred attention feature) is easily affected by environmental interference and environmental changes and thus is uncertain (i.e., the value of the inspection environmental interference feature in the dimension is larger); for another example, the dimension that can stably describe the bolt posture (i.e., the dimension with a smaller value in the inspection environmental interference feature) needs to be filtered out (i.e., the value of the preferred attention feature in the dimension is smaller); at this time, it is indicated that the noise interference in the reconstructed bolt posture feature obtained by reconstructing the bolt posture feature of F2 based on the preferred attention feature obtained by the last inspection is serious, and more inspection environmental interference features need to be removed.

[0096] On the contrary, the smaller the reconstruction error is, the more consistent the environmental interference doped with the distance features and the distribution direction features of different dimensions based on the last inspection is with the stable description of the bolt posture by the distance features and the distribution direction features of different dimensions, which indicates that the noise interference in the reconstructed bolt posture feature obtained by reconstructing the bolt posture feature of F2 based on the preferred attention feature obtained by the last inspection is not serious, and too many inspection environmental interference features do not need to be removed to ensure that the reconstructed bolt posture feature has enough ability to describe the posture of the bolt.

[0097] Based on this, in the preferred example, for the value of each dimension in the inspection environmental interference feature, the N1 dimensions with the largest values are selected, and the N1 dimensions are deleted from the reconstructed bolt posture feature of F2 to obtain the target bolt posture feature.

[0098] The N1 is positively correlated with the reconstruction error. The smaller the reconstruction error is, the smaller the N1 is, and too many inspection environmental interference features do not need to be removed to ensure that the reconstructed bolt posture feature has enough ability to describe the posture of the bolt; the greater the reconstruction error is, the greater the N1 is, and more inspection environmental interference features need to be removed to ensure that the reconstructed bolt posture feature avoids being interfered by the environment.

[0099] As an example, the N1 is positively correlated with the reconstruction error, and the formula includes: ; wherein g represents the reconstruction error, represents the upward rounding.

[0100] In step S107, the bolt loosening recognition is performed according to the target bolt posture feature.

[0101] For any one bolt fixing region whose bolt fastening index P2 is less than or equal to the first preset threshold th1, and all the inspection images F2 in the bolt fixing region, the reconstructed bolt posture feature of the matching view image of each inspection image F2 is recorded as Q1, and the target bolt posture feature of each inspection image F2 is recorded as Q2, wherein the dimension of Q2 and the dimension of Q1 are different, the deleted dimensions in Q2 are obtained, these dimensions in Q1 are also deleted, and then the cosine similarity of Q2 and Q1 is obtained, recorded as the second similarity of each inspection image F2. The average of the second similarities of all the inspection images F2 in the bolt fixing region is recorded as the bolt fastening index P3 of the bolt fixing region. When the bolt fastening index P3 is greater than the first preset threshold th1, it is determined that the bolt fixing region does not have the bolt loosening condition. Then, the preferred attention features, the reconstructed bolt posture features and the environmental features of each inspection image F2 and its view matching image are obtained according to the method of steps S103 and S104, and are stored in the vector database (equivalent to updating the preferred attention features, the reconstructed bolt posture features and the environmental features stored in the vector database), and then the next inspection process is performed according to step S105.

[0102] When the bolt fastening index P3 is less than or equal to the first preset threshold th1, it is determined that the bolt fixing region has the bolt loosening condition, and then the next inspection process is performed according to step S105.

[0103] Thus, the embodiment ends.

[0104] In step S101 of embodiment one, the following steps are included: segmenting the connected domain of the bolt fixing region in each template image, which is simply recorded as the bolt fixing connected domain; and segmenting the connected domain where each bolt in the bolt fixing connected domain is located.

[0105] As an example, the embodiment uses a semantic segmentation convolutional neural network (such as DeepLabV3 network) for segmentation. The specific segmentation process is known, and the embodiment will not be described in detail.

[0106] In step S101 of embodiment one, the following steps are included: drawing the distance feature of all the key point pairs in S1 into a histogram, recorded as a distance histogram. As an example, the specific process includes: The maximum value of the width and height of the minimum unconnected rectangle of the bolt fixing connected domain is recorded as V; The ratio of the distance feature of each key point pair in S1 to V is recorded as the normalized distance feature of each key point pair, wherein the purpose of taking V as the denominator to calculate the ratio is to normalize the distance feature.

[0107] The normalized distance features of all the key point pairs in S1 are divided into a plurality of first intervals, and the first intervals in this embodiment include: [0, 0.1], [0.1, 0.2], [0.2, 0.3], …, [0.9, 0.1].

[0108] The frequency of occurrence of all the normalized distance features in each first interval is counted, the center value of each first interval is taken as the abscissa (representing different distance features), and the frequency of occurrence is taken as the ordinate, so that the distance features of all the key point pairs in S1 are plotted into a histogram, denoted as a distance histogram.

[0109] The step S101 of the embodiment one includes: plotting the distribution direction features of all the key point pairs into a histogram, denoted as a distribution histogram, and as an example, the specific process includes: The ratio of the distribution direction feature of each key point pair in S1 to 360 degrees is denoted as the normalized distribution direction feature of each key point pair, and the purpose of taking 360 degrees as the denominator to calculate the ratio is to normalize the distribution direction feature.

[0110] The normalized distribution direction features of all the key point pairs in S1 are divided into a plurality of second intervals, and the second intervals in this embodiment include: [0, 0.1], [0.1, 0.2], [0.2, 0.3], …, [0.9, 0.1].

[0111] The frequency of occurrence of all the normalized distribution direction features in each second interval is counted, the center value of each second interval is taken as the abscissa (representing different distribution direction features), and the frequency of occurrence is taken as the ordinate, so that the distribution direction features of all the key point pairs in S1 are plotted into a histogram, denoted as a distribution histogram.

[0112] The step S101 of the embodiment one includes: detecting the key points in the template image, and as an example, the specific process includes: All the corner points in the template image are obtained using the Harris corner point detection algorithm, and each corner point is denoted as a key point.

[0113] In other examples, all the edge points in the template image are obtained using the Canny edge detection algorithm, and each edge point is denoted as a key point. In some examples, the key points are obtained using a key point detection network (such as a DKDNet network).

[0114] In other examples, the key points obtained by all the above examples can be used as the key points of this example.

[0115] Embodiment three: the embodiment provides a pre-assembled steel structure bolt loosening identification device, the device comprises a unmanned aerial vehicle, a camera carried by the unmanned aerial vehicle is used for collecting a template image and an inspection image, the unmanned aerial vehicle further comprises a memory, a processor and a computer program stored in the memory and executable on the processor, the processor reads the template image and the inspection image collected by the camera, and when the computer program is executed, all steps of all embodiments described above are executed.

[0116] The above merely describes preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for identifying loose bolts in pre-assembled steel structures, characterized in that: The method comprises the following steps: A template image is collected at each bolt fixing area, and the bolt posture features in the template image are extracted; a drone is used to inspect each bolt fixing area and collect inspection images, which are represented as F1; The similarity between the bolt posture features of the inspection image F1 and the bolt posture features of the template image is used as the bolt tightening index P1 of each bolt fixing area; when P1 is greater than a first preset threshold, each bolt fixing area is marked as a bolt-not-loose area; and the bolt posture features of F1 and the template image are reconstructed so that the similarity between the reconstructed bolt posture features of F1 and the reconstructed bolt posture features of the template image is greater than P1; After a preset time interval, each bolt fixing area is inspected again by a drone and an inspection image is collected, which is represented as F2. The similarity between the reconstructed bolt posture features of the inspection image F2 and the reconstructed bolt posture features of the template image is recorded as the bolt tightening index P2 of each bolt fixing area; when P2 is less than or equal to the first preset threshold, the difference between the bolt posture features after all reconstructions in the historical inspection process and the bolt posture features before all reconstructions is recorded as the inspection environment interference feature of the inspection image F2, and the bolt loosening is identified after removing the inspection environment interference feature from the reconstructed bolt posture features of the inspection image F2.

2. A method for identifying loose bolts in pre-assembled steel structures according to claim 1, characterized in that: The similarity between the bolt posture features of the inspection image F1 and the bolt posture features of the template image is used as the bolt tightening index P1 of each bolt fixing area, and the specific steps include the following: For each inspection image F1 in any bolt fixing area, the similarity between the bolt posture features of each inspection image F1 and the bolt posture features of the perspective matching image of each inspection image F1 is calculated. The average of the similarities between all inspection images F1 in any bolt fixing area and the perspective matching image is recorded as the bolt tightening index P1 of any bolt fixing area; the perspective matching image of each inspection image F1 refers to a template image at the same perspective as each inspection image F1.

3. A method for identifying loose bolts in pre-assembled steel structures according to claim 1, characterized in that: The reconstructing of the bolt posture features of F1 and the template image so that the similarity between the reconstructed bolt posture features of F1 and the reconstructed bolt posture features of the template image is greater than P1 includes the following specific steps: For any bolt fixing area that is determined to be not loose, and for each inspection image F1 within the bolt fixing area, a focus feature is randomly initialized for each inspection image F1; the bolt posture feature of the view matching image of each inspection image F1 is recorded as q2; the view matching image of each inspection image F1 refers to a template image at the same view angle as each inspection image F1; The first vector is obtained by multiplying the focus feature of each inspection image F1 by the bolt posture feature of the same dimension. The second vector is obtained by multiplying the focus feature of each inspection image F1 by the same dimension as q2. The cosine similarity between the first vector and the second vector is recorded as the first similarity. The difference between the mean of the first similarities of all inspection images F1 and the view matching image in the bolt fixing area and P1 is recorded as the target value of the focus feature. The focus feature that maximizes the target value and has a maximum value greater than 0 is recorded as the preferred focus feature of each inspection image F1, and also as the preferred focus feature of the view matching image of each inspection image F1. The bolt posture features of each inspection image F1 and the perspective matching image of each inspection image F1 are reconstructed based on the preferred focus features.

4. A method for identifying loose bolts in pre-assembled steel structures according to claim 1, characterized in that: The specific steps of recording the differences between the bolt posture features after all reconstructions and the bolt posture features before all reconstructions in the historical inspection process as the inspection environment interference features of the inspection image F2 include: For any bolt fixing area that is determined to be intact after each historical inspection, the difference between the reconstructed bolt posture feature and the pre-reconstructed bolt posture feature of each inspection image F1 in the bolt fixing area is recorded as the environmental feature of each inspection image F1, and also serves as the environmental feature of the perspective matching image of each inspection image F1. The perspective matching image of each inspection image F1 refers to a template image at the same perspective as each inspection image F1. The environmental features obtained for each template image after all historical inspections are averaged and normalized to obtain the inspection environment interference features of each template image, which are also used as the inspection environment interference features of the inspection image F2 at the same viewing angle as each template image.

5. A method for identifying loose bolts in pre-assembled steel structures according to claim 3, characterized in that: The specific steps of removing the inspection environment interference features from the reconstructed bolt posture features of the inspection image F2 include the following: For any bolt fixing area where P2 is less than or equal to the first preset threshold, the preferred focus feature of the perspective matching image of each inspection image F2 in the bolt fixing area is also used as the preferred focus feature of each inspection image F2, wherein the perspective matching image of each inspection image F2 refers to a template image at the same perspective as each inspection image F2; The reconstructed bolt posture feature of F2 is equal to the multiplication of the preferred focus feature of each inspection image F2 and the bolt posture feature in the same dimension; The cosine similarity between the preferred focus feature of each inspection image F2 and the inspection environment interference feature of each inspection image F2 is recorded as the reconstruction error; for the value of each dimension in the inspection environment interference feature, the N1 dimensions with the largest values ​​are selected and deleted from the reconstructed bolt posture feature of F2 to obtain the target bolt posture feature of each inspection image F2.

6. A method for identifying loose bolts in pre-assembled steel structures according to claim 5, characterized in that: The bolt loosening identification is performed after removing the inspection environment interference features from the reconstructed bolt posture features of the inspection image F2, and the specific steps include the following: For any bolt fixing area where P2 is less than or equal to the first preset threshold, the reconstructed bolt posture feature of the matching perspective image of each inspection image F2 in the bolt fixing area is recorded as Q1, and the target bolt posture feature of each inspection image F2 is recorded as Q2. After deleting the missing dimension of Q2 from Q1, the cosine similarity of Q2 and Q1 is obtained, which is recorded as the second similarity of each inspection image F2. The average of the second similarities of all inspection images F2 in the bolt fixing area is used as the bolt tightening index P3 of the bolt fixing area. When the bolt tightening index P3 is greater than the first preset threshold, it is determined that no bolt loosening has occurred in the bolt fixing area; when the bolt tightening index P3 is less than or equal to the first preset threshold, it is determined that bolt loosening has occurred in the bolt fixing area.

7. A method for identifying loose bolts in pre-assembled steel structures according to claim 1, characterized in that: The specific steps of extracting the bolt posture features from the template image include the following: Segment the connected domain of the bolt fixing area in each template image, which is referred to as the bolt fixing connected domain. Within the bolt fixing connected domain of each template image, detect the key points in the template image and segment the connected domains where all the bolts in the bolt fixing connected domain are located, which are referred to as the bolt connected domain. All key points in and outside the bolt fixed connected domain are combined in pairs to obtain a key point pair set, denoted as S1, and the posture features of the key point pair set S1 are obtained; all key points in and outside the bolt fixed connected domain are combined in pairs with all key points in the bolt connected domain, and the obtained key point pair set is denoted as S2; the posture features of the key point pair set S2 are obtained; the posture features of the key point pairs S1 and S2 are spliced ​​together end to end as the bolt posture features in each template image.

8. A method for identifying loose bolts in pre-assembled steel structures according to claim 3, characterized in that: The method of reconstructing the bolt posture features of each inspection image F1 and the perspective matching image of each inspection image F1 based on the preferred focus features includes the following specific steps: Multiply the preferred focus feature of each inspection image F1 by the bolt posture feature of each inspection image F1 in the same dimension to obtain the reconstructed bolt posture feature of each inspection image F1; The preferred focus feature of each inspection image is multiplied with the bolt posture feature of the view matching image of each inspection image in the same dimension to obtain the reconstructed bolt posture feature of the view matching image.

9. A method for identifying loose bolts in pre-assembled steel structures according to claim 7, characterized in that: The specific steps for obtaining the posture features are as follows: The distance between each key point pair in the key point pair set and the inclination of the straight line where the key point pair is located are recorded as the distance feature and distribution direction feature of each key point pair; the distance features of all key point pairs in the key point pair set and the distribution direction features of all key point pairs in the key point pair set are plotted into histograms and spliced ​​to obtain the posture features of the key point pair set.

10. A device for identifying loose bolts in pre-assembled steel structures, comprising a drone, a camera mounted on the drone for collecting template images and inspection images, a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: The processor reads the template image and the inspection image captured by the camera, and executes all steps of the method for identifying loose bolts in a pre-assembled steel structure as described in any one of claims 1 to 9 when running the computer program.

Images