Bolt looseness detection method, system and equipment and computer readable storage medium

Abstract

The invention discloses a bolt looseness detection method, system and device and a computer readable storage medium. The detection method comprises the steps that a to-be-detected image of a target bolt cluster is collected; a plurality of regions of interest in the to-be-detected image and the reference image of the target bolt cluster are obtained, and each region of interest is a region where a single bolt is installed; and calculating a rotation angle of the bolt in each region of interest of the to-be-detected image relative to the bolt in the region of interest corresponding to the reference image, and determining whether the bolt is loosened based on the rotation angle. A plurality of regions of interest obtained from an image to be detected and a reference image are used as mounting regions of a single bolt, and the rotation angle of the bolt in each region of interest of the image to be detected relative to the bolt in the region of interest corresponding to the reference image is calculated; whether the bolt in the to-be-detected image is loosened or not can be judged in time through the rotation angle.

Description

Technical Field

[0001] The present application relates to the technical field of bolt detection, and in particular to a method, system, device and computer-readable storage medium for detecting loose bolts. Background Art

[0002] In steel structure projects, high-strength bolts are often used as connecting devices between structures because of their ease of use. During the service life of steel structure projects, due to factors such as extreme temperature and humidity fluctuations and dynamic loads, various defects that affect the safety of the structure will occur. Among them, high-strength bolts, as one of the main connection methods, will become loose or even fall off due to the above factors in large steel structure facilities such as bridges. This defect has a serious impact on the safety and stability of the facility, so it is crucial to accurately and quickly identify the defect during the inspection of the facility.

[0003] In traditional inspection schemes, mature detection methods for bolt defects mainly include manual methods and sensor-based methods. Among them, manual methods mainly include visual inspection, tapping test and torque wrench methods, but these methods are labor-intensive and highly dependent on the inspector's work experience. Sensor-based methods mainly include acoustic wave detection, guided wave detection, circuit impedance, potential drop method and subharmonic resonance method. However, some of these sensors are designed with a large number of nonlinear operations and are prone to introduce various noises (such as acoustic waves, guided wave methods, and subharmonic resonance methods); some are difficult to deploy on the spot due to their complex structures (such as circuit impedance method and potential drop method).

[0004] In recent years, the inspection strategy of large facilities has gradually developed in the direction of intelligence. Methods based on computer vision and artificial intelligence technology are gaining more and more attention in the current intelligent inspection. In related technologies, in order to adapt to the complex scenes of bolts, detectors based on convolutional neural networks are used to detect and classify these bolts, but the above method still cannot detect the changes when the bolts rotate slightly in time, resulting in untimely detection and increasing the safety risk of bridges. Summary of the invention

[0005] The present application provides a method, system, device and computer-readable storage medium for detecting loose bolts, which can solve the technical problem in the prior art that a detector based on a convolutional neural network is used to detect and classify these bolts, but still cannot timely detect changes when the bolts undergo slight rotations, resulting in untimely detection and increased safety risks of the bridge.

[0006] In a first aspect, an embodiment of the present application provides a method for detecting loose bolts, the detection method comprising: acquiring an image to be tested of a target bolt cluster; obtaining the image to be tested of the target bolt cluster and multiple regions of interest in a reference image, wherein each region of interest is an area where a single bolt is installed; calculating a rotation angle of the bolt in each region of interest of the image to be tested relative to the bolt in the region of interest corresponding to the reference image, and determining whether the bolt is loose based on the rotation angle.

[0007] In combination with the first aspect, in one embodiment, before obtaining the multiple regions of interest in the image to be tested and the reference image of the target bolt cluster, it includes: scaling the reference image and the image to be tested to a uniform size, converting the data format and performing normalization preprocessing.

[0008] In combination with the first aspect, in one embodiment, the calculation of the rotation angle of the bolt in each region of interest of the image to be tested relative to the bolt in the region of interest corresponding to the reference image includes: performing bipartite graph matching on the region of interest in the image to be tested and the corresponding region of interest in the reference image, and identifying the region of interest in which bolts are not missing in multiple regions of interest of the image to be tested; calculating the rotation angle of the bolt in the region of interest in the image to be tested where bolts are not missing relative to the bolt in the region of interest corresponding to the reference image.

[0009] In combination with the first aspect, in one embodiment, the calculation of the rotation angle of the bolt in the area of ​​interest where the bolts are not missing in the image to be tested relative to the bolts in each area of ​​interest corresponding to the reference image includes: obtaining the corner point coordinates of the bolts in the area of ​​interest where the bolts are not missing in the image to be tested and the corner point coordinates of the bolts in the area of ​​interest corresponding to the reference image; using perspective transformation to transform the corner point coordinates of the bolt into a top surface contour image of the bolt; calculating the rotation angle of the top surface contour image of the bolt in the area of ​​interest where the bolts are not missing in the image to be tested relative to the top surface contour image of the bolt in the area of ​​interest corresponding to the reference image.

[0010] In combination with the first aspect, in one embodiment, the bipartite graph matching is performed on the region of interest in the image to be tested and the corresponding region of interest in the reference image, and the region of interest in the multiple regions of interest in the image to be tested where bolts are not missing is identified, including: performing bipartite graph matching on the region of interest in the image to be tested and the corresponding region of interest in the reference image to obtain multiple region of interest pairs; identifying region of interest pairs where bolts are missing from the multiple region of interest pairs; removing the region of interest pairs where bolts are missing from the multiple regions of interest to obtain the region of interest in the multiple regions of interest in which bolts are not missing from the multiple regions of interest in the image to be tested.

[0011] In combination with the first aspect, in one embodiment, before calculating the rotation angle of the bolt top surface contour image in the area of ​​interest where the bolts are not missing in the image to be tested relative to the area of ​​interest corresponding to the reference image, it includes: taking out the area of ​​interest corresponding to the area of ​​interest where the bolts are not missing from the image to be tested and the reference image respectively, and scaling the area of ​​interest to a uniform size, converting the data format and performing normalization preprocessing.

[0012] In combination with the first aspect, in one embodiment, the preprocessing of scaling the reference image and the image to be tested to a uniform size, converting the data format and performing normalization includes: using a first operator to scale the reference image and the image to be tested respectively; using a second operator to convert the scaled reference image and the scaled image to be tested into floating-point formats respectively, and continuing to scale to obtain data tensors of the reference image and the image to be tested respectively, and obtaining the RGB three channels of the reference image and the image to be tested data tensors respectively; using a third operator to normalize the RGB three channels of the reference image data tensor and the RGB three channels of the image to be tested data tensor respectively.

[0013] In a second aspect, an embodiment of the present application provides a bolt loosening detection system, the detection system comprising: an image acquisition device, which is used to acquire an image to be tested of a target bolt cluster; a first detector, which is used to obtain the image to be tested of the target bolt cluster and multiple regions of interest in a reference image, wherein each region of interest is an area where a single bolt is installed; and a second detector, which is used to calculate the rotation angle of the bolt in each region of interest of the image to be tested relative to the bolt in the region of interest corresponding to the reference image, and determine whether the bolt is loose based on the rotation angle.

[0014] In a third aspect, an embodiment of the present application provides a bolt loosening detection device, which includes a processor, a memory, and a bolt loosening detection program stored in the memory and executable by the processor, wherein when the bolt loosening detection program is executed by the processor, the steps of the bolt loosening detection method as described above are implemented.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a bolt loosening detection program is stored, wherein when the bolt loosening detection program is executed by a processor, the steps of the bolt loosening detection method as described above are implemented.

[0016] The beneficial effects brought by the technical solution provided in the embodiments of the present application include: By using multiple regions of interest obtained from the image to be tested and the reference image as the installation area of ​​a single bolt, and calculating the rotation angle of the bolt in each region of interest of the image to be tested relative to the region of interest corresponding to the reference image, the rotation angle can be used to promptly determine whether the bolt in the image to be tested is loose, thereby solving the technical problem that the detection method in the related art cannot promptly detect changes when the bolt undergoes slight rotation, resulting in untimely detection and increased safety risks of the bridge. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a flow chart of an embodiment of a method for detecting bolt loosening of the present application; Figure 2 This is a flow chart of the bolt missing and loosening detection method for this application; Figure 3 This is a flow chart of the bolt missing detection method of this application; Figure 4 A schematic diagram of a process flow of another embodiment of a method for detecting bolt loosening of the present application; Figure 5 This is a schematic diagram of the process structure of corner point coordinate detection in this application; Figure 6 This is a schematic diagram of the hardware structure of a bolt loosening detection device involved in the embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0019] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0020] In a first aspect, an embodiment of the present application provides a method for detecting loose bolts.

[0021] In one embodiment, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the method for detecting bolt looseness of the present application. Figure 1 As shown, the detection methods for bolt loosening include: S1: Acquire the image to be tested of the target bolt cluster. Before acquiring the image to be tested, the acquisition of the reference image may have been completed.

[0022] S2: Acquire multiple regions of interest in the test image and the reference image of the target bolt cluster, wherein each region of interest is the area where a single bolt is installed. The first detector can be used to generate multiple regions of interest, which can be recorded as ROIs. It should be understood that the regions of interest can include areas where bolts exist or areas where bolts are missing.

[0023] S3: Calculate the rotation angle of the bolt in each area of ​​interest of the image to be tested relative to the bolt in the area of ​​interest corresponding to the reference image, and determine whether the bolt is loose based on the rotation angle, that is, compare the rotation angle of the bolt in one of the areas of interest in the image to be tested with the bolt in the corresponding area of ​​interest in the reference image, so as to determine whether the bolt is loose based on the rotation angle. In the embodiment of the present application, a threshold value can be set. When the rotation angle is greater than the threshold value, it is judged that the bolt is not loose. When the rotation angle is less than the threshold value, it is judged that the bolt is loose, and the angle of looseness is given at the same time. Specifically, the threshold value can be set to 5°. Through this threshold value, the situation where noise is mistakenly reported as loose due to the algorithm being too sensitive can be effectively reduced, and the influence of human hand-rotation errors on the results when making a data set can be reduced.

[0024] In this embodiment, by using multiple regions of interest obtained from the image to be tested and the reference image as the installation area of ​​a single bolt, the rotation angle of the bolt in each region of interest of the image to be tested relative to the bolt in the corresponding region of interest of the reference image is calculated. That is, the rotation angle of a single bolt in the target bolt cluster of the image to be tested can be determined by using the corresponding bolt in the target bolt cluster of the reference image. The rotation angle can be used to promptly determine whether the bolt in the image to be tested is loose, which solves the technical problem that the detection method in the related art cannot promptly detect the changes when the bolt undergoes a slight rotation, resulting in untimely detection and increased safety risks of the bridge.

[0025] Furthermore, in one embodiment, before obtaining multiple regions of interest in the image to be tested and the reference image of the target bolt cluster, it can include: respectively scaling the reference image and the image to be tested to a uniform size, converting the data format and performing normalization preprocessing, and the preprocessed image to be tested and the reference image can respectively obtain the overall matrix of the image to be tested and the overall matrix of the reference image.

[0026] In this embodiment, by preprocessing the image to be tested and the reference image respectively, the overall matrix of the processed image to be tested and the overall matrix of the reference image can be input into the first detector to obtain multiple regions of interest of the image to be tested and multiple regions of interest of the reference image respectively. This solves the problem in the related art that the overall cluster state of the target bolt cluster is directly detected and the state of a single bolt in the target bolt cluster cannot be well reflected.

[0027] Furthermore, in one embodiment, the step of calculating the rotation angle of the bolt in each region of interest of the image to be measured relative to the bolt in the region of interest corresponding to the reference image may include: S31: Perform bipartite graph matching on the region of interest in the image to be tested and the corresponding region of interest in the reference image, and identify the region of interest in the multiple regions of interest in the image to be tested where the bolts are not missing. When obtaining the multiple regions of interest in the image to be tested and the reference image of the target bolt cluster, the detection frame of the single bolt installation area and its corresponding category can be obtained respectively. By performing bipartite graph matching on the detection frame of the reference image and the detection frame of the image to be tested based on IoU, the matching object of the detection frame of each image to be tested corresponding to the detection frame of the reference image can be obtained.

[0028] S32: Calculate the rotation angle of the bolt in the region of interest where the bolt is not missing in the image to be tested relative to the region of interest corresponding to the reference image.

[0029] In this embodiment, by first identifying the region of interest of the non-missing bolts in the image to be tested, the amount of calculation of the subsequent rotation angle can be reduced. That is to say, at this time, only the bolt rotation angle of the identified region of interest of the non-missing bolts can be calculated, and the region of interest opposite to the region of interest of the non-missing bolts is the region of interest of the missing bolts. This solves the problem of directly detecting the overall cluster state of the target bolt cluster in the related technology, failing to better identify the state of a single bolt in the target bolt cluster, and failing to timely monitor the bolts that are not missing but have been angularly deflected.

[0030] Further, in one embodiment, see Figure 2 As shown, the step of calculating the rotation angle of the bolt in the region of interest where the bolt is not missing in the image to be tested relative to the bolt in each region of interest corresponding to the reference image may include: S321: Obtain the corner point coordinates of the bolts in the region of interest where the bolts are not missing in the image to be tested and the corner point coordinates of the bolts in the region of interest corresponding to the reference image. In the embodiment of the present application, the second detector can be used to obtain the corner point coordinates of the bolts in the required region of interest. At this time, the bolt is considered to have six corner points, and the number of corner point coordinates obtained is six. In some other embodiments, when the bolts used have other numbers of corner points, the corresponding number of corner point coordinates can also be obtained. In some other embodiments, it is not necessary to obtain all the corner point coordinates of each bolt, as long as the number of corner point coordinates obtained is sufficient to calculate its rotation angle.

[0031] S322: Use perspective transformation to transform the corner coordinates of the bolt into a bolt top surface contour map. At this time, the corner coordinates can be transformed into a regular hexagon using perspective transformation. It should be understood that the bolt top surface contour map should be a front view contour map of the bolt top.

[0032] S323: Calculate the rotation angle of the bolt top surface contour image in the region of interest where the bolts are not missing in the image to be tested relative to the bolt top surface contour image in the region of interest corresponding to the reference image.

[0033] In this embodiment, by obtaining the coordinates of each corner point of the bolt in the area of ​​interest where no bolts are missing, that is, the corresponding areas in the image to be tested and the reference image are all obtained. At this time, the area in the image to be tested and the corresponding area in the reference image can be regarded as an area of ​​interest pair, and the corner point coordinates of each bolt in the area of ​​interest pair are transformed into a bolt top surface contour image using perspective transformation. By subtracting the hexagonal rotation angle of the bolts in the area in the image to be tested and the bolts in the corresponding area in the reference image, the rotation angle of the bolts in the area of ​​interest in the image to be tested can be obtained, which solves the problem that the looseness of each individual bolt in the bolt cluster cannot be obtained in time in the related technology.

[0034] Furthermore, the benchmark image can be an image taken when confirming that the bolts in the target bolt cluster are not loose or falling off. Before taking the benchmark image, all the bolts in the current bolt group can be manually corrected until they are not loose. After obtaining the corner point coordinates of the bolts in each area of ​​interest in the benchmark image, the results obtained for each bolt can be saved as a template. Subsequently, only the corner points of the bolts in the corresponding area of ​​interest can be inferred. This method effectively reduces the repeated reasoning process and saves the reasoning time for obtaining the corner point coordinates.

[0035] Further, in one embodiment, see Figure 3 As shown, the bipartite graph matching of the region of interest in the image to be tested and the corresponding region of interest in the reference image, and identifying the region of interest in which the bolts are not missing among the multiple regions of interest in the image to be tested, may include: S311: Perform bipartite graph matching on the region of interest in the image to be tested and the corresponding region of interest in the reference image to obtain multiple pairs of regions of interest. Identify the region of interest pairs with missing bolts in the multiple pairs of regions of interest. In the embodiment of the present application, bipartite graph matching based on IoU can be performed on the region of interest of the reference image and the region of interest of the image to be tested to obtain the matching object of each region of interest of the image to be tested for the region of interest of the reference image, that is, obtain multiple pairs of regions of interest matching the image to be tested and the reference image. Preferably, after scaling the reference image and the image to be tested to a uniform size, converting the data format and performing normalization preprocessing, the overall matrices of the image to be tested and the reference image can be obtained respectively, and each overall matrix is ​​sent to the first detector. The first detector can be a YOLOv10 detector at this time. 2D target detection is performed on the overall matrix of the reference image and the overall matrix of the image to be tested respectively, so as to obtain multiple detection frames in the overall matrix, each detection frame corresponds to an region of interest, and each detection frame corresponds to a bolt and a hole respectively. When the corresponding region is a bolt, it is considered that the detection frame has a bolt, and when the corresponding region is a hole, it is considered that the detection frame is a hole. It should be noted that if the detection frame is identified as a hole in the image to be tested, it does not mean that the bolt is missing, because there may also be holes in the reference image. Therefore, the missing of the bolt should be compared with the detection frame in the image to be tested and the detection frame corresponding to the reference image. The detection frame in the image to be tested and the detection frame corresponding to the reference image can be regarded as a detection frame pair. For each detection frame pair that has completed the matching, its corresponding category is determined. If there is a detection frame pair in which the detection frame category from the reference image is a bolt, and the detection frame category from the image to be tested is a hole, then a missing judgment is given for that position, which is considered to be a bolt missing. For the detection frame pairs that are not missing, they will be temporarily stored and sent to the next stage of detection.

[0036] S312: removing pairs of regions of interest where bolts are missing from the multiple regions of interest, and obtaining regions of interest where bolts are not missing from the multiple regions of interest of the image to be tested.

[0037] In this embodiment, the regions of interest where bolts are not missing in the multiple regions of interest of the image to be tested are obtained by directly removing the pairs of regions of interest where bolts are missing in the multiple regions of interest, which can reduce the repeated matching process.

[0038] Further, in one embodiment, see Figure 4 and Figure 5As shown, before calculating the rotation angle of the top surface contour image of the bolt in the interest area of ​​the non-missing bolts in the image to be tested relative to the top surface contour image of the bolt in the interest area corresponding to the reference image, it can include: taking out the interest area corresponding to the non-missing bolt interest area from the image to be tested and the reference image respectively, and scaling the interest area to a uniform size, converting the data format and performing normalization preprocessing, that is, in the embodiment of the present application, before calculating the looseness of the bolts in the image to be tested, the acquired image to be tested and the reference image are first preprocessed and the preprocessing results are input into the first detector to generate the overall matrix of the bolt cluster of high-strength bolts, and then the overall matrix is ​​subjected to bipartite graph matching, the missing bolts are identified, and the interest area of ​​the non-missing bolts is obtained. Thereafter, the interest area of ​​the non-missing bolts is directly obtained from the image to be tested and the reference image to obtain the area corresponding to the interest area of ​​the non-missing bolts for preprocessing.

[0039] Furthermore, in one embodiment, the preprocessing of scaling the reference image and the image to be measured to a uniform size, converting the data format and normalizing them may include: S11: Use the first operator to scale the reference image and the image to be measured respectively.

[0040] S12: using a second operator to convert the scaled reference image and the scaled image to be tested into floating point formats respectively, and continue scaling to obtain data tensors of the reference image and the image to be tested respectively, and respectively obtain RGB three channels of the data tensors of the reference image and the image to be tested.

[0041] S13: The third operator is used to normalize the three RGB channels of the reference image data tensor and the three RGB channels of the image data tensor to be tested. The first operator can be the Resize operator, the second operator is the ToTensor operator, and the third operator can be the Normalize operator. Specifically, the Resize operator is first used to scale the image to a uniform size to adapt to the input of the next stage, and then the ToTensor operator is used to convert the image data format from 8-bit RGB to a 3-channel floating point number, and the integers from 0 to 255 are mapped to between 0 and 1. Finally, the mean and variance parameters obtained by statistics on ImageNet are used, and the Normalize operator is used to normalize the logarithm to obtain an overall matrix.

[0042] The bolt loosening detection method in this case, that is, the high-strength bolt disease computer vision detection method, has strong robustness and high accuracy. This detection method can handle irregularly shaped bolt groups and complete the unified detection of bolt missing and bolt loosening in one process, so that it can timely identify the phenomenon of small bolt rotation and report the serious problem of bolt falling off. This detection method has a high tolerance for the viewing angle and distance of bolt image acquisition, and can meet the detection of bolt loosening and missing in different weather and different paint colors.

[0043] In a second aspect, an embodiment of the present application also provides a bolt loosening detection system.

[0044] In one embodiment, a bolt loosening detection system may include: an image acquisition device, which is used to acquire the image to be tested of the target bolt cluster. Both the image to be tested and the reference image can be acquired by the image acquisition device. When acquiring the image to be tested and the reference image respectively, the image acquisition device can be acquired at similar points. When acquiring, it should be ensured that all the connected bolts in the target bolt cluster can be acquired. It allows a certain shooting angle, but it should be ensured that the nut of each bolt acquired can fully display the outline. In the embodiment of the present application, the image acquisition device adopts an industrial camera, which has a relatively low cost. Specifically, during the acquisition process of the image to be tested and the reference image, in order to determine the best installation point of the image acquisition device, the shooting function of the image acquisition device can be turned on to test whether the installation of each installation point is reasonable, whether the bolt cap of each bolt in the bolt group photographed can fully display the hexagonal outline, whether the bolt cap photographed is located in the center of the image, whether the imaging is normal, etc.

[0045] The detection of loose bolts may also include a first detector, which is used to obtain multiple regions of interest in the test image and the reference image of the target bolt cluster, wherein each region of interest is an area where a single bolt is installed. In an embodiment of the present application, the first detector may use a YOLOv10 detector, which has been fine-tuned on a bolt group dataset including laboratory environments and actual scenarios. The bolt group dataset is an image set consisting of images collected from multiple bolt clusters, with rectangular frames and key point annotations; a second detector, which is used to calculate the rotation angle of the bolt in each region of interest of the image to be tested relative to the bolt in the region of interest corresponding to the reference image, and determine whether the bolt is loose based on the rotation angle. In an embodiment of the present application, the second detector may be a PiPNet key point detector, which has also been fine-tuned on a bolt group dataset including laboratory environments and actual scenarios.

[0046] Among them, the functional implementation of each structure in the above-mentioned bolt loosening detection system corresponds to the various steps in the above-mentioned bolt loosening detection method embodiment, and its functions and implementation processes will not be repeated here one by one.

[0047] In a third aspect, an embodiment of the present application provides a bolt loosening detection device, which may be a device with data processing function such as a personal computer (PC), a laptop computer, or a server.

[0048] Reference Figure 6 , Figure 6 The hardware structure diagram of the bolt loosening detection device involved in the embodiment of the present application is shown in FIG. In the embodiment of the present application, the bolt loosening detection device may include a processor, a memory, a communication interface, and a communication bus.

[0049] The communication bus may be of any type and is used to interconnect the processor, the memory, and the communication interface.

[0050] The communication interface includes input / output (I / O) interface, physical interface and logical interface, etc., which are used to interconnect the devices inside the bolt loosening detection device, and to interconnect the bolt loosening detection device with other devices (such as other computing devices or user devices). The physical interface can be an Ethernet interface, a fiber optic interface, an ATM interface, etc.; the user device can be a display, a keyboard, etc.

[0051] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0052] The processor may be a general-purpose processor, which may call a bolt loosening detection program stored in a memory and execute the bolt loosening detection method provided in the embodiment of the present application. For example, the general-purpose processor may be a central processing unit (CPU). The method executed when the bolt loosening detection program is called may refer to the various embodiments of the bolt loosening detection method of the present application, which will not be described in detail here.

[0053] Those skilled in the art will understand that Figure 6 The hardware structure shown in the figure does not constitute a limitation on the present application, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0054] In a fourth aspect, an embodiment of the present application also provides a computer-readable storage medium.

[0055] The computer-readable storage medium of the present application stores a bolt loosening detection program, wherein when the bolt loosening detection program is executed by a processor, the steps of the bolt loosening detection method as described above are implemented.

[0056] Among them, the method implemented when the bolt loosening detection program is executed can refer to the various embodiments of the bolt loosening detection method of the present application, and will not be repeated here.

[0057] It should be noted that the serial numbers of the above-mentioned embodiments of the present application are only for description and do not represent the advantages or disadvantages of the embodiments.

[0058] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. The terms "first", "second" and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit "first", "second" and "third" to different types.

[0059] In the description of the embodiments of the present application, "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a specific way.

[0060] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; the “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0061] In some processes described in the embodiments of the present application, multiple operations or steps that appear in a specific order are included, but it should be understood that these operations or steps may not be executed in the order in which they appear in the embodiments of the present application or in parallel, and the sequence number of the operation is only used to distinguish the different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed in sequence or in parallel, and these operations or steps may be combined.

[0062] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD) as described above, and includes a number of instructions for a terminal device to execute the methods described in each embodiment of the present application.

[0063] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for detecting loose bolts, characterized in that: The detection method comprises: Collect the image of the target bolt cluster to be tested; Acquire multiple regions of interest in the image to be tested and the reference image of the target bolt cluster, wherein each region of interest is a region where a single bolt is installed; The rotation angle of the bolt in each region of interest of the image to be tested relative to the bolt in the region of interest corresponding to the reference image is calculated, and whether the bolt is loose is determined based on the rotation angle.

2. The method for detecting bolt loosening according to claim 1, characterized in that: Before acquiring the image to be measured and the multiple regions of interest in the reference image of the target bolt cluster, the method includes: The reference image and the image to be tested are scaled to a uniform size, the data format is converted, and normalized preprocessing is performed.

3. The method for detecting bolt loosening according to claim 1, characterized in that: The step of calculating the rotation angle of the bolt in each region of interest of the image to be measured relative to the bolt in the region of interest corresponding to the reference image includes: Perform bipartite graph matching on the region of interest in the image to be tested and the corresponding region of interest in the reference image, and identify the region of interest in which the bolt is not missing among the multiple regions of interest in the image to be tested; The rotation angle of the bolt in the region of interest where the bolt is not missing in the image to be tested relative to the region of interest corresponding to the reference image is calculated.

4. The method for detecting bolt loosening according to claim 3, characterized in that: The step of calculating the rotation angle of the bolt in the region of interest where the bolt is not missing in the image to be tested relative to the bolt in each region of interest corresponding to the reference image includes: Obtaining the coordinates of the corner points of the bolts in the region of interest where the bolts are not missing in the image to be tested and the coordinates of the corner points of the bolts in the region of interest corresponding to the reference image; Use perspective transformation to transform the corner coordinates of the bolt into the top surface contour of the bolt; The rotation angle of the bolt top surface contour image in the region of interest where the bolts are not missing in the image to be tested relative to the bolt top surface contour image in the region of interest corresponding to the reference image is calculated.

5. The method for detecting bolt loosening according to claim 3, characterized in that: The bipartite graph matching is performed on the region of interest in the image to be tested and the corresponding region of interest in the reference image, and the region of interest in which the bolts are not missing among the multiple regions of interest in the image to be tested is identified, including: Perform bipartite graph matching on the region of interest in the image to be tested and the corresponding region of interest in the reference image to obtain multiple pairs of regions of interest; identifying a region of interest pair having a missing bolt among the plurality of region of interest pairs; The pairs of regions of interest where bolts are missing from the multiple regions of interest are removed to obtain regions of interest where bolts are not missing from the multiple regions of interest of the image to be tested.

6. The method for detecting bolt loosening according to claim 3, characterized in that: Before calculating the rotation angle of the bolt top surface profile in the region of interest where the bolts are not missing in the image to be tested relative to the bolt top surface profile in the region of interest corresponding to the reference image, the method includes: The regions of interest corresponding to the non-missing regions of interest of the bolts are respectively extracted from the image to be tested and the reference image, and the regions of interest are scaled to a uniform size, the data format is converted, and normalized preprocessing is performed.

7. The method for detecting bolt loosening according to claim 2, characterized in that: The preprocessing of scaling the reference image and the image to be tested to a uniform size, converting the data format and normalizing them respectively includes: The first operator is used to scale the reference image and the image to be measured respectively; The scaled reference image and the scaled image to be tested are converted into floating point formats by using a second operator, and the scaling is continued to obtain data tensors of the reference image and the image to be tested, respectively, and the three RGB channels of the data tensors of the reference image and the image to be tested are obtained respectively; The third operator is used to perform normalization processing on the three RGB channels of the reference image data tensor and the three RGB channels of the image data tensor to be tested.

8. A bolt loosening detection system, characterized in that: The detection system comprises: An image acquisition device, which is used to acquire an image of a target bolt cluster to be tested; A first detector, the first detector is used to obtain a plurality of regions of interest in the image to be tested and the reference image of the target bolt cluster, wherein each region of interest is a region where a single bolt is installed; The second detector is used to calculate the rotation angle of the bolt in each region of interest of the image to be measured relative to the bolt in the region of interest corresponding to the reference image, and determine whether the bolt is loose based on the rotation angle.

9. A bolt loosening detection device, characterized in that: The bolt loosening detection device includes a processor, a memory, and a bolt loosening detection program stored in the memory and executable by the processor, wherein when the bolt loosening detection program is executed by the processor, the steps of the bolt loosening detection method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a bolt loosening detection program, wherein when the bolt loosening detection program is executed by a processor, the steps of the bolt loosening detection method according to any one of claims 1 to 7 are implemented.

Images