Bolt fastening inspection device

Abstract

To provide a bolt fastening inspection device which has the high inspection accuracy of the fastening state of a bolt and can inspect the fastening states of many bolts in a short time.SOLUTION: A bolt fastening inspection device comprises: bolt image extraction means which extracts a bolt image of each bolt from a captured image; marking image extraction means which extracts a marking pattern in a bolt fastening part from the extracted bolt image; a learned marking determination model which has machine-learned a relation between the marking pattern and the fastening state of the bolt; and bolt fastening determination means which determines the fastening state of the bolt by inputting the extracted marking pattern to the marking determination model.SELECTED DRAWING: Figure 1

Description

Technical Field

[0003]

[0001] The present invention relates to a bolt fastening inspection device for detecting whether the fastening state of a bolt is normal or not.

Background Art

[0002] In structures such as bridges, towers, and steel frame buildings, high-strength bolts are used to join (connect) steel materials. In such fields, after primary tightening to ensure the fastening force (fastening quality) of the bolt, linear markings are applied across the shaft portion (threaded portion) of the bolt, the nut to be tightened, the washer, and the steel material (base portion) to be fastened. When tightening the nut using a tool such as a wrench, it is determined that a predetermined tension is obtained in this bolt by confirming that only this nut rotates and the bolt shaft portion and washer do not rotate together. Conventionally, it has been visually determined whether there is any deviation in the markings, but the inspection took time and misrecognition was likely to occur.

[0003] For example, Patent Document 1 discloses a detection system for determining the tightening state of a bolt by detecting the marking angle for the markings applied to each of a plate, washer, nut, and high-strength bolt based on an image taken after bolt fastening. Patent Document 2 discloses a bolt inspection device having a learned marking determination model that performs machine learning for determining the state of markings applied to a bolt and a learned pin tail determination model that performs machine learning for determining the presence or absence of a pin tail. <00,00017> However, in Patent Document 1, the process of detecting the marking angle from the taken image is complicated. Patent Document 2 inputs each bolt image into the nut determination model. In the method of directly detecting markings from the bolt image, in the recognition of the marking portion, it is easily affected by the surrounding image, and there is uncertainty in the determination accuracy.

Prior Art Documents

[0005] [Patent Document 1] Japanese Patent Publication No. 2018-9932 [Patent Document 2] Japanese Patent Publication No. 2021-113755 [Overview of the project] [Problems that the invention aims to solve]

[0006] The present invention aims to provide a bolt fastening inspection device that offers high accuracy in inspecting the fastening state of bolts and can inspect the fastening state of a large number of bolts in a short amount of time. [Means for solving the problem]

[0007] The bolt fastening inspection device according to the present invention is characterized by having a bolt image extraction means for extracting bolt images of individual bolts from captured images, a marking image extraction means for extracting marking patterns present in the bolt fastening portion from the extracted bolt images, and a trained marking determination model that has learned the relationship between the marking pattern and the fastening state of the bolt by machine learning, and a bolt fastening determination means for determining the fastening state of the bolt by inputting the extracted marking pattern into the marking determination model.

[0008] In this invention, the marking pattern refers to the positional relationship (pattern) of each marking that is applied in a straight line via a nut that fastens the bolt shaft to the base portion such as a plate on the steel material side before the bolt is fully tightened, and which indicates the positional relationship of each marking after the bolt is fully tightened. Specifically, for example, the marking pattern may be the positional relationship of the markings applied to the bolt shaft, nut, washer, and base.

[0009] In the present invention, it is preferable that the extraction of the marking pattern on the bolt fastening portion from the bolt image is performed by extracting only the marking pattern from the bolt image using semantic segmentation. Here, semantic segmentation is a deep learning algorithm that associates labels or categories with image elements (pixels). This improves accuracy because the marking pattern, which shows the positional relationship of only the markings after removing everything else from the bolt image, is input into a pre-trained marking judgment model that has learned the relationship between the marking pattern and the tightening state of the bolt. [Effects of the Invention]

[0010] The bolt fastening inspection device according to the present invention extracts and uses only the marking pattern after the bolt has been fully tightened, thus enabling high-precision inspection in a short time. In conventional visual inspection, for example, inspecting multiple bolts as shown in Figure 2 visually takes about 6 seconds per bolt and is prone to errors in judgment. However, the inspection device according to the present invention automatically detects bolt images based on images taken on-site and further detects only the marking pattern, so it takes only about 1 second per bolt, and has less noise and higher judgment accuracy than directly determining the marking status from the bolt image. [Brief explanation of the drawing]

[0011] [Figure 1] (a) shows an example of extracting bolt images from images taken of multiple bolts on a structure at the site, and (b) shows the state after extracting a marking pattern from the bolt images. [Figure 2] This shows the state of connection between structural elements using bolts. [Figure 3] (a) shows an example of marking before final tightening. (b) shows an example of a normal state after final tightening. (c) shows an example of an abnormal state where the bolt shaft rotates together. [Modes for carrying out the invention]

[0012] A configuration example of the bolt fastening inspection device according to the present invention will be described based on the drawings, but the present invention is not limited thereto.

[0013] Fig. 2 shows a state where steel materials are joined using high-strength bolts. Fig. 2 shows a state where the flange portion of an H-shaped steel is sandwiched between two plates and fastened and joined with a plurality of high-strength bolts. In this case, a bolt is inserted through the fastening hole from one side (the lower side in Fig. 2) of the joint portion, and a nut is screwed onto the base portion such as the other plate via a washer.

[0014] As a means for tightening the bolt, the bolt and the nut are initially tightened, and as shown in Fig. 3(a), markings are made linearly up to the shaft portion of the bolt, the nut, the washer, and the base portion. After that, final tightening is performed using a tool such as a wrench. In this field, some high-strength bolts are provided with a pigtail portion at the tip end, and when the nut is tightened with a wrench or the like, the pigtail portion breaks at a predetermined torque. Figs. 3(b) and (c) show the state after final tightening when the pigtail portion has broken. When the nut is properly finally tightened, only the nut rotates as shown in Fig. 3(b), but if the final tightening is not properly performed, the shaft portion of the bolt or the washer rotates together as shown in Fig. 3(c). Conventionally, this inspection has been performed visually.

[0015] The present invention photographs the state after final tightening of a plurality of bolts at the site shown in Fig. 2 as a moving image or a still image. The above image taken at the site contains many parts other than the vicinity of the bolts to be inspected. Therefore, as the first step, since the above image data is an aggregate of pixel unit information, for example, using an image recognition technology such as deep learning, a bolt image corresponding to the position of the bolt in Fig. 2 is extracted as shown in Fig. 1(a) (bolt image extraction means). In this first step, pattern recognition techniques such as recognizing and sorting the features of the bolt image may be used. In the image shown in Fig. 1(a), it takes time to determine whether the bolt is properly tightened. Therefore, in the present invention, as the second step, as shown in Fig. 1(b) from the bolt image shown in Fig. 1(a), only the positions of the markings of the shaft portion 11, nut 12, washer 13, and base portion 14 of the bolt are extracted as the marking pattern, which is characteristic. Semantic segmentation can be used as the means for extracting this marking pattern as a marking image. In the present invention, the above-mentioned extraction of the bolt image and the extraction of the marking image may be processed simultaneously, or the marking image may be directly extracted corresponding to the position of the bolt shown in Fig. 2.

[0016] Regarding the relationship between the rotational position of the marking of the nut 12 when the bolt (nut) is properly tightened and the marking positions of the shaft portion 11, washer 13, and base portion 14 of the bolt, various abnormal states such as an abnormal state where the marking of the shaft portion 11 of the bolt rotates together, a state where the marking of the washer 13 rotates together, and a state where both the marking of the shaft portion 11 of the bolt and the marking of the washer 13 rotate together are shown in the marking pattern. The relationship between the marking pattern and the tightening state of the bolt is input into a marking determination model that has been machine-learned using a neural network or the like. As a result, an automatic determination result is output and the inspection is completed. The bolts determined to be abnormal will be retightened.

Explanation of Reference Numerals

[0017] 11 Shaft portion of the bolt 12 Nut 13 Washer 14 Base portion

Claims

[Claim 1] A bolt image extraction means for extracting bolt images of individual bolts from captured images, A marking image extraction means for extracting a marking pattern from the extracted bolt image onto the bolt fastening portion, It has a pre-trained marking determination model that uses machine learning to determine the relationship between marking patterns and the tightening status of bolts. The aforementioned marking pattern is the relative misalignment of the markings applied to the bolt shaft, nut, washer, and base. The extraction of the marking pattern on the bolt fastening portion from the bolt image is performed by extracting only the marking pattern from the bolt image using semantic segmentation. A bolt fastening inspection device characterized by having a bolt fastening determination means that determines the fastening state of a bolt by inputting the extracted marking pattern into the marking determination model.

Images