Bolt fastening inspection system and tightening inspection method

Abstract

To provide a tightening inspection system and a tightening inspection method for a bolt tightening part, which can accurately and efficiently determine a tightening state of a bolt set in a bolt tightening part.SOLUTION: A tightening inspection system 100 for a bolt tightening part comprises a three-dimensional image acquisition unit 10 which acquires a three-dimensional image of the bolt tightening part, an inspection object image formation unit 15 which forms an inspection object image on the basis of the three-dimensional image acquired by the three-dimensional image acquisition unit 10, and a tightening state determination unit 50 which determines a bolt tightening state by three-dimensionally comparing marking positions after tightening of markings which are given to a high tension bolt, a nut, a washer and a coupling plate material before tightening, with reference marking positions.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a tightening inspection system and a tightening inspection method for bolt fastening parts.

Background Art

[0002] As is well known, when connecting two steel members during the construction of a steel structure, the two steel members are arranged side by side, and a connecting plate is arranged so as to straddle the two steel members. High-strength bolts (for example, torque-type high-strength bolts, hexagonal high-strength bolts) are inserted into a plurality of holes formed in the overlapping portion of the connecting plate and the steel member, and nuts are screwed and tightened through washers to form a bolt fastening part, thereby generally connecting the steel members.

[0003] When fastening a bolt set having these high-strength bolts, washers, and nuts to a bolt fastening part, first, a temporary bolt and a drift pin are inserted into 1 / 3 or more of the holes for temporary fixing. Then, a bolt set for final fastening is mounted in each hole, and the temporarily fixed temporary bolt and drift pin are replaced with the bolt set for final fastening, and the bolt set for final fastening is tightened once.

[0004] Then, after linearly marking the connecting plate, the once-tightened high-strength bolt, washer, and nut, the nut is finally tightened to complete the fastening of the bolt set of the bolt fastening part. At this time, for a torque-type high-strength bolt, when a predetermined torque is reached, the pigtail is rotated counterclockwise to break and remove the pigtail. Thereafter, a post-fastening inspection of the bolt fastening part is performed and recorded. Also, if necessary, predetermined measures are taken and these measures and inspection results are recorded.

[0005] In conventional bolt fastening inspections, for example, an appearance inspection (shape, appearance color inspection) and a fastening state inspection of the bolt set in the bolt fastening part are performed. Visual inspection detects issues such as missing bolt sets in bolted connections, missing washers, remaining drift pins, remaining pin tails on torque-shear type high-strength bolts, and discrepancies in bolt set specifications based on appearance color. Furthermore, in the fastening condition inspection, the fastening condition of the bolt set is inspected based on the relative positions of the markings that were applied when the bolt set was initially tightened after it was fully tightened.

[0006] However, conventional post-fastening inspections generally rely on visual judgment by workers, which is time-consuming and prone to inspection errors and omissions. Furthermore, recording test results was time-consuming and prone to errors, which reduced the reliability of the tests.

[0007] Therefore, technologies have been developed to accurately and efficiently ensure the fastening state of bolt sets after final tightening in bolt fastening sections (see, for example, Patent Documents 1 and 2). For example, the inspection system described in Patent Document 1 photographs the marked connecting plate material, washer, nut, and high-strength bolt after they have been fully tightened, identifies the pintail fracture surface of the high-strength bolt by color recognition, approximates the shape of the pintail fracture surface identified by color recognition with an ellipse, calculates the center position of the high-strength bolt from its center, detects the marking angle of the connecting plate material and bolt set from the marking position, and determines the bolt tightening state.

[0008] Furthermore, the inspection system described in Patent Document 2 analyzes the bolt fastening portion (target image) photographed from a direction along the axis of the bolt set after it has been fully tightened, identifies the angle between the position of the first marking attached to the connecting plate or washer and the position of the second marking attached to the nut with respect to the center point of the high-strength bolt, and inspects the fastening state of the bolt set based on the identified angle to see if the relative angles of the high-strength bolt, nut and washer are within an acceptable range. [Prior art documents] [Patent Documents]

[0009] [Patent Document 1] Patent No. 6877098 [Patent Document 2] Japanese Patent Publication No. 2020-186925 [Overview of the project] [Problems that the invention aims to solve]

[0010] However, the detection system described in Patent Document 1 may have a problem where, if a concave or convex curvature occurs on the pin tail fracture surface of a high-strength bolt, or if chips or fracture protrusions are formed at the fracture surface boundary, the contour shape becomes non-uniform, and errors in detecting the fracture surface contour can cause errors in the approximated center position, leading to errors in calculating the marking angle and potentially reducing the accuracy of the determination.

[0011] Furthermore, the inspection system described in Patent Document 2 inspects the bolt set by imaging it and based on the target image taken from a direction along the axis. As a result, the sides of the high-strength bolts, nuts, and washers cannot be seen, and the judgment is made based on only a portion of the total length of the marking, which reduces the accuracy of calculating the marking angle. In addition, if the marking is distorted on the side, it may be misinterpreted as a rotation angle difference, which reduces the accuracy of calculating the rotation angle difference.

[0012] On the other hand, with torque-shear type high-strength bolts, there are no markings on the pintail fracture surface, making it almost impossible to see the bolt markings. However, it is desirable to be able to accurately determine the fastening state of torque-shear type high-strength bolts as well.

[0013] This invention has been made in consideration of these circumstances, and aims to provide a tightening inspection system and tightening inspection method for bolt fastenings that can accurately and efficiently inspect the fastening state of bolt sets in bolt fastenings. [Means for solving the problem]

[0014] To solve the above problems, this invention proposes the following means. The inventors conducted extensive research on bolt fastening inspection in bolted connections and discovered that by acquiring an image of the object to be inspected based on a three-dimensional image (distance image) that can be viewed in three dimensions, and comparing this image with a three-dimensional reference model, it is possible to accurately and efficiently inspect the fastening state of the bolt set.

[0015] (1) The invention of configuration 1 is a tightening inspection system for a bolt fastening portion formed by fastening a bolt set having a high-strength bolt, a nut, and a washer to a connecting plate material, and is characterized by comprising: a three-dimensional image acquisition means for acquiring a three-dimensional image of the bolt fastening portion; an inspection target image forming unit for forming an inspection target image based on the three-dimensional image acquired by the three-dimensional image acquisition means; and a fastening state determination unit for acquiring a determination result for determining the bolt fastening state by three-dimensionally comparing the position of the markings applied to each of the high-strength bolt, the nut, the washer, and the connecting plate material before fastening with a preset reference marking position.

[0016] According to the bolt fastening inspection system of this invention, an inspection target image is formed based on a three-dimensional image that is visually apparent in three dimensions, which is created by combining multiple images captured with parallax acquired by a three-dimensional image acquisition means. The fastening state determination unit determines the bolt fastening state by comparing the marking position after fastening (final tightening) of the markings applied to each of the high-strength bolts, nuts, washers, and connecting plates after initial tightening with the marking position of a preset reference three-dimensional model, thereby enabling accurate and efficient identification of the marking position. As a result, the fastening status of the bolt set in the bolt fastening section can be determined accurately and efficiently.

[0017] Here, the three-dimensional image acquisition means is used to acquire a three-dimensional image of a bolt set in a bolt fastening section. This includes imaging devices such as cameras that move to multiple locations to capture images and then combine those images to acquire a three-dimensional image, stereo cameras that acquire a three-dimensional image by combining images captured by multiple imaging units, as used in creating 3D maps, and LiDAR (Light Detection and Ranging) equipped with light detection and distance measurement capabilities. Furthermore, the imaging unit includes imaging means provided in mobile terminals such as smartphones that transmit captured images wirelessly (electromagnetic waves (radio waves, infrared rays, etc.), ultrasound, etc.). Furthermore, a three-dimensional image refers to an image that is perceived three-dimensionally by combining multiple images (for example, two images) taken in a way that creates parallax, or an image that is perceived three-dimensionally by combining a three-dimensional point cloud that simultaneously acquires color tone and viewing distance. Furthermore, the inspection target image formed based on the three-dimensional image can be arbitrarily defined to the extent that it is possible to identify the configuration of the inspection item being inspected, and may include, for example, the three-dimensional image itself, wireframes, surfaces, solids, polygons, and other data.

[0018] (2) The invention of configuration 2 is a tightening inspection system for a bolt fastening portion of configuration 1, characterized by comprising: a shape comparison unit that three-dimensionally compares an inspection target image formed based on a three-dimensional image acquired by the three-dimensional image acquisition means with the shape of a preset reference model; and a shape determination unit that obtains a determination result by referring the shape comparison result in the shape comparison unit to a preset inspection item to determine the mounting state of the high-strength bolt in the bolt fastening portion.

[0019] According to the bolt fastening inspection system of this invention, the shape comparison unit compares the inspection target image formed based on the three-dimensional image acquired by the three-dimensional image acquisition means with the shape of a preset reference model in three dimensions, and the shape comparison result in the shape comparison unit is judged by referring to the inspection items preset in the shape determination unit, so that the mounting state of the bolt set in the bolt fastening unit can be determined accurately and efficiently.

[0020] (3) The invention of Configuration 3 is a tightening inspection system for the bolt fastening part of Configuration 1, characterized by comprising a bolt color determination unit that obtains a determination result for determining the appearance color of the bolt set based on the color tone of the bolt set in the image acquired by the three-dimensional image acquisition means.

[0021] According to the tightening inspection system for the bolt fastening part according to this invention, since the appearance color of the bolt set is determined based on the color tone of the bolt set in the image acquired by the three-dimensional image acquisition means, bolt sets with different surface treatments (for example, presence or absence of plating) having different colors can be detected. As a result, bolt sets with different specifications due to different surface treatments can be accurately and efficiently detected.

[0022] (4) The invention of Configuration 4 is a tightening inspection system for the bolt fastening part according to any one of Configurations 1 to 3, characterized by comprising a fastening state identifier setting unit that assigns an identifier for identifying at least any one of the determination results in an identifiable manner.

[0023] According to the tightening inspection system for the bolt fastening part according to this invention, since an identifier for identifying at least any one of the determination results in an identifiable manner is assigned, each bolt set and the determination result can be accurately and efficiently associated. As a result, the state of each bolt set within the arrangement of the bolt fastening part can be accurately identified.

[0024] (5) The invention of configuration 5 is a tightening inspection system for a bolt fastening portion of configuration 4, comprising an identifier array generation unit that generates an array of identifiers for each bolt set in the bolt fastening portion acquired by the three-dimensional image acquisition means, and a bolt fastening portion synthesis unit that, when the three-dimensional image acquisition means acquires the entire bolt fastening portion by dividing it into multiple sections such that the bolt sets of adjacent sections overlap, extracts an array of bolt sets whose identifier arrays match from the array of identifiers of bolt sets in each section generated by the identifier array generation unit, and synthesizes the entire bolt fastening portion by superimposing the matching arrays of bolt sets.

[0025] According to the bolt fastening inspection system of this invention, when bolt sets in adjacent sections are acquired in multiple sections with overlapping data, the bolt fastening section synthesis unit extracts matching bolt set sequences from the identifier sequences of bolt sets in each section generated by the identifier sequence generation unit, and synthesizes the entire bolt fastening section by superimposing the matching bolt set sequences. This allows for the efficient synthesis of the entire bolt fastening section from multiple images that have been divided and evaluated. As a result, the inspection results for the entire bolted section can be recorded accurately and efficiently.

[0026] (6) The invention of configuration 6 is a tightening inspection system for a bolt fastening portion of configuration 1, comprising a bolt fastening tool for fastening the bolt set, wherein at least a part of the imaging unit constituting the three-dimensional image acquisition means is arranged on the bolt fastening tool, and each time the bolt fastening tool fastens the bolt set, the three-dimensional image is acquired, and at least one of the following is acquired: the bolt fastening state determination result by the fastening state determination unit, and the shape comparison result acquired by comparing the inspection target image formed based on the three-dimensional image with the shape of a preset reference model, and the determination result of the mounting state of the high-strength bolt in the bolt fastening portion is determined by referring to a preset inspection item.

[0027] According to the bolt fastening inspection system of this invention, a bolt fastening tool is provided on which at least a part of the imaging unit constituting the three-dimensional image acquisition means is arranged. After fastening the bolt set, a three-dimensional image is acquired each time, and further, the fastening state determination unit acquires the determination result of at least one of the bolt fastening state or the high-strength bolt mounting state. Therefore, the position of the bolt set in the bolt fastening part can be accurately acquired, and the fastening state and mounting state of at least one of the bolt set can be accurately and efficiently determined in real time.

[0028] (7) The invention of configuration 7 is a bolt fastening inspection system for a bolt fastening portion described in any one of configurations 1 to 3 or 6, characterized in that it includes a bolt fastening portion result display unit that displays the determination result of the bolt set in accordance with each bolt set arranged throughout the bolt fastening portion.

[0029] According to the bolt fastening inspection system of this invention, since it is equipped with a bolt fastening result display unit, the judgment result of each bolt set can be displayed in correspondence with the arrangement of the entire bolt fastening section. As a result, the arrangement of the judgment results for each bolt set in the entire bolt fastening section can be easily and efficiently determined.

[0030] Furthermore, the bolt fastening result display unit may be provided on a mobile device or wearable device.

[0031] By equipping a mobile device with a bolt fastening result display unit, the judgment results can be output regardless of location. As a result, for example, it becomes possible to easily and efficiently compare and verify the construction status and judgment results at a construction site.

[0032] Equipping a wearable device with a bolt fastening result display unit allows for easy portability and efficient handling.

[0033] When used in conjunction with the bolt fastening inspection system described in Configuration 6, it is possible to visually inspect the image and display the results while holding the bolt tightening tool.

[0034] Furthermore, the bolt fastening result display unit may also include a report output unit that outputs the judgment result to a pre-set report, and a confirmation sign generation unit that generates a confirmation sign after confirming the judgment result.

[0035] If a report output unit is provided in the bolt fastening unit result display unit, the inspection results stored in the memory unit of the tightening inspection system can be output accurately and efficiently to a pre-configured report.

[0036] If the bolt fastening result display unit is equipped with a confirmation sign generation unit, then a confirmation sign can be generated after the judgment result has been confirmed. As a result, a high level of traceability and reliability can be ensured for the entire process, from the judgment result stored in the memory unit to the confirmation of the display result and the confirmation signature.

[0037] (8) The invention of configuration 8 is a tightening inspection system for a bolt fastening portion described in any one of configurations 1 to 3 or 6, wherein the three-dimensional image acquisition means includes a position detection means for detecting the relative position to the entire bolt fastening portion, and the system includes a bolt fastening portion synthesis unit that, when the three-dimensional image acquisition means acquires the entire bolt fastening portion by dividing it into a plurality of sections such that the bolt sets of adjacent sections overlap, synthesizes the entire bolt fastening portion by superimposing each section based on the relative position to the entire bolt fastening portion detected by the position detection means.

[0038] According to the bolt fastening inspection system of this invention, the bolt fastening part composite unit superimposes each section based on the relative position of the entire bolt fastening part detected by the position detection means, thereby composing the entire bolt fastening part accurately and efficiently. [Effects of the Invention]

[0039] According to the bolt fastening inspection system and method of this invention, the fastening state of the bolt set in the bolt fastening section can be accurately and efficiently inspected. [Brief explanation of the drawing]

[0040] [Figure 1] This is a block diagram illustrating an example of a schematic configuration of a bolt fastening inspection system according to the first embodiment of the present invention. [Figure 2] This is a flowchart illustrating the general outline of bolt fastening inspection using the bolt fastening inspection system according to the first embodiment. This is a front view illustrating the general configuration. [Figure 3] This is a front view illustrating the schematic configuration of the bolt fastening inspection system according to the first embodiment. [Figure 4] This is a plan view illustrating the schematic configuration of the bolt fastening inspection system according to the first embodiment. [Figure 5] This figure illustrates a schematic example of imaging of a bolt fastening part using the bolt fastening part inspection system according to the first embodiment. (A) shows the imaging range of a set of bolt fastening parts W by the imaging unit, and (B) shows the relationship between the divided images and the overlapping parts. [Figure 6] This diagram illustrates the general outline of a bolt set that is determined to be appropriate by the bolt fastening inspection system according to the first embodiment. [Figure 7] This figure illustrates an example of the items to be determined for determining the determination position and fastening shape by the bolt fastening inspection system according to the first embodiment. [Figure 8] This figure illustrates the schematic of the fastening shape determination by the bolt fastening inspection system according to the first embodiment. [Figure 9] This figure illustrates an example of the items to be judged in color determination and fastening condition determination by the bolt fastening inspection system according to the first embodiment. [Figure 10] This figure illustrates an example of a reference marking model used for determining the fastening state by the bolt fastening inspection system according to the first embodiment. [Figure 11] This diagram illustrates the general process for determining the fastening state using the bolt fastening inspection system according to the first embodiment. [Figure 12] This diagram illustrates the schematic configuration of identifiers in the bolt fastening inspection system according to the first embodiment. [Figure 13] This figure illustrates an example of identifiers for each judgment item of the bolt fastening inspection system according to the first embodiment. [Figure 14] This is a flowchart illustrating the general process of identifier formation in the bolt fastening inspection system according to the first embodiment. [Figure 15] This figure illustrates the schematic of identifier formation by the bolt fastening inspection system according to the first embodiment. [Figure 16] This figure illustrates the schematic of image synthesis by the bolt fastening inspection system according to the first embodiment. [Figure 17] This is a conceptual diagram illustrating the general process of imaging a bolted fastening portion using the bolted fastening portion inspection system according to the second embodiment of the present invention. [Figure 18] This is a plan view illustrating the schematic configuration of a tool-equipped imaging unit used in a bolt fastening inspection system according to the third embodiment of the present invention. [Figure 19] This is a front view illustrating the schematic configuration of the tool-equipped imaging unit used in the bolt fastening inspection system according to the third embodiment. [Figure 20] This is a side view illustrating the schematic configuration of the tool-equipped imaging unit used in the bolt fastening inspection system according to the third embodiment. [Figure 21] This diagram illustrates the general process of bolt fastening and imaging using a tool-equipped imaging unit according to the third embodiment. [Modes for carrying out the invention]

[0041] <First Embodiment> A first embodiment of the present invention will be described below with reference to Figures 1 to 15. First, with reference to Figures 1 to 5, the general configuration of the bolt fastening inspection system (bolt fastening inspection system) according to the first embodiment will be described. Figures 1 to 5 illustrate an example of the schematic configuration of the bolt fastening inspection system according to the first embodiment. Figure 1 is a block diagram illustrating the schematic configuration of the bolt fastening inspection system according to the first embodiment, and Figure 2 is a flowchart illustrating the schematic of bolt fastening inspection. Figure 3 is a front view illustrating the schematic configuration of the bolt fastening inspection system, Figure 4 is a top view, and Figure 5 illustrates an example of the schematic of bolt fastening imaging. In the figure, reference numeral 100 denotes the bolt fastening inspection system, reference numeral 11 (11L, 11R) denotes the imaging unit (three-dimensional image acquisition means), reference numeral T denotes the bolt set, reference numeral W denotes the bolt fastening part, and reference numeral W0 denotes the connecting plate material.

[0042] As shown in Figure 1, the bolt fastening inspection system (bolt fastening tightening inspection system) 100 includes, for example, a three-dimensional image acquisition unit (three-dimensional image acquisition means) 10, an inspection target image forming unit 15, a determination position identification unit 20, a shape determination unit 30, a color determination unit 40, a fastening state determination unit 50, a determination identifier forming unit 60, an image synthesis unit (bolt fastening synthesis unit) 70, an inspection result generation unit 80, a result confirmation unit 90, and a report printing unit (report output unit) 95.

[0043] As shown in Figure 2, the bolt fastening inspection system 100 performs the following steps: three-dimensional image acquisition (S101), inspection target image formation (S102), determination position identification (S103), shape determination (S104), color determination (S105), fastening state determination (S106), determination identifier formation (S107), image synthesis (S108), inspection result creation (S109), result confirmation (S109), and report printing (S110).

[0044] As shown in Figure 1, the three-dimensional image acquisition unit 10 includes an imaging unit 11 and a three-dimensional image forming unit 12. The imaging unit 11 includes imaging units 11L and 11R positioned to capture a set (or multiple, for example, two) of images from multiple directions (for example, two directions) that constitute a three-dimensional image due to parallax, for a bolt set T located in the bolt fastening section W. The imaging unit 11 is configured to send the captured images to the three-dimensional image forming unit 12.

[0045] The configuration of the imaging unit 11 can be set arbitrarily. In addition to a stereo camera having multiple optical systems that simultaneously image the bolt fastening part W from multiple directions, it may also be configured with imaging units 11L and 11R arranged in multiple directions (for example, two directions), or with imaging units 11L and 11R arranged with a timing difference by moving to positions corresponding to multiple directions (for example, two directions). Furthermore, the imaging unit 11 may be one that transmits images captured by an imaging means provided in a mobile terminal such as a smartphone to the three-dimensional image forming unit 12 using electromagnetic waves (radio waves, infrared rays, etc.).

[0046] Next, the imaging of the bolt fastening portion W will be explained with reference to Figures 5(A) and 5(B). Figure 5(A) shows the imaging range of a set of bolt fastening parts W by the imaging unit, and Figure 5(B) shows the relationship between the divided images and the overlapping parts. In Figure 5, the code WLN indicates the Nth row of bolts from the top of the bolt fastening part, the code PM indicates the Mth set of images, and the code PMLN indicates the Nth row of bolts from the top of the Mth set of images.

[0047] As shown in Figures 5(A) and 5(B), if the bolt fastening section W cannot be captured by the imaging section 11 as a set of images (for example, two images) in the vertical direction, the three-dimensional image acquisition unit 10 divides the bolt fastening section W into multiple adjacent sections, for example, so that at least one bolt set in the bolt fastening section W is duplicated, and captures the bolt fastening section W as four sets of images P1, P2, P3, and P4.

[0048] When imaging the bolt fastening section W by dividing it into four sets of images P1, P2, P3, and P4, as shown by the dashed lines in Figure 5(A), for example, the bolt fastening section W is divided from the top into WL1 to WL5, WL5 to WL9, WL8 to WL12, and WL12 to WL16. In this case, images P1, P2, P3, and P4 overlap at WL5, WL8, WL9, and WL12, respectively, as shown in Figure 5(A).

[0049] As shown by the dashed line in Figure 5(B), the fifth row WL5 from the top of the bolt fastening section W corresponds to the fifth row P1L5 from the top in image P1 and the first row P2L1 from the top in image P2. Similarly, WL8, the 8th row from the top of the bolt fastening section W, corresponds to P2L4, the 4th row from the top of image P2, and P3L1, the 1st row from the top of image P3; WL9, the 9th row from the top of the bolt fastening section W, corresponds to P2L5, the 5th row from the top of image P2, and P3L5, the 2nd row from the top of image P3; and WL12, the 12th row from the top of the bolt fastening section W, corresponds to P3L5, the 5th row from the top of image P3, and P4L1, the 1st row from the top of image P4. The imaging range of the imaging unit may be set arbitrarily.

[0050] The three-dimensional image forming unit 12 forms a stereoscopic image (distance image) based on a set of images sent from the imaging units 11L and 11R (11), and forms a three-dimensional image (3D image) corresponding to the stereoscopic image. Specifically, for example, a three-dimensional image is generated from a pair of images (two images) based on the parallax between them. The system is configured to send the formed three-dimensional image to the inspection target image forming unit 15.

[0051] The inspection target image forming unit 15 creates an inspection target image that can be compared using the judgment position identification unit 20, shape determination unit 30, color determination unit 40, and fastening state determination unit 50 from the sent three-dimensional image. The format of the image to be inspected can be set arbitrarily. For example, the image to be inspected may be formed as 3D-3D matching data, where a three-dimensional model and acquired three-dimensional data are compared three-dimensionally as three-dimensional images. Alternatively, the color determination unit 40 may form the image to be inspected as 2D-2D matching data, where two-dimensional images (2D images) obtained by converting the three-dimensional model and acquired three-dimensional data into planar projection images are matched with other two-dimensional image data.

[0052] Next, with reference to Figures 6 to 8, the determination position identification and fastening shape determination according to the first embodiment will be described. Figure 6 is a diagram illustrating the general outline of a bolt set that is determined to be appropriate by the bolt fastening inspection system according to the first embodiment, Figure 7 is a diagram illustrating the general outline of the items to be judged in the determination of the judgment position and the determination of the fastening shape, and Figure 8 is a diagram illustrating the general outline of the determination of the fastening shape.

[0053] As shown in Figure 1, the determination position identification unit 20 includes, for example, a determination address setting unit 21 and a search model storage unit 25. The determination address setting unit 21 identifies all planned bolt set fastening positions based on the appropriate and inappropriate bolt sets T, holes, and drift pins within each image P1, P2, P3, and P4. Specifically, the address of the determined bolt set T is set based on the columns A, B, C, D, E, F, G, H from the left of the bolt fastening part W in each image P1, P2, P3, P4, and the rows 1, 2, 3, 4, 5 from the top of the bolt fastening part W, as shown in Figure 5(B).

[0054] The search model storage unit 25 stores the search model. The search model is composed of a model similar to the shape determination criterion model corresponding to the items to be judged shown in Figure 7, which will be described later. The determination address setting unit 21 then sends the image to be inspected to the shape determination unit 30 and the determination address to the determination identifier formation unit 60.

[0055] As shown in Figure 1, the shape determination unit 30 includes, for example, a fastening shape determination unit (shape comparison unit) 31 and a reference model storage unit 35, the reference model storage unit 35 which stores a reference model for shape determination. The shape determination reference model includes, for example, a three-dimensional model corresponding to all the inspection items related to the shape of bolt set T shown in Figure 7, and a three-dimensional model that is suitable for the shape of bolt set T shown in Figure 7. These three-dimensional shape models are composed of, for example, three-dimensional models using CAD data (DXF, OBJ, OFF, PLY, STL format). Examples of inspection items related to the shape of bolt set T include, for example, missing bolts as shown in Figure 7(A), missing washers as shown in Figure 7(B), remaining drift pins (failure to remove drift pins DP) as shown in Figure 7(C), missing nuts as shown in Figure 7(D), and remaining pin tails (a state in which the pin tail PT remains unbroken) as shown in Figure 7(E). Note that these inspection items are just examples and may be set arbitrarily; for example, other items may be added, or some may be selected.

[0056] The fastening shape determination unit (shape comparison unit) 31 receives the inspection target image sent from the determination address setting unit 21 and adjusts the position, orientation, and scale of the six degrees of freedom in three-dimensional space from the inspection target image. When adjusting the position, orientation, and scale of the six degrees of freedom in three-dimensional space, for example, processing based on the three-dimensional functions of HALCON® can be applied. Then, by referring to the reference model storage unit 35, the system compares the image to be inspected with the reference model for shape determination (for example, shape-based matching (3D matching)) to detect a matching reference model for shape determination and determines whether the fastening shape of the bolt set T is appropriate.

[0057] As shown in Figure 8, the comparison between the image to be inspected and the reference model for shape determination is performed by comparing the image to be inspected with all the reference models for shape determination stored in the reference model storage unit 35, detecting a matching reference model for shape determination, and then determining the fastening shape. Here, the shape determination reference model shown in Figure 8 is an example of the pintail remaining model shown in Figure 7(E) and the shape determination conformance model shown in Figure 7.

[0058] The shape determination standard model consists of all shape determination standard models corresponding to the inspection target item, as well as a model of a conforming (OK) product. The fastening shape determination unit 31 then sends the inspection target image to the color determination unit 40 and the fastening shape determination result to the determination identifier formation unit 60.

[0059] Next, we will explain the color determination according to the first embodiment. As shown in Figure 1, the color determination unit 40 includes, for example, a bolt color determination unit 41 and a reference bolt color storage unit 45, where a reference bolt color model is stored. Then, the external color of the bolt set is determined based on the color tone of the bolt set in the acquired image. The format of the reference voltage color model can be set arbitrarily, and as for the color tone, for example, an RGB color image, a multi-channel image, grayscale, or shades of color may be applied.

[0060] The color determination unit 41 is configured to compare the color of the inspection target image of the bolt set that was matched by the shape determination unit with the reference bolt color obtained by referring to the reference bolt color storage unit 42, and to determine whether the color of the inspection target image is the same color as (for example, matches) the reference bolt color. The color of the image to be inspected is determined by applying the color of a pre-set area, but the color of the entire image to be inspected may be used, or an inspection color generated based on the color of the entire image to be inspected may be used. The color determination unit 41 then sends the inspection target image to the fastening state determination unit 50 and the color determination result to the determination identifier formation unit 60.

[0061] Next, the determination of the fastening state according to the first embodiment will be described with reference to Figures 9, 10, and 11. Figure 9 is a diagram illustrating the general items to be determined in the fastening state determination according to the first embodiment, Figure 10 is a diagram illustrating the general reference marking model according to the first embodiment, and Figure 11 is a diagram illustrating the general fastening state determination.

[0062] As shown in Figure 1, the fastening state determination unit 50 includes, for example, a rotation angle determination unit 51 and a reference marking model storage unit 55, and the reference marking model is stored in the reference marking model storage unit 55.

[0063] The rotation angle determination unit 51 compares the reference marking model stored in the reference marking model storage unit 55 with the image to be inspected, and calculates and determines the difference in the relative position (rotation angle) of the markings attached to the connecting plate material, washer, nut, and high-strength bolt. The rotation angle determination unit 51 then sends the fastening state determination result to the determination identifier formation unit 60. Alternatively, the determination may be made based on whether the rotation angle of the marking M2 on the nut T2 is within a set range.

[0064] Specifically, the reference marking model storage section 55 is divided into a connecting plate W0, a washer T1, a nut T2, and a high-strength bolt T3, as shown in Figure 10, and stores a reference marking model with markings M0, M1, M2, and M3 attached to each. For example, the reference marking model is composed of a three-dimensional model using CAD data (DXF, OBJ, OFF, PLY, STL formats).

[0065] The rotation angle determination unit 51 first performs 3D-3D matching of the reference marking models of the connecting plate W0, washer T1, nut T2, and high-strength bolt T3 with the image to be inspected, including position and orientation information of 6 degrees of freedom in three-dimensional space. Next, it rotates each reference marking model around the bolt axis to align the marking position with the image to be inspected.

[0066] Based on the rotation angles of each standard marking model at this time, for example, if there is a difference in the relative rotation angle between the nut T2 and the connecting plate W0, the fastening state is judged to be acceptable. Also, if there is a difference in the relative rotation angle between the connecting plate W0 and the washer T1, it is judged that the washer rotated together, and if there is a difference in the relative rotation angle between the connecting plate W0 and the high-strength bolt T3, it is judged that the bolt rotated together.

[0067] Next, with reference to Figures 12 to 15, the identification of identifiers in the bolt fastening inspection system according to the first embodiment will be described. Figure 12 is a diagram illustrating the schematic configuration of identifiers in the bolt fastening inspection system according to the first embodiment, and Figure 13 is a diagram illustrating an example of identifiers for each judgment item in the bolt fastening inspection system.

[0068] First, the schematic configuration of the identifier according to the first embodiment will be described with reference to Figures 12 and 13. The identifier is, for example, as shown in Figure 12, Identifier for each bolt set = [Address Identifier] + [Shape Identifier] + [Color Identifier] + [Consistency Identifier] This is defined as follows. The parts enclosed in brackets [] are the address of the determination position and the individual identifier for each determination result.

[0069] The address identifier is defined, for example, by the position (leftmost column, topmost row) in each image of the determined bolt set T. Furthermore, the shape identifier and color identifier are defined by the classifications shown in Figure 13, and the fastening state identifier is defined by the quantitative value of the nut's rotation angle. The classifications and methods for handling the quantitative values ​​of the shape identifier and color identifier are stored in the shape identifier generation unit 64, the color identifier generation unit 65, and the fastening identifier generation unit 66, respectively.

[0070] In the above embodiment, the shape identifier and color identifier were defined by classification, and the fastening state identifier was defined by a quantitative value based on the rotation angle due to the misalignment of markings between the high-strength bolt, nut, washer, and connecting plate material. However, the classification and format of each identifier can be set arbitrarily. For example, the fastening state identifier may be defined by classifications based on the misalignment of markings between the high-strength bolt, nut, washer, and connecting plate material (e.g., OK, NG).

[0071] As shown in Figure 1, the determination identifier formation unit 60 includes, for example, a new record setting unit 61, an address identifier generation unit 62, a determination identifier addition unit 63, a shape identifier generation unit 64, a color identifier generation unit 65, a conclusion identifier generation unit 66, and a determination result storage unit 67. Then, an array of identifiers is formed for each bolt set.

[0072] When the new record setting unit 61 receives a signal from the determination address setting unit 21 that identifies the location of the determined bolt set, it sets a new record to store the determination identifier for that address. Then, the new record is passed to the address identifier generation unit 62.

[0073] The address identifier generation unit 62 generates an address identifier by storing the determination position in the three-dimensional image in the new record received from the new record setting unit 61. The address identifier generation unit 62 then sends the generated address identifier to the determination identifier addition unit 63.

[0074] The determination identifier addition unit 63 receives an address identifier from the address identifier generation unit 62 and adds the identifiers of the respective determination results sent from the shape identifier generation unit 64, the color identifier generation unit 65, and the fastening identifier generation unit 66 to the address identifier. The generated identifier is then sent to the determination result storage unit 67 and the image synthesis unit (bolt fastening unit synthesis unit) 70.

[0075] The shape identifier generation unit 64 generates a shape determination identifier by referring to the shape determination identifier classification shown in Figure 13 based on the fastening shape determination result received from the fastening shape determination unit 31. Then, the shape determination identifier is sent to the determination identifier addition unit 63.

[0076] The color identifier generation unit 65 generates a color identifier by referring to the color identifier classification shown in Figure 13 based on the color determination result received from the color determination unit 41. Then, the color determination identifier is sent to the determination identifier addition unit 63.

[0077] The fastening identifier generation unit 66 generates a fastening identifier based on the fastening state determination result received from the rotation angle determination unit 51. Then, the fastening identifier is sent to the determination identifier addition unit 63.

[0078] The determination result storage unit 67 stores the identifier sent from the determination identifier addition unit 63.

[0079] Next, with reference to Figures 14 and 15, an overview of identifier formation according to the first embodiment will be described. As shown in Figure 14, identifier formation involves creating a new record (S201), setting a determination address (S202), adding a shape identifier (S203), adding a color identifier (S204), and adding a fastening identifier (S205) to form an identifier for each bolt set.

[0080] (1) First, when a new record is set, a new record is set with the fields [Address Identifier], [Shape Identifier], [Color Identifier], and [Contract Status Identifier] left blank (null), as shown in the new record setting in Figure 15 (S201). (2) Next, when the determination address setting is performed, as shown in the determination address setting in Figure 15, an identifier record is formed in which the [address identifier] is set and the [shape identifier], [color identifier], and [fastening status identifier] fields are left blank (null) (S202). (3) Next, when the shape identifier is added, as shown in Figure 15, the shape identifier is entered in addition to the address identifier, and an identifier record is formed in which the color identifier and fastening status identifier are left blank (null) (S203). (4) Next, when the color identifier is added, as shown in the color identifier addition in Figure 15, an identifier record is formed in which the [color identifier] is entered in addition to the [address identifier] and [shape identifier], and the [fastening status identifier] is left blank (null) (S204). (5) Next, when the fastening identifier is added, as shown in the fastening identifier addition in Figure 15, the [address identifier], [shape identifier], and [color identifier] are entered in addition to the [fastening status identifier], and an identifier record without blanks (null) as shown in the fastening identifier addition is formed (S205).

[0081] As shown in Figure 1, the image synthesis unit (bolt fastening unit synthesis unit) 70 comprises an identifier array generation unit 71, an identifier array comparison unit 72, and an image merging unit 73. The identifiers received from the determination identifier addition unit 63 are associated with bolt sets in multiple images, and these multiple images are synthesized to form an image of the entire bolt fastening unit. Here, the identifier array refers to the array of identifiers of bolt sets T belonging to the same row.

[0082] The identifier array generation unit 71 associates the identifiers received from the determination identifier addition unit 63 with the bolt sets in each image of the bolt fastening section, and generates an identifier array for each bolt row in each image. It then sends the identifiers and identifier array data for each bolt set in each image to the identifier array comparison unit 72.

[0083] The identifier sequence comparison unit 72 compares the identifier and identifier sequence data for each bolt set sent from the identifier sequence generation unit 71 with adjacent images to detect bolt sequences that match. Then, the data regarding the identifier and identifier sequence for each bolt set in each image, along with the data for bolt rows whose identifier sequences match, are sent to the image merging unit 73.

[0084] The image merging unit 73 forms an image of the entire bolt fastening section by superimposing bolt rows with matching identifier sequences from adjacent images and merging adjacent images, based on the identifier and identifier sequence data for each bolt set sent from the identifier sequence comparison unit 72 and the data for bolt rows with matching identifier sequences.

[0085] Next, with reference to Figure 16, an overview of the image synthesis according to the first embodiment will be described. Figure 16 is a diagram illustrating the schematic of image synthesis according to the first embodiment. In the diagram, the symbol PMLN indicates the Nth bolt row from the top of the Mth image.

[0086] As shown in Figure 16, images P1 to P4 correspond to the identifier sequences P1L5 (5th row from the top of image P1) and P2L1 (1st row from the top of image P2), P2L4 (4th row from the top of image P2) and P3L1 (1st row from the top of image P3), P2L5 (5th row from the top of image P2) and P3L5 (2nd row from the top of image P3), and P3L5 (5th row from the top of image P3) and P4L1 (1st row from the top of image P4), respectively. Then, by superimposing the corresponding identifier sequences mentioned above, an image of the entire bolted section is synthesized.

[0087] As shown in Figure 1, the inspection result generation unit 80 includes, for example, an overall judgment result generation unit 81, a report generation unit 82, a signature generation unit 83, and an inspection result storage unit 84. The overall judgment result generation unit 81 associates the identifier received from the image merging unit 73 with the image of the entire bolt fastening section synthesized by the image merging unit 73, and links the inspection results of each bolt set in the entire bolt fastening section. The overall judgment result generation unit 81 then sends the inspection results for the entire bolt fastening section to the report generation unit 82.

[0088] The report generation unit 82 creates report data by matching the inspection results of the entire bolt fastening section received from the overall judgment result generation unit 81 to pre-set input fields. The report generation unit 82 then sends the created report data to the result confirmation unit 90.

[0089] The signature generation unit 83 is configured, for example, by a program, and creates signed document data by associating the signature data received from the result verification unit 90 with the document data containing the inspection results. The signature generation unit (verification signature generation unit) 83 then sends the signed form data to the inspection result storage unit 84 and the form printing unit 95. The inspection result storage unit 84 stores the signed form data sent from the signature generation unit 83.

[0090] As shown in Figure 1, the result confirmation unit 90 includes, for example, a report display unit (report output unit) 91 and a signature input unit 92. The report display unit (report output unit) 91 is configured, for example, by a wearable terminal (mobile terminal) and is capable of displaying reports sent from the report generation unit 82. Configuring the report display unit (report output unit) 91 with a mobile terminal is suitable for quality control, comparison, and verification at construction sites (remote locations). The format of the report display unit 91 can be set arbitrarily. For example, mobile devices such as tablets, smartphones, and wearable devices such as head-mounted displays, as well as desktop LCD displays, may be used.

[0091] The signature input unit 92 is composed of, for example, a liquid crystal touch panel of the form display unit 91 and touch buttons displayed on the liquid crystal touch panel. After the result verifier confirms the inspection results, they sign with a stylus and press the touch button to input the signature data. The signature input unit 92 then sends the input signature data to the signature generation unit (confirmation signature generation unit) 83. The format of the signature input section 92 may be set arbitrarily, and security methods such as electronic certificate cards or passwords may be used, or other methods may be used.

[0092] The form printing unit (form output unit) 95 prints a form containing the inspection results received from the result confirmation unit 90 and the signature confirming the results.

[0093] According to the bolt fastening inspection system 100 of the first embodiment, the bolt set T is shaped by comparing the inspection target image formed based on a three-dimensional image with a reference model for shape determination in three dimensions, so that the mounting state of the bolt set in the bolt fastening part W can be determined accurately and efficiently.

[0094] Furthermore, the bolt fastening inspection system 100 determines the appearance color of the bolt set T by comparing the inspection target image formed based on a three-dimensional image with a reference color model, thus enabling accurate detection of bolt sets T with different surface treatments (e.g., with or without plating) that have different colors. As a result, bolt sets with different specifications due to varying surface treatments can be efficiently detected.

[0095] Furthermore, the bolt fastening inspection system 100 allows for accurate and efficient identification of the marking location because the inspection target image formed based on a three-dimensional image is compared three-dimensionally with a reference marking model. As a result, the fastening state of the bolt set T in the bolt fastening section W can be determined accurately and efficiently.

[0096] Furthermore, the bolt fastening inspection system 100 assigns an identifier to each bolt set T, allowing for accurate and efficient linking of each bolt set T with the judgment result. As a result, the condition of each bolt set T in the bolt fastening section W can be accurately identified.

[0097] Furthermore, the bolt fastening inspection system 100 extracts the sequence of bolt sets T whose identifier sequences match in images of adjacent sections, and then superimposes the sequences of the matching bolt sets T to synthesize the entire bolt fastening section. This allows for efficient synthesis of the entire bolt fastening section from multiple divided images. As a result, the inspection results for the entire bolted section can be recorded accurately and efficiently.

[0098] Furthermore, the bolt fastening inspection system 100 is equipped with a bolt fastening result display unit, which allows the judgment result of each bolt set to be displayed in accordance with the arrangement of the entire bolt fastening section. As a result, the arrangement of the judgment results for each bolt set in the entire bolt fastening section can be easily and efficiently determined.

[0099] Furthermore, according to the bolt fastening inspection system 100, the bolt fastening composite section combines the entire bolt fastening section by overlapping each section based on the relative position of the entire bolt fastening section detected by the position detection means, thus enabling accurate and efficient combination of the entire bolt fastening section.

[0100] Furthermore, the bolt fastening inspection system 100 includes a mobile terminal as a report display unit 91, allowing the judgment results to be displayed regardless of location. As a result, for example, it becomes possible to compare the construction status and the judgment results at a construction site, making verification work easy and efficient.

[0101] Furthermore, the bolt fastening inspection system 100 allows for easy portability and efficient handling by using a wearable device as the mobile terminal.

[0102] Furthermore, the bolt fastening inspection system 100 is equipped with a report generation unit 82, which allows for the accurate and efficient creation and output (storage, display, and printing) of inspection results in a pre-configured report.

[0103] Furthermore, the bolt fastening inspection system 100 is equipped with a signature generation unit (confirmation signature generation unit) 83, which allows a signature to be added to the form after confirming the judgment result. As a result, the reliability of traceability can be improved throughout the entire process, from the judgment result to confirmation.

[0104] <Second Embodiment> A second embodiment of the present invention will be described below with reference to Figure 17. Figure 17 is a diagram illustrating the schematic of imaging of a bolt fastening part using a bolt fastening part inspection system according to a second embodiment of the present invention. In the figure, reference numeral 200 denotes the bolt fastening part inspection system, and reference numeral 211 denotes the imaging unit with an acceleration sensor.

[0105] As shown in Figure 17, the imaging unit 211 with an acceleration sensor includes, for example, an imaging unit 11 and an acceleration sensor (position detection means, not shown). The imaging unit 11 is the same as in the first embodiment, so the same reference numerals are used and its description is omitted. For example, a 6-degree-of-freedom accelerometer is used as the acceleration sensor (position detection means).

[0106] Specifically, as shown in Figure 17, it is a 6-degree-of-freedom accelerometer capable of detecting acceleration in the directions of mutually orthogonal three-dimensional coordinate axes X, Y, and Z, and acceleration in the circumferential directions αX, αY, and αZ around the X, Y, and Z axes. The type of accelerometer can be arbitrarily configured.

[0107] The bolt fastening inspection system 200 then calculates the velocity in the X, Y, and Z axes, the distance traveled, the rotational speed, and the rotational angle in the X, Y, and Z axes by integrating the acceleration in the three-dimensional coordinate axis directions (X, Y, and Z axes) detected by the acceleration sensor with respect to time. As a result, the imaging position and imaging direction of the imaging unit 11 can be accurately determined.

[0108] According to the bolt fastening inspection system 200 of the second embodiment, the imaging position and imaging direction of the imaging unit 11 can be accurately determined, so when imaging the bolt fastening W, multiple images captured can be combined to form an image of the entire bolt fastening without having to set overlapping bolt rows.

[0109] In the above embodiment, a case in which an accelerometer is used as a position detection means was described, but GPS, a linear scale, etc. may be used instead of an accelerometer, or in conjunction with an accelerometer.

[0110] <Third Embodiment> The bolt fastening inspection system according to the third embodiment of the present invention will be described below with reference to Figures 18 to 21. Figure 18 is a plan view illustrating the schematic configuration of the imaging unit with a tool used in the bolt fastening inspection system according to the third embodiment, Figure 19 is a front view, and Figure 20 is a side view. Figure 21 is a diagram illustrating the schematic of bolt fastening and imaging by the imaging unit with a tool according to the third embodiment. In the figure, reference numeral 305 indicates the fastening tool with imaging unit, and reference numeral 306 indicates the bolt tightening tool. Also, in Figure 20, the imaging unit is represented by a dashed line for ease of representation.

[0111] The bolt fastening inspection system according to the third embodiment includes, for example, a fastening tool 305 with imaging units, on which a plurality of imaging units 11L, 11R (311) are arranged by mounting means (e.g., brackets) (not shown) on a bolt tightening tool 306, as shown in Figures 18 to 20. Since the imaging units 11L and 11R are the same as in the first embodiment, they are given the same reference numerals and their description is omitted.

[0112] The bolt tightening tool 306 includes, for example, a nut runner section 307 for fastening a bolt set T, a rechargeable battery 308 for supplying power to the nut runner section 307, and a gripping operation section 309 on which an operation switch (not shown) is located and which is held by the operator. Furthermore, the bolt set T of the bolt fastening section W is fastened, and after fastening the bolt set T, it is possible to image the fastened bolt set T each time. The type of bolt tightening tool 306 can be arbitrarily set, and various well-known types can be applied.

[0113] Next, the method of using the bolt tightening tool 306 will be explained with reference to Figures 21(A) and 21(B). (1) First, as shown in Figure 21(A), the bolt set T is fastened to the connecting plate W0 using the nut runner portion 307. (2) Next, as shown in Figure 21(B), once the bolt set T is fastened with the nut runner section 307, the fastening tool with imaging unit 305 is moved away from the bolt set T, and the bolt set T is imaged by the imaging unit 311 each time. Then, based on the captured image, fastening shape determination, color determination, and fastening state determination are performed.

[0114] According to the bolt fastening inspection system of the third embodiment, the bolt fastening tool 305 is equipped with imaging units 11L and 11R (311) that acquire three-dimensional images on the bolt tightening tool 306. Three-dimensional images are acquired each time a bolt set T is fastened, and fastening shape determination, color determination, and fastening state determination are performed. As a result, the position information of the bolt set T in the bolt fastening section W, fastening shape determination, color determination, and fastening state determination can be grasped accurately and efficiently in real time.

[0115] The types of imaging units 11L and 11R (311) can be arbitrarily configured. For example, the imaging unit may be positioned at a distance from the bolt set T after the bolt tightening tool 306 has fastened the bolt set T, and then approach the fastened bolt set T to take an image. Alternatively, one imaging unit may move to take images from multiple directions around the bolt set T.

[0116] It should be noted that the present invention is not limited to the embodiments described above, and various modifications can be made without departing from the spirit of the invention. Furthermore, the configuration of the bolt fastening inspection system is not limited to the block diagram shown in Figure 1, and the configuration of the bolt fastening inspection system can be set arbitrarily.

[0117] For example, in the above embodiment, we have described a case in which the bolt fastening state is determined based on the position of each marking after fastening, which is applied to each of the high-strength bolts, nuts, washers, and connecting plates before fastening; the mounting state of the bolt set in the bolt fastening part is determined based on the shape comparison result of the inspection target image formed based on the three-dimensional image with a preset reference model; and the appearance color of the bolt set is determined based on the color tone of the bolt set in the image acquired by the three-dimensional image acquisition means. However, it is possible to arbitrarily set which of the following is performed: determination of the bolt fastening state, determination of the mounting state of the bolt set in the bolt fastening part, and determination of the appearance color of the bolt set.

[0118] Furthermore, in the above embodiment, for example, the case in which the image to be inspected and the reference models (reference model for shape determination and reference marking model) are three-dimensional images and comparison (3D-3D matching) is performed between the three-dimensional images has been described. However, for color determination, for example, the image to be inspected and the reference models (reference model for shape determination and reference marking model) may each be composed of two-dimensional images (planar projection images, etc.) generated based on three-dimensional images, and these may be compared (2D-2D matching) between the two-dimensional images.

[0119] Furthermore, in the above embodiment, the case in which the bolt fastening inspection system 100 performs fastening shape determination, color determination, and fastening state determination has been described. However, for example, one or two of the fastening shape determination, color determination, and fastening state determination may be performed, or other determinations may be added to these determinations.

[0120] Furthermore, in the above embodiment, a case was described in which the bolt fastening inspection system 100 generates identifiers for the determination results of fastening shape determination, color determination, and fastening state determination. However, for example, the system may be configured not to generate identifiers after the determination is performed, or it may be configured to generate identifiers for one or two of the determination results of fastening shape determination, color determination, and fastening state determination after the determination is performed.

[0121] Furthermore, in the above embodiment, when the entire bolted fastening section is acquired by dividing it into multiple sections such that the bolt sets T of adjacent sections overlap, an identifier sequence is generated from the identifiers of the fastening shape determination, color determination, and fastening state determination of each bolt set T in the bolted fastening section W, and the entire bolted fastening section is synthesized by superimposing the bolt set sequences in which the identifier sequences of each image match. However, the entire bolted fastening section may also be synthesized based on an identifier sequence generated from identifiers from one or two of the fastening shape determination, color determination, and fastening state determination identifiers, or other methods other than identifier sequences may be used when synthesizing the entire bolted fastening section. Alternatively, the entire bolted fastening section may be acquired as a single image.

[0122] Furthermore, although the above embodiment described the case in which the judgment result is incorporated into a report, whether or not to incorporate the judgment result into a report can be set arbitrarily. Furthermore, while the above embodiment described a case where the judgment result is combined with a form, a signature is entered, a signature is generated, and then printed or stored in the inspection result storage unit, whether or not to enter a signature with the judgment result and generate a signature can be arbitrarily set, and inspection results without signature generation may be stored in the inspection result storage unit. Alternatively, the judgment result may be stored in the inspection result storage unit without being combined with a form.

[0123] Furthermore, although the above embodiment described the case where the high-strength bolt is a torque-shear type high-strength bolt, it goes without saying that it may also be applied to high-strength hexagonal bolts that do not produce pin tail residue, such as torque-shear type high-strength bolts.

[0124] Furthermore, the bolt fastening inspection system and method may be used not only for bolt fastenings in structures such as bridges, but also for bolt fastenings in industrial products.

[0125] Furthermore, in the above embodiment, the marking may be performed manually, or a known marking device may be used. For example, it is efficient and preferable to apply a marking device, such as the one described in Japanese Patent Publication No. 2016-74087, which comprises a stamping roller, a cup-shaped holder having a housing portion that can accommodate the stamping roller on its inner circumference, and a link connecting the pivotally supported stamping roller to the holder, wherein as the holder moves around the bolt set from the top side in the axial direction of the bolt set toward the connecting plate material, the stamping roller moves radially toward the bolt set to form an axial marking on the outer surface of the bolt set. Furthermore, since it can form distortion-free, straight markings, it is more preferable in that it allows for more accurate determination of the rotation angle. [Industrial applicability]

[0126] The bolt fastening inspection system and method according to this invention can accurately and efficiently inspect the fastening state of bolt sets in bolt fastening sections, making them industrially applicable. [Explanation of symbols]

[0127] T-bolt set T1 Washer T2 Nut T3 High-Strength Bolt (Tor-Shear Type High-Strength Bolt) PT Pintail DP Drift Pin M marking (bolt set) M0 marking (connecting plate material) M1 marking (washer) M2 marking (nut) M3 marking (high-strength bolt) W0 Connecting plate material W bolt fastening section 100 Bolt Fastening Inspection System (Bolt Fastening Tightening Inspection System) 10 Three-dimensional image acquisition unit (three-dimensional image acquisition means) 11, 11L, 11R, 311 Imaging Unit 12 Three-dimensional image forming unit 15. Image forming unit for inspection 20 Judgment position identification section 21. Determination Address Setting Section 25 Search Model Storage Unit 30 Shape determination section 31 Fastening shape determination section 35 Reference Model Storage Unit 40 Color judgment section 41 Bolt color determination section 45 Reference bolt color storage section 50 Fastening status determination unit 51 Rotation angle determination unit 55 Reference Marking Model Storage Unit 60 Judgment identifier formation section 61 New Record Setting Section 62 Address Identifier Generation Unit 63. Determination identifier addition section 64 Shape identifier generation unit 65 Color identifier generation unit 66. Conclusion Identifier Generation Unit 67. Judgment Result Storage Unit 70 Image synthesis section (bolt fastening section synthesis section) 71 Identifier Array Generation Unit 72 Identifier Array Comparison Section 73 Image merging section 80. Test Result Generation Unit 81 Overall judgment result generation section 82. Document Generation Unit 83 Signature Generation Unit (Confirmation Signature Generation Unit) 84 Inspection Result Storage Unit 90 Result Confirmation Section 91. Report display unit (report output unit) 92 Signature Input Section 95. Form Printing Section (Form Output Section) 305 Fastening tool with imaging unit 306 Bolt tightening tool (tool)

Claims

1. A tightening inspection system for bolt fastenings formed by fastening a bolt set having high-strength bolts, nuts, and washers to a connecting plate material, A three-dimensional image acquisition means for acquiring a three-dimensional image of the bolt fastening portion, An inspection target image forming unit that forms an inspection target image based on the three-dimensional image acquired by the three-dimensional image acquisition means, A fastening state determination unit obtains a determination result by comparing the position of the markings applied to each of the high-strength bolt, nut, washer, and connecting plate material before fastening with a preset reference marking position in three dimensions to determine the bolt fastening state. A tightening inspection system for bolt fastening parts, characterized by comprising the following features.

2. A tightening inspection system for bolt fastening parts according to claim 1, A shape comparison unit that compares the inspection target image formed based on the three-dimensional image acquired by the three-dimensional image acquisition means with the shape of a pre-set reference model in three dimensions, A shape determination unit obtains a determination result by referring the shape comparison result from the shape comparison unit to a preset inspection item to determine the mounting state of the high-strength bolt in the bolt fastening unit, A tightening inspection system for bolt fastening parts, characterized by comprising the following features.

3. A tightening inspection system for bolt fastening parts according to claim 1, The bolt color determination unit acquires a determination result that determines the external color of the bolt set based on the color tone of the bolt set in the image acquired by the three-dimensional image acquisition means. A tightening inspection system for bolt fastenings, characterized by the following features.

4. A tightening inspection system for bolt fastening parts according to any one of claims 1 to 3, A tightening inspection system for bolt fastenings, characterized by comprising a fastening status identifier setting unit that assigns an identifier to identify at least one of the aforementioned determination results.

5. A tightening inspection system for bolt fastening parts according to claim 4, The three-dimensional image acquisition means includes an identifier array generation unit that generates an array of identifiers for each bolt set in the bolt fastening portion acquired by the three-dimensional image acquisition means, When the three-dimensional image acquisition means acquires the entire bolt fastening portion by dividing it into multiple sections such that the bolt sets of adjacent sections overlap, A bolt fastening part synthesis unit extracts bolt set sequences that match the identifier sequences from the sequence of bolt set identifiers for each section generated by the identifier sequence generation unit, and synthesizes the entire bolt fastening part by superimposing the matching bolt set sequences. A tightening inspection system for bolt fastenings, characterized by the presence of the following features.

6. A tightening inspection system for bolt fastening parts according to claim 1, The bolt set is equipped with a bolt tightening tool, At least a portion of the imaging unit constituting the three-dimensional image acquisition means is arranged on the bolt tightening tool. After the bolt tightening tool fastens the bolt set, the three-dimensional image is acquired each time. The bolt fastening state determination result by the fastening state determination unit, The shape comparison result obtained by comparing the image of the object to be inspected, formed based on the three-dimensional image, with the shape of a pre-set reference model, is used to determine the mounting state of the high-strength bolt in the bolt fastening section by referring to a pre-set inspection item. Get at least one of the following A tightening inspection system for high-strength bolts, characterized by the following features.

7. A tightening inspection system for bolt fastening parts according to any one of claims 1 to 3 or 6, A bolt fastening section result display unit displays the determination result of the bolt set in accordance with each bolt set arranged throughout the bolt fastening section. A tightening inspection system for bolt fastenings, characterized by the presence of the following features.

8. A tightening inspection system for bolt fastening parts according to any one of claims 1 to 3 or 6, The three-dimensional image acquisition means includes a position detection means for detecting the relative position to the entire bolt fastening portion, When the three-dimensional image acquisition means acquires the entire bolt fastening portion by dividing it into multiple sections such that the bolt sets of adjacent sections overlap, Based on the relative position of the entire bolt fastening portion detected by the position detection means, a bolt fastening portion composite portion is formed by superimposing each section to combine the entire bolt fastening portion. A tightening inspection system for bolt fastenings, characterized by the presence of the following features.

Images