Bolt tightening position confirmation system and bolt tightening position confirmation method

Abstract

To provide a new bolt tightening position confirmation system and bolt tightening position confirmation method that allows the tightening status of each bolt passing through at least one of multiple through holes formed in a plate to be confirmed for each bolt position, while reducing the burden on on-site workers. [Solution] The plate is photographed in color from the side opposite to the direction in which the bolts are inserted. Image processing of the color image is performed to detect the position of each bolt that has passed through, and different color coloring results are identified for each bolt's tightening state on the opposite side of the tip where the position was detected. Based on the processing results, the position and tightening state of each bolt that has passed through are confirmed, and the confirmation results from the confirmation unit are presented to persons other than the on-site workers.

Description

Technical Field

[0004] , , , , , , , , , , , , , ,

[0005]

[0001] The present invention relates to a bolt tightening position confirmation system and a bolt tightening position confirmation method.

Background Art

[0002] Conventionally, various technologies have been proposed as technologies for improving the efficiency of bolt tightening work for various structures in the complicated working environment at construction sites such as bridges. As one of these proposed technologies, a technology of a bolt tightening management system that can confirm the supply rotation and omission of tightening during the tightening work and can save the trouble of performing an extraction inspection after the completion of the tightening work has been proposed (see Patent Document 1; hereinafter referred to as "Conventional Example 1").

[0003] In addition, technologies of a detection method and a system for detecting the tightening position of bolts in a structure by performing image processing have been proposed (see Patent Document 2; hereinafter referred to as "Conventional Example 2"). Furthermore, even for high-output bolts in an environment where it is difficult to obtain a clear image, technologies of a high-output bolt inspection system and a high-output bolt inspection method that can objectively determine the quality of tightening have also been proposed (see Patent Document 3; hereinafter referred to as "Conventional Example 3"). In Conventional Example 3, it is disclosed that image processing using a learned model by machine learning is performed when detecting the tightening position of a bolt.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the conventional example 1 described above, a barcode or similar document containing the tightening procedure is attached to the tightening side of a plate such as a splice plate, and a field worker reads the barcode by operating a code reader. The tightening operation is then performed using a nut runner according to the read procedure. Furthermore, in the conventional example 1, the field worker can confirm the completion of both the preliminary tightening (initial tightening) and the final tightening by determining the completion.

[0006] As described above, in the conventional example 1, the field worker is responsible for operating the code reader and confirming the completion of preliminary and final tightening. Therefore, from the perspective of reducing the burden on field workers, there is room for improvement considering the recent advancements in digital technology.

[0007] In the conventional example 2 described above, it is assumed that the plate is marked before tightening. The end of the tightened bolt is photographed, including the marking, and the bolt position is detected from the binarized result when the conformance condition is met, while changing the threshold for binarization of the photographic result. As a result, the amount of computation required to meet the conformance condition tends to be large. In addition, shadows are easily captured in the photographic result, and when such shadows are captured, they seriously affect the binarization result of the photographic result. Therefore, depending on the shooting environment, it may not be possible to detect the bolt position with accuracy.

[0008] In the conventional example 3 described above, images including lines attached to the bolt, nut, washer, and base material (plate) are taken in advance, and the center of the bolt end, bolt line, nut line, washer line, and base material line included in the captured images are extracted. Then, the quality of the tightening state of the high-power bolt is determined based on the extracted bolt line, nut line, washer line, and base material line. Furthermore, when applying conventional example 3, it is disclosed that it is preferable to use multispectral illumination for the lighting of the object being photographed in order to extract the various lines with high accuracy.

[0009] The present invention has been made in view of the above circumstances, and aims to provide a new bolt tightening position confirmation system and bolt tightening position confirmation method that can confirm the tightening status of each bolt that has passed through at least one of a plurality of through holes formed in a plate, while reducing the burden on on-site workers. [Means for solving the problem]

[0010] The first method is, A bolt tightening position confirmation system for confirming the tightening state of each bolt that has passed through at least one of a plurality of through holes formed in a plate, for each bolt position, A photographing unit that takes color photographs of the plate from the side opposite to the direction in which the bolt is inserted, An image processing unit performs image processing on the color images acquired by the aforementioned imaging unit to detect the position of each of the through bolts, and identifies different colors for each tightening state of the tip on the opposite side of each bolt whose position has been detected. A confirmation unit that checks the position and tightening state of each of the through bolts based on the processing results from the image processing unit, A presentation unit that presents the verification results from the aforementioned verification unit to persons other than on-site workers, It is characterized by being equipped with [the following features].

[0011] The second method is, in the first method, The image processing unit is characterized by using a machine learning model to recognize the bolt that has penetrated the body, to detect the position of the recognized bolt, and to identify the color of the tip of the recognized bolt.

[0012] The third method is, in the first method, The aforementioned tightening state has three states: pre-tightening state, pre-tightening state, and final tightening state. In the pre-tightening state, the color of the tip of the recognized bolt is the first color. In the pre-tightened state, the color of the tip is a second color, different from the first color, which is applied to the tip when the pre-tightening is complete. In the final tightening state, the color of the tip is a third color, different from both the first and second colors, which was applied to the tip upon completion of the final tightening. It is characterized by the following:

[0013] The fourth method is, in the third method, The final tightening is determined to be complete when the torque change gradient value, calculated as the ratio of the change in torque applied to the nut to the change in rotation angle of the nut, becomes less than or equal to a predetermined ratio of the maximum value after the start of the final tightening, either after the completion of the preliminary tightening or as the bolt tightening progresses from the state before tightening. It is characterized by the following:

[0014] The fifth method is, in the first method, The system is further characterized by comprising a transmission unit that transmits information of the shooting results of the shooting unit to the image unit located at a remote location from the shooting unit.

[0015] The sixth method is, A bolt tightening position confirmation method used in a bolt tightening position confirmation system, which comprises a shooting unit, an image processing unit, a confirmation unit, and a presentation unit, for confirming the tightening state of each bolt that has passed through at least one of a plurality of through holes formed in a plate, for each bolt position, The aforementioned imaging unit performs an imaging step of taking a color photograph of the plate from the side opposite to the direction in which the bolt is inserted, The image processing unit performs image processing on the color image acquired in the shooting step to detect the position of each of the through bolts, and an image processing step to identify different colors for each tightening state of the tip on the opposite side of each bolt whose position has been detected, A confirmation step in which the confirmation unit confirms the position and tightening state of each of the penetrated bolts based on the processing result in the image processing step; A presentation step in which the confirmation unit presents the confirmation result in the confirmation step to persons concerned other than the on-site workers using the presentation unit; It is characterized by comprising.

Effect of the Invention

[0016] According to the first means, the photographing unit performs color photography of the plate from the side opposite to the insertion direction of the bolt. Subsequently, the image processing unit performs image processing on the acquired color image to detect the position of each of the penetrated bolts, and discriminates different colors for each tightening state of the tip portion on the opposite side of each bolt at the detected position.

[0017] Next, the confirmation unit confirms the position and tightening state of each of the penetrated bolts based on the processing result by the image processing unit. Such a confirmation result is presented by the presentation unit to persons concerned other than the on-site workers (for example, office workers, etc.).

[0018] For this reason, after extracting a two-dimensional region of the bolt tip portion, which is easier to extract than a one-dimensional pattern such as a line mark, the position of the bolt is detected, and the color colored on the bolt tip portion is discriminated. Therefore, it is possible to accurately confirm the tightening position and tightening state of the bolt without adopting multi-spectral light as illumination light.

[0019] In addition, persons concerned other than the on-site workers (for example, office workers, etc.) can confirm the tightening position and tightening state of the bolts.

[0020] Therefore, it is possible to confirm the tightening state of the bolts for each bolt position while reducing the burden on the on-site workers.

[0021] According to the second method, the image processing unit uses a machine learning model to recognize the penetrating bolt, detect the position of the recognized bolt, and identify the color of the tip of the recognized bolt. Therefore, the presence of the bolt can be recognized efficiently and accurately while reducing dependence on the site environment, and the position of the bolt can be detected with high accuracy.

[0022] According to the third method, there are three tightening states: pre-tightening state, pre-tightening state, and final tightening state. In the pre-tightening state, the color of the tip of the recognized bolt is the first color. In the pre-tightening state, the color of the tip is the second color, which is different from the first color, and which is applied to the tip when the pre-tightening is completed. In the final tightening state, the color of the tip is the third color, which is different from both the first and second colors, and which is applied to the tip when the final tightening is completed. Therefore, when pre-tightening or final tightening is performed, which is frequently used when bolting splice plates in bridges and the like, the tightening state of the bolt can be checked for each bolt position.

[0023] According to the fourth method, the final tightening is determined to be complete when the torque change gradient value, calculated as the ratio of the change in torque applied to the nut to the change in the rotation angle of the nut as the bolt tightening progresses after the completion of the preliminary tightening or from the state before tightening, becomes less than or equal to a predetermined ratio of the maximum value after the start of the final tightening. In other words, the final tightening of the screw is performed using the so-called yield point method. This makes it possible to achieve bolt tightening with a rational introduction axial force.

[0024] According to the fifth method, the system further includes a transmission unit that transmits information on the shooting results of the shooting unit to an image processing unit located remotely from the shooting unit. Therefore, the image processing unit, which requires high information processing capabilities, as well as the confirmation unit and display unit, can be installed in an environment with good operating conditions (for example, an office environment). As a result, the tightening status of each bolt can be reliably confirmed.

[0025] According to the sixth method, in the shooting step, the shooting unit takes a color photograph of the plate from the side opposite to the direction in which the bolts are inserted. Subsequently, in the image processing step, the image processing unit processes the color image acquired in the shooting step to detect the position of each bolt that has passed through, and identifies different colors for each bolt and its tightening state on the opposite side of the tip where the position was detected. Next, in the confirmation step, the confirmation unit confirms the position and tightening state of each of the bolts that have passed through based on the processing results in the image processing step. Then, in the presentation step, the confirmation unit presents the confirmation results from the confirmation step to persons other than the on-site workers.

[0026] Therefore, it is possible to check the tightening status of each bolt at each bolt location while reducing the burden on on-site workers. [Brief explanation of the drawing]

[0027] [Figure 1] This figure shows a schematic configuration of a bolt tightening position confirmation system according to one embodiment of the present invention. [Figure 2] This is a diagram showing a scene of bolt tightening work. [Figure 3] This is a block diagram showing the functional configuration of the controller in Figure 1. [Figure 4] This diagram illustrates the registered contents within the work table shown in Figure 3. [Figure 5] Figure 1 is a block diagram showing the functional configuration of the office layout equipment. [Figure 6] This is a flowchart illustrating the procedure for tightening the bolts. [Figure 7] Figure 6 is a flowchart illustrating the procedure for tightening the bolts. [Figure 8] This figure shows an example of a captured image used to derive a working coordinate system. [Figure 9] This figure shows an example of an image taken after the preliminary tightening work is completed. [Figure 10]This figure shows an example of an image taken when the final tightening process is complete. [Modes for carrying out the invention]

[0028] One embodiment of the present invention will be described below with reference to Figures 1 to 10. In the following description and figures, the same elements will be denoted by the same reference numerals, and redundant explanations will be omitted.

[0029] [composition] Figure 1 shows a schematic configuration of the bolt tightening position confirmation system 300 according to this embodiment. In the following explanation, we will assume that the bolt tightening position confirmation system 300 is used to confirm the bolt tightening position and tightening state at the connection point of the bridge girder (plate girder) of an I-girder (large bridge) with a total length of several hundred meters, for example.

[0030] In this embodiment, the object-oriented OBJ is a structure in which high-strength hexagonal bolts BLT (hereinafter also simply referred to as "bolts") are tightened via the first connecting plate CP1 and the second connecting plate CP2 at the joint where the first bridge girder BG1 and the second bridge girder BG2 are joined. This tightening is performed by sequentially passing the high-strength hexagonal bolt BLT through the washer WS1, the first connecting plate CP1, the bridge girder BG1 or bridge girder BG2, the second connecting plate CP2 and the washer WS2 in the Z-axis direction, and then screwing a nut NT onto the protruding portion on the Z-axis side of the high-strength hexagonal bolt BLT and tightening it.

[0031] As shown in Figure 1, the bolt tightening position confirmation system 300 comprises a group of on-site equipment 100 located at the tightening site and an office-based device 200 located in an office separate from the tightening site. The controller 110 included in the on-site equipment group 100 and the office-based device 200 are able to communicate with each other via a wide-area communication network (WAN: Wide Area Network; hereinafter also simply referred to as the "communication network") 500 such as the Internet.

[0032] The field-installed equipment group 100 includes a controller 110, a nut runner 120, a tablet terminal 130, and an electric impact wrench 140. The configuration of the controller 110 will be described later.

[0033] The nut runner 120 is a tool (tightening tool) for tightening bolts. The nut runner 120 is connected to the controller 110 for information communication and can perform bolt tightening work (final tightening work) according to the tightening conditions registered (set) within the nut runner 120. In addition, the nut runner 120 can be switched between right rotation (clockwise) and left rotation (counterclockwise).

[0034] Furthermore, as mentioned above, the nut runner 120 is intended for use at construction sites, and it is preferable that it be equipped with a waterproof cover and a protective cover for damage, etc., that is suitable for such construction sites.

[0035] Furthermore, the nut runner 120 is configured to report to the controller 110 whether or not predetermined tightening conditions have been met when the bolts are fully tightened. The nut runner 120 is also equipped with a notification mechanism to inform on-site workers of the tightening results.

[0036] In this embodiment, the notification means consists of a green lamp, a yellow lamp, a red lamp, and a speaker (all not shown) located on the head of the nut runner 120. The notification means illuminates the green lamp and emits a predetermined chime sound when a tightening operation is performed on each bolt that satisfies the tightening conditions. The notification means illuminates the red lamp and emits a predetermined beep sound when a final tightening operation that satisfies the tightening conditions is not performed.

[0037] Furthermore, the nut runner 120 is equipped with an automatic marking socket. When tightening is successful and the tightening conditions are met, the tip of the high-strength hexagonal bolt BLT on the tightening side is colored a predetermined color (yellow in this embodiment).

[0038] Furthermore, the nut runner 120 has a built-in nut rotation amount sensor and a torque sensor. Based on the detection results from the nut rotation amount sensor and torque sensor, if the torque change gradient value, which is calculated as the ratio of the change in torque value applied to the nut NT rotated during the tightening work to the change in rotation angle of the nut NT, falls below a predetermined ratio to the maximum value after the start of the final tightening, it is determined that the tightening has been completed, so-called load-bearing point tightening has been successful, the final tightening work for the nut NT is terminated, and this fact is reported to the controller 110. If the final tightening work has not elapsed for a predetermined time from the start, or if the total rotation angle from the start exceeds a predetermined angle, it is determined that load-bearing point tightening has failed, the final tightening work for the nut NT is terminated, and this fact is reported to the controller 110.

[0039] The predetermined percentage, time, or angle are determined in advance through experiments, simulations, experience, etc. These tightening condition parameters are registered within the nut runner 120.

[0040] Furthermore, as shown in Figure 2, the nut runner 120 is held in a slidable position in the vertical and horizontal directions by a guide mechanism 129. More specifically, the guide mechanism 129 is detachably fixed to the flange of the first bridge girder BG1. In addition, this embodiment has a first rail that is long in the vertical direction, a second rail that is long in the horizontal direction and movable vertically along the first rail, and a holding part provided on the second rail and movable horizontally along the second rail, and the nut runner 120 is held in this holding part. This makes it possible to move the nut runner 120 as appropriate using the guide mechanism 129 when performing tightening work on each bolt.

[0041] The guide mechanism 129 is not limited to the above configuration, and may be, for example, an arm-type guide mechanism having a fixing part that can be detachably fixed to the first bridge girder BG1, a link mechanism extending from the fixing part, and a holding part provided at the tip of the link mechanism for holding the nut runner 120.

[0042] In this embodiment, the bolt fastening positions are arranged in an 8x6 matrix, as shown in Figure 2. Furthermore, two-dimensional markers (e.g., ArUco markers) MK0 to MK3, each with a different pattern, are attached to the four corners of the connecting plate CP2.

[0043] Returning to Figure 1, the tablet terminal 130 described above is a terminal device used by field workers at a work site where bolt tightening work is performed. The tablet terminal 130 is connected to the controller 110 via wireless LAN, and by receiving operations from the field worker on the control panel of the terminal 130, it is possible to input various instructions corresponding to those operations to the controller 110. In addition, the tablet terminal 130 can display information received from the controller 110 on a display unit (not shown).

[0044] Furthermore, the instruction input sent from the tablet terminal 130 to the controller 110 includes a report of the completion of preliminary tightening for each individual bolt position.

[0045] A general-purpose electric impact wrench 140 can be used as the electric impact wrench 140, and it is used for pre-tightening bolts (BLTs). When pre-tightening is performed, an automatic marking socket is attached to the electric impact wrench 140. When pre-tightening is complete, the tip of the high-strength hexagonal bolt BLT on the tightening side is colored a predetermined color (in this embodiment, "white") that is different from the color at the end of the final tightening as described above. In this embodiment, before pre-tightening, the color of the tip of the high-strength hexagonal bolt BLT on the tightening side is different from both "white" and "yellow".

[0046] <Configuration of Controller 110> Next, we will describe the configuration of the controller 110 mentioned above.

[0047] As shown in Figure 3, the controller 110 includes a processing control unit 111 with a built-in timekeeping unit for timekeeping, a storage unit 112, a shooting unit 113, and an operation input unit 114. The controller 110 also includes a display unit 115, a sound output unit 116, and a sound input unit 117. Furthermore, the controller 110 includes a WAN communication unit 118 and a local communication unit 119.

[0048] The processing control unit 111 is comprised of a central processing unit (CPU) and peripheral circuits. By executing a program, the processing control unit 111 performs various processes and controls connected units, thereby realizing the functional operation of the controller 110.

[0049] The storage unit 112 is equipped with a storage device capable of writing and reading data, such as an HDD (Hard Disk Drive) or SSD (Solid State Drive), and stores various information data used by the processing control unit 111. The information data stored in the storage unit 112 includes each program executed by the processing control unit 111 and the work table WTB.

[0050] As shown in Figure 4, the work table WTB allows registration of the number of rows M and the number of columns N of the bolt arrangement, associated with the identifier to be checked. Furthermore, the work table WTB allows registration of the bolt position (Xmn, Ymn), which is the XY coordinate value of the bolt position, the current status STSmn, the preliminary tightening order PFmn, the preliminary tightening time PFTmn, the final tightening order PFmn, and the final tightening time PFTmn, associated with the bolt arrangement position (m (=1~M), n (=1~N)), which is the arrangement position of each bolt. These registrations are updated according to the registration update information received from the office-based device 200.

[0051] Here, the bolt position (Xmn, Ymn) is the coordinate position in the XY coordinate system defined as described later. The current status STSmn is one of the following: bolt not inserted, pre-tightening, pre-tightening completed, or final tightening completed. The current status STSmn changes sequentially according to the progress of the tightening work.

[0052] The preliminary tightening sequence PFmn and preliminary tightening time PFTmn are registered according to the specifications from the office-based device 200. Similarly, the final tightening sequence FFmn and final tightening time FFTmn are also registered according to the specifications from the office-based device 200.

[0053] The shooting unit 113 is equipped with a camera and takes color images of the object to be inspected from the opposite direction of the bolt insertion. The resulting color video is then sent as digital data to the processing control unit 111. In this embodiment, the camera is mounted on a helmet worn by the field worker. The camera is also capable of aberration correction.

[0054] The operation input unit 114 is equipped with various function keys and accepts input from field workers by pressing each key, and the operation information is sent to the processing control unit 111. In this embodiment, the operation input unit 114 has a touch panel or the like with transparent electrodes arranged in a grid pattern to cover the display surface of the display unit 115, and detects the position where a finger or stylus is pressed, and sends that position information as operation information to the processing control unit 111.

[0055] The display unit 115 is composed of an LCD (Liquid Crystal Display) or the like, and displays various information on the display screen according to the display control signals from the processing control unit 111.

[0056] The sound output unit 116 receives output sound data from the processing control unit 111. The sound output unit 116 then outputs sound corresponding to the output sound data. In this embodiment, the sound output unit 116 is configured as an earphone type and is worn on the ear of a field worker.

[0057] The sound input unit 117 is equipped with a microphone. The sound collected by the microphone is sent to the processing control unit 111 as input sound data.

[0058] In this embodiment, the sound output unit 116 and the sound input unit 117 are configured as headsets worn by field workers. Under the control of the processing control unit 111, they can communicate with office workers using the office-based device 200 via the communication network 500.

[0059] The WAN communication unit 118, under the control of the processing control unit 111, connects to the communication network 500 wirelessly and communicates with external devices such as the office-based equipment 200 connected to the communication network 500. The data transmitted from the controller 110 to the office-based equipment 200 through such communication includes image data obtained from the shooting unit 113, conversational audio data spoken by field workers and collected by the sound input unit 117, and reports of completion of preliminary and final tightening for each bolt. The data received by the controller 110 from the office-based equipment 200 includes conversational audio data spoken by office workers and sent from the office-based equipment 200, and data for updating the registered contents of the work table WTB.

[0060] The local communication unit 119 communicates with the nut runner 120 and the tablet terminal 130 under the control of the processing control unit 111. In this embodiment, the local communication unit 119 and the nut runner 120 are connected via LAN, and the local communication unit 119 and the tablet terminal 130 are connected wirelessly as described above.

[0061] The information sent from the nut runner 120 to the local communication unit 119 includes whether the final tightening operation was successful and the hole arrangement positions (m,n) that were targeted by the final tightening operation.

[0062] The communication data sent from the local communication unit 119 to the tablet terminal 130 includes, as appropriate, work instruction information generated by the processing control unit 111, search results information on the current work status, and image data obtained from the shooting unit 113. Furthermore, the communication data sent from the tablet terminal 130 to the local communication unit 119 includes the aforementioned preliminary tightening completion report.

[0063] <Configuration of office-based device 200> Next, the configuration of the office setup device 200 described above will be explained.

[0064] As shown in Figure 5, the office-based device 200 includes a processing control unit 210, a storage unit 220, and an operation input unit 230. The office-based device 200 also includes a display unit 240, an audio output unit 250, and an audio input unit 260. Furthermore, the office-based device 200 includes a WAN communication unit 270.

[0065] The processing control unit 210 is configured with a central processing unit (CPU) and peripheral circuits including a timing element. The processing control unit 210 executes a program to perform various processes and control connected units, thereby realizing the functional operation of the office-based device 200.

[0066] The storage unit 220 is equipped with a storage device capable of writing and reading data, such as an HDD (Hard Disk Drive) or SSD (Solid State Drive), and stores various information data used by the processing control unit 210. The information data stored in the storage unit 220 includes each program executed by the processing control unit 210 and the work history table WHT.

[0067] The work table WHT is configured to register the same information as the work table WTB in the controller 110 shown in Figure 4 above, in synchronization with the work table WTB.

[0068] The operation input unit 230 is equipped with various function keys and accepts key press input from office workers, sending the operation information to the processing control unit 210. In this embodiment, the operation input unit 230 has a touch panel or the like with transparent electrodes arranged in a grid pattern to cover the display surface of the display unit 240, and detects the position pressed by a finger or stylus, sending the position information as operation information to the processing control unit 111. The operation input unit 230 can also be equipped with a mouse as a pointing device.

[0069] The display unit 240 is composed of an LCD (Liquid Crystal Display) or the like, and displays various information on the display screen according to the display control signals from the processing control unit 210. Information according to the work progress and the shooting results sent from the controller 110 are displayed on the display screen in real time as appropriate.

[0070] The sound output unit 250 receives output sound data from the processing control unit 210. The sound output unit 250 then outputs sound corresponding to the output sound data. In this embodiment, the sound output unit 250 is part of a headset that can be worn by an office worker, and is configured to output sound near the office worker's ear.

[0071] The sound input unit 260 is equipped with a microphone. The sound collected by the microphone is sent to the processing control unit 210 as input sound data. In this embodiment, the sound input unit 260 is part of a headset that can be worn by an office worker, and the microphone can be positioned near the office worker's mouth. Under the control of the processing control unit 210, communication with field workers is possible via the communication network 500.

[0072] The WAN communication unit 270, under the control of the processing control unit 210, connects to the communication network 500 wirelessly and communicates with external devices (e.g., the controller 110) connected to the communication network 500. The communication data transmitted from the WAN communication unit 270 to the controller 110 through such communication includes work instruction data for field workers, conversational audio data spoken by office workers and collected by the sound input unit 260, and registration update data for updating the registered contents of the work table WTB.

[0073] Furthermore, the communication data that the WAN communication unit 270 receives from the controller 110 includes, as described above, image data which is the result of shooting by the shooting unit 113, conversational audio data spoken by field workers and collected by the sound input unit 117, preliminary tightening completion reports and final tightening completion reports, etc.

[0074] [Operation] Next, we will explain the bolt tightening position confirmation operation, which is achieved through the cooperation of the on-site equipment group 100 and the office-based device 200.

[0075] In the bolt tightening position confirmation operation, as shown in Figure 6, preparatory work is first performed in step S11. This preparatory work includes preparation work performed outside the tightening site and preparation work performed at the tightening site.

[0076] <Preparation work outside the work site> (1) Machine learning of the shape of the high-strength hex bolt BLT and bolt through hole HL used for fastening Machine learning for high-strength hexagonal bolts (BLTs) is performed on color images of the portion of the bolt that protrudes from the connecting plate CP2, after pre-tightening is performed with a washer WS2 and nut NT on the tightening side from the connecting plate CP2. The machine learning also takes into account variations in the shooting angle, changes in weather, and the effects of sunlight. Deep learning techniques can be suitably employed for this type of machine learning.

[0077] Furthermore, machine learning for bolt through-holes HL (see Figure 8) is performed on color images of locations where through-holes equivalent to bolt through-holes HL are drilled in a structure equivalent to the one formed by combining the first connecting plate CP1 and the second connecting plate CP2 at the joint where the first bridge girder BG1 and the second bridge girder BG2 are joined. In this machine learning as well, similar to the machine learning for high-strength hexagonal bolts BLT described above, machine learning is performed that takes into account changes in the shooting angle, changes in weather, and the effects of sunlight. In addition, deep learning methods can be suitably employed in the machine learning for bolt through-holes HL.

[0078] Furthermore, depending on the type of work site, machine learning can be performed on other bolts (temporary bolts, plated bolts) and the shape of pin components.

[0079] (2) Preparation of design data The distance between the center positions of adjacent high-strength hexagonal bolts (BLTs), that is, the distance between the center positions of adjacent bolt through-holes, is registered in the storage unit 220 of the office placement device 200.

[0080] In this embodiment, the bolt arrangement is in the form of an 8x6 matrix.

[0081] <Preparation work at the fastening site> First, the first connecting plate CP1 and the second connecting plate CP2 are temporarily fixed to the joint where the first bridge girder BG1 and the second bridge girder BG2 meet, using bolts BLT that pass through at predetermined bolt arrangement positions, thereby preventing the first bridge girder BG1, the second bridge girder BG2, the first connecting plate CP1, and the second connecting plate CP2 from separating from each other (see Figure 8).

[0082] Next, attach the base marker MK0 to the lower left of the second connecting plate CP2, and the general marker MK1 to the lower right. Also, attach the general marker MK2 to the upper left of the second connecting plate CP2, and the general marker MK3 to the upper right. These markers MK0 to MK3 are 2D markers with different patterns, making it easy to identify their central position from the captured image.

[0083] Once the preparation work in step S11 is completed as described above, in step S12, an image including the base marker MK0 is captured by the shooting unit 113. This capture is performed by the field worker adjusting the position of the camera mounted on their helmet and the shooting field of view direction, etc., in accordance with voice instructions from the office worker monitoring the captured image displayed on the display unit 240, or in accordance with the captured image displayed on the tablet terminal 130. The image data of the captured result is transmitted to the processing control unit 210 of the office-based device 200.

[0084] Subsequently, in step S13, an image including the base marker MK0 and general markers MK1 to MK3 is captured. During this capture, the field worker adjusts the camera position, field of view, etc., based on voice instructions from the office worker to the field worker, or according to the captured image displayed on the tablet terminal 130. An example of the image obtained in this way is shown in Figure 8.

[0085] Next, in step S14, the processing control unit 210 of the office setup device 200 derives a working coordinate system based on the captured image obtained in step S13. In deriving this working coordinate system, the processing control unit 210 first sets the central position of the base marker MK0 as the coordinate origin.

[0086] The processing control unit 210 then extracts the center position of the tip of the bolt used for temporary fastening, and the center position of the bolt through-hole HL that was not used for temporary fastening, using the machine learning model obtained through the machine learning described above. The unit then determines the alignment direction of the extracted center positions along the rightward direction in Figure 8 as the X-axis direction. For determining this X-axis direction, statistical methods such as the least squares method can be used. The unit also determines the direction perpendicular to the X-axis direction as the Y-axis direction. Figure 8 shows the determined X-axis and Y-axis directions.

[0087] Furthermore, the direction towards the viewer on the paper, which is perpendicular to both the X-axis and Y-axis directions, is determined as the Z-axis direction.

[0088] Next, in step S15, the processing control unit 210 identifies the bolt tightening positions (i.e., the center positions mentioned above) in the XY coordinate system within the entire area to be checked. In doing so, the processing control unit 210 first identifies the bolt arrangement (m,n) corresponding to each bolt tightening position.

[0089] The processing control unit 210 then uses the average value of the row-direction spacing of the bolt tightening positions shown in Figure 8 as the row-direction distance of the bolt tightening positions in the design, and the average value of the column-direction spacing as the column-direction distance of the bolt tightening positions in the design data. It then identifies each bolt position (Xmn, Ymn) in the XY coordinate system that conforms to the actual dimensions, as well as the center positions of the base marker MK0 and the general markers MK1 to MK3.

[0090] The processing control unit 210 registers the bolt sequence (m,n) and bolt position (Xmn,Ymn) corresponding to each identified bolt tightening position, the identifier to be checked, the number of rows M (=8) and the number of columns N (=6) of the bolt sequence in the work history table WHT, and further transmits this registration information to the processing control unit 111 of the controller 110 as registration update information. The work table WTB in the storage unit 112 is updated according to the registration update information transmitted in this way.

[0091] Next, the processing control unit 210 generates a current status STSmn for the bolt arrangement (m,n) at that time, based on the imaging results from the imaging unit 113. The current status STSmn thus generated is registered in the work history table WHT and is further transmitted from the processing control unit 210 to the processing control unit 111 of the controller 110 as registration update information. The work table WTB in the storage unit 112 is updated according to the registration update information thus transmitted.

[0092] Next, in step S16, bolt tightening is performed. In this bolt tightening, as shown in Figure 7, first, in step S21, it is determined whether or not bolts have been inserted into all the positions where bolt tightening is required. This determination is made by the processing control unit 210 based on the images captured by the imaging unit 113 and / or the contents of the current status STSmn registered in the work history table WHT.

[0093] If the result of the determination in step S21 is negative (step 21: N), the process proceeds to step S22. In step S22, the field worker inserts the missing bolts in response to voice instructions from the office worker, or voluntarily. Then, the process returns to step S21.

[0094] When a missing bolt is inserted, the processing control unit 210 recognizes the bolt sequence in which the new bolt was inserted based on the image captured by the imaging unit 113. It then updates the current status of the recognized bolt sequence to "pre-tightening" in the work history table WHT and transmits this updated information to the processing control unit 111 of the controller 110. The work table WTB in the storage unit 112 is updated according to the transmitted updated information.

[0095] Furthermore, if the captured image contains either the base marker MK0 or any one of the general markers MK1 to MK3, the processing control unit 210 identifies the bolt sequence in which a new insertion has occurred based on the positional relationship between the central position of that marker and the recognized bolt sequence.

[0096] If the result of the determination in step S21 is positive (step 21: Y), the process proceeds to step S23. In step S23, the processing control unit 210 determines the preliminary closing sequence PFmn. The processing control unit 210 registers the result of this determination in the work history table WHT in the storage unit 220 and transmits it to the processing control unit 111 of the controller 110 as registration update information. The contents of the work table WTB in the storage unit 112 are updated according to the registration update information thus transmitted.

[0097] Furthermore, the progress of inserting bolts into locations where bolt tightening is required is shown to office workers by displaying images captured by the imaging unit 113 on the display unit 240.

[0098] Next, in step S24, pre-tightening is performed. This pre-tightening is carried out by a field worker using an electric impact wrench 140, following the pre-tightening sequence.

[0099] Once the pre-tightening of each bolt is complete, the field worker operates the tablet terminal 130 to report the bolt arrangement that has been pre-tightened and that the pre-tightening has been completed to the processing control unit 111. The processing control unit 111 then transmits the report that the pre-tightening has been completed to the processing control unit 210 in the office-based device 200.

[0100] Upon receiving a report that the preliminary tightening has been completed, the processing control unit 210 identifies, based on the imaging results from the imaging unit 113, that the tips of the bolts in the bolt arrangement corresponding to the report of the completion of the preliminary tightening have been colored white, and confirms that the preliminary tightening is complete. The processing control unit 210 identifies the bolt arrangement in which the preliminary tightening has been completed in the same manner as in step S22 described above.

[0101] Once this confirmation is made, the processing control unit 210 determines whether the pre-tightening was performed in accordance with the pre-tightening sequence. If the result of this determination is positive, the processing control unit 210 identifies the confirmation time as the pre-tightening completion time for the identified bolt sequence. The processing control unit 210 then sets the identified pre-tightening completion as the new current status and registers the identified pre-tightening completion time as the pre-tightening time in association with the corresponding bolt sequence in the work history table WHT. It also transmits this registration information to the processing control unit 111 of the controller 110 as registration update information for the corresponding bolt sequence. The contents of the work table WTB in the storage unit 112 are updated according to the transmitted registration update information.

[0102] If the result of the judgment is negative, the processing control unit 210 immediately notifies the office worker that the pre-tightening sequence was incorrect by sound output from the sound output unit 250 and / or by display on the display unit 240. It also transmits the content of the notification as warning data to the processing control unit 111 of the controller 110. Upon receiving this warning data, the processing control unit 111 notifies the field worker of the warning content using the sound output unit 115 and / or the tablet terminal 130.

[0103] Upon receiving the notification, field workers are instructed to use an electric impact wrench 140 or similar tool to remove any bolts that have been pre-tightened incorrectly, and then perform pre-tightening according to the correct sequence. In this embodiment, for any bolts removed from a given bolt arrangement, new bolts are inserted and pre-tightened again when it is the correct pre-tightening sequence.

[0104] Next, in step S25, it is determined whether all pre-tightening work has been completed. This determination is made by the processing control unit 210, which checks whether all the tips of the bolts to be pre-tightened are colored white in the image captured by the imaging unit 113, and / or whether all of the current status STSmn registered in the work history table WHT are in the state of pre-tightening completed.

[0105] If the result of the judgment in step S25 is negative (step S25:N), the process returns to step S24. Then, steps S24 and S25 are repeated until the result of the judgment in step S25 is positive.

[0106] Furthermore, the progress of the preliminary tightening is presented to office workers by displaying images captured by the shooting unit 113 on the display unit 240.

[0107] Figure 9 shows an example of an image taken by the imaging unit 113 for verification after all bolts have been pre-tightened. In Figure 9, the fact that pre-tightening is complete and the bolt tips are colored white is indicated by the addition of diagonal hatching to the bolt tips.

[0108] If the result of the determination in step S25 is positive (step S25: Y), the process proceeds to step S26. In step S26, the processing control unit 210 determines the final tightening sequence FFmn of the bolts. The processing control unit 210 registers the result of this determination in the work history table WHT in the storage unit 220 and transmits it to the processing control unit 111 of the controller 110 as registration update information. The contents of the work table WTB in the storage unit 112 are updated according to the registration update information thus transmitted.

[0109] Next, in step S27, the final tightening is performed. This final tightening is carried out using the nut runner 120, following the final tightening sequence.

[0110] As described above, the success or failure of the final tightening of individual bolts is determined when the torque change gradient value, calculated as the ratio of the change in torque value applied to the nut NT being tightened to the change in rotation angle of the nut NT rotated during the tightening work, is less than or equal to a predetermined ratio of the maximum value after the start of the final tightening, and it is determined that the tightening has been completed. In this case, the tip of the corresponding bolt is colored yellow, the final tightening work for that nut NT is completed, and this fact is reported to the controller 110. On the other hand, if the final tightening work is not completed after a predetermined time has elapsed from the start, or if the total rotation angle from the start exceeds a predetermined angle, it is determined that the final tightening has failed, the final tightening work for that nut NT is completed, and this fact is reported to the controller 110.

[0111] If it is determined that the load-bearing point tightening has failed, as described above, the red lamp on the notification device of the nut runner 120 will light up and a predetermined beep sound will be emitted to notify the on-site worker that the load-bearing point tightening has failed. Upon recognizing this notification, the on-site worker will use the nut runner 120, electric impact wrench 140, etc., to remove the bolt that failed to be tightened at the load-bearing point, and then sequentially insert, pre-tighten, and final-tighten the new bolts in relation to the bolt arrangement of the removed bolts.

[0112] On the other hand, if it is determined that the load-bearing point tightening has been successful, as described above, the green lamp on the notification device provided by the nut runner 120 lights up and a predetermined beep sound is emitted, notifying the on-site worker that the load-bearing point tightening has been successful. In addition, the fact that the final tightening has been successful is sent to the processing control unit 111 of the controller 110. The processing control unit 111 then transmits a report that the final tightening has been successful to the processing control unit 210 of the office-based device 200.

[0113] Upon receiving a report that the final tightening was successful, the processing control unit 210 identifies, based on the imaging results from the imaging unit 113, that the tip of the bolt in the bolt arrangement corresponding to the report of successful final tightening is colored yellow, and confirms that the final tightening was successful. The processing control unit 210 identifies the bolt arrangement (m,n) of the bolt that was successfully tightened in the same manner as in steps S22 and S24 described above.

[0114] Once this confirmation is made, the processing control unit 210 determines whether the final tightening was performed in accordance with the final tightening sequence. If the result of this determination is positive, the processing control unit 210 identifies the confirmation time as the final tightening completion time for the identified bolt arrangement. The processing control unit 210 then sets the identified final tightening completion as the new current status and registers the identified final tightening completion time as the final tightening time in association with the corresponding bolt arrangement in the work history table WHT. It also transmits this registration information to the processing control unit 111 of the controller 110 as registration update information for the corresponding bolt arrangement. The contents of the work table WTB in the storage unit 112 are updated according to the transmitted registration update information.

[0115] If the result of the judgment is negative, the processing control unit 210 immediately notifies the office worker that the tightening sequence was incorrect by sound output from the sound output unit 250 and / or by display on the display unit 240. It also transmits the content of the notification as warning data to the processing control unit 111 of the controller 110. Upon receiving this warning data, the processing control unit 111 notifies the field worker of the warning content using the sound output unit 115 and / or the tablet terminal 130.

[0116] Upon receiving the notification, field workers are instructed to use a nut runner 120 or similar tool to remove any bolts that have been tightened incorrectly, and then perform the final tightening in the correct order. In this embodiment, for any bolts removed from a given bolt arrangement, when it is time for final tightening, new bolts are inserted, pre-tightened, and then fully tightened in sequence.

[0117] Furthermore, the progress of the final closing process is presented to office workers by displaying images captured by the shooting unit 113 on the display unit 240.

[0118] Figure 10 shows an example of an image taken by the imaging unit 113 when all bolts have been fully tightened. In Figure 10, the bolt tips are colored yellow to indicate that the bolts have been fully tightened, by filling in the bolt tips with black.

[0119] Next, in step S28, it is determined whether all the final tightening work has been completed. This determination is made by the processing control unit 210, which determines whether all the tips of the bolts to be finalized are colored yellow in the image captured by the imaging unit 113, and / or whether all of the current status STSmn registered in the work history table WHT are finalized.

[0120] If the result of the judgment in step S28 is negative (step S28:N), the process returns to step S27. Then, steps S27 and S28 are repeated until the result of the judgment in step S28 becomes positive.

[0121] If the result of the judgment in step S28 is positive (step S28: Y), the verification process for the subject of this verification is terminated.

[0122] As described above, in this embodiment, the imaging unit 113 in the field-based equipment group 100 takes a color photograph of the connecting plate CP2 from the side opposite to the direction in which the bolts BLT are inserted. Subsequently, the processing control unit 210 of the office-based device 200 performs image processing on the color image to detect the position of each of the penetrating bolts BLT, and identifies different colors for each tightening state of the tip on the opposite side for each bolt BLT whose position has been detected.

[0123] Next, the processing control unit 210 confirms the position and tightening status of each of the through bolts based on the image processing results. These confirmation results are then presented to relevant parties other than the field workers via the display unit 240 of the office-based device 200.

[0124] Therefore, it is possible to accurately check the tightening position and tightening status of bolts while reducing the burden on on-site workers, and to present the check results to relevant parties other than on-site workers.

[0125] Furthermore, in this embodiment, the processing control unit 210 uses a machine learning model to recognize the bolt that has penetrated the material, detects the position of the recognized bolt, and identifies the color of the tip of the recognized bolt. Therefore, the presence of the bolt can be recognized efficiently and accurately while reducing dependence on the site environment, and thus the position of the bolt can be detected with high accuracy.

[0126] Furthermore, in this embodiment, there are three tightening states: pre-tightening state, pre-tightening state, and final tightening state. In the pre-tightening state, the color of the tip of the recognized bolt is the first color. In the pre-tightening state, the color of the tip is the second color, which is different from the first color, and which is applied to the tip when the pre-tightening is completed. In the final tightening state, the color of the tip is the third color, which is different from both the first and second colors, and which is applied to the tip when the final tightening is completed. Therefore, when pre-tightening or final tightening is performed, which is frequently used when bolting splice plates in bridges and the like, the tightening state of the bolt can be checked for each bolt position.

[0127] Furthermore, in this embodiment, the final tightening is determined to be complete when the torque change gradient value, calculated as the ratio of the change in torque applied to the nut NT to the change in rotation angle of the nut NT, becomes less than or equal to a predetermined ratio of the maximum value after the start of the final tightening, as the final tightening work of the bolt progresses from the state before the preliminary tightening is completed or before the final tightening is completed. In other words, the final tightening of the bolt BLT is performed using the so-called yield point method. Therefore, bolt tightening with an easy, stable, and rational introduction axial force can be achieved.

[0128] Furthermore, this embodiment includes a transmission unit that transmits information of the shooting results from the shooting unit 113 to an image unit located remotely from the shooting unit 113. This allows the processing control unit, which requires high information processing capabilities, to be installed in an environment with good operating conditions (for example, an office environment). As a result, the tightening status of each bolt can be reliably checked.

[0129] [Variations of the Embodiment] The present invention is not limited to the embodiments described above, and various modifications are possible within the scope of the spirit of the invention.

[0130] For example, in the above embodiment, the base marker and three general markers are attached to the outside near the four corners of the bolt tightening work area. In contrast, the markers may be attached to any position as long as the entire bolt tightening work area can be covered by changing the target area in such a way that at least one marker is included in the image captured by the imaging unit and the central position of the marker can be detected. Furthermore, the number or position of the base marker and general markers is not limited to this. For example, if the bolt tightening work area is large, the number of general markers may be four or more, and furthermore, multiple base markers may be provided to provide multiple work coordinate systems.

[0131] Furthermore, according to the above embodiment, the processing control unit 210 sets the central position of the base marker as the coordinate origin, and then extracts the center position of the tip of the bolt used for temporary fastening and the center position of the bolt through-hole HL that was not used for temporary fastening using the machine learning model obtained by the machine learning described above. However, it is also possible to obtain the center position of the bolt tip and the center position of the bolt through-hole using a machine learning model through image processing without setting a coordinate origin, and then extract the relative position of the bolt and the relative position of the bolt through-hole.

[0132] Furthermore, in the above embodiment, the present invention was applied to confirming the tightening position and tightening state when using high-strength hexagonal bolts. In contrast, the present invention may also be applied to confirming the tightening position and tightening state when using types other than high-strength hexagonal bolts.

[0133] Furthermore, in the above embodiment, the present invention was applied to the bolt tightening when connecting the first and second bridge girders via a connecting plate at the joint where the first and second bridge girders meet. In contrast, the present invention may also be applied to bolt tightening during the assembly of steel frames for high-rise buildings or during the construction of relatively large buildings.

[0134] Furthermore, in the above embodiment, it was assumed that a wired method was used for communication between the controller and the nut runner. However, a wireless method may also be used for such communication.

[0135] Furthermore, in the above embodiment, the movement of the nut runner and the start of the tightening operation are primarily controlled by the on-site worker. In contrast, the movement of the nut runner and the tightening operation may be controlled by an office-based device in response to instructions from an office worker. Moreover, the nut runner may also be equipped to perform a pre-tightening function.

[0136] Furthermore, in the above embodiment, the shooting unit is attached to a helmet worn by a field worker. Alternatively, the shooting unit may be mounted on a so-called drone in a manner that allows for changes in the shooting position, shooting field of view, and shooting field of view direction, and the office-based equipment may control these functions.

[0137] Furthermore, in the above embodiment, the arrangement pattern of the bolt tightening positions was a matrix pattern. In contrast, the arrangement pattern of the bolt tightening positions may be a different arrangement pattern from the matrix pattern, such as a staggered pattern.

[0138] Furthermore, in the above embodiment, it was not assumed that the object to be examined would be illuminated when photographing it. However, when the sunlight environment is extremely poor and it is difficult to effectively photograph the object to be examined, an illumination unit may be provided to illuminate the object to be examined.

[0139] Furthermore, in the above embodiment, the completion of final tightening of the bolt was determined by coloring the tip of the bolt, but the determination may also be made by taking into account the detection of the bolt tightening sound (a characteristic sound in the case of final tightening). In this case, the confirmation unit can more appropriately confirm the bolt position and tightening state based on the processing results of the image processing unit and the sound analysis unit. Alternatively, the determination may be made by detecting characteristic points other than coloring, such as scribbles.

[0140] Furthermore, although the above embodiment shows the case where final tightening is performed by the load-bearing point method, it is also applicable when final tightening is performed by the torque method.

[0141] Furthermore, in the above embodiment, white coloring was used for preliminary tightening and yellow coloring for final tightening, but the colors are not limited to these. [Industrial applicability]

[0142] As described above, the present invention is useful in the field of bolt fastening. [Explanation of Symbols]

[0143] 100 … On-site equipment group 110 ... Controller 111 … Processing control unit 112 ... Memory Unit 113 … Filming Unit 114 ... Operation input unit 115 ... Display unit 116… Sound output unit 117 … Audio input unit 118 … WAN communication unit (transmitting unit) 119 … Local communication unit 120 ... Nutliner 129… Guide mechanism 130 … Tablet devices 140… Electric impact wrench 200 … Office layout equipment 210 ... Processing control unit (image processing unit, verification unit) 220 ... Memory Unit 230 ... Operation input unit 240 ... Display unit (presentation unit) 250… Sound output unit 260 … Audio input unit 270 … WAN communication unit 300… Bolt tightening position confirmation system 500 … WAN communication network BG1, BG2… Bridge girders BLT… High-strength hexagonal bolt (bolt) CP1, CP2… Connecting plates NT... Nut WS1, WS2… Washers WHT… Work History Table WTB… Work Table

Claims

1. A bolt tightening position confirmation system for confirming the tightening state of each bolt that has passed through at least one of a plurality of through holes formed in a plate, for each bolt position, A photographing unit that takes color photographs of the plate from the side opposite to the direction in which the bolt is inserted, An image processing unit performs image processing on the color image acquired by the aforementioned shooting unit to detect the position of each of the through bolts, and identifies different color coloring results for each tightening state of the tip on the opposite side of each bolt whose position has been detected. A confirmation unit that checks the position and tightening state of each of the through bolts based on the processing results from the image processing unit, A presentation unit that presents the verification results from the aforementioned verification unit to persons other than on-site workers, A bolt tightening position confirmation system characterized by comprising the following:

2. The bolt tightening position confirmation system according to claim 1, characterized in that the image processing unit uses a machine learning model to recognize the through bolt, detect the position of the recognized bolt, and identify the color of the tip of the recognized bolt.

3. The aforementioned tightening state has three states: pre-tightening state, pre-tightening state, and final tightening state. In the pre-tightening state, the color of the tip of the recognized bolt is the first color. In the pre-tightened state, the color of the tip is a second color, different from the first color that was applied to the tip when the pre-tightening was completed. In the final tightening state, the color of the tip is a third color, which is different from both the first and second colors, and which was applied to the tip when the final tightening was completed. The bolt tightening position confirmation system according to feature 1.

4. The final tightening is determined to be complete when the torque change gradient value, calculated as the ratio of the change in torque applied to the nut to the change in rotation angle of the nut, becomes less than or equal to a predetermined ratio of the maximum value after the start of the final tightening, either after the completion of the preliminary tightening or as the bolt tightening progresses from the state before tightening. The bolt tightening position confirmation system according to feature 3.

5. The bolt tightening position confirmation system according to claim 1, further comprising a transmitting unit that transmits information of the shooting results of the shooting unit to the image processing unit located at a remote position from the shooting unit.

6. A bolt tightening position confirmation method used in a bolt tightening position confirmation system, which comprises a shooting unit, an image processing unit, a confirmation unit, and a presentation unit, for confirming the tightening state of each bolt that has passed through at least one of a plurality of through holes formed in a plate, for each bolt position, The aforementioned imaging unit performs an imaging step of taking a color photograph of the plate from the side opposite to the direction in which the bolt is inserted, The image processing unit performs image processing on the color image acquired in the shooting step to detect the position of each of the through bolts, and an image processing step to identify different colors for each tightening state of the tip on the opposite side of each bolt whose position has been detected, The verification unit performs a verification step in which it confirms the position and tightening state of each of the through bolts based on the processing results in the image processing step, The verification unit performs a presentation step in which it uses the presentation unit to present the verification results from the verification step to persons other than the on-site workers, A method for confirming the bolt tightening position, characterized by comprising the following:

Images