Bolt tightening position confirmation system and bolt tightening position confirmation method

Abstract

To provide a new bolt tightening position confirmation system and bolt tightening position confirmation method that can confirm the tightening state of each bolt that has passed through at least one of a plurality of through holes formed in a plate, for each bolt position, while reducing the burden on on-site workers. [Solution] The plate is photographed in color from the side opposite to the bolt insertion direction, and the color image is processed to detect the position of each of the penetrated bolts.The coloring results are identified using different colors depending on the tightening state of the tip on the opposite side of each bolt whose position has been detected.Based on the processing results, the position and tightening state of each of the penetrated bolts are confirmed, and the confirmation results from the confirmation unit are presented to relevant parties other than on-site workers.

Description

[Technical Field]

[0001] The present invention relates to a bolt tightening position confirmation system and a bolt tightening position confirmation method. [Background technology]

[0002] Various technologies have been proposed to improve the efficiency of bolt tightening work for a wide variety of structures in the complicated work environment of construction sites such as bridges. One of these proposed technologies is a bolt tightening management system that can check for misalignment and missed tightening during tightening work and eliminates the need for random inspections after tightening work is completed (see Patent Document 1; hereinafter referred to as "Prior Art 1").

[0003] Also proposed is a detection method and system technology that uses image processing to detect the tightening position of bolts in a structure (see Patent Document 2; hereinafter referred to as "Prior Art 2"). Furthermore, even in the case of high-power bolts, where it is difficult to obtain clear images, a high-power bolt inspection system and high-power bolt inspection method technology that can objectively determine whether the tightening is good or bad has also been proposed (see Patent Document 3; hereinafter referred to as "Prior Art 3"). Note that Prior Art 3 discloses that image processing is performed using a trained model based on machine learning when detecting the tightening position of a bolt. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 7133738 [Patent Document 2] Japanese Patent Application Publication No. 2018-194475 [Patent Document 3] Japanese Patent Publication No. 2023-176245 Summary of the Invention [Problem to be solved by the invention]

[0005] In the technology of the above-mentioned conventional example 1, a field worker operates a code reader to read a barcode or the like that records the tightening work procedure and is affixed to the tightening work side surface of a plate such as a splice plate, and tightening work is performed using a nut runner according to the read work procedure. Also, in the technology of conventional example 1, the field worker can determine and confirm the completion of pre-tightening (primary tightening) and final tightening.

[0006] As described above, in the technology of Conventional Example 1, the field worker is required to operate the code reader and confirm the completion of pre-tightening and final tightening, etc. Therefore, from the perspective of reducing the burden on the field worker, there is room for improvement in light of recent advances in digital technology.

[0007] The technology in the above-mentioned prior art example 2 assumes that a marking is applied to the plate before tightening, and photographs the end of the tightened bolt, including the marking. The threshold for binarizing the photographed result is varied, and the bolt position is detected from the binarized result when the compatibility condition is met. This tends to increase the amount of calculation required to meet the compatibility condition. In addition, shadows are easily reflected in the photographed result, which has a serious impact on the binarization result of the photographed result. As a result, depending on the photographing environment, the bolt position cannot be detected accurately.

[0008] In the technology of the above-mentioned Prior Art Example 3, an image including lines on the bolt, nut, washer, and base material (plate) is captured in advance, and the center of the bolt end, bolt line, nut line, washer line, and base material line included in the captured image are extracted.Then, the quality of the tightening state of the high-power bolt is judged based on the extracted bolt line, nut line, washer line, and base material line.In addition, when applying Prior Art Example 3, it is disclosed that it is preferable to use multispectral lighting for illuminating the object to be captured in order to extract various lines with high accuracy.

[0009] The present invention has been made in consideration of the above circumstances, and aims to provide a new bolt tightening position confirmation system and bolt tightening position confirmation method that can check the tightening state of each bolt that has passed through at least one of a plurality of through holes formed in a plate, for each bolt position, while reducing the burden on on-site workers. [Means for solving the problem]

[0010] The first method is A bolt tightening position confirmation system for confirming a tightening state of each bolt passing through at least one of a plurality of through holes formed in a plate, for each bolt position, comprising: a photographing unit for photographing the plate in color from a side opposite to the insertion direction of the bolt; an image processing unit that processes the color image acquired by the photographing unit to detect the positions of the penetrated bolts and identifies different colors for the fastening states of the tip ends on the opposite side of each bolt whose positions have been detected; a confirmation unit that confirms the position and fastening state of each of the passed bolts based on the processing results by the image processing unit; a presentation unit that presents the confirmation result by the confirmation unit to a person concerned other than the on-site worker; The present invention is characterized by comprising:

[0011] The second means is the first means, the image processing unit is characterized in that it uses a learning model based on machine learning to recognize the penetrated bolt, detect the position of the recognized bolt, and identify the color of the tip of the recognized bolt.

[0012] The third method is Second Option In The tightening state includes three states: pre-tightening state, pre-tightening state, and final tightening state. In the pre-tightened state, the recognized color of the tip portion of the bolt is a first color, In the pre-tightening state, the color of the tip portion is a second color that is different from the first color and that is colored to the tip portion when pre-tightening is completed, In the final tightening state, the color of the tip portion is a third color that is different from both the first color and the second color and that is colored to the tip portion when final tightening is completed. It is characterized by:

[0013] The fourth means is the third means, The final tightening is performed after the completion of the preliminary tightening or as the bolt tightening progresses from the pre-tightening state. The bolt The torque change gradient value calculated as the ratio of the torque value change applied to the nut to the rotation angle change amount of the nut is determined to be completed when the torque change gradient value is equal to or less than a predetermined ratio of the maximum value after the start of the final tightening. It is characterized by:

[0014] The fifth means is the first means, The image capturing apparatus further comprises a transmission unit that transmits information on the photographing results of the photographing unit to the image unit that is located at a remote location from the photographing unit.

[0015] The sixth measure is A bolt tightening position confirmation method for use in a bolt tightening position confirmation system that includes a photographing unit, an image processing unit, a confirmation unit, and a presentation unit, and that confirms, for each bolt position, a tightening state of each bolt that has passed through at least one of a plurality of through holes formed in a plate, the method comprising: a photographing step in which the photographing unit photographs the plate in color from a side opposite to the bolt insertion direction; an image processing step in which the image processing unit performs image processing on the color image acquired in the photographing step to detect the positions of the penetrated bolts and identify different colors for the fastening states of the tip ends on the opposite side of each bolt whose positions have been detected; a confirmation step in which the confirmation unit confirms the position and fastening state of each of the passed bolts based on the processing result of the image processing step; a presentation step in which the confirmation unit presents the confirmation result in the confirmation step to a person other than the field worker using the presentation unit; The present invention is characterized by comprising: [Effects of the Invention]

[0016] According to the first method, the photographing unit photographs the plate in color from the side opposite to the bolt insertion direction. The image processing unit then processes the captured color image to detect the position of each of the inserted bolts and distinguishes between different colors depending on the tightening state of the tip end on the opposite side of each bolt whose position has been detected.

[0017] Next, a confirmation unit confirms the position and fastening state of each of the penetrated bolts based on the processing results by the image processing unit.The confirmation results are then presented by a presentation unit to relevant parties other than the on-site workers (e.g., office workers, etc.).

[0018] Therefore, by extracting the two-dimensional area of ​​the bolt tip, which is easier to extract than the one-dimensional pattern of the line mark, the position of the bolt is detected and the color of the bolt tip is identified. This makes it possible to accurately check the bolt tightening position and tightening status without using multispectral light as illumination light.

[0019] Furthermore, persons other than the on-site workers (for example, office workers) can check the tightening position and tightening state of the bolts.

[0020] Therefore, the tightening state of the bolts can be checked for each bolt position while reducing the burden on the field workers.

[0021] According to the second aspect, the image processing unit uses a learning model based on machine learning to recognize the penetrated bolt, detect the position of the recognized bolt, and identify the color of the tip of the recognized bolt. This allows the presence of the bolt to be recognized efficiently and accurately while reducing dependency on the site environment, thereby enabling the position of the bolt to be detected with high accuracy.

[0022] According to the third means, there are three tightening states: a pre-tightening state, a pre-tightening state, and a final tightening state, and in the pre-tightening state, the color of the recognized bolt tip is a first color, in the pre-tightening state the color of the tip is a second color different from the first color that is colored on the tip when pre-tightening is complete, and in the final tightening state the color of the tip is a third color different from both the first color and the second color that is colored on the tip when final tightening is complete. Therefore, when pre-tightening or final tightening, which are often used in bolt tightening of splice plates on bridges, etc., is performed, the tightening state of the bolt can be confirmed for each bolt position.

[0023] According to the fourth method, final tightening is determined to be complete when the torque change gradient, calculated as the ratio of the change in torque value applied to the nut to the amount of change in the rotation angle of the nut as bolt tightening progresses after the completion of pre-tightening or from the pre-tightening state, becomes equal to or less than a predetermined ratio of the maximum value after the start of final tightening. In other words, the so-called yield point method is used to final tighten the screw. This makes it possible to achieve bolt tightening with a rational applied axial force.

[0024] According to the fifth aspect, the apparatus further includes a transmission unit that transmits information on the photographing results of the photographing unit to an image processing unit located remotely from the photographing unit, so that the image processing unit, which requires high information processing capabilities, as well as the confirmation unit and display unit, can be installed in an environment with good operating conditions (for example, an office environment), making it possible to stably check the bolt tightening status for each bolt position.

[0025] According to a sixth aspect of the present invention, in the photographing step, the photographing unit photographs the plate in color from the side opposite to the bolt insertion direction. Subsequently, in the image processing step, the image processing unit processes the color image acquired in the photographing step to detect the position of each of the bolts that have penetrated the plate and identifies different colors for the fastening state of the tip of the opposite side of each bolt whose position has been detected. Next, in the confirmation step, the confirmation unit confirms the position and fastening state of each of the penetrated bolts based on the processing results of the image processing step. Then, in the presentation step, the confirmation unit presents the confirmation results of the confirmation step to relevant parties other than the on-site workers.

[0026] Therefore, the tightening state of the bolts can be checked for each bolt position while reducing the burden on the field workers. [Brief explanation of the drawings]

[0027] [Figure 1] 1 is a diagram showing a schematic configuration of a bolt tightening position confirmation system according to an embodiment of the present invention; [Figure 2] FIG. 10 is a diagram showing a scene of bolt tightening work. [Figure 3] FIG. 2 is a block diagram showing the functional configuration of a controller in FIG. 1. [Figure 4] 4 is a diagram for explaining the registered contents in the working table of FIG. 3. FIG. [Figure 5] FIG. 2 is a block diagram showing the functional configuration of the office layout device of FIG. 1. [Figure 6] 10 is a flowchart illustrating a procedure for fastening a bolt. [Figure 7] 7 is a flowchart for explaining the bolt tightening procedure of FIG. 6. [Figure 8] FIG. 10 is a diagram showing an example of a captured image for deriving a task coordinate system. [Figure 9] FIG. 10 is a diagram showing an example of a captured image when the pretightening work is completed. [Figure 10]FIG. 10 is a diagram showing an example of a captured image when the final tightening work is completed. DETAILED DESCRIPTION OF THE INVENTION

[0028] An embodiment of the present invention will be described below with reference to Figures 1 to 10. In the following description and drawings, the same elements are denoted by the same reference numerals, and duplicated description will be omitted.

[0029] [composition] 1 shows a schematic configuration of a bolt tightening position confirmation system 300 according to this embodiment. In the following, the bolt tightening position confirmation system 300 will be described assuming that the object to be confirmed is, for example, a connection portion of an I-girder (plate girder) bridge girder that constitutes a large bridge with a total length of several hundred meters, and that the bolt tightening position confirmation system 300 is used to confirm the bolt tightening position and tightening state of the object to be confirmed.

[0030] In this embodiment, the object OBJ to be confirmed is a structure where a high-strength hex bolt BLT (hereinafter simply referred to as "bolt") is tightened via a first connecting plate CP1 and a second connecting plate CP2 at a joint where a first bridge girder BG1 and a second bridge girder BG2 are joined. The tightening is performed by passing the high-strength hex bolt BLT through a washer WS1, the first connecting plate CP1, the bridge girder BG1 or BG2, the second connecting plate CP2, and the washer WS2 in that order in the Z-axis direction, and then screwing a nut NT onto the protruding portion of the high-strength hex bolt BLT in the Z-axis direction and tightening it.

[0031] 1, the bolt tightening position confirmation system 300 comprises a field-located device group 100 that is placed at the tightening site, and an office-located device 200 that is placed in an office remote from the tightening site. A controller 110 included in the field-located device group 100 and the office-located device 200 are capable of communicating with each other via a wide area network (WAN; hereinafter simply referred to as a "communication network") 500 such as the Internet.

[0032] The on-site device group 100 includes a controller 110, a nut runner 120, a tablet terminal 130, and an electric impact machine 140. The configuration of the controller 110 will be described later.

[0033] The nut runner 120 is a tool (tightening tool) for tightening a bolt. The nut runner 120 is connected to the controller 110 so as to be able to communicate information, and is capable of performing bolt tightening work (final tightening work) according to tightening conditions registered (set) in the nut runner 120. The rotation direction of the nut runner 120 is also switchable between a right rotation direction (clockwise) and a left rotation direction (counterclockwise).

[0034] As described above, the nut runner 120 is intended for use at a construction site, and is preferably provided with a waterproof cover and a cover for protection against damage, etc., that are suitable for the construction site.

[0035] Furthermore, when the bolt has been fully tightened, the nut runner 120 reports the tightening result, which indicates whether or not the predetermined tightening conditions have been met, to the controller 110. The nut runner 120 also includes a notification means for notifying the tightening result to a site worker.

[0036] In this embodiment, the notification means is composed of a green lamp, a yellow lamp, a red lamp, and a speaker (all not shown) provided on the head of the nut runner 120. When the tightening operation for each bolt has been performed in a manner that satisfies the tightening conditions, the notification means turns on the green lamp and outputs a predetermined chime sound. When the tightening operation has not been performed in a manner that satisfies the tightening conditions, the notification means turns on the red lamp and outputs a predetermined beep sound.

[0037] An automatic marking socket is also attached to the nut runner 120. When the tightening is successful and the tightening conditions are satisfied, the tip of the tightening side of the high-strength hexagon bolt BLT is colored a predetermined color (yellow in this embodiment).

[0038] The nut runner 120 also has a built-in nut rotation amount sensor and a torque sensor. Based on the detection results of the nut rotation amount sensor and the torque sensor, if it is determined that the torque change gradient value calculated as the ratio of the torque value change imparted to the nut NT to the rotation angle change amount related to the nut NT rotated during the tightening operation is equal to or less than a predetermined ratio of the maximum value after the start of the tightening operation, it is determined that the so-called load-bearing point tightening has been successful, and the tightening operation for the nut NT is terminated, and this is reported to the controller 110. If the tightening operation has not elapsed for a predetermined time from the start, or if the total rotation angle from the start exceeds a predetermined angle, it is determined that the load-bearing point tightening has failed, and the tightening operation for the nut NT is terminated, and this is reported to the controller 110.

[0039] The predetermined ratio and the predetermined time or angle are determined in advance through experiments, simulations, experience, etc. These tightening condition parameters are registered in the nut runner 120.

[0040] 2, the nut runner 120 is held by a guide mechanism 129 so as to be slidable in the vertical and horizontal directions. More specifically, the guide mechanism 129 is detachably fixed to the flange of the first bridge girder BG1. Furthermore, in this embodiment, there is provided a first rail that is long in the vertical direction, a second rail that is long in the horizontal direction and is movable in the vertical direction along the first rail, and a holding part that is provided on the second rail and is movable in the horizontal direction along the second rail, and the nut runner 120 is held by this holding part. As a result, when performing the tightening operation on each bolt, the nut runner 120 can be moved appropriately using the guide mechanism 129.

[0041] The guide mechanism 129 is not limited to the above configuration, and may be, for example, an arm-type guide mechanism having a fixing part that can be detachably fixed to the first bridge girder BG1, a link mechanism extending from the fixing part, and a holding part provided at the tip of the link mechanism that holds the nut runner 120.

[0042] In this embodiment, the bolt fastening positions are arranged in a matrix of 8 rows and 6 columns, as shown in Fig. 2. Two-dimensional markers (for example, ArUco markers) MK0 to MK3, each with a different pattern, can be attached to the four corners of the connecting plate CP2.

[0043] Returning to Fig. 1, the tablet terminal 130 is a terminal device used by a site worker at a work site where bolt tightening work is performed. The tablet terminal 130 is connected to the controller 110 via a wireless LAN, and by accepting operations by the site worker on an operation unit of the terminal 130, it is possible to input various instructions corresponding to the operations to the controller 110. Furthermore, the tablet terminal 130 is able to display information received from the controller 110 on a display unit (not shown).

[0044] The instruction input sent from the tablet terminal 130 to the controller 110 includes a preliminary tightening completion report for each bolt position.

[0045] A general electric impact can be used as the electric impact 140, and is used when pretightening the bolt BLT. When pretightening is performed, an automatic marking socket is attached to the electric impact 140. When pretightening is completed, the tip of the high-strength hex bolt BLT on the tightening side is colored a predetermined color (in this embodiment, "white") that is different from the color at the time of completion of the main tightening described above. In this embodiment, before pretightening, the color of the tip of the high-strength hex bolt BLT on the tightening side is different from both "white" and "yellow."

[0046] <Configuration of Controller 110> Next, the configuration of the controller 110 will be described.

[0047] 3, the controller 110 includes a processing control unit 111 that includes a built-in timer unit for measuring time, a storage unit 112, a photographing unit 113, and an operation input unit 114. The controller 110 also includes a display unit 115, a sound output unit 116, and a sound input unit 117. The controller 110 also includes a WAN communication unit 118 and a local communication unit 119.

[0048] The processing control unit 111 is configured to include a central processing unit (CPU) and peripheral circuits. The processing control unit 111 executes programs to perform various processes and control connected units, thereby realizing the functional operation of the controller 110.

[0049] The storage unit 112 includes a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive) that can write and read data, and stores various information data used by the processing control unit 111. The information data stored in the storage unit 112 includes each program executed by the processing control unit 111 and a working table WTB.

[0050] 4, the number of rows M and the number of columns N of the bolt arrangement can be registered in the working table WTB in association with the identifier of the confirmation target. The working table WTB can also register the bolt position (Xmn, Ymn), which is the XY coordinate value of the bolt position, the current status STSmn, the pre-tightening sequence PFmn, the pre-tightening time PFTmn, the final tightening sequence PFmn, and the final tightening time PFTmn, in association with the bolt arrangement position (m (= 1 to M), n (= 1 to N)), which is the arrangement position of each bolt. These registrations are updated according to the registration update information received from the office layout device 200.

[0051] Here, the bolt position (Xmn, Ymn) is a coordinate position in an XY coordinate system determined as described below. The current status STSmn is either "bolt not inserted," "not yet pretightened," "pretightened completed," or "final tightening completed." The current status STSmn changes successively according to the progress of the tightening work.

[0052] The preliminary tightening sequence PFmn and the preliminary tightening time PFTmn are registered in response to a designation from the office arrangement device 200. The final tightening sequence FFmn and the final tightening time FFTmn are also registered in response to a designation from the office arrangement device 200.

[0053] The photographing unit 113 includes a camera unit, and takes color photographs of the object to be confirmed from the direction opposite to the bolt insertion direction. The photographed color moving images are then sent as digital data to the processing control unit 111. In this embodiment, the camera unit is attached to a helmet worn by a site worker. The camera unit is also capable of aberration correction.

[0054] The operation input unit 114 has various function keys, receives input by pressing each key by a field worker, and sends the operation information to the processing control unit 111. In this embodiment, the operation input unit 114 has a touch panel or the like in which transparent electrodes are arranged in a grid pattern so as to cover the display surface of the display unit 115, detects a position pressed by a finger, a touch pen, or the like, and sends the position information to the processing control unit 111 as operation information.

[0055] The display unit 115 is configured with an LCD (Liquid Crystal Display) or the like, and displays various information on the display screen in accordance with a display control signal from the processing control unit 111 .

[0056] The sound output unit 116 receives output sound data from the processing control unit 111. Then, the sound output unit 116 outputs a sound corresponding to the output sound data. In this embodiment, the sound output unit 116 is configured as an earphone and is worn on the ear of a field worker.

[0057] The sound input unit 117 is configured to include a microphone. The sound collected by the microphone is sent to the processing control unit 111 as input sound data.

[0058] In this embodiment, the sound output unit 116 and the sound input unit 117 are configured as headsets worn by the field worker, and under the control of the processing control unit 111, they can converse with the office worker using the office layout device 200 via the communication network 500.

[0059] The WAN communication unit 118, under the control of the processing control unit 111, wirelessly connects to the communication network 500 and communicates with external devices such as the office layout device 200 connected to the communication network 500. Data transmitted from the controller 110 to the office layout device 200 through such communication includes image data captured by the photography unit 113, conversational voice data uttered by field workers and collected by the sound input unit 117, preliminary tightening completion reports and final tightening completion reports for each bolt, etc. Data received by the controller 110 from the office layout device 200 includes conversational voice data uttered by office workers and sent from the office layout device 200, and data for updating the registered contents of the work table WTB, etc.

[0060] The local communication unit 119 communicates with the nut runner 120 and the tablet terminal 130 under the control of the processing control unit 111. In this embodiment, the local communication unit 119 and the nut runner 120 are connected via a LAN, and the local communication unit 119 and the tablet terminal 130 are connected wirelessly as described above.

[0061] The information sent from the nut runner 120 to the local communication unit 119 includes the success or failure of the final fastening operation and the holt arrangement position (m, n) that was the target of the final fastening operation.

[0062] The communication data sent from the local communication unit 119 to the tablet terminal 130 includes, as appropriate, work instruction information generated in the processing control unit 111, search result information for the current work status, image data captured by the photographing unit 113, etc. The communication data sent from the tablet terminal 130 to the local communication unit 119 also includes the above-mentioned preliminary tightening completion report.

[0063] <Configuration of office layout device 200> Next, the configuration of the office layout device 200 will be described.

[0064] 5, the office layout device 200 includes a processing control unit 210, a storage unit 220, and an operation input unit 230. The office layout device 200 also includes a display unit 240, a sound output unit 250, and a sound input unit 260. The office layout device 200 also includes a WAN communication unit 270.

[0065] The processing control unit 210 is configured with a central processing unit (CPU) and peripheral circuits including a timer element. The processing control unit 210 executes programs to perform various processes and control connected units, thereby realizing the functional operation of the office layout device 200.

[0066] The storage unit 220 includes a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive) that can write and read data, and stores various information data used by the processing control unit 210. The information data stored in the storage unit 220 includes each program executed by the processing control unit 210 and the work history table WHT.

[0067] In the working table WHT, the same contents as those registered in the working table WTB in the controller 110 shown in FIG. 4 are registered in synchronization with the working table WTB.

[0068] The operation input unit 230 is equipped with various function keys, accepts input by pressing each key by an office worker, and sends the operation information to the processing control unit 210. In this embodiment, the operation input unit 230 has a touch panel or the like in which transparent electrodes are arranged in a grid pattern so as to cover the display surface of the display unit 240, detects a position pressed by a finger, a touch pen, or the like, and sends the position information to the processing control unit 111 as operation information. The operation input unit 230 can also be equipped with a mouse as a pointing device.

[0069] The display unit 240 is configured with an LCD (Liquid Crystal Display) or the like, and displays various information on the display screen in accordance with display control signals from the processing control unit 210. Information according to the progress of work and the photographing results sent from the controller 110 are displayed on the display screen in real time as appropriate.

[0070] The sound output unit 250 receives output sound data from the processing control unit 210. Then, the sound output unit 250 outputs sound corresponding to the output sound data. In this embodiment, the sound output unit 250 is part of a headset that can be worn by an office worker, and is capable of outputting sound near the ears of the office worker.

[0071] The sound input unit 260 is configured to include a microphone. The sound collected by the microphone is sent to the processing control unit 210 as input sound data. In this embodiment, the sound input unit 260 is part of a headset that can be worn by an office worker, and the microphone can be placed near the mouth of the office worker. Under the control of the processing control unit 210, it is possible to have a conversation with the field worker via the communication network 500.

[0072] The WAN communication unit 270, under the control of the processing control unit 210, wirelessly connects to the communication network 500 and communicates with external devices (such as the controller 110) connected to the communication network 500. The communication data from the WAN communication unit 270 to the controller 110 through such communication includes work instruction data for field workers, as well as the above-mentioned conversation voice data uttered by office workers and collected by the sound input unit 260, and registration update data for updating the registered contents of the work table WTB.

[0073] In addition, the communication data that the WAN communication unit 270 receives from the controller 110 includes, as described above, image data that is the result of photography by the photography unit 113, conversational voice data uttered by on-site workers and collected by the sound input unit 117, preliminary tightening completion reports, and final tightening completion reports.

[0074] [Operation] Next, a bolt tightening position confirmation operation that is realized by the field arrangement device group 100 and the office arrangement device 200 working together will be described.

[0075] In such a bolt tightening position confirmation operation, first, preparation work is performed in step S11 as shown in Fig. 6. This preparation work includes preparation work outside the tightening site and preparation work on the tightening site.

[0076] <Preparation work outside the work site> (1) Machine learning of the shapes of the high-strength hex bolts BLT and bolt through holes HL used for fastening Machine learning for the high-strength hex bolt BLT is performed on color images of the portion of the bolt that protrudes from the connecting plate CP2 after pretightening, etc., is performed with the washer WS2 and nut NT on the tightening side of the connecting plate CP2. The machine learning also takes into account variations in the imaging angle, weather changes, and sunlight. Deep learning techniques can be suitably employed for this type of machine learning.

[0077] Furthermore, machine learning for the bolt through-hole HL (see FIG. 8) is performed on color images of a location where a through-hole equivalent to the bolt through-hole HL is drilled in a structure equivalent to the combination of the first connecting plate CP1 and the second connecting plate CP2 at the joint where the first bridge girder BG1 and the second bridge girder BG2 are butted together. As with the machine learning for the high-strength hex bolt BLT described above, this machine learning also takes into account changes in the shooting angle, weather, and sunlight. Furthermore, deep learning techniques can be suitably employed for machine learning for the bolt through-hole HL.

[0078] Depending on the type of work site, machine learning can be performed on other bolts (temporary bolts, plated bolts), the shape of the pin member, etc.

[0079] (2) Preparing design data The distance between the center positions of adjacent high-strength hexagon bolts BLT, that is, the distance between the center positions of adjacent bolt through holes, is registered in the storage unit 220 of the office layout device 200.

[0080] In this embodiment, the bolts are arranged in a matrix of 8 rows and 6 columns.

[0081] <Preparation work at the tightening site> First, the first connecting plate CP1 and the second connecting plate CP2 are temporarily fixed to the joint where the first bridge girder BG1 and the second bridge girder BG2 are joined using bolts BLT that are inserted at a predetermined bolt arrangement position, thereby preventing the first bridge girder BG1, the second bridge girder BG2, the first connecting plate CP1 and the second connecting plate CP2 from separating from each other (see Figure 8).

[0082] Next, a base marker MK0 is attached to the lower left of the second connecting plate CP2, and a general marker MK1 is attached to the lower right. Furthermore, a general marker MK2 is attached to the upper left of the second connecting plate CP2, and a general marker MK3 is attached to the upper right. These markers MK0 to MK3 are two-dimensional markers with different patterns, and their central positions can be easily identified from the captured image.

[0083] When the preparation work in step S11 is completed as described above, in step S12, an image including the base marker MK0 is captured by the photographing unit 113. This photographing is performed by the field worker in response to voice instructions from the office worker monitoring the photographed image displayed on the display unit 240, or by the field worker adjusting the position of the camera attached to the field worker's helmet, the photographing field direction, etc., in accordance with the photographed image displayed on the tablet terminal 130. Image data of the photographed result thus captured is transmitted to the processing control unit 210 of the office layout device 200.

[0084] Subsequently, in step S13, an image including the base marker MK0 and the general markers MK1 to MK3 is captured. During this capture, the field worker adjusts the camera position, the field of view, etc., in response to voice instructions from the office worker to the field worker, or the captured image displayed on the tablet terminal 130. An example of an image thus obtained is shown in FIG. 8.

[0085] Next, in step S14, the processing control unit 210 of the office layout device 200 derives a working coordinate system based on the captured image obtained in step S13. When deriving the working coordinate system, the processing control unit 210 first sets the center position of the base marker MK0 as the coordinate origin.

[0086] Next, the processing control unit 210 extracts the center positions of the tip ends of the bolts used for temporary fastening and the center positions of the bolt through holes HL that were not used for temporary fastening using the machine learning model obtained by the above-mentioned machine learning. Then, the arrangement direction of the extracted center positions along the right side in FIG. 8 is determined as the X-axis direction. A statistical method such as the least squares method can be used to determine the X-axis direction. Furthermore, the direction perpendicular to the X-axis direction is determined as the Y-axis direction. The determined X-axis and Y-axis directions are shown in FIG. 8.

[0087] The direction toward the front of the paper, which is perpendicular to both the X-axis and Y-axis directions, is determined as the Z-axis direction.

[0088] Next, in step S15, the processing control unit 210 identifies the bolt tightening positions (i.e., the above-mentioned center positions) in the XY coordinate system within the entire area to be confirmed. In this identification, the processing control unit 210 first identifies the bolt arrangement (m, n) corresponding to each bolt tightening position.

[0089] 8 as the row-wise distance between the bolt fastening positions in the design, and the average column-wise distance as the column-wise distance between the bolt fastening positions in the design data.Then, the processing control unit 210 identifies each bolt position (Xmn, Ymn) in the XY coordinate system along the actual dimensions, as well as the center positions of the base marker MK0 and the general markers MK1 to MK3.

[0090] The processing control unit 210 registers the bolt arrangement (m, n) and bolt position (Xmn, Ymn) corresponding to each bolt tightening position thus identified, the identifier of the object to be confirmed, the number of rows M (= 8) and the number of columns N (= 6) of the bolt arrangement in the work history table WHT, and further transmits the registration information to the processing control unit 111 of the controller 110 as registration update information. The work table WTB in the storage unit 112 is updated according to the registration update information thus transmitted.

[0091] Subsequently, the processing control unit 210 generates a current status STSmn for the bolt arrangement (m, n) at that time based on the photographing results by the photographing unit 113. The current status STSmn thus generated is registered in the work history table WHT, and is further transmitted as registration update information from the processing control unit 210 to the processing control unit 111 of the controller 110. The work table WTB in the storage unit 112 is updated in accordance with the registration update information thus transmitted.

[0092] Next, in step S16, bolts are tightened. In this bolt tightening, as shown in Fig. 7, first, in step S21, it is determined whether bolts have been inserted into all positions that require bolt tightening. This determination is made by the processing control unit 210 based on images captured by the photographing unit 113 and / or the contents of the current status STSmn registered in the work history table WHT.

[0093] If the result of the determination in step S21 is negative (step S21: N), the process proceeds to step S22. In step S22, the field worker inserts the missing bolts in response to a voice instruction from the office worker or spontaneously. Then, the process returns to step S21.

[0094] When a missing bolt is newly inserted, the processing control unit 210 recognizes the bolt arrangement in which the newly inserted bolt has been performed based on the image captured by the photographing unit 113, and updates the work history table WHT by changing the current status corresponding to the recognized bolt arrangement to "before pre-tightening," and further transmits this registration update information to the processing control unit 111 of the controller 110. The work table WTB in the storage unit 112 is updated according to the registration update information thus transmitted.

[0095] If the captured image contains the base marker MK0 and any one of the general markers MK1 to MK3, the processing control unit 210 identifies the bolt arrangement into which the new insertion has been performed based on the positional relationship between the center position of that marker and the recognized bolt arrangement.

[0096] If the result of the determination in step S21 is affirmative (step S21: Y), the process proceeds to step S23. In this step S23, the process control unit 210 determines the preliminary tightening sequence PFmn. The process control unit 210 registers the result of the determination in the work history table WHT in the storage unit 220, and transmits it as registration update information to the process control unit 111 of the controller 110. In accordance with the registration update information thus transmitted, the contents of the work table WTB in the storage unit 112 are updated.

[0097] The progress of inserting the bolts into the positions where bolt tightening is required is presented to the office worker by displaying the images captured by the photographing unit 113 on the display unit 240.

[0098] Next, in step S24, pre-tightening is performed by a site worker using the electric impact 140 in accordance with the pre-tightening sequence.

[0099] When the pre-tightening of each bolt is completed, the field worker operates the tablet terminal 130 to report the bolt arrangement for which pre-tightening has been completed and the fact that pre-tightening has been completed to the processing control unit 111. The processing control unit 111 transmits a report to the processing control unit 210 of the office layout device 200 that the pre-tightening has been completed.

[0100] When the report that the pre-tightening has been completed is received, the processing control unit 210 determines that the tips of the bolts in the bolt arrangement corresponding to the report that the pre-tightening has been completed have been colored white based on the photographing results by the photographing unit 113, and confirms the completion of pre-tightening. Note that the processing control unit 210 determines the bolt arrangement for which pre-tightening has been completed in the same manner as in step S22 described above.

[0101] Once this confirmation is made, the processing control unit 210 determines whether the pre-tightening was performed in accordance with the pre-tightening sequence. If the result of this determination is affirmative, the processing control unit 210 specifies the confirmation time as the pre-tightening completion time for the specified bolt arrangement. The processing control unit 210 then sets the specified pre-tightening completion as the new current status, registers the specified pre-tightening completion time as the pre-tightening time in the work history table WHT in association with the corresponding bolt arrangement, and transmits the registered content to the processing control unit 111 of the controller 110 as registration update information for the corresponding bolt arrangement. The content of the work table WTB in the storage unit 112 is updated according to the registration update information thus transmitted.

[0102] If the result of the determination is negative, the processing control unit 210 immediately notifies the office worker that the pre-tightening sequence is incorrect by a sound output from the sound output unit 250 and / or a display on the display unit 240, and also transmits the content of the notification as warning data to the processing control unit 111 of the controller 110. Upon receiving the warning data, the processing control unit 111 notifies the site worker of the warning content using the sound output unit 115 and / or the tablet terminal 130.

[0103] The on-site worker who receives the notification uses the electric impact 140 or the like to remove the bolts that have been pre-tightened in the wrong pre-tightening order, and then performs pre-tightening work in the correct pre-tightening order. Note that in this embodiment, for the bolt arrangement from which the bolt was removed, a new bolt is inserted and new pre-tightening is performed when it is the pre-tightening order.

[0104] Next, in step S25, it is determined whether all pre-tightening work has been completed. This determination is made by the processing control unit 210 by determining whether all of the tip portions of the bolts to be pre-tightened in the photographing results taken by the photographing unit 113 are colored white, and / or whether all of the current statuses STSmn registered in the work history table WHT indicate that pre-tightening has been completed.

[0105] If the result of the determination in step S25 is negative (step S25: N), the process returns to step S24. Then, the processes in steps S24 and S25 are repeated until the result of the determination in step S25 becomes positive.

[0106] The progress of the pre-tightening is presented to the office worker by displaying the image captured by the photographing unit 113 on the display unit 240.

[0107] Fig. 9 shows an example of an image of the object to be checked taken by the photographing unit 113 when pre-tightening for all bolts has been completed. In Fig. 9, the completion of pre-tightening and the coloring of the tip of the bolt in white are indicated by hatching the tip of the bolt with diagonal lines.

[0108] If the result of the determination in step S25 is affirmative (step S25: Y), the process proceeds to step S26. In this step S26, the processing control unit 210 determines the final bolt tightening order FFmn. The processing control unit 210 registers the result of this determination in the work history table WHT in the storage unit 220, and also transmits it as registration update information to the processing control unit 111 of the controller 110. In accordance with the registration update information thus transmitted, the contents of the work table WTB in the storage unit 112 are updated.

[0109] Subsequently, in step S27, the final tightening is performed using the nut runner 120 according to the final tightening sequence.

[0110] As described above, when it is determined that the final tightening of each bolt has been completed when the torque change gradient value calculated as the ratio of the torque value change imparted to the nut NT to be tightened relative to the amount of change in the rotation angle of the nut NT rotated during the tightening operation is equal to or less than a predetermined ratio of the maximum value after the start of the final tightening, it is determined that the so-called load-bearing point tightening has been successful, the tip of the corresponding bolt is colored yellow, the final tightening operation for the nut NT is terminated, and this is reported to the controller 110. Also, if the final tightening operation has not been completed within a predetermined time from the start, or if the total rotation angle from the start exceeds a predetermined angle, it is determined that the load-bearing point tightening has failed, the final tightening operation for the nut NT is terminated, and this is reported to the controller 110.

[0111] If it is determined that the tightening at the load point has failed, as described above, the red lamp in the notification means provided in the nut runner 120 lights up and a predetermined beep is output, thereby notifying the site worker that the tightening at the load point has failed. Upon recognizing this notification, the site worker uses the nut runner 120, the electric impact 140, etc. to remove the bolt that failed to be tightened at the load point, and then inserts a new bolt, pre-tightens it, and tightens it in that order in the bolt arrangement of the removed bolt.

[0112] On the other hand, if it is determined that the tightening at the load-bearing point has been successful, as described above, the green lamp of the notification means provided in the nut runner 120 lights up and a predetermined beep is output, thereby notifying the site worker that the tightening at the load-bearing point has been successful. In addition, a message that the final tightening has been successful is sent to the processing control unit 111 of the controller 110. The processing control unit 111 transmits a report that the final tightening has been successful to the processing control unit 210 of the office layout device 200.

[0113] When the report that the final tightening was successful is received, the processing control unit 210 determines that the tip of the bolt in the bolt arrangement corresponding to the report that the final tightening was successful has been colored yellow based on the photographing results by the photographing unit 113, and confirms the success of the final tightening. Note that the processing control unit 210 determines the bolt arrangement (m, n) of the bolt that was successfully tightened in the same manner as in steps S22 and S24 described above.

[0114] Once this confirmation is made, the processing control unit 210 determines whether the final tightening was performed in accordance with the final tightening order. If the result of this determination is affirmative, the processing control unit 210 specifies the confirmation time as the final tightening completion time for the specified bolt arrangement. The processing control unit 210 then sets the specified final tightening completion as the new current status, registers the specified final tightening completion time as the final tightening time in the work history table WHT in association with the corresponding bolt arrangement, and transmits the registered content to the processing control unit 111 of the controller 110 as registration update information for the corresponding bolt arrangement. The content of the work table WTB in the storage unit 112 is updated according to the registration update information thus transmitted.

[0115] If the result of the determination is negative, the processing control unit 210 immediately notifies the office worker that the final tightening order is incorrect by a sound output from the sound output unit 250 and / or a display on the display unit 240, and transmits the content of the notification as warning data to the processing control unit 111 of the controller 110. Upon receiving the warning data, the processing control unit 111 notifies the field worker of the warning content using the sound output unit 115 and / or the tablet terminal 130.

[0116] Upon receiving the notification, the on-site worker removes the bolts that were tightened in the wrong order using the nut runner 120, etc., and then tightens them in the correct order. In this embodiment, for the bolt arrangement from which the bolt was removed, when it is the turn for tightening, new bolts are inserted, pre-tightened, and tightened in this order.

[0117] The progress of the final tightening is presented to the office worker by displaying the image captured by the photographing unit 113 on the display unit 240.

[0118] Fig. 10 shows an example of an image of the object to be checked taken by the photographing unit 113 when final tightening of all bolts has been completed. In Fig. 10, the fact that final tightening has been completed and the tip of the bolt has been colored yellow is indicated by filling the tip of the bolt with black.

[0119] Next, in step S28, it is determined whether all of the final tightening work has been completed. This determination is made by the processing control unit 210 determining whether all of the tip portions of the bolts to be final tightened have been colored yellow based on whether the tip portions of all of the bolts to be final tightened have been colored yellow in the photographing results taken by the photographing unit 113 and / or whether all of the current statuses STSmn registered in the work history table WHT indicate that final tightening has been completed.

[0120] If the result of the determination in step S28 is negative (step S28: N), the process returns to step S27. Then, the processes in steps S27 and S28 are repeated until the result of the determination in step S28 becomes positive.

[0121] If the result of the determination in step S28 is affirmative (step S28: Y), the confirmation process for the current confirmation target is completed.

[0122] As described above, in this embodiment, the photographing unit 113 in the field installation equipment group 100 photographs the connecting plate CP2 in color from the side opposite to the insertion direction of the bolts BLT. Subsequently, the processing control unit 210 of the office installation device 200 performs image processing on the color image to detect the positions of the penetrated bolts BLT and to identify different colors according to the tightening state of the tip end on the opposite side of each bolt BLT whose position has been detected.

[0123] Next, the processing control unit 210 checks the position and fastening state of each of the inserted bolts based on the image processing results. The check results are displayed on the display unit 240 of the office layout device 200 to relevant parties other than the field workers.

[0124] Therefore, the bolt tightening position and tightening state can be checked with high accuracy while reducing the burden on the field workers, and the confirmation results can be presented to relevant parties other than the field workers.

[0125] In this embodiment, the processing control unit 210 uses a learning model based on machine learning to recognize a penetrated bolt, detect the position of the recognized bolt, and identify the color of the tip of the recognized bolt. This allows the presence of a bolt to be recognized efficiently and accurately while reducing dependency on the site environment, thereby enabling the bolt's position to be detected with high accuracy.

[0126] Furthermore, in this embodiment, there are three tightening states: a pre-tightening state, a pre-tightening state, and a final tightening state. In the pre-tightening state, the color of the recognized bolt tip is a first color. In the pre-tightening state, the color of the tip is a second color different from the first color that is applied to the tip when pre-tightening is complete. In the final tightening state, the color of the tip is a third color different from both the first color and the second color that is applied to the tip when final tightening is complete. Therefore, when pre-tightening or final tightening, which are often used in bolt tightening of splice plates on bridges, etc., is performed, the tightening state of the bolt can be confirmed for each bolt position.

[0127] In this embodiment, the final tightening is determined to be complete when the torque change gradient calculated as the ratio of the torque value change applied to the nut NT to the amount of change in the rotation angle of the nut NT as the final tightening operation of the bolt progresses after the completion of the pre-tightening or from the pre-tightening state becomes equal to or less than a predetermined ratio of the maximum value after the start of the final tightening. In other words, the final tightening of the bolt BLT is performed using the so-called yield point method. This allows for easy, stable, and rational bolt tightening with an applied axial force.

[0128] Furthermore, in this embodiment, a transmission unit is further provided that transmits information on the photographing results of the photographing unit 113 to an image unit located remotely from the photographing unit 113, so the processing control unit, which requires high information processing capabilities, can be installed in a good operating environment (for example, an office environment). This allows the tightening status of the bolts to be stably checked for each bolt position.

[0129] [Modification of the embodiment] The present invention is not limited to the above-described embodiment, and various modifications are possible within the scope of the present invention.

[0130] For example, in the above embodiment, the base marker and three general markers are affixed to the outside near the four corners of the bolt tightening work area. However, the markers may be affixed to any position that can cover the entire bolt tightening work area by including any one of the markers in the image captured by the camera unit and changing the capture target area in a manner that allows the center position of the marker to be detected. Furthermore, the number and positions of the base markers and general markers are not limited to this. For example, if the bolt tightening work area is large, the number of general markers may be four or more, and further, multiple base markers may be provided to provide multiple work coordinate systems.

[0131] Furthermore, according to the above embodiment, the processing control unit 210 sets the center position of the base marker as the coordinate origin, and then extracts the center position of the tip of the bolt used for temporary fastening and the center position of the bolt through hole HL that was not used for temporary fastening by using the machine learning model obtained by the above-mentioned machine learning. However, it is also possible to find the center position of the tip of the bolt and the center position of the bolt through hole by using a machine learning model through image processing without setting a coordinate origin, and extract the relative positions of the bolt and the bolt through hole.

[0132] In addition, in the above embodiment, the present invention is applied to checking the tightening position and tightening state when using a high-strength hexagon bolt, but the present invention may also be applied to checking the tightening position and tightening state when using a type of bolt other than a high-strength hexagon bolt.

[0133] In the above embodiment, the present invention is applied to bolt tightening when connecting a first bridge girder and a second bridge girder via a connecting plate at a joint where the first bridge girder and the second bridge girder are butted together. However, the present invention may also be applied to bolt tightening when assembling the steel frames of high-rise buildings or constructing relatively large buildings.

[0134] In the above embodiment, it is assumed that a wired system is used for communication between the controller and the nutrunner, but a wireless system may also be used for this communication.

[0135] In the above embodiment, the movement of the nut runner and the start of the tightening operation are controlled primarily by the on-site worker. However, the movement of the nut runner and the tightening operation may be controlled by an office-based device in response to instructions from an office worker. Furthermore, the nut runner may also be configured to perform a pre-tightening function.

[0136] In the above embodiment, the camera unit is attached to a helmet worn by a field worker. Alternatively, the camera unit may be mounted on a drone so that the camera position, field of view, and direction can be changed, and the office-based device may control these.

[0137] In the above embodiment, the bolt tightening positions are arranged in a matrix pattern. However, the bolt tightening positions may be arranged in a pattern other than the matrix pattern, such as a staggered pattern.

[0138] In the above embodiment, it is not assumed that the target to be confirmed is illuminated when photographing the target to be confirmed. However, when it is difficult to effectively photograph the target to be confirmed due to poor lighting conditions or the like, a lighting unit may be provided to illuminate the target to be confirmed.

[0139] In the above embodiment, completion of final tightening of the bolt is determined by the coloring of the tip of the bolt, but it may also be determined by detecting the sound of the bolt being tightened (the characteristic sound of final tightening). In this case, the confirmation unit can more appropriately confirm the bolt position and tightening state based on the processing results of the image processing unit and the sound analysis unit. Furthermore, completion of final tightening of the bolt may also be determined by detecting characteristic points such as marks other than the coloring.

[0140] Furthermore, in the above embodiment, the case where the final tightening is performed by the proof stress point method is shown, but the present invention is also applicable to the case where the final tightening is performed by the torque method.

[0141] In the above embodiment, the pretightening is colored white and the final tightening is colored yellow, but the colors are not limited to this. [Industrial Applicability]

[0142] As described above, the present invention is useful in the field of bolt tightening. [Explanation of symbols]

[0143] 100… Field equipment group 110 ... Controller 111 ... Processing control unit 112 ... Memory Unit 113... Shooting unit 114 ... Operation input unit 115 ... Display unit 116 ... Sound output unit 117...Sound Input Unit 118 … WAN communication unit (transmission unit) 119 ... Local communication unit 120... Nutliner 129 ... Guide mechanism 130...tablet device 140... Electric impact 200 … Office layout equipment 210... Processing control unit (image processing unit, confirmation unit) 220...Memory Unit 230 ... Operation input unit 240 ... Display unit (presentation unit) 250...sound output unit 260...Sound input unit 270 … WAN communication unit 300...Bolt tightening position confirmation system 500 … WAN communication network BG1, BG2 ... Bridge girders BLT...High strength hex bolt (bolt) CP1, CP2 ... Connecting plates NT... Nut WS1, WS2 ... Washers WHT: Work history table WTB: Work table

Claims

1. A bolt tightening position confirmation system for confirming a tightening state of each bolt passing through at least one of a plurality of through holes formed in a plate, for each bolt position, comprising: a photographing unit for photographing the plate in color from a side opposite to the insertion direction of the bolt; an image processing unit that performs image processing on the color image acquired by the photographing unit to detect the positions of the penetrated bolts and identifies the coloring results of the different colors according to the fastening state of the tip end portion on the opposite side of each bolt whose position has been detected; a confirmation unit that confirms the position and fastening state of each of the passed bolts based on the processing results by the image processing unit; a presentation unit that presents the confirmation result by the confirmation unit to a person concerned other than the on-site worker; A bolt tightening position confirmation system comprising:

2. The bolt tightening position confirmation system described in Claim 1, characterized in that the image processing unit uses a learning model based on machine learning to recognize the penetrated bolt, detect the position of the recognized bolt, and identify the color of the tip of the recognized bolt.

3. The tightening state includes three states: a pre-tightening state, a pre-tightening state, and a final tightening state. In the pre-tightened state, the recognized color of the tip portion of the bolt is a first color, In the pre-tightening state, the color of the tip portion is a second color that is different from the first color and that is colored to the tip portion when pre-tightening is completed, In the final fastening state, the color of the tip portion is a third color that is different from both the first color and the second color and that is colored to the tip portion when final fastening is completed.

3. The bolt tightening position confirmation system according to claim 2.

4. The final tightening is determined to be completed when a torque change gradient value calculated as a ratio of a change in torque value applied to the nut to a change in rotation angle of the nut associated with the bolt after the completion of the preliminary tightening or as the bolt tightening progresses from the pre-tightening state becomes equal to or less than a predetermined ratio of a maximum value after the start of the final tightening.

4. The bolt tightening position confirmation system according to claim 3.

5. 2. The bolt tightening position confirmation system according to claim 1, further comprising a transmission unit that transmits information on the photographing results of the photographing unit to the image processing unit located at a remote position from the photographing unit.

6. A bolt tightening position confirmation method for use in a bolt tightening position confirmation system that includes a photographing unit, an image processing unit, a confirmation unit, and a presentation unit, and that confirms, for each bolt position, a tightening state of each bolt that has passed through at least one of a plurality of through holes formed in a plate, the method comprising: a photographing step in which the photographing unit photographs the plate in color from a side opposite to the bolt insertion direction; an image processing step in which the image processing unit performs image processing on the color image acquired in the photographing step to detect the positions of the penetrated bolts and identify different colors for the fastening states of the tip ends on the opposite side of each bolt whose positions have been detected; a confirmation step in which the confirmation unit confirms the position and fastening state of each of the passed bolts based on the processing result of the image processing step; The confirmation unit uses the presentation unit to present the confirmation result in the confirmation step. a presentation step of presenting the information to related parties other than the field workers; A bolt tightening position confirmation method comprising:

Images