High-strength bolt tightening state evaluation device

Abstract

To accurately evaluate a fastening state of a high strength bolt.SOLUTION: Inputting an inputted image in which a high strength bolt to be evaluated is imaged into an extra length evaluation learned model 13M that has learned, as learned data, an inputted image for learning in which a high strength bolt is imaged, which is prepared in advance, and an extra length evaluation result, which is a result of evaluating an extra length that is a length of a shaft part of the high strength bolt imaged in the inputted image for learning, which protrudes from a nut surface to output the extra length evaluation result of the high strength bolt imaged in the inputted image; An extra length evaluation part 13 for evaluating the extra length of the high strength bolt detects each of the high strength bolt, the nut, the washer, and the marking attached to the member on the basis of the input image, calculates a rotation angle of each of the high strength bolt, the nut, and the washer with respect to the member, and evaluates the fastening state of the high strength bolt on the basis of each of the rotation angles.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a high-strength bolt tightening condition evaluation device for evaluating the tightening condition of a high-strength bolt. [Background technology]

[0002] When constructing architectural structures, high-strength bolts are widely used to join, for example, steel frames with filler plates, splice plates, etc. When high-strength bolts are used, tension is introduced into the high-strength bolts when they are tightened, generating frictional forces between the members to be joined, thereby forming a frictional joint between the members.

[0003] To properly achieve a friction joint using a high-strength bolt, the bolt must be tightened according to a predetermined procedure. Tightening a high-strength bolt is performed mainly in three steps: primary tightening, marking, and final tightening. In the primary tightening, the high-strength bolt is tightened to the extent that the components to be joined are in close contact with each other. In marking, a line is drawn using a line marker or similar tool so that it is continuous with the surfaces of the components to be joined, the washer, the nut, and the shank of the high-strength bolt protruding from the nut. In final tightening, the high-strength bolt is tightened and tension is introduced into the high-strength bolt. In this way, the components are frictionally joined by the high-strength bolt. Because the back surface of the nut is lubricated, during final tightening, the nut rotates relative to both the high-strength bolt and the washer. Therefore, after final tightening, it is possible to confirm that the high-strength bolt has been properly tightened by, for example, checking that the markings on the nut are not misaligned with the markings on the high-strength bolt and washer.

[0004] For example, when joining steel frames with filler plates or splice plates, a large number of high-strength bolts are installed in a matrix on the surface of the components. Since the tightening condition check described above must be performed on all of these high-strength bolts, it is extremely time-consuming. Therefore, automatic checks of the tightening condition of high-strength bolts are being considered. In this regard, Patent Document 1 discloses that when inspecting for loose bolts that fasten a structure, markings are applied to the bolts and the steel material at the bolt attachment point with a highly weather-resistant paint or the like beforehand when the bolts are fastened, and when inspecting for loose bolts, the bolt portion is enlarged in an image of the structure taken from the ground, and looseness of the bolt being inspected is determined by the deviation between the marking on the bolt and the marking on the steel material at the bolt attachment point. Furthermore, for example, Patent Document 2 describes a bolt position detection method for detecting the bolt position from an image of the bolt for inspection after the bolt has been tightened. Furthermore, Patent Document 3 discloses a system that detects the tightening state of a high-strength bolt fastened to a plate via a washer and a nut, using markings made on each of them before tightening. The system includes an imaging means that photographs the tightening state of the washer, nut, and high-strength bolt to the plate, a marking angle detection means that detects the marking angles of the markings made on the plate, washer, nut, and high-strength bolt based on the photographed images, and a determination means that determines the tightening state based on the detected marking angles. In Patent Document 3, the marking angles are detected by analyzing and processing the images photographed by the camera of the imaging means.

[0005] It is desirable to more accurately evaluate the tightening condition of high-strength bolts. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2005-3658 [Patent Document 2] Japanese Patent Application Publication No. 2018-194475 [Patent Document 3] Japanese Patent Application Publication No. 2018-9932 Summary of the Invention [Problem to be solved by the invention]

[0007] The problem to be solved by the present invention is to provide a high-strength bolt tightening condition evaluation device that can accurately evaluate the tightening condition of a high-strength bolt. [Means for solving the problem]

[0008] The present invention employs the following means to solve the above-mentioned problems: That is, the present invention is a high-strength bolt tightening state evaluation device that evaluates the tightening state of a high-strength bolt when joining members with the high-strength bolt, a washer, and a nut, and inputs an input image of the high-strength bolt to be evaluated into a loose length evaluation trained model that has been trained using, as training data, a previously prepared learning input image of the high-strength bolt and a loose length evaluation result that is an evaluation result regarding the loose length, which is the length of the shank of the high-strength bolt that protrudes from the surface of the nut, as imaged in the learning input image, and calculates the loose length evaluation result of the high-strength bolt that is imaged in the input image. The present invention provides a high-strength bolt tightening state evaluation device comprising: an excess length evaluation unit that outputs the results and evaluates the excess length of the high-strength bolt; a rotation angle evaluation unit that detects each of the high-strength bolt, the nut, the washer, and markings on the member based on the input image, calculates the rotation angle of each of the high-strength bolt, the nut, and the washer relative to the member, and evaluates the tightening state of the high-strength bolt based on each of the rotation angles; and a comprehensive evaluation unit that comprehensively evaluates the tightening state of the high-strength bolt based on the evaluation of the excess length and an evaluation based on the rotation angle. With the above configuration, the rotation angle evaluation unit detects the markings on the high-strength bolt, nut, washer, and other components based on an input image of the high-strength bolt to be evaluated, and calculates the rotation angle of the high-strength bolt, nut, and washer relative to each component. Based on each of the rotation angles calculated in this way, it is possible to detect, for example, the occurrence of washer corotation, where the washer rotates together with the nut, or the occurrence of rotation around the bolt axis, where only the high-strength bolt rotates. In this way, the tightening condition of the high-strength bolt can be appropriately evaluated based on the rotation angle. Furthermore, with high-strength bolts, after final tightening, if the excess length of the shank (the length that protrudes from the surface of the nut) is too short, for example, so that the tip of the shank is positioned closer to the component than the surface of the nut and the entire shank is contained within the nut, the groove in the nut located outside the tip of the shank is not used to tighten the high-strength bolt. Furthermore, if the excess length is too long, the shank of the high-strength bolt may have stretched. In either of these cases, sufficient tension may not be introduced into the high-strength bolt, resulting in an inadequate friction joint. In contrast, the above configuration includes a margin evaluation unit that inputs an input image into a margin evaluation trained model that has been trained using a pre-prepared learning input image of a high-strength bolt and a margin evaluation result that is an evaluation result regarding the margin of the high-strength bolt imaged in the learning input image as training data, outputs the margin evaluation result of the high-strength bolt imaged in the input image, and evaluates the margin of the high-strength bolt. This allows the margin of the high-strength bolt to be evaluated with high accuracy. The tightening condition of the high-strength bolt is evaluated comprehensively based on the evaluation of the excess length and the evaluation based on the rotation angle, thereby enabling the tightening condition of the high-strength bolt to be evaluated with high accuracy.

[0009] In one aspect of the present invention, the rotation angle evaluation unit includes a marking detection trained model that has been machine-trained to detect, from the learning input image, portions corresponding to each of the markings affixed to the high-strength bolt, the nut, the washer, and the member, as well as the end face of the shank of the high-strength bolt, the surface of the nut, the surface of the washer, and the side face of the washer, and the rotation angle evaluation unit inputs an adjusted image obtained by adjusting the input image into the marking detection trained model to detect the markings affixed to the high-strength bolt, the nut, the washer, and the member, as well as portions corresponding to the end face of the shank of the high-strength bolt, the surface of the nut, the surface of the washer, and the side face of the washer, and detecting the markings on the shank of the high-strength bolt, the nut, the washer, and the member, and the portions corresponding to the end face of the shank of the high-strength bolt, the surface of the nut, the surface of the washer, and the side face of the washer, and then detecting a high-strength bolt point in the adjustment image that is the center of the end face of the shank of the high-strength bolt, based on the portions corresponding to the end face of the shank of the high-strength bolt, the surface of the nut, the surface of the washer, and the side face of the washer. The positions of the bolt center, the nut center which is the center of the surface of the nut, the washer center which is the center of the surface of the washer, and the component center which is the position where the axis of the shank intersects on the surface of the component are calculated, the high-strength bolt angle which is the angle between the high-strength bolt center and the marking affixed to the high-strength bolt in the adjusted image is calculated, the nut angle which is the angle between the nut center and the marking affixed to the nut in the adjusted image is calculated, the washer angle which is the angle between the washer center and the marking affixed to the washer in the adjusted image is calculated, and the component angle which is the angle between the component center and the marking affixed to the component in the adjusted image is calculated, and the rotation angle of each of the high-strength bolt, the nut, and the washer relative to the component is calculated based on the high-strength bolt angle, the nut angle, the washer angle, and the component angle. According to the above configuration, by inputting an adjusted image obtained by adjusting the input image into a marking detection trained model, each of the markings attached to the high-strength bolt, nut, washer, and component, as well as the parts corresponding to the end face of the shank of the high-strength bolt, the surface of the nut, the surface of the washer, and the side of the washer, are detected. First, based on the portions corresponding to the end face of the shank of the high-strength bolt, the surface of the nut, and the surface of the washer, for example, three points are arbitrarily selected from the periphery of each of these end faces or surfaces, and the equation of a circle is applied to calculate the positions of the high-strength bolt center (the center of the end face of the shank of the high-strength bolt), the nut center (the center of the surface of the nut), and the washer center (the center of the surface of the washer) in the adjusted image. Furthermore, the outer periphery of the side of the washer captured in the adjusted image is the portion of the area corresponding to the washer in the adjusted image that is closest to the component and forms the boundary between the washer and the surface of the component. The shank of the bolt passes through the center of the area enclosed by this boundary. Therefore, by arbitrarily selecting three points from this boundary, i.e., the outer periphery of the side of the washer, and applying the equation of a circle, the position of the component center in the adjusted image, which is the position where the center of the shank of the high-strength bolt, i.e., the axis center, intersects with the surface of the component, can be calculated. This allows the following to be calculated: the high-strength bolt angle, which is the angle formed by the line connecting the center of the high-strength bolt and the marking on the high-strength bolt in the adjusted image; the nut angle, which is the angle formed by the line connecting the center of the nut and the marking on the nut in the adjusted image; the washer angle, which is the angle formed by the line connecting the center of the washer and the marking on the washer in the adjusted image; and the component angle, which is the angle formed by the line connecting the center of the component and the marking on the component in the adjusted image. Based on the high-strength bolt angle, nut angle, washer angle, and member angle calculated in this manner, the rotation angles of the high-strength bolt, nut, and washer relative to each member can be calculated. In this way, the rotation angle evaluation unit can be appropriately realized.

[0010] In another aspect of the present invention, the device further includes a nut rotation angle average difference evaluation unit that calculates the average value of the rotation angle of the nut calculated by the rotation angle evaluation unit for each of the plurality of input images, and evaluates, for each of the plurality of input images, whether the difference between the rotation angle of the nut calculated for that input image and the average value is greater than or equal to a lower limit angle threshold and less than or equal to an upper limit angle threshold. According to the above configuration, by determining whether the difference between the rotation angle of the nut of the high-strength bolt captured in the input image and the average value of the rotation angles of the nuts of the high-strength bolts captured in each of the other multiple input images including the input image is greater than or equal to a lower angle threshold and less than or equal to an upper angle threshold, it is possible to detect if there is a high-strength bolt that has been tightened so that the nut has rotated particularly more or less than other high-strength bolts. [Effects of the Invention]

[0011] According to the present invention, it is possible to provide a high-strength bolt tightening state evaluation device that can accurately evaluate the tightening state of a high-strength bolt. [Brief explanation of the drawings]

[0012] [Figure 1] This is a perspective view of a high-strength bolt after primary tightening and before final tightening. [Figure 2] FIG. 1 is a perspective view of a high-strength bolt after it has been properly tightened. [Figure 3] 1 is a block diagram of a high-strength bolt tightening state evaluation system and a high-strength bolt tightening state evaluation device according to an embodiment of the present invention. [Figure 4] FIG. 10 is a diagram showing an example of an evaluation target image in which a plurality of high-strength bolts are captured. [Figure 5] This is an oblique view of a high-strength bolt after final tightening, in which the excess length is shorter than the lower threshold value and the end face of the shank of the high-strength bolt is located within the internal space of the nut. [Figure 6]FIG. 10 is a perspective view of a high-strength bolt after final tightening, in which the excess length is longer than the upper limit threshold value. [Figure 7] FIG. 10 is a diagram illustrating an example of an adjusted image obtained by adjusting an input image. [Figure 8] FIG. 8 is a diagram showing an example of a segment extraction image obtained by detecting markings and surfaces of each member in the adjusted image shown in FIG. 7. [Figure 9] 1 is an explanatory diagram of a high-strength bolt angle, a nut angle, a washer angle, and a component angle. [Figure 10] This is a plan view of a high-strength bolt in which an installer has added a post-tightening marking after final tightening. [Figure 11] This is a plan view of a high-strength bolt after it has been fully tightened by an installer without making any markings. [Figure 12] FIG. 1 is a plan view of a high-strength bolt in a state where the washer rotates together with the bolt during final tightening. [Figure 13] FIG. 1 is a plan view of a high-strength bolt in a state where rotation of the bolt axis occurs during final tightening. [Figure 14] FIG. 10 is a plan view of a high-strength bolt when the nut is turned too little during final tightening. [Figure 15] 10 is a flowchart of a method for evaluating the tightening state of a high-strength bolt in the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] The present invention is a high-strength bolt tightening condition evaluation device that evaluates the tightening condition of high-strength bolts. The high-strength bolt tightening condition evaluation device comprehensively evaluates the tightening condition of high-strength bolts based on an image of the excess length of the high-strength bolt and the marking information attached to the high-strength bolt, nut, washer, and member. Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a perspective view of a high-strength bolt after primary tightening and before final tightening. When constructing an architectural structure, multiple components 50, such as steel frames, filler plates, and splice plates, are abutted against one another and joined by high-strength bolts 60. In FIG. 1 , a shank 61 of the high-strength bolt 60 penetrates from the bottom to the top of the component 50 through a bolt through-hole (not shown) opened in the component 50. A washer 80 is abutted against a surface 50f of the component 50. The shank 61 penetrates the washer 80. The washer 80 has a front and a back, and the back faces the component 50, while the front forms a surface 80f facing outward (upward in FIG. 1 ), opposite the component 50. A nut 70 is attached to the shank 61 at a position opposite the washer 80 with respect to the component 50. The nut 70 is threaded onto the shank 61 so as to abut against the surface 80f of the washer 80. The nut 70 has a front and a back, and is provided so that the back faces the washer 80 and the front forms a surface 70f facing outward, opposite the washer 80. A grade mark 72 is provided on the front of the nut 70.

[0014] When high-strength bolts 60 are used, tension introduced into the high-strength bolts 60 as they are tightened generates frictional force between the components 50, thereby frictionally joining the components 50 together. To properly achieve this frictional joining, the high-strength bolts 60 are tightened mainly in three stages: primary tightening, marking, and final tightening. In the primary tightening, the high-strength bolts 60 are tightened to the extent that they bring the components 50 into tight contact with each other. In the marking, continuous lines are drawn using a line marker or the like on the components 50, the washer 80, the nut 70, and the shank 61 of the high-strength bolt 60 protruding from the nut 70. 1, a marking 63 is provided on the side surface 61s of the shank 61 of the high-strength bolt 60, a marking 73 is provided on the surface 70f and side surface 70s of the nut 70, a marking 83 is provided on the surface 80f and side surface 80s of the washer 80, and a marking 53 is provided on the surface 50f of the member 50. These markings 53, 63, 73, and 83 are provided so as to be continuous with each other in a stage after the primary tightening.

[0015] In this embodiment, a torsion-type high-strength bolt is used as the high-strength bolt 60. A torsion-type high-strength bolt has a pintail 65 provided at the tip of a shank 61. The pintail 65 breaks during final tightening. The high-strength bolt tightening state evaluation device 3 may be configured to target high-strength hexagon bolts. In this embodiment, since the target is a torsion-type high-strength bolt as described above, the high-strength bolt tightening state evaluation device 3 is configured to perform processing to determine whether or not the pintail 65 breaks during final tightening, as will be described later. However, when targeting high-strength hexagon bolts, the high-strength bolt tightening state evaluation device 3 may be configured so as not to perform this processing.

[0016] FIG. 2 is a perspective view of a high-strength bolt after it has been properly tightened. During final tightening, the high-strength bolt 60 is tightened, tension is introduced into the high-strength bolt 60, and the components 50 are frictionally joined. Because the back surface of the nut 70 is lubricated, during final tightening, the nut 70 rotates relative to the high-strength bolt 60 and also rotates relative to the washer 80. As a result, when final tightening is performed correctly, as shown in FIG. 2, the marking 73 on the nut 70 is rotated and misaligned with the markings 53, 63, and 83 on the high-strength bolt 60, washer 80, and component 50. Also, FIG. 2 illustrates that, as a result of correct final tightening, the pintail 65 is broken, and an end face 61f is depicted as a fracture surface located at the tip of the shank 61.

[0017] FIG. 3 is a block diagram of a high-strength bolt tightening state evaluation system and a high-strength bolt tightening state evaluation device according to this embodiment. The high-strength bolt tightening state evaluation system 1 includes a mobile terminal 2 and a high-strength bolt tightening state evaluation device 3. The high-strength bolt tightening state evaluation system 1 and the high-strength bolt tightening state evaluation device 3 evaluate the tightening state of a high-strength bolt 60 when joining members 50 together using the high-strength bolt 60, a washer 80, and a nut 70. The high-strength bolt tightening state evaluation device 3 is a computer equipped with hardware such as a processor and memory. The high-strength bolt tightening state evaluation device 3 functionally comprises an evaluation target image acquisition unit 11, an input image generation unit 12, a slack evaluation unit 13, a pintail fracture evaluation unit 14, a nut front / back evaluation unit 15, a washer front / back evaluation unit 16, a rotation angle evaluation unit 17, a nut rotation angle average difference evaluation unit 18, and an overall evaluation unit 19. The input image generation unit 12 comprises a high-strength bolt detection trained model 12M. The slack evaluation unit 13 comprises a slack evaluation trained model 13M. The pintail fracture evaluation unit 14 comprises a pintail fracture evaluation trained model 14M. The nut front / back evaluation unit 15 comprises a nut front / back evaluation trained model 15M. The washer front / back evaluation unit 16 comprises a washer front / back evaluation trained model 16M. The rotation angle evaluation unit 17 comprises a marking detection trained model 17M.

[0018] The mobile terminal 2 is a device such as a tablet terminal or smartphone that can be carried to a construction site by an evaluation worker who evaluates the tightening condition of the high-strength bolt 60, and is equipped with a camera that can take images of the high-strength bolt 60 being evaluated at the construction site. FIG. 4 is a diagram showing an example of an evaluation target image in which a plurality of high-strength bolts are captured. The evaluation target image 30, which is an image of the high-strength bolt 60 to be evaluated after final tightening, taken by the evaluation worker using the camera of the mobile terminal 2, is transmitted to the high-strength bolt tightening condition evaluation device 3 via a public communication network or the like that allows wireless or wired communication. The evaluation target image acquisition unit 11 acquires the evaluation target image 30 transmitted in this manner.

[0019] As shown in FIG. 4 , when joining components 50, a large number of high-strength bolts 60 are provided in a matrix. Therefore, a plurality of high-strength bolts 60 are captured in the evaluation target image 30. The high-strength bolt tightening state evaluation device 3 evaluates the tightening state of each of the plurality of high-strength bolts 60 captured in the evaluation target image 30. To this end, the input image generation unit 12 extracts and generates an input image 31, which will be input to each evaluation unit described later, from the evaluation target image 30 so that only one high-strength bolt 60 is captured. The input image generation unit 12 generates input images 31 for all of the high-strength bolts 60 captured in the evaluation target image 30. In FIG. 4, for example, for a high-strength bolt 60A imaged in the upper left and a high-strength bolt 60B imaged to the immediate right, the outline of an input image 31 that can be generated for these is shown by a two-dot chain line.

[0020] The input image generation unit 12 detects the high-strength bolt 60 captured in the evaluation target image 30 using the high-strength bolt detection trained model 12M. The high-strength bolt detection trained model 12M is realized, for example, by multiple convolutional layers followed by multiple deconvolutional layers, and is configured to detect high-strength bolts 60 in the evaluation target image 30 through semantic segmentation. Specifically, the high-strength bolt detection trained model 12M is trained using training data that includes a pre-prepared training input image of a high-strength bolt 60 and labeled images, in which each object captured in the training input image is labeled, as training data corresponding to the training input image. The high-strength bolt detection trained model 12M is trained by adjusting the values ​​of parameters, weights, etc., using backpropagation or gradient descent so that when a training input image is input, the output result is similar to the labeled image corresponding to the training input image. As a result, the high-strength bolt detection trained model 12M is trained to output an inference result, i.e., a labeled image, that is similar to the corresponding training data when a training input image is input.

[0021] In this way, a high-strength bolt detection trained model 12M is generated as a trained model that is used as a program module that is part of artificial intelligence software, with learning parameters such as filter parameters and weights learned. The high-strength bolt detection trained model 12M, which has been deep-trained as described above, is configured to output a labeled image in which objects in the image 30 to be evaluated are labeled when the image 30 to be evaluated is input.

[0022] The input image generation unit 12 inputs the evaluation target image 30 into the high-strength bolt detection trained model 12M to generate a labeled image. The input image generation unit 12 sets an area in the labeled image so as to surround a portion labeled corresponding to the high-strength bolt 60 (and nut 70, washer 80), and cuts out the image along that area to generate the input image 31. If there are multiple portions labeled corresponding to the high-strength bolt 60 (and nut 70, washer 80) in the evaluation target image 30, the input image generation unit 12 generates multiple input images 31 corresponding to each of those portions.

[0023] The excess length evaluation unit 13 evaluates the excess length of the high-strength bolt 60 after final tightening. The excess length of the high-strength bolt 60 is the length of the part of the shank 61 of the high-strength bolt 60 that protrudes from the surface 70f of the nut 70. Fig. 5 is a perspective view of the high-strength bolt after final tightening, in which the excess length is shorter than the lower threshold value and the end face of the shank of the high-strength bolt is located within the internal space of the nut. Fig. 6 is a perspective view of the high-strength bolt after final tightening, in which the excess length is longer than the upper threshold value. As shown in Figure 5, if the excess length is too short, for example, and the end face 61f of the shank 61 of the high-strength bolt 60 is located within an insertion hole formed in the center of the nut 70 into which the shank 61 is threaded, making the inner wall 70t of the insertion hole visible, the groove of the nut 70 formed in the visible portion of the inner wall 70t is not used to tighten the high-strength bolt. Also, if the excess length is too long, as shown in Figure 6, the shank 61 of the high-strength bolt 60 may have stretched. In either of these cases, sufficient tension may not be introduced into the high-strength bolt 60, resulting in an inadequate friction joint.

[0024] The excess length evaluation unit 13 uses the excess length evaluation trained model 13M to evaluate the excess length of the high-strength bolt 60 captured in each of the multiple input images 31 generated by the input image generation unit 12. The slack evaluation trained model 13M is configured to evaluate the slack of the high-strength bolt 60 captured in the input image 31, for example, by using a convolutional neural network (CNN) realized by multiple convolution layers followed by a fully connected layer. Specifically, the slack evaluation trained model 13M is trained using, as training data, a previously prepared training input image in which the high-strength bolt 60 is captured, and a slack evaluation result, which is an evaluation result regarding the slack of the high-strength bolt 60 captured in the training input image, as teacher data corresponding to the training input image. If the margin in the image input during learning is shorter than the lower threshold and is therefore abnormal, the margin evaluation result will be, for example, "the margin is shorter than the lower threshold." If the margin in the image input during learning is normal and is equal to or greater than the lower threshold and equal to or less than the upper threshold, the margin evaluation result will be, for example, "the margin is equal to or greater than the lower threshold and equal to or less than the upper threshold." If the margin in the image input during learning is longer than the upper threshold and is therefore abnormal, the margin evaluation result will be, for example, "the margin is longer than the upper threshold." In this way, the margin evaluation trained model 13M is trained to evaluate margin using three-class classification. Accordingly, the margin evaluation trained model 13M has three output nodes in the output layer of the fully connected layer, each corresponding to a margin evaluation result, such as "the margin is shorter than the lower threshold," "the margin is equal to or greater than the lower threshold and equal to or less than the upper threshold," and "the margin is longer than the upper threshold." When a training input image is input, the margin evaluation trained model 13M undergoes machine learning by adjusting the values ​​of parameters, weights, etc. using backpropagation or gradient descent so that the output node corresponding to the margin evaluation result of the training input image takes a value close to 1 and the other output nodes take values ​​close to 0. As a result, the margin evaluation trained model 13M is trained so that when a training input image is input, it outputs an inference result close to the corresponding training data as a margin evaluation result.

[0025] In this way, a spare length evaluation trained model 13M is generated as a trained model that is used as a program module that is part of artificial intelligence software, with learning parameters such as filter parameters and weights learned. The excess length evaluation trained model 13M, which has been deep-learned as described above, is configured to output the excess length evaluation result of the high-strength bolt 60 in the input image 31 when the input image 31 is input.

[0026] The slack evaluation unit 13 sequentially inputs input images 31 of the high-strength bolt 60 to be evaluated, generated by the input image generation unit 12, into the above-described slack evaluation trained model 13M. As a result of inputting the input images 31 into the slack evaluation trained model 13M, the slack evaluation unit 13 detects the output node with the largest value closest to 1 from among the multiple output nodes of the slack evaluation trained model 13M, and outputs the slack evaluation result corresponding to that output node as the slack evaluation result for the high-strength bolt 60 captured in the input image 31. In this way, the slack evaluation unit 13 evaluates the slack of the high-strength bolt 60 for each of the multiple input images 31. In this embodiment, the excess length is evaluated as the number of threads formed on the shank 61 protruding from the surface 70f of the nut 70. In this case, the lower threshold value can be set to, for example, 1 thread, and the upper threshold value can be set to, for example, 6 threads.

[0027] The pintail breakage evaluation unit 14 evaluates whether or not the pintail 65 of the high-strength bolt 60 has broken after final tightening. The pintail breakage evaluation unit 14 uses the pintail breakage evaluation learned model 14M to evaluate whether the pintail 65 of the high-strength bolt 60 imaged in each of the multiple input images 31 generated by the input image generation unit 12 is broken. The pintail fracture evaluation trained model 14M is configured to evaluate whether or not the pintail 65 of the high-strength bolt 60 captured in the input image 31 is fractured, for example, by using a CNN realized by a plurality of convolutional layers followed by a fully connected layer. Specifically, the pintail fracture evaluation trained model 14M is trained using, as training data, a training input image in which the high-strength bolt 60 is captured, and a pintail fracture evaluation result, which is an evaluation result as to whether or not the pintail 65 of the high-strength bolt 60 captured in the training input image, as teacher data corresponding to the training input image. If the pintail 65 is broken in the learning input image, which is normal, the pintail breakage evaluation result will be, for example, "no pintail (broken)." If the pintail 65 is not broken in the learning input image, which is abnormal, the pintail breakage evaluation result will be, for example, "pintail present (not broken)." In this way, the pintail breakage evaluation trained model 14M is trained to evaluate whether or not the pintail 65 is broken using two-class classification. Accordingly, the pintail breakage evaluation trained model 14M has two output nodes in the output layer of the fully connected layer, each corresponding to a pintail breakage evaluation result, for example, "no pintail (broken)" and "no pintail (broken)." When a training input image is input, the pintail breakage evaluation trained model 14M undergoes machine learning by adjusting the values ​​of parameters, weights, etc. using backpropagation or gradient descent so that, when a training input image is input, the output node corresponding to the pintail breakage evaluation result of the training input image has a value close to 1, and the other output nodes have values ​​close to 0. As a result, the pintail breakage evaluation trained model 14M is trained so that, when a training input image is input, it outputs an inference result close to the corresponding training data as a pintail breakage evaluation result.

[0028] In this way, a pintail breakage evaluation trained model 14M is generated as a trained model that is used as a program module that is part of artificial intelligence software, with learning parameters such as filter parameters and weights learned. The pintail fracture evaluation learned model 14M, which has been deep-learned as described above, is configured to output the pintail fracture evaluation result of the high-strength bolt 60 in the input image 31 when the input image 31 is input.

[0029] The pintail fracture evaluation unit 14 sequentially inputs input images 31 of the high-strength bolt 60 to be evaluated, generated by the input image generation unit 12, to the pintail fracture evaluation trained model 14M described above. As a result of inputting the input images 31 to the pintail fracture evaluation trained model 14M, the pintail fracture evaluation unit 14 detects the output node with the largest value closest to 1 from among the multiple output nodes of the pintail fracture evaluation trained model 14M, and outputs the pintail fracture evaluation result corresponding to that output node as the pintail fracture evaluation result for the high-strength bolt 60 captured in the input image 31. In this way, the pintail fracture evaluation unit 14 evaluates whether the pintail 65 of the high-strength bolt 60 is fractured for each of the multiple input images 31.

[0030] The nut front / back evaluation unit 15 evaluates whether the front and back positions of the nut 70 screwed onto the high-strength bolt 60 are normal. That is, the nut front / back evaluation unit 15 evaluates whether the nut 70 is in a normal position, with the surface 70f of the nut 70 facing away from the member 50 being the "front" side, on which the grade mark 72 is provided and whose edge is chamfered, or whether the nut is in an inverted position, which is different from the normal position, with the "back" side appearing on the surface 70f.

[0031] The nut front and back evaluation unit 15 uses the nut front and back evaluation trained model 15M to evaluate whether the front and back postures of the nut 70 captured in each of the multiple input images 31 generated by the input image generation unit 12 are normal. The nut front / back evaluation trained model 15M is configured to evaluate whether the front and back postures of the nut 70 captured in the input image 31 are normal or not, for example, by using a CNN realized by multiple convolutional layers followed by a fully connected layer. Specifically, the nut front / back evaluation trained model 15M is trained using, as training data, a previously prepared training input image in which the high-strength bolt 60 is captured, and a nut front / back evaluation result, which is an evaluation result as to whether the front and back postures of the nut 70 captured in the training input image are normal or not, as teacher data corresponding to the training input image. In the learning input image, if the "front" side of the nut 70 appears on the surface 70f, which is normal, the nut front / back evaluation result will be, for example, "the posture of the nut is normal." In the learning input image, if the "back" side of the nut 70 appears on the surface 70f, which is abnormal, the nut front / back evaluation result will be, for example, "the posture of the nut is abnormal." In this way, the nut front / back evaluation trained model 15M is trained to evaluate whether the posture of the front and back of the nut 70 is normal or not by two-class classification. Accordingly, the nut front / back evaluation trained model 15M has two output nodes in the output layer of the fully connected layer, each corresponding to a nut front / back evaluation result, for example, "nut posture is normal" and "nut posture is abnormal." When a training input image is input, the nut front / back evaluation trained model 15M performs machine learning by adjusting the values ​​of parameters, weights, etc. using backpropagation or gradient descent so that, when a training input image is input, the output node corresponding to the nut front / back evaluation result of the training input image becomes a value close to 1, and the other output nodes become values ​​close to 0. As a result, the nut front / back evaluation trained model 15M is trained to output, as a nut front / back evaluation result, an inference result close to the corresponding training data when a training input image is input.

[0032] In this way, a nut front and back evaluation trained model 15M is generated as a trained model that is used as a program module that is part of artificial intelligence software, with learning parameters such as filter parameters and weights learned. The nut front / back evaluation trained model 15M, which has been deep learned as described above, is configured to output the nut front / back evaluation result of the nut 70 in the input image 31 when the input image 31 is input.

[0033] The nut front / back evaluation unit 15 sequentially inputs input images 31 of the high-strength bolt 60 to be evaluated, generated by the input image generation unit 12, to the nut front / back evaluation trained model 15M described above. As a result of inputting the input images 31 to the nut front / back evaluation trained model 15M, the nut front / back evaluation unit 15 detects the output node with the largest value closest to 1 from among the multiple output nodes of the nut front / back evaluation trained model 15M, and outputs the nut front / back evaluation result corresponding to that output node as the nut front / back evaluation result for the nut 70 captured in the input image 31. In this way, the nut front / back evaluation unit 15 evaluates whether the posture of the front and back of the nut 70 is normal for each of the multiple input images 31.

[0034] The washer front / back evaluation unit 16 evaluates whether the front and back orientations of the washer 80 are normal. That is, the washer front / back evaluation unit 16 evaluates whether the washer 80 is in a normal orientation, in which the surface 80f of the washer 80 facing away from the member 50 is the "front" side with a chamfered edge, or whether the washer 80 is in an inverted orientation, which is different from the normal orientation, in which the "back" side appears on the surface 80f.

[0035] The washer front / back evaluation unit 16 uses the washer front / back evaluation learned model 16M to evaluate whether the front and back postures of the washer 80 captured in each of the multiple input images 31 generated by the input image generation unit 12 are normal. The washer front-back evaluation trained model 16M is configured to evaluate whether the orientation of the front and back of the washer 80 captured in the input image 31 is normal or not, for example, by using a CNN realized by a plurality of convolution layers and subsequent fully connected layers. Specifically, the washer front-back evaluation trained model 16M is trained using, as training data, a training input image in which a high-strength bolt 60 is captured, and a washer front-back evaluation result, which is an evaluation result regarding whether the orientation of the front and back of the washer 80 captured in the training input image is normal or not, as teacher data corresponding to the training input image. In the learning input image, if the "front" side of the washer 80 appears on the surface 80f, which is normal, the washer front / back evaluation result will be, for example, "the washer posture is normal." In the learning input image, if the "back" side of the washer 80 appears on the surface 80f, which is abnormal, the washer front / back evaluation result will be, for example, "the washer posture is abnormal." In this way, the washer front / back evaluation trained model 16M is trained to evaluate whether the front / back posture of the washer 80 is normal or not by two-class classification. Accordingly, the washer front / back evaluation trained model 16M has two output nodes in the output layer of the fully connected layer, each corresponding to a washer front / back evaluation result, for example, "washer posture is normal" and "washer posture is abnormal." When a learning input image is input, the washer front / back evaluation trained model 16M undergoes machine learning by adjusting the values ​​of parameters, weights, etc. using backpropagation or gradient descent so that, when a learning input image is input, the output node corresponding to the washer front / back evaluation result of the learning input image has a value close to 1, and the other output nodes have values ​​close to 0. As a result, the washer front / back evaluation trained model 16M is trained to output, as a washer front / back evaluation result, an inference result close to the corresponding training data when a learning input image is input.

[0036] In this way, a washer front and back evaluation trained model 16M is generated as a trained model that is used as a program module that is part of artificial intelligence software, with learning parameters such as filter parameters and weights learned. The washer front / back evaluation trained model 16M, which has been deep-learned as described above, is configured to output the washer front / back evaluation result of the washer 80 in the input image 31 when the input image 31 is input.

[0037] The washer front / back evaluation unit 16 sequentially inputs input images 31 of the high-strength bolt 60 to be evaluated, generated by the input image generation unit 12, to the washer front / back evaluation trained model 16M described above. As a result of inputting the input images 31 to the washer front / back evaluation trained model 16M, the washer front / back evaluation unit 16 detects the output node with the largest value closest to 1 from among the multiple output nodes of the washer front / back evaluation trained model 16M, and outputs the washer front / back evaluation result corresponding to that output node as the washer front / back evaluation result for the washer 80 captured in the input image 31. In this way, the washer front / back evaluation unit 16 evaluates whether the front / back orientation of the washer 80 is normal for each of the multiple input images 31.

[0038] The rotation angle evaluation unit 17 detects each of the markings 53, 63, 73, 83 attached to the high-strength bolt 60, nut 70, washer 80, and member 50 for each of the input images 31, calculates the rotation angle of each of the high-strength bolt 60, nut 70, and washer 80 relative to the member 50, and evaluates the tightening state of the high-strength bolt 60 based on each of the rotation angles. FIG. 7 is a diagram showing an example of an adjusted image obtained by adjusting an input image. First, the rotation angle evaluation unit 17 adjusts the input image 31 to generate an adjusted image 32. Specifically, the rotation angle evaluation unit 17 generates the adjusted image 32 by performing a projective transformation on the input image 31 in order to accurately perform angle-related calculations as will be described later.

[0039] To perform the evaluation of the rotation angle, the rotation angle evaluation unit 17 detects each of the markings 53, 63, 73, and 83 attached to the high-strength bolt 60, the nut 70, the washer 80, and the member 50, and portions corresponding to the end face 61f of the shank 61 of the high-strength bolt 60, the surface 70f of the nut 70, the surface 80f of the washer 80, and the side surface 80s of the washer 80, in the adjusted image 32. The rotation angle evaluation unit 17 performs these detections using the marking detection trained model 17M. The marking detection trained model 17M is realized by, for example, multiple convolutional layers followed by multiple deconvolutional layers, and is configured to detect, by semantic segmentation, each of the markings 53, 63, 73, and 83 attached to the high-strength bolt 60, the nut 70, the washer 80, and the component 50, as well as portions corresponding to the end face 61f of the shank 61 of the high-strength bolt 60, the surface 70f of the nut 70, the surface 80f of the washer 80, and the side surface 80s of the washer 80, in the adjusted image 32. Specifically, the marking detection trained model 17M is trained using, as training data, a previously prepared training input image in which the high-strength bolt 60 is captured, and labeled images in which each of the objects captured in the training input image is labeled, as training data corresponding to the training input image. The marking detection trained model 17M is machine-learned by adjusting the values ​​of parameters, weights, etc. using backpropagation or gradient descent so that when a learning input image is input, the output result is close to a labeled image corresponding to the learning input image. As a result, the marking detection trained model 17M is trained so that when a learning input image is input, it outputs an inference result, i.e., a labeled image, that is close to the corresponding training data.

[0040] In this way, a marking detection trained model 17M is generated as a trained model that is used as a program module that is part of artificial intelligence software, with learning parameters such as filter parameters and weights learned. The marking detection learned model 17M, which has been deep-learned as described above, is configured to, when an adjusted image 32 is input, output a segment extraction image in which each of the markings 53, 63, 73, 83 attached to each of the high-strength bolt 60, nut 70, washer 80, and component 50 in the adjusted image 32, and each of the portions corresponding to the end face 61f of the shank 61 of the high-strength bolt 60, the surface 70f of the nut 70, the surface 80f of the washer 80, and the side surface 80s of the washer 80 are labeled and extracted as segments.

[0041] The rotation angle evaluation unit 17 inputs the adjusted image 32 into the marking detection learned model 17M to generate a segment extraction image, thereby obtaining the positions (pixel coordinates) of the marking 63 on the high-strength bolt 60, the marking 73 on the nut 70, the marking 83 on the washer 80, the marking 53 on the member 50, the end face 61f of the shank 61 of the high-strength bolt 60, the surface 70f of the nut 70, the surface 80f of the washer 80, and the side surface 80s of the washer 80 in the adjusted image 32. Fig. 8 is an example of a segment extraction image. In the segment extraction image 33 shown in Fig. 8, the portions corresponding to the marking 63 on the high-strength bolt 60, the marking 73 on the nut 70, the marking 83 on the washer 80, and the marking 53 on the member 50 are displayed surrounded by rectangles.

[0042] FIG. 9 is an explanatory diagram of the high-strength bolt angle, nut angle, washer angle, and member angle. Next, the rotation angle evaluation unit 17 calculates the position of the high-strength bolt center 61c, which is the center of the end face 61f of the shank 61 of the high-strength bolt 60, in the adjusted image 32, based on the portion corresponding to the end face 61f of the shank 61 of the high-strength bolt 60 in the segment extraction image 33. The rotation angle evaluation unit 17 calculates the high-strength bolt center 61c, for example, by arbitrarily acquiring three points from the outer periphery of the end face 61f and applying the equation of a circle. Furthermore, the rotation angle evaluation unit 17 calculates the position of the nut center 70c, which is the center of the surface 70f of the nut 70, in the adjusted image 32, based on the portion corresponding to the surface 70f of the nut 70 in the segment extraction image 33. The rotation angle evaluation unit 17 calculates the nut center 70c, for example, by arbitrarily acquiring three points from the outer periphery of the surface 70f and applying the equation of a circle. Furthermore, the rotation angle evaluation unit 17 calculates the position of a washer center 80c, which is the center of the surface 80f of the washer 80, in the adjusted image 32, based on the portion corresponding to the surface 80f of the washer 80 in the segment extraction image 33. The rotation angle evaluation unit 17 calculates the washer center 80c, for example, by arbitrarily acquiring three points from the outer periphery of the surface 80f and applying an equation of a circle. Furthermore, the rotation angle evaluation unit 17 calculates the position of the component center 50c, which is the position where the axis of the shank 61 of the high-strength bolt 60 intersects with the surface 50f of the component 50. The component center 50c is the same as the center of the surface of the washer 80 opposite the surface 80f (the lower side, back side in FIG. 9). Therefore, in this embodiment, the rotation angle evaluation unit 17 calculates the component center 50c by arbitrarily acquiring three points from the edge (outer periphery) 80t of the portion corresponding to the side surface 80s of the washer 80 in the segment extraction image 33, which is the outer periphery of the surface of the washer 80 opposite the surface 80f, and applying the equation of a circle.

[0043] Next, the rotation angle evaluation unit 17 calculates the high-strength bolt angle, which is the angle between a horizontal line 61l that passes through the high-strength bolt center 61c and extends horizontally, and a line 61m that connects the marking 63 affixed to the high-strength bolt 60 and the high-strength bolt center 61c in the adjusted image 32. More precisely, the rotation angle evaluation unit 17 calculates the high-strength bolt angle as the angle between the horizontal line 61l and a line 61m that connects a portion of the marking 63 located on the outer periphery of the end face 61f of the shaft portion 61 and the high-strength bolt center 61c. Furthermore, the rotation angle evaluation unit 17 calculates the nut angle, which is the angle between a horizontal line 70l that passes through the nut center 70c and extends horizontally, and a line 70m that connects the marking 73 on the nut 70 and the nut center 70c in the adjusted image 32. More precisely, the rotation angle evaluation unit 17 calculates the nut angle as the angle between the horizontal line 70l and a line 70m that connects the nut center 70c and a portion of the marking 73 located on the outer periphery of the surface 70f. Furthermore, the rotation angle evaluation unit 17 calculates the washer angle, which is the angle between a horizontal line 80l that passes through the washer center 80c and extends horizontally, and a line 80m that connects the marking 83 on the washer 80 and the washer center 80c in the adjusted image 32. More precisely, the rotation angle evaluation unit 17 calculates the washer angle as the angle between the horizontal line 80l and a line 80m that connects the portion of the marking 83 located on the outer periphery of the surface 80f and the washer center 80c. Furthermore, the rotation angle evaluation unit 17 calculates the member angle, which is the angle between a horizontal line 50l that passes through the member center 50c and extends horizontally in the adjusted image 32, and a line 50m that connects the marking 83 affixed to the member 50 and the member center 50c. More precisely, the rotation angle evaluation unit 17 calculates the member angle as the angle between the horizontal line 50l and a line 50m that connects the part of the marking 53 located on the edge (outer periphery) 80t of the side surface 80s of the washer 80 and the member center 50c.

[0044] The rotation angle evaluation unit 17 calculates the high-strength bolt rotation angle, nut rotation angle, and washer rotation angle, which are the rotation angles of the high-strength bolt 60, nut 70, and washer 80 relative to each member 50, based on the high-strength bolt angle, nut angle, washer angle, and member angle calculated as described above. The rotation angle evaluation unit 17 evaluates the tightening state of the high-strength bolt 60 by using the high-strength bolt rotation angle, the nut rotation angle, and the washer rotation angle as the case may be.

[0045] The rotation angle evaluation unit 17 detects post-tightening markings made by a builder at a location where a marking should be placed if the final tightening of the high-strength bolt 60 has been completed normally, in order to falsely claim that the final tightening of the high-strength bolt 60 has been completed normally, even though the final tightening of the high-strength bolt 60 has not been completed normally. FIG. 10 is a plan view of a high-strength bolt in which a builder has provided a post-tightening marking after final tightening. When a post-tightening marking is made, two markings, one made in the correct procedure after the primary tightening and one made in the post-tightening procedure, are made on either the high-strength bolt 60, the nut 70, or the washer 80. For example, in the example of FIG. 10 , the washer 80 is made with a marking 83 made in the correct procedure after the primary tightening and a post-tightening marking 83d. The rotation angle evaluation unit 17 checks whether two or more markings 63, 73, 83 are made on either the high-strength bolt 60, the nut 70, or the washer 80. If two or more markings 63, 73, 83 are not made on any of the high-strength bolt 60, the nut 70, and the washer 80, the rotation angle evaluation unit 17 evaluates that the high-strength bolt 60 captured in the input image 31, from which the adjusted image 32 was generated, was in a normal state, with no post-tightening marking made during the final tightening. If two or more markings 63, 73, 83 are provided on any of the high-strength bolt 60, nut 70, and washer 80, the rotation angle evaluation unit 17 evaluates that this is an abnormal state in which the markings were added later when the high-strength bolt 60 captured in the input image 31, which was the basis for generating the adjustment image 32, was fully tightened. In this way, the rotation angle evaluation unit 17 outputs an evaluation result regarding the post-added marking for each of the input images 31.

[0046] The rotation angle evaluation unit 17 detects a state in which marking is not performed after the primary tightening and the final tightening is performed, that is, the marking is omitted. FIG. 11 is a plan view of a high-strength bolt after it has been fully tightened by an installer without marking it. If marking is omitted, the high-strength bolt 60, nut 70, and washer 80 will have no markings. The rotation angle evaluation unit 17 checks whether markings 63, 73, and 83 are provided on the high-strength bolt 60, nut 70, and washer 80. If markings 63, 73, and 83 are provided on the high-strength bolt 60, nut 70, and washer 80, the rotation angle evaluation unit 17 evaluates that markings were provided during final tightening of the high-strength bolt 60 captured in the input image 31 from which the adjusted image 32 was generated, and that this is a normal state. If markings 63, 73, and 83 are not provided on the high-strength bolt 60, nut 70, and washer 80, the rotation angle evaluation unit 17 evaluates that marking was omitted during final tightening of the high-strength bolt 60 captured in the input image 31 from which the adjusted image 32 was generated, and that this is an abnormal state. In this way, the rotation angle evaluation unit 17 outputs an evaluation result regarding omission of marking for each of the input images 31.

[0047] The rotation angle evaluation unit 17 detects washer co-rotation, that is, when the washer 80 rotates together with the nut 70 during final tightening. FIG. 12 is a plan view of a high-strength bolt in a state where the washer rotates together with the bolt during final tightening. When washer co-rotation occurs, the washer 80 rotates together with the nut 70, reducing the difference between the nut rotation angle and the washer rotation angle. Therefore, the rotation angle evaluation unit 17 determines whether the difference between the nut rotation angle and the washer rotation angle is equal to or less than a predetermined washer co-rotation determination threshold. The washer co-rotation determination threshold may be set to, for example, 5°. The washer co-rotation determination threshold may also be set to another angle. When the difference between the nut rotation angle and the washer rotation angle is greater than the washer co-rotation determination threshold, the rotation angle evaluation unit 17 determines that washer co-rotation did not occur during final tightening of the high-strength bolt 60 captured in the input image 31 from which the adjustment image 32 was generated, and that the state is normal. If the difference between the nut rotation angle and the washer rotation angle is equal to or less than the washer co-rotation judgment threshold, the rotation angle evaluation unit 17 evaluates that an abnormal state has occurred in which the washer co-rotated during the final tightening of the high-strength bolt 60 captured in the input image 31 from which the adjustment image 32 was generated. In this way, the rotation angle evaluation unit 17 outputs an evaluation result regarding the rotation with the washer for each input image 31.

[0048] The rotation angle evaluation unit 17 detects the rotation of only the high-strength bolt 60 around the bolt axis during final tightening. FIG. 13 is a plan view of a high-strength bolt in a state where rotation of the bolt axis occurs during final tightening. When rotation around the bolt axis occurs, only the high-strength bolt 60 rotates, so the high-strength bolt rotation angle is larger than the nut rotation angle. Therefore, the rotation angle evaluation unit 17 determines whether the high-strength bolt rotation angle is smaller than the nut rotation angle. If the high-strength bolt rotation angle is smaller than the nut rotation angle, the rotation angle evaluation unit 17 evaluates that no rotation around the bolt axis occurred during final tightening of the high-strength bolt 60 captured in the input image 31 from which the adjusted image 32 was generated, indicating a normal state. If the high-strength bolt rotation angle is equal to or greater than the nut rotation angle, the rotation angle evaluation unit 17 evaluates that rotation around the bolt axis occurred during final tightening of the high-strength bolt 60 captured in the input image 31 from which the adjusted image 32 was generated, indicating an abnormal state. In this way, the rotation angle evaluation unit 17 outputs an evaluation result regarding the rotation around the bolt axis for each of the input images 31.

[0049] The rotation angle evaluation unit 17 detects whether the rotation angle of the nut 70 is too small or too large during final tightening, ie, whether the nut is over-rotated or over-rotated. FIG. 14 is a plan view of a high-strength bolt in the case where the nut is turned too little during final tightening. When a nut under-rotation or over-rotation occurs, the nut rotation angle is either under-rotated or over-rotated. Therefore, the rotation angle evaluation unit 17 determines whether the nut rotation angle is equal to or greater than a predetermined nut under-rotation threshold and equal to or less than a predetermined nut over-rotation threshold. When the nut rotation angle is equal to or greater than the nut under-rotation threshold and equal to or less than the nut over-rotation threshold, the rotation angle evaluation unit 17 evaluates that a nut under-rotation or over-rotation did not occur during the final tightening of the high-strength bolt 60 captured in the input image 31 from which the adjusted image 32 was generated, and that this is a normal state. When the nut rotation angle is equal to or less than the nut under-rotation threshold or equal to or greater than the nut over-rotation threshold, the rotation angle evaluation unit 17 evaluates that a nut under-rotation or over-rotation occurred during the final tightening of the high-strength bolt 60 captured in the input image 31 from which the adjusted image 32 was generated, and that this is an abnormal state. In this way, the rotation angle evaluation unit 17 outputs an evaluation result regarding under-rotation and over-rotation of the nut for each input image 31.

[0050] As already explained, the rotation angle evaluation unit 17 calculates the nut rotation angles for all the high-strength bolts 60 captured in the evaluation target image 30. The nut rotation angle average difference evaluation unit 18 evaluates the degree of variation in nut rotation angles among all high-strength bolts 60 captured in the evaluation target image 30 based on the nut rotation angles calculated for all of these high-strength bolts 60. To do this, the nut rotation angle average difference evaluation unit 18 first calculates the average value of the nut rotation angles calculated by the rotation angle evaluation unit 17 for each of the multiple input images 31. Then, for each of the multiple input images 31, the nut rotation angle average difference evaluation unit 18 determines whether the difference between the nut rotation angle calculated for that input image 31 and the average value is greater than or equal to a lower limit angle threshold and less than or equal to an upper limit angle threshold. The lower limit threshold can be set to an angle 30° smaller than the average value. The upper limit threshold can be set to an angle 30° larger than the average value. If the nut rotation angle is greater than or equal to the lower limit threshold and less than or equal to the upper limit threshold, the nut rotation angle average difference evaluation unit 18 evaluates that the rotation angle of the nut 70 is appropriate and normal for final tightening of the high-strength bolt 60. If the nut rotation angle is smaller than the lower limit threshold or greater than the upper limit threshold, the nut rotation angle average difference evaluation unit 18 evaluates that the rotation angle of the nut 70 is inappropriate and abnormal for final tightening of the high-strength bolt 60. In this way, the nut rotation angle average difference evaluation unit 18 outputs, for each of the input images 31, an evaluation result regarding the difference from the average value of the nut rotation angle.

[0051] The overall evaluation unit 19 performs an overall evaluation of the fastening state of the high-strength bolt 60 based on the various evaluation results described above. For each input image 31, the comprehensive evaluation unit 19 obtains the following: the excess length evaluation result from the excess length evaluation unit 13, the pintail breakage evaluation result from the pintail breakage evaluation unit 14, the nut front / back evaluation result from the nut front / back evaluation unit 15, the washer front / back evaluation result from the washer front / back evaluation unit 16, the evaluation result regarding retrofit marking from the rotation angle evaluation unit 17, the evaluation result regarding forgotten marking, the evaluation result regarding washer co-rotation, the evaluation result regarding bolt shaft rotation, the evaluation result regarding nut under-rotation / over-rotation, and the evaluation result regarding the difference from the average value of the nut rotation angle from the nut rotation angle average difference evaluation unit 18. If the above evaluation results include any input image 31 that has received a negative evaluation such as inappropriate or problematic, the comprehensive evaluation unit 19 creates an output image in which the portion of the evaluation target image 30 corresponding to the high-strength bolt 60 corresponding to that input image 31 is distinguished from other high-strength bolts 60 by appropriate means such as coloring it in red or surrounding it with a frame. In this output image, for example, for a high-strength bolt 60 that has received a negative evaluation and is displayed in a distinctive manner, the detailed evaluation results may be added, for example, as a text file. The overall evaluation unit 19 transmits the generated output image to the mobile terminal 2.

[0052] The mobile terminal 2 receives the output image from the high-strength bolt tightening state evaluation device 3. By viewing the output image on the mobile terminal 2, the evaluator can identify and understand the high-strength bolts 60 that have been negatively evaluated, i.e., that have been evaluated as not being properly tightened. Furthermore, in cases where the evaluation results are attached to the output image as a text file, if, for example, the evaluator selects a high-strength bolt 60 in the output image that has been evaluated as having an inappropriate tightening state, the reason why the tightening state has been evaluated as being inappropriate can be understood by appropriate means, such as by displaying the text file on the screen.

[0053] 1 to 14 and 15, a method for evaluating the tightening state of a high-strength bolt using the tightening state evaluation system 1 and the tightening state evaluation device 3 for a high-strength bolt will be described. Fig. 15 is a flowchart of the method for evaluating the tightening state of a high-strength bolt. First, an evaluation worker at a construction site takes an image of a high-strength bolt 60, the fastening state of which is to be evaluated, using the camera of the mobile terminal 2 to generate an evaluation target image 30. The mobile terminal 2 transmits the evaluation target image 30 to the high-strength bolt fastening state evaluation device 3. The evaluation target image acquisition unit 11 acquires the transmitted evaluation target image 30 (step S1).

[0054] The input image generation unit 12 inputs the evaluation target image 30 into the high-strength bolt detection trained model 12M to generate a labeled image. The input image generation unit 12 sets an area in the labeled image to surround a portion labeled corresponding to the high-strength bolt 60 (and nut 70, washer 80), and cuts out the image along that area to generate the input image 31. If there are multiple portions labeled corresponding to the high-strength bolt 60 (and nut 70, washer 80) in the evaluation target image 30, the input image generation unit 12 generates multiple input images 31 corresponding to each of those portions (step S2).

[0055] The slack evaluation unit 13 sequentially inputs input images 31 of the high-strength bolt 60 to be evaluated, generated by the input image generation unit 12, to the slack evaluation trained model 13M. As a result of inputting the input images 31 to the slack evaluation trained model 13M, the slack evaluation unit 13 detects the output node with the largest value closest to 1 from among the multiple output nodes of the slack evaluation trained model 13M, and outputs the slack evaluation result corresponding to that output node as the slack evaluation result for the high-strength bolt 60 captured in the input image 31. In this way, the slack evaluation unit 13 evaluates the slack of the high-strength bolt 60 for each of the multiple input images 31 (step S3).

[0056] The pintail fracture evaluation unit 14 sequentially inputs input images 31 of the high-strength bolt 60 to be evaluated, which have been generated by the input image generation unit 12, to the pintail fracture evaluation trained model 14M. As a result of inputting the input images 31 to the pintail fracture evaluation trained model 14M, the pintail fracture evaluation unit 14 detects the output node with the largest value closest to 1 from among the multiple output nodes of the pintail fracture evaluation trained model 14M, and outputs the pintail fracture evaluation result corresponding to that output node as the pintail fracture evaluation result for the high-strength bolt 60 captured in the input image 31. In this way, the pintail fracture evaluation unit 14 evaluates whether the pintail 65 of the high-strength bolt 60 is fractured for each of the multiple input images 31 (step S4).

[0057] The nut front / back evaluation unit 15 sequentially inputs input images 31 of the high-strength bolt 60 to be evaluated, generated by the input image generation unit 12, to the nut front / back evaluation trained model 15M. As a result of inputting the input images 31 to the nut front / back evaluation trained model 15M, the nut front / back evaluation unit 15 detects the output node with the largest value closest to 1 from among the multiple output nodes of the nut front / back evaluation trained model 15M, and outputs the nut front / back evaluation result corresponding to that output node as the nut front / back evaluation result of the nut 70 imaged in the input image 31. In this way, the nut front / back evaluation unit 15 evaluates whether the posture of the front and back of the nut 70 is normal for each of the multiple input images 31 (step S5).

[0058] The washer front / back evaluation unit 16 sequentially inputs input images 31 of the high-strength bolt 60 to be evaluated, which have been generated by the input image generation unit 12, to the washer front / back evaluation trained model 16M. As a result of inputting the input images 31 to the washer front / back evaluation trained model 16M, the washer front / back evaluation unit 16 detects the output node with the largest value closest to 1 from among the multiple output nodes of the washer front / back evaluation trained model 16M, and outputs the washer front / back evaluation result corresponding to that output node as the washer front / back evaluation result for the washer 80 captured in the input image 31. In this way, the washer front / back evaluation unit 16 evaluates whether the orientation of the front and back of the washer 80 is normal for each of the multiple input images 31 (step S6).

[0059] The rotation angle evaluation unit 17 inputs the adjusted image 32 into the marking detection learned model 17M to generate a segment extraction image, thereby obtaining the positions (pixel coordinates) of the marking 63 on the high-strength bolt 60, the marking 73 on the nut 70, the marking 83 on the washer 80, the marking 53 on the member 50, the end face 61f of the shank 61 of the high-strength bolt 60, the surface 70f of the nut 70, the surface 80f of the washer 80, and the side surface 80s of the washer 80 in the adjusted image 32 (step S7). The rotation angle evaluation unit 17 calculates the positions of the high-strength bolt center 61c, the nut center 70c, the washer center 80c, and the member center 50c. Based on this, the rotation angle evaluation unit 17 calculates the high-strength bolt rotation angle, the nut rotation angle, and the washer rotation angle, which are the rotation angles of the high-strength bolt 60, the nut 70, and the washer 80 relative to the member 50 (step S8). The rotation angle evaluation unit 17 evaluates the tightening state of the high-strength bolt 60 based on the high-strength bolt rotation angle, the nut rotation angle, and the washer rotation angle (step S9). Specifically, the rotation angle evaluation unit 17 evaluates whether or not any of the following has occurred: post-marking, omission of marking, co-rotation with the washer, rotation around the bolt axis, or under- or over-rotation of the nut.

[0060] The nut rotation angle average difference evaluation unit 18 calculates the average value of the nut rotation angles calculated by the rotation angle evaluation unit 17 for each of the multiple input images 31. Then, the nut rotation angle average difference evaluation unit 18 determines whether the difference between the nut rotation angle calculated for each of the multiple input images 31 and the average value is equal to or greater than a lower limit angle threshold and equal to or less than an upper limit angle threshold. If the nut rotation angle is smaller than the lower limit threshold or larger than the upper limit threshold, the nut rotation angle average difference evaluation unit 18 evaluates that the rotation angle of the nut 70 is inappropriate and is in an abnormal state when finally tightening the high-strength bolt 60 (step S10). The overall evaluation unit 19 performs an overall evaluation of the fastening state of the high-strength bolt 60 based on the various evaluation results described above.

[0061] The high-strength bolt tightening state evaluation device 3 as described above is a high-strength bolt tightening state evaluation device 3 that evaluates the tightening state of a high-strength bolt 60 when joining members 50 using the high-strength bolt 60, a washer 80, and a nut 70, and inputs an input image 31 in which the high-strength bolt 60 to be evaluated is captured to a margin evaluation trained model 13M that has been trained using as training data a previously prepared learning input image in which the high-strength bolt 60 is captured and a margin evaluation result that is an evaluation result regarding the margin that is the length of the shank 61 of the high-strength bolt 60 captured in the learning input image as teacher data. the rotation angle evaluation unit 17 that detects each of the markings 53, 63, 73, 83 on the high-strength bolt 60, nut 70, washer 80, and member 50 based on the input image 31, calculates the rotation angle of each of the high-strength bolt 60, nut 70, and washer 80 relative to the member 50, and evaluates the tightening state of the high-strength bolt 60 based on each of the rotation angles; and the overall evaluation unit 19 that comprehensively evaluates the tightening state of the high-strength bolt 60 based on the evaluation of the excess length and an evaluation based on the rotation angle. With the above configuration, the rotation angle evaluation unit 17 detects each of the markings 53, 63, 73, and 83 on the high-strength bolt 60, nut 70, washer 80, and component 50 based on the input image 31 capturing the high-strength bolt 60 to be evaluated, and calculates the rotation angle of each of the high-strength bolt 60, nut 70, and washer 80 relative to the component 50. Based on each of the rotation angles calculated in this manner, it is possible to detect, for example, the occurrence of washer corotation, in which the washer 80 rotates together with the nut 70, or the occurrence of rotation around the bolt axis, in which only the high-strength bolt 60 rotates. In this way, the tightening state of the high-strength bolt 60 can be appropriately evaluated based on the rotation angle. Furthermore, in the case of the high-strength bolt 60, after final tightening, if the excess length of the shank 61, which is the length of the shank 61 that protrudes from the surface 70f of the nut 70, is too short, for example, so that the tip of the shank 61 is located closer to the component 50 than the surface 70f of the nut 70, and the entire shank 61 is housed within the nut 70, the groove of the nut 70 located outside the tip of the shank 61 is not used to tighten the high-strength bolt 60. Furthermore, if the excess length is too long, the shank 61 of the high-strength bolt 60 may have stretched. In either of these cases, sufficient tension may not be introduced into the high-strength bolt 60, resulting in an inadequate friction joint. In contrast, the above configuration includes a slack evaluation unit 13 that inputs an input image 31 into a slack evaluation trained model 13M that has been trained using, as training data, a previously prepared learning input image of the high-strength bolt 60 and a slack evaluation result that is an evaluation result regarding the slack of the high-strength bolt 60 imaged in the learning input image, and outputs the slack evaluation result of the high-strength bolt 60 imaged in the input image 31, thereby evaluating the slack of the high-strength bolt 60. This allows the slack of the high-strength bolt 60 to be evaluated with high accuracy. Based on the evaluation of the excess length and the evaluation based on the rotation angle thus performed, the tightening state of the high-strength bolt 60 is comprehensively evaluated. Therefore, the tightening state of the high-strength bolt 60 can be evaluated with high accuracy.

[0062] The rotation angle evaluation unit 17 also includes a marking detection trained model 17M that has been machine-trained to detect, from the learning input image, portions corresponding to the markings 53, 63, 73, and 83 attached to the high-strength bolt 60, the nut 70, the washer 80, and the member 50, as well as the end face 61f of the shank 61 of the high-strength bolt 60, the surface 70f of the nut 70, the surface 80f of the washer 80, and the side surface 80s of the washer 80. The marking detection trained model 17M is trained to detect, from the learning input image, an adjusted image 32 obtained by adjusting the input image 31, The markings 53, 63, 73, 83 on each of the high-strength bolt 60, the nut 70, the washer 80, and the member 50, and the portions corresponding to the end face 61f of the shank 61 of the high-strength bolt 60, the surface 70f of the nut 70, the surface 80f of the washer 80, and the side surface 80s of the washer 80 are detected. Based on the portions corresponding to the end face 61f of the shank 61 of the high-strength bolt 60, the surface 70f of the nut 70, the surface 80f of the washer 80, and the side surface 80s of the washer 80, the center of the end face 61f of the shank 61 of the high-strength bolt 60 in the adjusted image 32 is determined. The positions of the high-strength bolt center 61c, the nut center 70c which is the center of the surface 70f of the nut 70, the washer center 80c which is the center of the surface 80f of the washer 80, and the member center 50c which is the position where the axis of the shank 61 intersects on the surface 50f of the member 50 are calculated, and the high-strength bolt angle which is the angle formed by the line 61m connecting the high-strength bolt center 61c and the marking 63 on the high-strength bolt 60 in the adjusted image 32 is calculated, and the nut center 70c and the marking 73 on the nut 70 in the adjusted image 32 are calculated. The nut angle is calculated as the angle formed by the line 70m connecting the washer center 80c and the marking 83 on the washer 80 in the adjustment image 32, the washer angle is calculated as the angle formed by the line 80m connecting the washer center 80c and the marking 83 on the washer 80 in the adjustment image 32, and the component angle is calculated as the angle formed by the line 50m connecting the component center 50c and the marking 53 on the component 50 in the adjustment image 32, and the rotation angles of each of the high-strength bolt 60, nut 70, and washer 80 relative to the component 50 are calculated based on the high-strength bolt angle, nut angle, washer angle, and component angle. According to the above configuration, by inputting the adjusted image 32 obtained by adjusting the input image 31 into the marking detection trained model 17M, each of the markings 53, 63, 73, 83 attached to each of the high-strength bolt 60, nut 70, washer 80, and component 50, and each of the portions corresponding to the end face 61f of the shaft portion 61 of the high-strength bolt 60, the surface 70f of the nut 70, the surface 80f of the washer 80, and the side surface 80s of the washer 80 are detected. First, based on the portions corresponding to the end face 61f of the shank 61 of the high-strength bolt 60, the surface 70f of the nut 70, and the surface 80f of the washer 80, for example, three points are arbitrarily selected from the outer peripheries of the end face 61f or the surfaces 70f, 80f for each of these end face 61f and surfaces 70f, 80f, and a circle equation is applied to them to calculate the positions of the high-strength bolt center 61c, which is the center of the end face 61f of the shank 61 of the high-strength bolt 60, the nut center 70c, which is the center of the surface 70f of the nut 70, and the washer center 80c, which is the center of the surface 80f of the washer 80, in the adjusted image 32. In addition, the outer periphery 80t of the side surface 80s of the washer 80 captured in the adjusted image 32 is the portion located closest to the component 50 within the region corresponding to the washer 80 in the adjusted image 32, and forms the boundary line between the washer 80 and the surface 50f of the component 50. In the component 50, the shank 61 of the high-strength bolt 60 passes through the center of the portion surrounded by this boundary line. Therefore, by arbitrarily obtaining three points from this boundary line, i.e., the outer periphery 80t of the side surface 80s of the washer 80, and applying the equation of a circle, it is possible to calculate the position of the component center 50c, which is the position where the center of the shank 61 of the high-strength bolt 60, i.e., the axis, intersects with the surface 50f of the component 50 in the adjusted image 32. This allows the following to be calculated: the high-strength bolt angle, which is the angle formed by the line 61m connecting the high-strength bolt center 61c and the marking 63 on the high-strength bolt 60 in the adjusted image 32; the nut angle, which is the angle formed by the line 70m connecting the nut center 70c and the marking 73 on the nut 70 in the adjusted image 32; the washer angle, which is the angle formed by the line 80m connecting the washer center 80c and the marking 83 on the washer 80 in the adjusted image 32; and the component angle, which is the angle formed by the line 50m connecting the component center 50c and the marking 53 on the component 50 in the adjusted image 32. Based on the high-strength bolt angle, nut angle, washer angle, and member angle calculated in this manner, the rotation angles of the high-strength bolt 60, nut 70, and washer 80 relative to each member 50 can be calculated. In this way, the rotation angle evaluation unit 17 can be appropriately realized.

[0063] The device further includes a nut rotation angle average difference evaluation unit 18 that calculates the average value of the rotation angles of the nut 70 calculated by the rotation angle evaluation unit 17 for each of the multiple input images 31, and evaluates whether the difference between the rotation angle of the nut 70 calculated for each of the multiple input images 31 and the average value is greater than or equal to a lower limit angle threshold and less than or equal to an upper limit angle threshold. According to the above configuration, by determining whether the difference between the rotation angle of the nut 70 of the high-strength bolt 60 captured in the input image 31 and the average value of the rotation angles of the nuts 70 of the high-strength bolts 60 captured in each of the other multiple input images 31 including the input image 31 is greater than or equal to the lower limit angle threshold and less than or equal to the upper limit angle threshold, it is possible to detect if there is a high-strength bolt that has been tightened so that the nut 70 has rotated particularly more or less than other high-strength bolts.

[0064] In addition, the excess length evaluation result indicates whether the number of threads formed on the shaft portion 61 protruding from the surface 70f of the nut 70, as excess length, is greater than the lower threshold value and less than the upper threshold value, or is shorter than the lower threshold value, or is longer than the upper threshold value. According to the above configuration, the extra length evaluation unit 13 can be appropriately realized.

[0065] The high-strength bolt tightening state evaluation device 3 also includes a nut front / back evaluation unit 15 that inputs an input image 31 into a nut front / back evaluation trained model 15M that has been trained using the learning input image and, as training data, a nut front / back evaluation result, which is an evaluation result as to whether the front and back postures of the nut 70 captured in the learning input image, and outputs the nut front / back evaluation result of the nut 70 captured in the input image 31, and evaluates whether the front and back postures of the nut 70 are normal. According to the above-described configuration, the front and back postures of the nut 70 can be evaluated.

[0066] The model also includes a washer front / back evaluation unit 16 that inputs an input image 31 into a washer front / back evaluation trained model 16M that has been trained using as training data an input image during learning and a washer front / back evaluation result, which is an evaluation result as to whether the front and back postures of the washer 80 captured in the learning input image as training data, and outputs the washer front / back evaluation result of the washer 80 captured in the input image 31, thereby evaluating whether the front and back postures of the washer 80 are normal. According to the above configuration, the orientation of the front and back of the washer 80 can be evaluated.

[0067] In addition, when the high-strength bolt 60 is a torsion-type high-strength bolt having a pintail 65, the high-strength bolt tightening state evaluation device 3 further includes a pintail fracture evaluation unit 14 that inputs an input image 31 into a pintail fracture evaluation trained model 14M that has been trained using as training data the input image at the time of learning and, as teacher data, a pintail fracture evaluation result that is an evaluation result as to whether or not the pintail 65 in the input image at the time of learning is fractured, outputs the pintail fracture evaluation result of the high-strength bolt 60 captured in the input image 31, and evaluates whether or not the pintail 65 is fractured. According to the above-described configuration, when the high-strength bolt 60 is a torsion-type high-strength bolt having a pintail 65, it is possible to evaluate whether the pintail 65 has broken during final tightening.

[0068] The high-strength bolt tightening condition evaluation device of the present invention is not limited to the above-described embodiment explained with reference to the drawings, and various other modifications are conceivable within the technical scope thereof. For example, in the above embodiment, each trained model is realized by semantic segmentation or CNN, but this is not limiting. Each trained model may be realized by another type of machine learning device. Furthermore, in the above embodiment, the pintail fracture evaluation unit inputs photographed images of high-strength bolts into a machine-learned pintail fracture evaluation trained model, which is a program module of artificial intelligence software, and evaluates whether or not the pintail of the high-strength bolt is fractured for each input image. However, without being limited to a software program module, the pintail fracture evaluation unit may also be configured so that an operator visually inspects photographed images of the high-strength bolt, evaluates whether or not the pintail of the high-strength bolt is fractured, and inputs the pintail fracture evaluation information into the high-strength bolt tightening condition evaluation device. In addition to this, it is possible to select and discard the configurations given in the above embodiments, or to change them to other configurations as appropriate. [Explanation of symbols]

[0069] 3 High-strength bolt tightening condition evaluation device 50c Member center 12 Input image generation unit 50f Surface of component 12M High-strength bolt detection trained model 53 Markings on components 13 Extra length evaluation section 60 High strength bolt 13M Extra length evaluation trained model 61 Shaft 14 Pintail fracture evaluation section 61c High strength bolt center 14M Pintail fracture evaluation trained model 61f End face of the shaft of a high-strength bolt 15 Nut front and back evaluation section 63 Markings on high strength bolts 15M Nut front and back evaluation trained model 65 pin tail 16 Washer front and back evaluation section 70 Nut 16M Washer front and back evaluation trained model 70c nut center 17 Rotation angle evaluation section 70f Nut surface 17M Marking detection trained model 73 Markings on nuts 18 Nut rotation angle average difference evaluation section 80 Washer 19 Overall evaluation section 80c Washer center 31 Input image 80f Surface of washer 32 Adjustment image 80s Side of washer 50 Component 83 Marking on washer 50m A line connecting the center of the component and the marking on the component 61m A line connecting the center of the high-strength bolt and the marking on the high-strength bolt 70m The line connecting the center of the nut and the marking on the nut 80m Line connecting the center of the washer and the marking on the washer

Claims

1. A high-strength bolt tightening condition evaluation device that evaluates the tightening condition of a high-strength bolt when members are joined together using a high-strength bolt, a washer, and a nut, comprising: a margin evaluation unit that inputs an input image of the high-strength bolt to be evaluated into a margin evaluation trained model that has been trained using, as training data, a previously prepared learning input image of the high-strength bolt, and a margin evaluation result that is an evaluation result regarding the margin that is the length of the shank of the high-strength bolt that protrudes from the surface of the nut, as imaged in the learning input image, and outputs the margin evaluation result of the high-strength bolt that is imaged in the input image, thereby evaluating the margin of the high-strength bolt; a rotation angle evaluation unit that detects each of the high-strength bolt, the nut, the washer, and markings on the member based on the input image, calculates a rotation angle of each of the high-strength bolt, the nut, and the washer relative to the member, and evaluates the tightening state of the high-strength bolt based on each of the rotation angles; a comprehensive evaluation unit that comprehensively evaluates the tightening state of the high-strength bolt based on the evaluation of the excess length and the evaluation based on the rotation angle; A high-strength bolt tightening condition evaluation device comprising:

2. The rotation angle evaluation unit a marking detection trained model that has been machine-trained to detect, from the learning input image, portions corresponding to each of the markings attached to the high-strength bolt, the nut, the washer, and the member, the end face of the shank of the high-strength bolt, the surface of the nut, the surface of the washer, and a side surface of the washer; an adjusted image obtained by adjusting the input image is input into the marking detection trained model to detect each of the markings attached to each of the high-strength bolt, the nut, the washer, and the member, and the portions corresponding to each of the end face of the shank of the high-strength bolt, the surface of the nut, the surface of the washer, and the side surface of the washer; calculate the positions of a high-strength bolt center, which is the center of the end face of the shank of the high-strength bolt, a nut center, which is the center of the surface of the nut, a washer center, which is the center of the surface of the washer, and a member center, which is the position where the axis of the shank intersects on the surface of the member, in the adjusted image, based on the portions corresponding to the end face of the shank of the high-strength bolt, the surface of the nut, the surface of the washer, and the side surface of the washer; Calculating a high-strength bolt angle, which is the angle formed by a line connecting the center of the high-strength bolt and the marking on the high-strength bolt in the adjusted image; Calculating a nut angle, which is the angle formed by a line connecting the nut center and the marking on the nut in the adjusted image; calculating a washer angle, which is the angle formed by a line connecting the center of the washer and the marking on the washer in the adjustment image; calculating a component angle, which is the angle formed by a line connecting the component center and the marking attached to the component in the adjusted image; Calculating the rotation angles of the high-strength bolt, the nut, and the washer relative to the member based on the high-strength bolt angle, the nut angle, the washer angle, and the member angle.

2. The high-strength bolt tightening condition evaluation device according to claim 1.

3. The system further includes a nut rotation angle average difference evaluation unit that calculates an average value of the rotation angle of the nut calculated by the rotation angle evaluation unit for each of the plurality of input images, and evaluates whether or not the difference between the rotation angle of the nut calculated for each of the plurality of input images and the average value is equal to or greater than a lower limit angle threshold and equal to or less than an upper limit angle threshold.

3. The high-strength bolt tightening condition evaluation device according to claim 1 or 2.

Images