Determining system

Abstract

A determining system 1 comprises: an imaging unit 132 that is provided on a moving body which travels along a track and that images fasteners and fastening sites provided on or around the track while moving in conjunction with the movement of the moving body; a storage unit 133 that stores a relationship defining an allowable range of positional displacement between the fasteners and the fastening sites, the relationship being defined in advance on the basis of, among coordinates set along the track, the coordinates of the fasteners and coordinates corresponding to a trajectory along which the imaging unit 132 moves in conjunction with the movement of the moving body along the track; and a determining unit 137 that determines loosening of the fasteners with respect to the fastening sites on the basis of images captured by the imaging unit 132 and the relationship stored in the storage unit 133.

Description

Judgment System

[0001] The present invention relates to a determination system for determining looseness of a fastener.

[0002] A known conventional judgment system is, for example, that described in Patent Document 1. The judgment system described in Patent Document 1 equips an automatic traveling means that autonomously travels a predetermined course at an inspection work site where loosening of bolts that fasten a structure is inspected with a bolt loosening inspection function and a communication function, positions an inspection head unit mounted on the automatic traveling means with respect to the bolt to be inspected, extracts information related to the fastening state of the bolt to be inspected, judges whether or not the bolt is loose based on the extracted information and reference information for the bolt to be inspected in a normal fastening state, and communicates the judgment result and the location of the loose bolt to an external party.

[0003] Japanese Patent Application Publication No. 8-278116

[0004] As described above, the determination system photographs fasteners (bolts, nuts, etc.) as the mobile body (automated driving means) moves, and determines whether the fasteners are loose based on the captured images (video). Such a determination system determines whether the fasteners are loose based on a pair of marks affixed to the fastener and the fastening portion where the fastener is fastened, but in images taken as the mobile body moves, depending on the position of the fastener in the image, there may appear to be an apparent misalignment between the pair of marks. In this case, there is a risk that the system will erroneously determine that the fastener is loose, even when it is not actually loose.

[0005] Specifically, as shown in Figure 12(a), when the fastener (nut) N and the fastened portion (bolt) B are viewed from the front (from the front of the marks M1 and M2) in a state in which no loosening has occurred in the fastener N, there appears to be no misalignment between the mark M1 on the fastener N and the mark M2 on the fastened portion B. As shown in Figure 12(b), when the fastener N and the fastened portion B are viewed from an oblique direction in a state in which no loosening has occurred in the fastener N, there appears to be a misalignment between the mark M1 on the fastener N and the mark M2 on the fastened portion B. In this way, even if no misalignment has actually occurred in the fastener N, the mark M1 and the mark M2 may appear to be misaligned depending on the angle from which the fastener N and the fastened portion B are viewed.

[0006] An object of one aspect of the present invention is to provide a determination system that can improve determination accuracy.

[0007] (1) A judgment system according to one aspect of the present invention is a judgment system that judges the looseness of a fastener based on the positional relationship between a fastener and a fastening portion to which the fastener is fastened, and includes: a photographing unit that is provided on a moving body that travels along a track and that moves as the moving body moves while photographing the fasteners and fastening portions provided on the track or around the track; a memory unit that stores, among coordinates set along the track, a relationship that determines the allowable range of positional deviation between the fastener and the fastening portion, the relationship being predetermined based on the coordinates of the fastener and the trajectory along which the photographing unit moves as the moving body moves along the track; and a judgment unit that judges the looseness of the fastener relative to the fastening portion based on the image photographed by the photographing unit and the relationship stored in the memory unit.

[0008] In a determination system according to one aspect of the present invention, the determination unit determines the looseness of the fastener relative to the fastening portion based on the image captured by the photographing unit and the relationship stored in the memory unit. Thus, in the determination system, the memory unit stores a relationship that determines the allowable range of positional deviation between the fastener and the fastening portion, which is predefined based on the coordinates of the fastener and the coordinates corresponding to the trajectory of the photographing unit as the moving body moves along the trajectory, and the coordinates of the fastener, among coordinates set along the trajectory, and the looseness of the fastener is determined using this allowable range. This allows the determination system to determine the looseness of the fastener regardless of the relative positional relationship between the moving body and the fastener. Therefore, the determination system can improve the accuracy of the determination.

[0009] (2) In the judgment system of (1) above, when there are multiple images of a single fastener and multiple judgment results are obtained for the single fastener based on the multiple images, the judgment unit may judge the looseness of the fastener based on the judgment result that is closest to the median of the tolerance range. In this configuration, when multiple judgment results are obtained for a single fastener, a highly accurate judgment result can be obtained by adopting the judgment result that is closest to the median of the tolerance range.

[0010] (3) In the determination system of (1) above, when there are multiple images of a single fastener and multiple determination results are obtained for the single fastener based on the multiple images, the determination unit may determine the looseness of the fastener based on the proportion of the multiple determination results indicating whether the fastener is loose. In this configuration, when multiple determination results are obtained for a single fastener, a highly accurate determination result can be obtained by determining the proportion of the fastener that is loose (for example, a predetermined number or 60% or more).

[0011] (4) In any one of the determination systems (1) to (3) above, when multiple fasteners are present in the image, the determination unit may select a fastener to be determined and determine the looseness of that fastener. In this configuration, when multiple fasteners are present in the image, highly accurate determination results can be obtained by specifying the fastener to be determined.

[0012] (5) In the determination system of (4) above, if multiple fasteners are present in the image, the determination unit may select a fastener located near the center of the moving direction of the moving object in the image and determine the looseness of that fastener. Fasteners located near the center of the moving direction of the moving object in the image have less apparent misalignment. Therefore, by selecting and determining that fastener, it is possible to obtain a highly accurate determination result.

[0013] (6) In the determination system of any one of (1) to (5) above, the fastener and the fastening portion may each be provided with a mark for visually determining whether the fastener is loose, and the determination unit may determine whether the fastener is loose based on a positional deviation of the mark. In this configuration, the looseness of the fastener can be determined with high accuracy by checking the positional deviation of the mark.

[0014] (7) In the determination system of any one of (1) to (5) above, the fastener and the fastening portion may each be provided with a two-dimensional code for determining whether the fastener is loose, and the determination unit may determine whether the fastener is loose based on a positional deviation of the two-dimensional code. In this configuration, the fastener can be specified (identified) by using the two-dimensional code.

[0015] (8) In any one of the determination systems (1) to (7) above, the moving body may be an overhead vehicle that travels along a track that is suspended from the ceiling of the facility.

[0016] According to one aspect of the present invention, it is possible to improve the accuracy of determination.

[0017] FIG. 1 is a schematic diagram of a conveyance system equipped with a determination system according to one embodiment. FIG. 2 is a diagram illustrating the configuration of the conveyance system. FIG. 3(a) is a diagram illustrating an example of fasteners and fastened portions viewed from a direction perpendicular to the direction of movement of the inspection vehicle, and FIG. 3(b) is a diagram illustrating an example of fasteners and fastened portions viewed from the direction of movement of the inspection vehicle. FIG. 4 is a diagram illustrating the configuration of an overhead conveyance vehicle. FIG. 5 is a diagram illustrating the configuration of an overhead conveyance vehicle. FIG. 6 is a diagram illustrating an example of a relationship that determines an allowable range of positional deviation between a nut and a rail, which is stored in a memory unit. FIG. 7 is a diagram illustrating a method for determining nut looseness. FIG. 8 is a diagram illustrating an example of an image captured by a photographing unit. FIG. 9 is a diagram illustrating an example of a screen displayed in a management system. FIGS. 10(a) and 10(b) are diagrams illustrating examples of fasteners and fastened portions according to another embodiment. FIG. 11 is a diagram illustrating examples of fasteners and fastened portions according to another embodiment. FIG. 12(a) is a front view of the fastener and fastening portion, and FIG. 12(b) is a perspective view of the fastener and fastening portion.

[0018] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same or corresponding elements are designated by the same reference numerals, and redundant description will be omitted.

[0019] Fig. 1 is a diagram schematically illustrating a conveyance system including a determination system according to an embodiment. As shown in Fig. 1, the conveyance system 100 is a system for conveying an article using an overhead transport vehicle 120 that is movable along a rail (track) T. The rail T is a member that allows the overhead transport vehicle 120 to travel, and is suspended from the ceiling.

[0020] Fig. 2 is a diagram showing the configuration of the transport system 100. As shown in Fig. 2, the transport system 100 includes a management system 110, an overhead transport vehicle 120, and a rail T. Although two overhead transport vehicles 120 are shown in Fig. 1, one or more overhead transport vehicles 120 may be included. In the transport system 100, power is supplied to the overhead transport vehicle 120 in a non-contact manner from a power supply line (not shown) provided along the rail T.

[0021] The management system 110 manages the transport system 100. The management system 110 is a host controller of the ceiling transport vehicle 120, and outputs transport commands to the ceiling transport vehicle 120. The management system 110 is capable of communicating (wirelessly communicating) with the ceiling transport vehicle 120.

[0022] The overhead transport vehicle 120 includes, for example, an OHT (Overhead Hoist Transfer), a ceiling-suspended crane, etc. The goods include, for example, a container for storing multiple semiconductor wafers, a container for storing glass substrates, a reticle pod, general parts, etc.

[0023] The conveyance system 100 is equipped with an inspection vehicle (mobile body) 130. The inspection vehicle 130 is an overhead traveling vehicle. The inspection vehicle 130 is also called a doctor vehicle, etc. The inspection vehicle 130 judges whether a plurality of nuts (fasteners) N ​​(see FIG. 3(a)) are loosened, which are provided along the rail T of the conveyance system 100. The inspection vehicle 130 constitutes the judgment system 1. The inspection vehicle 130 only needs to enter the rail T when judging whether the nuts N are loose. In other words, when the inspection vehicle 130 is not judging whether the nuts N are loosened, the inspection vehicle 130 is located in an area outside the rail T. When the inspection vehicle 130 receives a command from the management system 110 to judge whether the nuts N are loosened, it enters the rail T and performs an inspection for whether the nuts N are loosened.

[0024] FIG. 3( a) is a diagram showing an example of the fastener and the fastening portion as viewed from a direction perpendicular to the direction of movement of the inspection vehicle 130 (the width direction of the inspection vehicle 130). FIG. 3( b) is a diagram showing an example of the fastener and the fastening portion as viewed from the direction of movement of the inspection vehicle 130. As shown in FIGS. 3( a) and 3(b), a nut N is fastened to a bolt B. The bolt B is fixed to, for example, a rail T. In the conveyance system 100, a plurality of nuts N are provided along the rail T. In this embodiment, an example will be described in which the fastener is the nut N and the fastening portion where the bolt B to which the nut N is fastened is fixed is the rail T. The nut N and the bolt B can be used, for example, to suspend the rail T from a ceiling. The fastening portion may be configured separately from the rail T and fixed in position relative to the rail T.

[0025] As shown in Fig. 3(b), the nut N and the rail T are disposed in different positions. Specifically, the nut N is disposed closer to the inside of the rail T than the side surface of the rail T. In Fig. 3(a), the nut N is located further back than the rail T in the depth direction of the paper.

[0026] As shown in Figures 3(a) and 3(b), a mark M1 is provided on the nut N. A mark M2 is provided on the rail T. Each of the marks M1 and M2 is a mark for visually determining whether the nut N is loose. The mark M1 is provided at a position corresponding to the mark M2 when the nut N is sufficiently tightened to the bolt B. The position where the marks M1 and M2 correspond can be a position where the centers of the marks M1 and M2 are on approximately the same line when viewed from the direction of Figure 3(a), i.e., a direction perpendicular to the direction of movement of the inspection vehicle 130. The marks M1 and M2 can be provided by applying paint such as ink.

[0027] Fig. 4 is a diagram showing the configuration of the inspection vehicle 130. As shown in Fig. 4, the inspection vehicle 130 is equipped with a communication unit 131, an imaging unit 132, a memory unit 133, a sensor 134, an encoder 135, a position acquisition unit 136, and a determination unit 137. The inspection vehicle 130 further includes a control unit and the like that controls the running of the inspection vehicle 130.

[0028] The communication unit 131 communicates with the management system 110. The communication unit 131 receives a driving command transmitted from the management system 110. The communication unit 131 transmits the determination result output from the determination unit 137 to the management system 110.

[0029] The photographing unit 132 is integrally attached to the inspection vehicle 130 and photographs the object while moving in accordance with the movement of the inspection vehicle 130. The photographing unit 132 may be a two-dimensional camera or a monocular camera (e.g., a CCD). As shown in FIG. 5, the photographing unit 132 is provided on the vehicle body 140 of the inspection vehicle 130. The photographing unit 132 is positioned so as to photograph the nut N. While two photographing units 132 are shown in FIG. 5, four photographing units 132 may be provided on the inspection vehicle 130, for example. Specifically, four photographing units 132 are provided on the vehicle body 140 of the inspection vehicle 130, two on the left and right sides of the front (the left and right sides in the direction of movement of the inspection vehicle 130) and two on the left and right sides of the rear. The photographing unit 132 photographs, for example, 30 images per second. The photographing unit 132 outputs the photographed photographed data to the determination unit 137.

[0030] The memory unit 133 stores various types of information. The memory unit 133 stores the coordinates (position information) of all nuts N provided along the rail T. The coordinates are set along the rail T. The memory unit 133 stores a relationship that determines the allowable range of positional deviation between the nuts N and the rail T, which is predefined based on the coordinates of the nuts N and the coordinates corresponding to the trajectory along which the photographing unit 132 moves as the inspection vehicle 130 moves along the rail T.

[0031] FIG. 6 is a diagram showing an example of a relationship stored in the memory unit 133 that determines the allowable range of positional misalignment between the nut N and the rail T. In FIG. 6, the vertical axis represents the amount of misalignment between the nut N and the rail T, and the horizontal axis represents the left-right position in the image. The horizontal axis corresponds to the extension direction of the rail T and also corresponds to the movement direction of the inspection vehicle 130. The amount of misalignment is expressed as the number of pixels in the image. In the example shown in FIG. 6, the area A (filled in portion) surrounded by the two lines L1 and L2 indicates the allowable range. In the example shown in FIG. 6, the two lines L1 and L2 are shown as straight lines, but the lines L1 and L2 may also be curved. The allowable range can be appropriately set by the user based on experimental values, etc. The allowable range can be set, for example, by plotting the above relationship based on an image for setting the allowable range and determining the lines L1 and L2 by fitting the plot. For example, the area A in Fig. 6 is determined by photographing the nut N and the rail T in advance with various amounts of misalignment between them, and determining a range corresponding to the amount of misalignment in an image of the nut N and the rail T that is determined to be within a humanly acceptable range. Conversely, the range of area A in Fig. 6 may be determined based on the amount of misalignment in an image of the nut N and the rail T that is determined to be outside the acceptable range.

[0032] The sensor 134 detects marker members (not shown) for position detection that are provided on the rail T. A plurality of marker members are provided along the rail T. The marker members are, for example, bar codes, two-dimensional codes, etc. The sensor 134 outputs the detection result to the position acquisition unit 136.

[0033] The encoder 135 measures the distance traveled by the inspection vehicle 130. The encoder 135 is provided, for example, on the drive shaft of a drive source (such as a servo motor) of the inspection vehicle 130. The encoder 135 outputs the measurement result to the position acquisition unit 136.

[0034] The position acquisition unit 136 acquires the position (coordinates) of the inspection vehicle 130. The position acquisition unit 136 acquires the position of the inspection vehicle 130 based on the detection results detected by the sensor 134 and the measurement results measured by the encoder 135. The position acquisition unit 136 outputs position information related to the position of the inspection vehicle 130 to the determination unit 137.

[0035] 4, the determination unit 137 determines whether the nut N is loose. The determination unit 137 may include a processor such as a central processing unit (CPU) or a graphics processing unit (GPU), a memory such as a read-only memory (ROM) or a random access memory (RAM), and a storage such as a solid-state drive (SSD).

[0036] The determination unit 137 determines whether the nut N is loose based on the image captured by the photographing unit 132. The determination unit 137 may make the determination at the timing when the image is captured by the photographing unit 132, or at other timing.

[0037] The determination unit 137 determines whether the nut N is loose based on the image captured by the imaging unit 132 and the above relationship stored in the storage unit 133. As shown in FIG. 7 , the determination unit 137 acquires the center coordinates (white circles in FIG. 7 ) of the mark M1 of the nut N and the mark M2 of the rail T in the image. The determination unit 137 performs predetermined image processing based on the position information acquired by the position acquisition unit 136 to acquire the center coordinates of the mark M1 of the nut N and the mark M2 of the rail T. The determination unit 137 calculates the difference D between the center coordinates of the mark M1 and the mark M2, and if the difference D is within the allowable range of the above relationship, the determination unit 137 determines that the nut N is loose (NG determination) if the difference D is not within the allowable range of the above relationship (if it is outside the allowable range). The determination unit 137 outputs the determination result (NG determination) to the communication unit 131.

[0038] In this embodiment, when multiple determination results are obtained for one nut N, the determination unit 137 uses the determination result that is closest to the median within the allowable range of the above relationship stored in the storage unit 133. In the example shown in Fig. 6, the median is "0" for the amount of deviation and "0" for the left / right position.

[0039] FIG. 8 is a diagram showing an example of an image captured by the photographing unit 132. As shown in FIG. 8, two nuts N are captured in image G. To distinguish between the two nuts N, in FIG. 8, the nut N on the right side is designated as nut N1, and the nut N on the left side is designated as nut N2. When there are multiple nuts N in image G, the determination unit 137 selects the nut N to be determined and determines the looseness of that nut N. In this embodiment, the determination unit 137 determines the nut N closest to the center (dashed line) of image G as the determination target. The center of image G is the center in the direction of movement of the inspection vehicle 130. In the example shown in FIG. 8, the determination unit 137 determines the nut N1 closest to the center of image G as the determination target.

[0040] The management system 110 stores the judgment results sent from the inspection vehicle 130. The judgment results can be viewed in the management system 110. Fig. 9 is a diagram showing an example of a display screen displayed in the management system 110. As shown in Fig. 9, for example, the position information of a nut N that has been judged to be loose in the judgment results is displayed on the rail T. In the example shown in Fig. 9, the position of the nut N that has been judged to be loose is indicated by a black circle. Fig. 9 shows that three nuts N are loose. When any nut N is selected in the image, an image including the nut N is displayed.

[0041] As described above, in the determination system 1 according to this embodiment, the determination unit 137 determines whether the nut N is loose on the rail T based on the image captured by the photographing unit 132 and the relationship stored in the memory unit 133. In this way, in the determination system 1, the memory unit 133 stores a relationship that determines the allowable range of positional deviation between the nut N and the rail T, which is predefined based on the coordinates corresponding to the trajectory that the photographing unit 132 moves as the inspection vehicle 130 moves along the rail T and the coordinates of the nut N, and the determination system 1 determines whether the nut N is loose using this allowable range. This allows the determination system 1 to determine whether the nut N is loose regardless of the relative positional relationship between the inspection vehicle 130 and the nut N. Therefore, the determination system 1 can improve the accuracy of determination.

[0042] In the determination system 1 according to the present embodiment, when there are multiple images of a single nut N and multiple determination results are obtained for the single nut N, the determination unit 137 determines whether the nut N is loose based on the determination result that is closest to the median value in the tolerance range. In this configuration, when multiple determination results are obtained for a single nut N, it is possible to obtain a highly accurate determination result by adopting the determination result that is closest to the median value in the tolerance range.

[0043] In the determination system 1 according to this embodiment, when multiple nuts N are present in an image, the determination unit 137 selects a nut N to be determined and determines the looseness of that nut N. In this embodiment, the determination unit 137 determines the nut N closest to the center of the image as the determination target. With this configuration, when multiple nuts N are present in an image, it is possible to obtain a highly accurate determination result by identifying the target to be determined.

[0044] In the determination system 1 according to this embodiment, the nut N and the rail T are provided with marks M1 and M2, respectively, for visually determining whether the nut N is loose. The determination unit 137 determines whether the nut N is loose based on the positional deviation between the marks M1 and M2. In this configuration, the looseness of the nut N can be determined with high accuracy by checking the positional deviation between the marks M1 and M2.

[0045] Although the embodiments of the present invention have been described above, the present invention is not necessarily limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present invention.

[0046] In the above embodiment, an example has been described in which looseness of the nuts N is determined in the inspection vehicle 130. However, the looseness of the nuts N may be determined in the ceiling transport vehicle 120. In this case, it is sufficient that the ceiling transport vehicle 120 is equipped with the imaging unit 132 and the like.

[0047] In the above embodiment, an example has been described in which the inspection vehicle 130 is equipped with the determination unit 137. However, the determination of looseness of fasteners may be performed in the management system 110. In this case, images taken by the inspection vehicle 130 are transmitted to the management system 110, and the determination is performed in the management system 110.

[0048] In the above embodiment, an example has been described in which the mark M1 is provided on the nut N and the mark M2 is provided on the rail T. However, as shown in Fig. 10(a) , the mark M1 may be provided on the nut N and the mark M2 may be provided on the bolt B.

[0049] In the above embodiment, the fastener is a nut N. However, the fastening part may be a bolt B (the head of the bolt) or the like, as shown in FIG.

[0050] In the above embodiment, an example was described in which, when multiple judgment results are obtained for one nut N, the judgment unit 137 adopts the judgment result closest to the median within the allowable range of the above relationship stored in the memory unit 133. However, when there are multiple images of one fastener and multiple judgment results are obtained for one fastener, the judgment unit 137 may also judge the looseness of the fastener based on the proportion of the multiple judgment results indicating whether the fastener is loose. The judgment unit 137 may also determine that the fastener is loose when, for example, there are a predetermined number or more of NG judgments in the multiple judgment results.

[0051] In the above embodiment, an example has been described in which a mark M1 is provided on the nut N and a mark M2 is provided on the rail T. However, as shown in FIG. 11 , two-dimensional codes C1 and C2 may be provided on the nut N1 and the rail T, respectively. The two-dimensional codes C1 and C2 contain information that can identify the nut N1. This allows the determination system 1 to uniquely recognize the nut N by reading the two-dimensional codes C1 and C2. This allows the nut N to be identified without using the coordinates of the nut N and the position of the inspection vehicle 130.

[0052] 1...determination system, 130...inspection vehicle (mobile body), 132...photographing unit, 133...storage unit, 137...determination unit, C1, C2...two-dimensional code, M1, M2...mark, T...rail (track).

Claims

1. A determination system for determining looseness of a fastener based on the positional relationship between the fastener and a fastening portion to which the fastener is fastened, comprising: a photographing unit that is provided on a moving body that travels along a track and moves as the moving body moves to photograph the fastener and the fastening portion provided on the track or around the track; a memory unit that stores a relationship that determines the allowable range of positional deviation between the fastener and the fastening portion, the relationship being predetermined based on the coordinates set along the track, which correspond to the trajectory that the photographing unit moves as the moving body moves along the track, and the coordinates of the fastener; and a determination unit that determines looseness of the fastener with respect to the fastening portion based on the image taken by the photographing unit and the relationship stored in the memory unit.

2. The judgment system according to claim 1, wherein when there are multiple images showing one fastener and multiple judgment results are obtained for one fastener based on the multiple images, the judgment unit judges the looseness of the fastener based on the judgment result that is closest to the median value in the tolerance range.

3. The judgment system of claim 1, wherein when there are multiple images showing one fastener and multiple judgment results are obtained for one fastener based on the multiple images, the judgment unit judges whether the fastener is loose based on the proportion of the multiple judgment results indicating whether the fastener is loose or not.

4. A determination system as described in claim 1 or 2, wherein, when a plurality of fasteners are present in the image, the determination unit selects the fastener to be determined and determines the looseness of that fastener.

5. The determination system of claim 4, wherein, when multiple fasteners are present in the image, the determination unit selects the fastener located near the center of the moving direction of the moving object in the image and determines whether the fastener is loose.

6. A determination system as described in claim 1 or 2, wherein each of the fasteners and the fastening portions is provided with a mark for visually determining whether the fasteners are loose, and the determination unit determines whether the fasteners are loose based on a positional deviation of the marks.

7. A determination system as described in claim 1 or 2, wherein each of the fasteners and the fastening portions is provided with a two-dimensional code for determining whether the fasteners are loose, and the determination unit determines whether the fasteners are loose based on a positional deviation of the two-dimensional code.

8. A determination system according to claim 1 or 2, wherein the moving body is an overhead vehicle that travels along the track that is suspended from the ceiling of the facility.

Images