Inspection device for facility construction work, inspection method for facility construction work, and inspection program for facility construction work

Abstract

To provide an inspection device for equipment construction, an inspection method for equipment construction, and an inspection program for equipment construction that execute an easy and highly accurate inspection for equipment construction.SOLUTION: A construction inspection server device for equipment construction includes: an acquisition section (API section) that acquires inspection item information indicating that an item to be inspected is a component of a fence, an inspection image obtained by imaging the components of the fence, and guide information on a guide image for imaging the components of the fence included in the inspection image, from a user terminal device 100; an inspection object recognition section that calculates a degree of certainty indicating certainty of the components of the fence on the basis of the inspection image, and recognizes whether the inspection image includes the components of the fence on the basis of the degree of certainty; an inspection section that inspects whether the number of components of the fence is the same as the number of recognized components of the fence based on the guide information; and a notification section that notifies a terminal device of a result of inspection executed by the inspection section.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an inspection device for facility construction work, an inspection method for facility construction work, and an inspection program for facility construction work.

Background Art

[0002] Conventionally, techniques for inspecting the construction state have been known. For example, Patent Documents 1 to 3 describe techniques for inspecting the construction state with high accuracy using a plurality of members. Patent Document 1 describes a technique for inspecting the construction state of hardware for branch lines of utility poles, and Patent Document 2 describes a technique for inspecting the construction state of support members of utility poles. Furthermore, Patent Document 3 describes displaying a guide frame or mask of an inspection object according to a plurality of inspection items included in facility construction work.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the inspection of the construction state described in Patent Documents 1 to 3 mentioned above, in many cases, the inspection items for each inspection object and the required accuracy for each inspection item are specified, and it is desired to perform the inspection simply and with high accuracy.

[0005] The present invention has been made in view of the above problems, and an object thereof is to provide an inspection device for facility construction work, an inspection method for facility construction work, and an inspection program for facility construction work that can inspect facility construction work simply and with high accuracy. [Means for solving the problem]

[0006] (1) One aspect of the present invention is a method for detecting a fence component from a terminal device, comprising: receiving, from a terminal device, inspection item information indicating that an inspection item is a fence component; an inspection image of the fence component; and an image of the fence component included in the inspection image. After obtaining the inspection image, re-image the components of the fence included in the inspection image the number of fence components based on the guide information is the same as the number of fence components recognized; and a notification unit that notifies the terminal device of the results of the inspection performed by the inspection unit.

[0008] (3) One aspect of the present invention is an inspection device for facility construction as described above, wherein the components of the fence are fasteners of the fence, and the guide image may have areas corresponding to each of a predetermined number of fasteners.

[0009] (4) One aspect of the present invention is the above-mentioned facility construction inspection device, wherein the guide image may be one guide image selected by a contractor from a plurality of guide images that differ in the number of fasteners.

[0010] (5) One aspect of the present invention is an inspection device for facility construction as described above, wherein the fasteners of the fence include an insulating portion, the acquisition unit acquires inspection item information indicating that the inspection item is the insulating portion, an inspection image capturing an image of the insulating portion, and guide information regarding a guide image for capturing an image of the insulating portion included in the inspection image, the recognition unit calculates a certainty factor indicating the likelihood of the insulating portion and recognizes whether or not the insulating portion is included in the inspection image based on the certainty factor, and the inspection unit inspects whether or not the insulating portion is included in the inspection image based on the guide information.

[0011] (6) One aspect of the present invention is The computer From the terminal device, inspection item information indicating that the inspection item is a component of the fence, an inspection image obtained by imaging the component of the fence, and a guide image for imaging the component of the fence included in the inspection image After obtaining the inspection image, re-image the components of the fence included in the inspection image Obtaining guide information related to the guide image, calculating a confidence level indicating the likelihood of the component of the fence based on the inspection image, recognizing whether the inspection image includes the component of the fence based on the confidence level, and determining whether the number of components of the fence based on the guide information is the same as the number of recognized components of the fence Execute And a step of notifying the terminal device of the inspection result, which is an inspection method for facility construction.

[0012] (7) One aspect of the present invention is to cause a computer to obtain, from a terminal device, inspection item information indicating that the inspection item is a component of the fence, an inspection image obtained by imaging the component of the fence, and guide information related to a guide image for imaging the component of the fence included in the inspection image, calculate a confidence level indicating the likelihood of the component of the fence based on the inspection image, recognize whether the inspection image includes the component of the fence based on the confidence level, check whether the number of components of the fence based on the guide information is the same as the number of recognized components of the fence, and notify the terminal device of the inspection result. It is a program for inspecting facility construction. After obtaining the inspection image, re-image the components of the fence included in the inspection image

Effect of the Invention

[0013] According to one aspect of the present invention, facility construction can be inspected simply and with high precision.

Brief Description of the Drawings

[0014]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Figure 34

Figure 35

Figure 36

Figure 37

Figure 38

Figure 39

Figure 40

Figure 41

Figure 42

Figure 43

Figure 44

Figure 45

Figure 46

Embodiments for Carrying Out the Invention

[0015] Hereinafter, an inspection apparatus for facility construction, an inspection method for facility construction, and an inspection program for facility construction to which the present invention is applied will be described with reference to the drawings.

[0016] The facility construction inspection system according to the embodiment performs the inspection of the facility construction using an image captured by a field worker at the site of the facility construction of the base station. The facility construction inspection system according to the embodiment simply and highly accurately inspects by determining whether a construction member as an inspection target exists in the image captured by the field worker and whether the construction member is properly constructed. Further, the facility construction inspection system according to the embodiment can reduce the labor of taking images by the field worker, the labor of inspection by the inspector of the facility construction, and the labor of re-construction when there are defects in the facility construction by presenting the determination results (recognition results and inspection results of the construction member) to the field worker.

[0017] FIG. 1 is a block diagram showing an example of the configuration of the facility construction inspection system according to the embodiment. The facility construction inspection system includes, for example, one or more user terminal devices 100, a construction inspection server device 200, and a management terminal 300. The user terminal device 100, the construction inspection server device 200, and the management terminal 300 are connected to a communication network NW, for example. Each device connected to the communication network NW is provided with a communication interface such as a NIC (Network Interface Card) or a wireless communication module (not shown in FIG. 1). The communication network NW includes, for example, the Internet, a WAN (Wide Area Network), a LAN (Local Area Network), a cellular network, and the like.

[0018] The user terminal device 100 is a portable terminal device equipped with a camera device such as a smartphone or a tablet terminal, for example. The user terminal device 100 starts a UA (User Agent) such as a browser or an application program. The UA is an application for communicating with the construction inspection server device 200, for example. The user terminal device 100 uses the construction inspection application as the UA, and performs display processing, operation reception processing, and the like using the content received from the construction inspection server device 200. Further, the user terminal device 100 provides operation information, captured images, and the like based on the user's operation to the construction inspection server device 200 using the construction inspection application.

[0019] The construction inspection server device 200 is an information processing device that receives inspection requests for facility construction work and provides inspection results for facility construction work. The construction inspection server device 200 includes, for example, a cooperation API (Application Programming Interface) unit 210 and an inspection target recognition unit 220. Functional units such as the cooperation API unit 210 and the inspection target recognition unit 220 are realized, for example, by a processor such as a CPU (Central Processing Unit) executing a program stored in a program memory. Also, some or all of these functional units may be realized by hardware such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable Gate Array), or may be realized by the cooperation of software and hardware.

[0020] The cooperation API unit 210 includes, for example, an API unit 212, an image processing unit 214, an inspection unit 216, and a control unit 218. The API unit 212 performs processes such as receiving various types of information from the user terminal device 100 and providing content to the user terminal device 100. The image processing unit 214 performs predetermined image processing on the image data acquired from the user terminal device 100. Hereinafter, an image used for inspection is described as an inspection image. The inspection unit 216 performs inspection processing for facility construction work using the inspection image acquired by the API unit 212 and / or the inspection image processed by the image processing unit 214. The control unit 218 controls the activation of the image recognition engine in the inspection target recognition unit 220 and the like.

[0021] The inspection target recognition unit 220 includes, for example, a plurality of image recognition engines 222A, 222B, ··· 222N (N is a natural number). When collectively referring to a plurality of image recognition engines, they are simply described as the image recognition engine 222. Each image recognition engine 222 corresponds to each inspection item among a plurality of inspection items. The image recognition engine 222 performs recognition processing for recognizing a construction member as an inspection target in the inspection item.

[0022] FIG. 2 is a block diagram showing an example of an inspection target recognition unit. The inspection target recognition unit 220 includes, for example, a learning image database 230, a learning processing unit 232, a recognition processing unit 234, and a result output unit 236.

[0023] The learning image database 230 is a database that stores, as learning images, for example, images acquired from a management terminal 300. The learning image database 230 stores, as learning images, for example, positive example images. A positive example image is an image obtained by imaging a target object in a normal state. The positive example image is, for example, an image obtained by imaging a construction member used in the construction of base station facilities. The construction members are, for example, components of base station facilities such as precast tubes (pipes), fences, insulating bushings, foundations, boundary markers, paint cans, etc. The positive example image is associated with tag information including the construction member name. Note that information indicating, for example, a process name and an inspection item name may be associated with the positive example image. Note that the learning image database 230 may store, as learning images, not only positive example images but also images obtained by imaging objects different from the construction members (objects in an abnormal state or defective products).

[0024] The learning image database 230 supplies learning images to the learning processing unit 232. The learning processing unit 232 performs recognition processing using the learning images to obtain a determination result. The learning processing unit 232 learns the processing parameters of the recognition processing unit 234 (recognition model 234A) so as to obtain a determination result that a construction member has been detected, and generates learning result data. The processing parameters of the recognition processing unit 234 (recognition model 234A) are, for example, filters (also referred to as weights and biases) included in a neural network. The learning result data is stored in the learning result data storage unit 2341.

[0025] The recognition processing unit 234 includes, for example, a learning result data storage unit 2341 and a determination unit 2342. In the learning result data storage unit 2341, the processing parameters of the recognition processing unit 234 as the learning result are stored, and the processing parameters of the recognition processing unit 234 are updated by the learning processing unit 232. When the recognition processing unit 234 acquires an inspection image, it extracts feature amounts using a recognition model 234A based on the processing parameters of the recognition processing unit 234. The determination unit 2342 detects construction members based on the extracted feature amounts. The recognition processing unit 234 outputs the determination result by the determination unit 2342 as a recognition result.

[0026] Figure 3 is a sequence diagram showing an example of the inspection process of facility construction in the facility construction inspection system. The user terminal device 100 transmits inspection item information to the construction inspection server device 200 based on the user's operation, and transmits an inspection image and guide information corresponding to the inspection item to the construction inspection server device 200. Next, the construction inspection server device 200 transmits the recognition result of the inspection object included in the inspection image to the user terminal device 100, and further transmits the inspection result of the inspection object to the user terminal device 100. Thereby, it is possible to notify the user whether the construction inspection server device 200 can accurately recognize the inspection object and whether the construction of the inspection object has been appropriately performed for the inspection item and the inspection object specified by the user terminal device 100.

[0027] FIG. 4 is a flowchart showing an example of inspection processing of facility construction in the construction inspection server device. First, the cooperation API unit 210 acquires the name of an inspection item in facility construction based on the information received from the user terminal device 100 (step S100). When the cooperation API unit 210 acquires the name of the inspection item, it acquires the activation list of the inspection target recognition unit 220. The cooperation API unit 210 compares the name of the inspection item acquired in step S100 with the activation list (step S104), and determines whether the image recognition engine 222 corresponding to the acquired name of the inspection item is activated (step S106). When the image recognition engine 222 corresponding to the name of the inspection item is not activated (step S106: NO), the cooperation API unit 210 controls to activate the image recognition engine 222 corresponding to the name of the inspection item (step S108).

[0028] When the image recognition engine 222 corresponding to the name of the inspection item is activated (step S106: YES), or after the image recognition engine 222 corresponding to the name of the inspection item is activated in step S108, the cooperation API unit 210 acquires the image file and the inspection information file (step S110). Next, the cooperation API unit 210 performs predetermined image processing on the inspection image included in the acquired image file (step S112). The predetermined image processing includes, for example, edge detection (sharpening) processing and brightness correction processing. The user terminal device 100 delivers the inspection image subjected to the predetermined image processing to the image recognition engine 222 corresponding to the name of the inspection item.

[0029] The image recognition engine 222 performs recognition processing to determine whether there is a construction member (object to be inspected) corresponding to the inspection item name based on the image acquired from the cooperation API unit 210 (step S114). At this time, the image recognition engine 222 calculates a confidence level for each construction member. Also, the image recognition engine 222 calculates the confidence level of the construction member for each image process. When the confidence level of the construction member is equal to or higher than a predetermined value, the image recognition engine 222 determines that the construction member exists. When the recognition result is that the construction member exists (step S116: YES), the image recognition engine 222 proceeds to step S120. When the recognition result is that the construction member does not exist (step S116: NO), the image recognition engine 222 re-acquires the inspection image (step S118). When re-acquiring the inspection image, the construction inspection server device 200 transmits a request to re-take the image to the user terminal device 100.

[0030] The cooperation API unit 210 compares the confidence level of the construction member calculated by the image recognition engine 222 with a predetermined value (step S120). The cooperation API unit 210 performs determination processing of the inspection item using the inspection image in which the construction member is recognized with a confidence level higher than the predetermined value (step S122). Note that the determination processing of the inspection item will be described later. As a result of the determination processing of the inspection item, the cooperation API unit 210 transmits the inspection result to the user terminal device 100 (step S124).

[0031] (Inspection of the pre-cube (pipe)) Hereinafter, the inspection of the pre-cube (pipe) in the base station equipment will be described. FIG. 5 is a diagram showing an example of a screen for selecting an object to be inspected. The user terminal device 100 displays a selection screen of the object to be inspected shown in FIG. 5 based on the user's operation, and recognizes which object to be inspected has been selected by receiving an operation on the selection screen. When a pre-catheter is selected, the user terminal device 100 displays the screen shown in FIG. 6. FIG. 6 is a diagram showing an example of a screen for selecting a guide image for inspecting the pre-catheter. In the embodiment, for example, guide images corresponding to four patterns can be selected as the specifications of the pre-catheter. The specification of the pre-catheter is, for example, the thickness of the diameter of the pre-catheter, and the user selects the thickness of the diameter of the pre-catheter constructed. Thereby, the facility construction inspection system can inspect the bending radius corresponding to the thickness of the diameter of the pre-catheter constructed.

[0032] FIG. 7 is a diagram showing an example of an inspection image of a pre-catheter and a recognition result of the pre-catheter, FIG. 8 is a diagram showing an example of tag information of the pre-catheter, FIG. 9 is a diagram showing an example of a guide image of the pre-catheter, and FIG. 10 is a diagram showing an example of guide information about the pre-catheter. The user terminal device 100 captures the inspection image shown in FIG. 7(A) and transmits it to the construction inspection server device 200. The construction inspection server device 200 performs a recognition process of the pre-catheter on the inspection image, and assigns tag information #1 to the region in the inspection image recognized as the pre-catheter shown in FIG. 7(B). The tag information includes the tag name "u-tube" as the recognition result of the image recognition engine 222. Note that when a plurality of pre-catheters are recognized in one inspection image, the construction inspection server device 200 may generate a recognition result indicating that the recognition of the pre-catheter has failed.

[0033] The user terminal device 100 displays a guide image of a pre-catheter shown in FIG. 9, and when receiving an operation for determining the position of the guide image, for example, it transmits guide information to the construction inspection server device 200. The guide image includes an inner diameter side guide (first image) arranged on the inner diameter side of the pre-catheter, an outer diameter side guide (second image) arranged on the outer diameter side of the pre-catheter, and a pair of positioning guides (third image) overlapping the inner diameter side guide and the outer diameter side guide. The inner diameter side guide and the outer diameter side guide have shapes corresponding to the bending radius (specification) of the pre-catheter selected by the user (constructor). The positioning guides are a pair of rectangular regions separated by the distance between the inner diameter part and the outer diameter part of the selected pre-catheter. When the user terminal device 100 receives an operation in which the guide image is positioned, it transmits transmission information indicating the center coordinates (x, y) of the inner diameter side guide (guide circle) and the radius (r) of the guide circle to the construction inspection server device 200.

[0034] FIG. 11 is a diagram showing an example of the inspection process of the pre-catheter. The construction inspection server device 200 reproduces the inner diameter side guide in the inspection image using the center coordinates C(x, y) of the inner diameter side guide (guide circle) received from the user terminal device 100 and the radius r of the inner diameter side guide. The construction inspection server device 200 determines that the construction of the pre-catheter is appropriately performed when the reproduced inner diameter side guide does not overlap the region recognized as the pre-catheter, and determines that the construction of the pre-catheter is not appropriately performed when the reproduced inner diameter side guide overlaps the region recognized as the pre-catheter.

[0035] FIG. 12 is a diagram showing another example of the inspection process of the precast tube. The construction inspection server device 200 may perform an inspection by excluding a partial range from the range where the reproduced inner diameter side guide and the area recognized as the precast tube are adjacent. The construction inspection server device 200 sets, for example, a range that is a predetermined distance below the center coordinate C of the inner diameter side guide as the exclusion range. The predetermined distance is, for example, a length obtained by multiplying the radius r by a predetermined set value (for example, 0.7). When the reproduced inner diameter side guide does not overlap with the area recognized as the precast tube outside the exclusion range, the construction inspection server device 200 determines that the construction of the precast tube is properly performed, and when the reproduced inner diameter side guide overlaps with the area recognized as the precast tube, it determines that the construction of the precast tube is not properly performed. Although the range below the distance obtained by multiplying the radius r by 0.7 from the center coordinate C of the inner diameter side guide is set as the exclusion range, the present invention is not limited to this, and the exclusion range may be changed as long as it is possible to reduce the possibility that the lowermost part of the inner diameter side guide is accidentally overlapped due to hand tremors or the like.

[0036] FIG. 13 is a diagram showing an example of an inspection result screen in which the construction of the precast tube meets a predetermined standard, and FIG. 14 is a diagram showing an example of an inspection result screen in which the construction of the precast tube does not meet a predetermined standard. The construction inspection server device 200 transmits the inspection result to the user terminal device 100, and the user terminal device 100 displays the inspection result screen shown in FIG. 13 or FIG. 14 based on the inspection result.

[0037] (Fence inspection) Hereinafter, the inspection of the fence in the base station facility will be described. When the fence is selected as the inspection object by the user terminal device 100, the screen shown in FIG. 15 is displayed. FIG. 15 is a diagram showing an example of a screen for selecting a guide image for inspecting the fence. In the embodiment, for example, a guide image corresponding to the location of the lattice structure of the fence or the shape of the joint in the fence can be selected. Thereby, the facility construction inspection system can perform the inspection of the fence corresponding to the location and the shape of the joint of the constructed fence.

[0038] FIG. 16 is a diagram showing an example of an inspection image and a guide image of a lattice structure in a fence, FIG. 17 is a diagram showing an example of a recognition result of the lattice structure, FIG. 18 is a diagram showing an example of tag information of the fence, and FIG. 19 is a diagram showing an example of guide information of the lattice structure. The user terminal device 100 captures an inspection image including a lattice structure as shown in FIG. 16 and transmits it to the construction inspection server device 200. The construction inspection server device 200 performs a recognition process of the lattice structure on the inspection image, and assigns tag information (#1 or #2) to the area in the inspection image recognized as the lattice structure shown in FIG. 17. The tag information includes the tag names "chidori - gin" or "chidori - cha" as the recognition results of the image recognition engine 222.

[0039] The user terminal device 100 displays a guide image of the lattice structure shown in FIG. 16, and when an operation for determining the position of the guide image, for example, is received, it transmits the guide information to the construction inspection server device 200. The guide image includes a plurality of circular images arranged at predetermined intervals in one direction. The area of each circular image corresponds to an inspection area where it is inspected whether the lattice structure is included. When the user terminal device 100 receives an operation in which the guide image is positioned, it transmits transmission information indicating the center coordinates (x1, y1, x2, y2, ···) and radius (r) of each circular image to the construction inspection server device 200.

[0040] The construction inspection server device 200 uses the guide information to inspect whether the number of recognized lattice structures is the same as the number of circular images (inspection areas). Since the number of guide images shown in FIG. 16 is 4 and the number of areas to which the tag information of the lattice structure shown in FIG. 17 is assigned is 4, the construction inspection server device 200 generates an inspection result indicating that the construction of the lattice structure meets the predetermined standards.

[0041] FIG. 20 is a diagram showing an example of an inspection result screen in which the construction of the lattice-shaped object meets the standard. The construction inspection server device 200 transmits the inspection result to the user terminal device 100, and the user terminal device 100 displays the inspection result screen shown in FIG. 20 based on the inspection result.

[0042] Next, the inspection of the joint of the fence will be described. FIG. 21 is a diagram showing an example of the shape of the joint in the fence, and FIG. 22 is a diagram showing an example of a guide image corresponding to the shape of the joint in the fence. There are multiple types of joint shapes in the fence as shown in FIG. 21, and one of them is selected based on the user's operation. The user terminal device 100 transmits information indicating the shape of the joint in the selected fence to the construction inspection server device 200 as inspection item information, and further captures an inspection image including the joint and transmits it to the construction inspection server device 200. The construction inspection server device 200 performs recognition processing of nuts (fasteners) at the joint on the inspection image, and assigns tag information (#3 or #4 in FIG. 18) to the area in the inspection image recognized as a nut. The tag information includes the tag names "nut-gin" or "nut-cha" as the recognition results of the image recognition engine 222. FIG. 23 is a diagram showing an example of the recognition result of the nut.

[0043] FIG. 24 is a diagram showing an example of a screen in which a guide image is superimposed on an inspection image including the joint in the fence, and FIG. 25 is a diagram showing an example of the guide information of the joint. The guide image includes a column guide image having a shape corresponding to the column supporting the lattice-shaped object of the fence and a nut guide image corresponding to the nut (fastener) fastening the columns together. When the user terminal device 100 receives an operation in which the guide image is positioned, it transmits transmission information indicating the center coordinates (x1, y1, x2, y2, ···) and radius (r) of the nut guide image to the construction inspection server device 200.

[0044] The construction inspection server device 200 uses the guidance information to inspect whether the number of nuts (fasteners) recognized from the inspection image is the same as the number of circular images (inspection areas). As shown in FIG. 24, when the number of nut guide images is two and the number of areas with tag information of nuts is two, the construction inspection server device 200 generates an inspection result indicating that the construction of the lattice-shaped object meets the predetermined criteria. The construction inspection server device 200 transmits the inspection result to the user terminal device 100, and the user terminal device 100 displays an inspection result screen based on the inspection result.

[0045] FIG. 26 is a diagram showing an example of an inspection result screen indicating that the construction of the fence joint meets the criteria. The construction inspection server device 200 transmits the inspection result to the user terminal device 100, and the user terminal device 100 displays the inspection result screen shown in FIG. 26 based on the inspection result.

[0046] FIG. 27 is a diagram showing an example of the inspection process at the joint of the fence. The construction inspection server device 200 reproduces the nut guide image in the inspection image using the center coordinates C(x, y) of the nut guide image received from the user terminal device 100 and the radius r of the inner diameter side guide. When the centroid coordinates of the area recognized as one or more nuts exist in the reproduced nut guide image, the construction inspection server device 200 determines that the nut meets the predetermined criteria for the reproduced nut guide image. The construction inspection server device 200 generates the inspection result of the fence joint by making a determination for each reproduced nut guide image.

[0047] Hereinafter, the inspection of the insulating bushing will be described. The insulating bushing is attached together with bolts at the joint, for example. FIG. 28 is a diagram showing an example of a screen for selecting a guide image for inspecting the insulating bushing, FIG. 29 is an example of an inspection image including the insulating bushing and the recognition result, and FIG. 30 is a diagram showing an example of the tag information of the insulating bushing.

[0048] When an insulating bushing is selected as the object to be inspected, the user terminal device 100 displays the screen shown in FIG. 28. When, for example, "bolt side with insulation" or "nut side with insulation" is selected, the user terminal device 100 transmits an inspection image including the insulating bushing as shown in FIG. 29(A) to the construction inspection server device 200 together with the inspection item information. The construction inspection server device 200 performs recognition processing of the insulating bushing on the inspection image, and assigns tag information (#2) to the region in the inspection image recognized as the insulating bushing as shown in FIG. 29(B). The tag information includes the tag name "bush-black" or "bush-white" as the recognition result of the image recognition engine 222. In the embodiment, when an item other than "bolt side with insulation" and "nut side with insulation" is selected, the user terminal device 100 does not need to transmit the inspection item information to the construction inspection server device 200, and the construction inspection server device 200 does not need to perform the recognition processing and inspection processing.

[0049] FIG. 31 is a diagram showing an example of the inspection process of the insulating bushing, and FIG. 32 is a diagram showing an example of the guide information about the insulating bushing. The user terminal device 100 generates information representing the coordinate range (x1, y1, x2, y2) where the insulating bushing exists in the inspection image, for example, based on the operation of the user, and transmits it to the construction inspection server device 200. When there is one or more centroid coordinates of the region recognized as the insulating bushing in the coordinate range received from the user terminal device 100, the construction inspection server device 200 generates an inspection result indicating that the construction of the insulating bushing meets the predetermined standard.

[0050] (Inspection of boundary markers) Hereinafter, the inspection of boundary markers in base station facilities will be described. The user terminal device 100 displays a selection screen for the inspection object shown in FIG. 5 based on the user's operation, and accepts an operation on the selection screen to recognize which inspection object has been selected. When a boundary plate (boundary marker) is selected, the user terminal device 100 displays the screen shown in FIG. 33. FIG. 33 is a diagram showing an example of a screen for selecting a guide image for inspecting the boundary plate. In the embodiment, for example, guide images corresponding to three composition patterns can be selected as the relationship between the boundary plate and the base member. These composition patterns are called, for example, "horizontal", "right angle", and "reverse right angle", and the user selects the composition pattern constructed. Thereby, the facility construction inspection system can perform an inspection of the boundary plate corresponding to the composition pattern.

[0051] FIG. 34 is a diagram showing an example of an inspection image of the boundary plate and the base member, FIG. 35 is a diagram showing an example of a guide image and a recognition result of the boundary plate, FIG. 36 is a diagram showing an example of tag information of the boundary plate and the base member, and FIG. 37 is a diagram showing an example of guide information. The user terminal device 100 captures the inspection image shown in FIG. 34 and transmits it to the construction inspection server device 200. The construction inspection server device 200 performs recognition processing of the boundary plate and the base member on the inspection image, and assigns tag information #1 to the region in the inspection image recognized as the boundary plate shown in FIG. 35. The tag information includes the tag name "pura" as the recognition result of the image recognition engine 222.

[0052] When the user terminal device 100 displays a guide image of the boundary plate and receives an operation for determining, for example, the orientation of the guide image, it transmits guide information to the construction inspection server device 200. The guide image includes a guide image corresponding to the boundary plate and guide images corresponding to two boundary lines extending from the boundary plate. The relationship between the guide image corresponding to the boundary plate and the guide images corresponding to the boundary lines is determined by the orientation selected by the user, and the orientation information selected by the user is transmitted to the construction inspection server device 200 as guide information. For example, when the user selects "right angle" and "left", the user terminal device 100 displays the guide image shown in Fig. 35(A) and transmits the guide information indicating "left" to the construction inspection server device 200. For example, when the user selects "right angle" and "right", the user terminal device 100 displays the guide image shown in Fig. 35(B) and transmits the guide information indicating "right" to the construction inspection server device 200. The boundary line parallel to the X direction in Fig. 35(A) is located at the coordinate y1 which is 1 / 3 from the lower end in the Y direction, and the boundary line parallel to the Y direction is located at the coordinate x1 which is 2 / 3 from the right end in the X direction. The boundary line parallel to the X direction in Fig. 35(B) is located at the coordinate y1 which is 1 / 3 from the lower end in the Y direction, and the boundary line parallel to the Y direction is located at the coordinate x2 which is 1 / 3 from the right end in the X direction.

[0053] When the construction inspection server device 200 selects a guide image of a right angle or an inverted right angle, it determines that the boundary plate is present at a predetermined location when the intersection coordinates of the boundary lines overlap with the region recognized as the boundary plate. When the construction inspection server device 200 selects a horizontal guide image, it determines that the boundary plate is present at a predetermined location when the boundary lines intersect with the region recognized as the boundary plate. When the construction inspection server device 200 determines that the boundary plate does not exist at a predetermined location, it notifies the user terminal device 100 that the boundary plate could not be detected.

[0054] FIG. 38 is a diagram showing the relationship between the boundary line and the base member. The construction inspection server device 200 determines the positional relationship between the base member and the boundary line based on the selected composition pattern and the orientation information. As shown in FIG. 38(A), when "right angle" is selected as the composition pattern and the orientation is left, the construction inspection server device 200 determines that the positional relationship between the boundary plate and the base member is appropriate if the maximum value x12 in the X direction of the base member is smaller than the X-direction position x11 of the boundary line parallel to the Y axis, and the maximum value y12 in the Y direction of the base member is smaller than the Y-direction position y11 of the boundary line parallel to the X axis. As shown in FIG. 38(B), when "horizontal" is selected as the composition pattern and the orientation is left, the construction inspection server device 200 determines that the positional relationship between the boundary plate and the base member is appropriate if the maximum value y12 in the Y direction of the base member is smaller than the Y-direction position y11 of the boundary line parallel to the X axis. As shown in FIG. 38(C), when "reverse right angle" is selected as the composition pattern and the orientation is left, the construction inspection server device 200 determines that the positional relationship between the boundary plate and the base member is appropriate if the base member does not exist inside the rectangle having the intersection of the two boundary lines and the point of the maximum values of X and Y (the lower right point in the image) as the diagonal line.

[0055] As shown in FIG. 38(D), when the construction inspection server device 200 selects "right angle" as the composition pattern and the orientation is right, if the minimum value x22 of the base member in the X direction is smaller than the X-direction position x21 of the boundary line parallel to the Y axis, and the maximum value y22 of the base member in the Y direction is smaller than the Y-direction position y21 of the boundary line parallel to the X axis, it is determined that the positional relationship between the boundary plate and the base member is appropriate. As shown in FIG. 38(E), when the construction inspection server device 200 selects "horizontal" as the composition pattern and the orientation is right, if the maximum value y22 of the base member in the Y direction is smaller than the Y-direction position y21 of the boundary line parallel to the X axis, it is determined that the positional relationship between the boundary plate and the base member is appropriate. As shown in FIG. 38(F), when the construction inspection server device 200 selects "reverse right angle" as the composition pattern and the orientation is right, if the base member does not exist inside the rectangle with the intersection of the two boundary lines and the point of the minimum values of X and Y (the lower left point in the image) as the diagonal line, it is determined that the positional relationship between the boundary plate and the base member is appropriate.

[0056] FIG. 39 is a diagram showing another example of a guide image for the boundary plate, FIG. 40 is a diagram showing an example of the recognition results of the boundary plate and the base member, and FIG. 41 is a diagram showing an example of the inspection results regarding the boundary plate. As described above, the user terminal device 100 may select a composition pattern, superimpose a guide image including a boundary line on the inspection image for display, generate orientation information of the guide image, and transmit it to the construction inspection server device 200. The construction inspection server device 200 recognizes the region of the boundary plate and the region of the base member by recognizing the inspection image, and based on the composition pattern and the orientation information, it can determine that there is a boundary plate near the boundary line captured by the user in accordance with the guide image, and that there is no base member beyond the boundary line based on the boundary plate. As a result, the construction inspection server device 200 transmits the inspection results to the user terminal device 100, and the user terminal device 100 displays the inspection result screen shown in FIG. 41 based on the inspection results.

[0057] (Inspection of paint cans) The inspection of paint cans in the base station equipment will be described below. FIG. 42 is a diagram showing an example of a screen for selecting the type of guide for an empty paint can, FIG. 43 is a diagram showing an example of a screen for setting the quantity of the selected empty paint can, FIG. 44 is a diagram showing an example of an inspection image including a paint can, FIG. 45 is a diagram showing an example of the recognition result of a paint can, and FIG. 46 is a diagram showing an example of the inspection result of a paint can. The user terminal device 100 causes the screen of FIG. 42 to be displayed to allow the user to select the use of the paint cans used in the construction of the base station, causes the screen of FIG. 43 to be displayed to allow the user to select the required quantity of paint cans, and transmits the use and required quantity of the selected paint cans to the construction inspection server device 200 as guide information. Further, as shown in FIG. 44, the user terminal device 100 captures an inspection image and transmits it to the construction inspection server device 200. The construction inspection server device 200 recognizes the paint can as shown in FIG. 45, and when the construction using the paint can satisfies a predetermined standard based on the guide information and the recognition result, it displays an inspection result as shown in FIG. 46.

[0058] (Effect of the Embodiment) According to the embodiment described above, from the user terminal device 100, an API unit 212 that acquires inspection item information indicating that the inspection item is a pipe, an inspection image obtained by imaging a pipe installed by bending construction, and guide information regarding a guide image for imaging the pipe included in the inspection image, an inspection target recognition unit 220 that calculates a confidence level indicating the certainty of the pipe based on the inspection image and recognizes whether or not the inspection image includes a pipe based on the confidence level, an inspection unit 216 that performs an inspection of the facility construction based on the region recognized as a pipe and the guide information, and a cooperation API unit 210 that notifies the user terminal device 100 of the result of the inspection performed by the inspection unit 216, it is possible to realize an inspection device for facility construction. According to the embodiment, it is possible to easily and highly accurately inspect the facility construction regarding the pipe installed by bending construction.

[0059] According to an embodiment, from the user terminal device 100, an API unit 212 that acquires inspection item information indicating that the inspection item is a fence component, an inspection image obtained by imaging the fence component, and guide information regarding a guide image for imaging the fence component included in the inspection image; an inspection target recognition unit 220 that calculates a confidence level indicating the likelihood of the fence component based on the inspection image and recognizes whether or not the inspection image includes the fence component based on the confidence level; an inspection unit 216 that inspects whether or not the number of fence components based on the guide information is the same as the number of recognized fence components; and an API unit 212 that notifies the user terminal device 100 of the result of the inspection performed by the inspection unit 216 are provided, and an inspection device for facility construction can be realized. According to the embodiment, facility construction regarding lattice-shaped objects, fasteners, and insulating bushings as fence components can be inspected simply and with high precision.

[0060] According to an embodiment, from the user terminal device 100, an API unit 212 that acquires inspection item information indicating that the inspection item is a boundary marker, an inspection image obtained by imaging the boundary marker and a part of the facility, and guide information regarding a guide image for imaging the boundary marker included in the inspection image; an inspection target recognition unit 220 that calculates a confidence level indicating the likelihood of the boundary marker based on the inspection image and recognizes whether or not the inspection image includes the boundary marker based on the confidence level, calculates a confidence level indicating the likelihood of the facility based on the inspection image, and recognizes whether or not the inspection image includes the facility based on the confidence level; an inspection unit 216 that performs an inspection of facility construction based on the positional relationship between a boundary line extending from a region recognized as a boundary marker based on the orientation information of the captured inspection image and the region recognized as the facility; and an API unit 212 that notifies the user terminal device 100 of the result of the inspection performed by the inspection unit 216 are provided, and an inspection device for facility construction can be realized. According to the embodiment, facility construction regarding the boundary marker and the facility can be inspected simply and with high precision.

[0061] Although each embodiment and modification example have been described, these are merely examples and are not limited thereto. For example, any one of the embodiments or modification examples, or a part of an embodiment or a part of a modification example, may be combined with one or more other embodiments or one or more other modification examples to realize an aspect of the present invention.

[0062] A program for executing each process of the user terminal device 100 and the construction inspection server device 200 in the present embodiment is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed, whereby the above-described various processes related to the user terminal device 100 and the construction inspection server device 200 may be performed.

[0063] Note that the "computer system" referred to here may include hardware such as an OS and peripheral devices. Also, the "computer system" shall include a homepage providing environment (or display environment) if the WWW system is used. Further, the "computer-readable recording medium" refers to a writable non-volatile memory such as a flexible disk, a magneto-optical disk, a ROM, a flash memory, a portable medium such as a CD-ROM, and a storage device such as a hard disk incorporated in a computer system.

[0064] Furthermore, the "computer-readable recording medium" also includes a volatile memory (e.g., DRAM (Dynamic Random Access Memory)) inside a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and that holds the program for a certain period of time. Also, the above program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium.

[0065] Here, the "transmission medium" for transmitting a program refers to a medium having a function of transmitting information, such as a network (communication network) like the Internet or a communication line (communication wire) like a telephone line. Also, the above program may be for realizing a part of the functions described above. Further, it may be a so-called difference file (difference program) that can be realized in combination with a program already recorded in a computer system for the functions described above.

[0066] As described above, the embodiments of the present invention have been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and designs within the scope not departing from the gist of the present invention are also included. For example, as an embodiment of the present invention, it may be configured only by the user terminal device 100.

Description of Reference Numerals

[0067] 100 User terminal device 200 Construction inspection server device 210 Cooperation API section 212 API section 214 Image processing section 216 Inspection section 218 Control section 220 Inspection target recognition section 222 Image recognition engine

Claims

1. An acquisition unit that acquires, from a terminal device, inspection item information indicating that the inspection item is a component of a fence, an inspection image obtained by imaging the component of the fence, and guide information regarding a guide image for re-imaging the component of the fence included in the inspection image after obtaining the inspection image by imaging the component of the fence included in the inspection image; A recognition unit that calculates a confidence level indicating the likelihood of the component of the fence based on the inspection image, and recognizes whether or not the inspection image includes the component of the fence based on the confidence level; An inspection unit that inspects whether or not the number of components of the fence based on the guide information is the same as the number of the recognized components of the fence; A notification unit that notifies the terminal device of the result of the inspection performed by the inspection unit; An inspection device for facility construction, comprising the above components.

2. The component of the fence is a fastening of the fence, and the guide image has regions corresponding to each of a predetermined number of fastenings. The inspection device for facility construction according to Claim 1, comprising the above components.

3. The guide image is one guide image selected by a constructor from a plurality of guide images having different numbers of fastenings. The inspection device for facility construction according to Claim 2.

4. The fastening of the fence includes an insulating part. The acquisition unit acquires inspection item information indicating that the inspection item is the insulating part, an inspection image obtained by imaging the insulating part, and guide information regarding a guide image for imaging the insulating part included in the inspection image. The recognition unit calculates a confidence level indicating the likelihood of the insulating part, and recognizes whether or not the inspection image includes the insulating part based on the confidence level. The inspection unit inspects whether or not the inspection image includes the insulating part based on the guide information. The inspection device for facility construction according to Claim 1.

5. A computer performs: a step of acquiring, from a terminal device, inspection item information indicating that the inspection item is a component of a fence, an inspection image obtained by imaging the component of the fence, and guide information regarding a guide image for re-imaging the component of the fence included in the inspection image after obtaining the inspection image by imaging the component of the fence included in the inspection image; Calculating a confidence level indicating the likelihood of the components of the fence based on the inspection image, and recognizing whether the inspection image includes the components of the fence based on the confidence level; Inspecting whether the number of components of the fence based on the guide information is the same as the number of the recognized components of the fence; Notifying the terminal device of the inspection result; An inspection method for facility construction, which executes the above steps.

6. Causing a computer to: Obtain from a terminal device inspection item information indicating that the inspection item is a component of a fence, an inspection image obtained by imaging the components of the fence, and guide information regarding a guide image for re-imaging the components of the fence included in the inspection image after obtaining the inspection image; Calculating a confidence level indicating the likelihood of the components of the fence based on the inspection image, and recognizing whether the inspection image includes the components of the fence based on the confidence level; Inspecting whether the number of components of the fence based on the guide information is the same as the number of the recognized components of the fence; Notifying the terminal device of the inspection result; An inspection program for facility construction.

Images