Inspection system, inspection method, and program

Abstract

To provide an inspection system, an inspection method and a program which can more correctly set an inspection range in a photographed image.SOLUTION: An inspection system 1 comprises: an image acquisition unit 1b; a display control unit 1c; an inspection range setting unit 1e; and an inspection unit 1h. The image acquisition unit 1b acquires data of a photographed image that is the image obtained by imaging an inspection object 9 in a three-dimensional shape and includes three-dimensional position information. The display control unit 1c displays the photographed image on a screen D1. The inspection range setting unit 1e sets an inspection range indicating a range in which the inspection object 9 is captured in the photographed image by the operation of a person. The inspection unit 1h inspects the inspection object 9 by performing image recognition processing on the inspection range.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an inspection system, an inspection method, and a program. [Background technology]

[0002] In Patent Document 1, at a construction site of a reinforced concrete structure, the diameter, pitch, number, etc. of the arranged rebars are inspected to confirm that the rebars are arranged correctly before they are buried inside the poured concrete.

[0003] Specifically, the reinforcing bar inspection device of Patent Document 1 includes a stereoscopic photography unit, an arrangement plane identification unit, and an inspection unit. The stereoscopic photography unit acquires positional information of the reinforcing bars in three-dimensional space by capturing images of the reinforcing bars arranged on a plane in three-dimensional space from two different points. The arrangement plane identification unit identifies the plane on which the reinforcing bars are arranged based on the positional information of the reinforcing bars acquired by the stereoscopic photography unit. The inspection unit inspects the reinforcing bars, with the reinforcing bars arranged on the plane identified by the arrangement plane identification unit as the inspection target. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-2737 Summary of the Invention [Problem to be solved by the invention]

[0005] When multiple components are arranged in a three-dimensional space, other components may appear in the image (captured image) captured by the stereo imaging unit (imaging unit) in addition to the component to be inspected. In this case, it may be difficult to distinguish the area in the captured image where the component to be inspected appears (inspection area) from the area where other components appear. As a result, it is difficult to accurately set the inspection area.

[0006] An object of the present disclosure is to provide an inspection system, an inspection method, and a program that can more accurately set an inspection range within a captured image. [Means for solving the problem]

[0007] An inspection system according to one aspect of the present disclosure includes an image acquisition unit, a display control unit, an inspection range setting unit, and an inspection unit. The image acquisition unit acquires captured image data, which is an image of a three-dimensional inspection object and includes three-dimensional position information. The display control unit displays the captured image on a screen. The inspection range setting unit sets an inspection range indicating the range in which the inspection object is captured in the captured image through a human operation. The inspection unit inspects the inspection object by performing image recognition processing on the inspection range. The inspection range setting unit sets, as the inspection range, an imaging area in which an image of a rectangular body is captured, the four vertices of which correspond to four points specified by the user's operation in the captured image displayed on the screen. One of the four points corresponds to a vertex common to three mutually perpendicular faces of the outline of the rectangular body. The remaining three points correspond to ends of three sides extending from the vertex common to the three faces. An inspection system according to one aspect of the present disclosure includes an image acquisition unit, a display control unit, an inspection range setting unit, and an inspection unit. The image acquisition unit acquires data of an image of a three-dimensional inspection object, the captured image including three-dimensional position information. The display control unit displays the captured image on a screen. The inspection range setting unit sets an inspection range indicating the range in which the inspection object is captured in the captured image through a human operation. The inspection unit inspects the inspection object by performing image recognition processing on the inspection range. The inspection range setting unit sets the inspection range using information on the dimensions of the inspection object. An inspection system according to one aspect of the present disclosure includes an image acquisition unit, a display control unit, an inspection range setting unit, an inspection unit, and a marker generation unit. The image acquisition unit acquires captured image data, which is an image of a three-dimensional inspection object and includes three-dimensional position information. The display control unit displays the captured image on a screen. The inspection range setting unit sets an inspection range indicating the range in which the inspection object is captured in the captured image through a human operation. The inspection unit inspects the inspection object by performing image recognition processing on the inspection range. The marker generation unit generates a marker image indicating the inspection range in the captured image. The display control unit displays the marker image on the screen by superimposing it on the captured image. The marker generation unit generates a three-dimensional model of the inspection range based on the three-dimensional position information included in the inspection range, and sets a projected image of the three-dimensional model as the marker image. An inspection system according to one aspect of the present disclosure includes an image acquisition unit, a display control unit, an inspection range setting unit, an inspection unit, and a marker generation unit. The image acquisition unit acquires captured image data, which is an image of a three-dimensional inspection target and includes three-dimensional position information. The display control unit displays the captured image on a screen. The inspection range setting unit sets an inspection range indicating the range in which the inspection target is captured in the captured image through a human operation. The inspection unit inspects the inspection target by performing image recognition processing on the inspection range. The marker generation unit generates a marker image indicating the inspection range in the captured image. The display control unit displays the marker image on the screen by superimposing it on the captured image. The marker image is a projection image of a three-dimensional model. The marker generation unit displays the marker image on the screen by converting three-dimensional coordinates of the three-dimensional model into two-dimensional coordinates based on coordinate information indicating the position of the inspection target in a three-dimensional space in which the inspection target exists. An inspection system according to one aspect of the present disclosure includes an image acquisition unit, a display control unit, an inspection range setting unit, an inspection unit, a marker generation unit, and a marker processing unit. The image acquisition unit acquires captured image data, which is an image of a three-dimensional inspection object and includes three-dimensional position information. The display control unit displays the captured image on a screen. The inspection range setting unit sets an inspection range indicating the range in which the inspection object is captured in the captured image through a human operation. The inspection unit inspects the inspection object by performing image recognition processing on the inspection range. The marker generation unit generates a marker image indicating the inspection range in the captured image. The display control unit displays the marker image on the screen by superimposing it on the captured image. The marker processing unit performs at least one of processing, including movement, rotation, enlargement, and reduction, on the marker image displayed on the screen in response to the human operation. The inspection range setting unit modifies the inspection range in response to the at least one processing performed on the marker image.

[0008] An inspection method according to one aspect of the present disclosure includes an image acquisition step, a display step, an inspection range setting step, and an inspection step. The image acquisition step is an image of a three-dimensional inspection object, and data of the captured image including three-dimensional position information is acquired. The display step displays the captured image on a screen. The inspection range setting step sets an inspection range indicating the range in which the inspection object is captured in the captured image by human operation. The inspection step inspects the inspection object by performing image recognition processing on the inspection range. In the inspection range setting step, an imaging area in which an image of a rectangular body is captured, the four vertices of which correspond to the four points specified by the user in the captured image displayed on the screen, is set as the inspection range. One of the four points corresponds to a vertex common to three mutually perpendicular faces of the outline of the rectangular body. The remaining three points correspond to the ends of three sides extending from the vertex common to the three faces. An inspection method according to one aspect of the present disclosure includes an image acquisition step, a display step, an inspection range setting step, and an inspection step. The image acquisition step is an image of a three-dimensional inspection object, and data of the captured image including three-dimensional position information is acquired. The display step displays the captured image on a screen. The inspection range setting step sets an inspection range indicating the range in which the inspection object is captured in the captured image by human operation. The inspection step inspects the inspection object by performing image recognition processing on the inspection range. The inspection range setting step sets the inspection range using information on the dimensions of the inspection object. An inspection method according to one aspect of the present disclosure includes an image acquisition step, a display step, an inspection range setting step, an inspection step, and a marker image setting step. The image acquisition step is an image of a three-dimensional inspection object, and data of the captured image including three-dimensional position information is acquired. The display step displays the captured image on a screen. The inspection range setting step sets an inspection range indicating the range in which the inspection object is captured in the captured image through human operation. The inspection step inspects the inspection object by performing image recognition processing on the inspection range. The marker image setting step generates a marker image indicating the inspection range in the captured image. The marker image setting step displays the marker image on the screen while superimposing it on the captured image. The marker image setting step generates a three-dimensional model of the inspection range based on the three-dimensional position information included in the inspection range, and a projected image of the three-dimensional model is used as the marker image. An inspection method according to one aspect of the present disclosure includes an image acquisition step, a display step, an inspection range setting step, an inspection step, and a marker image setting step. The image acquisition step is an image of a three-dimensionally shaped inspection target, and data of the captured image including three-dimensional position information is acquired. The display step displays the captured image on a screen. The inspection range setting step sets an inspection range indicating the range in which the inspection target is captured in the captured image through human operation. The inspection step inspects the inspection target by performing image recognition processing on the inspection range. The marker image setting step generates a marker image indicating the inspection range in the captured image. The marker image setting step superimposes the marker image on the captured image and displays it on the screen. The marker image is a projection image of a three-dimensional model. The marker image setting step displays the marker image on the screen by converting three-dimensional coordinates of the three-dimensional model into two-dimensional coordinates based on coordinate information indicating the position of the inspection target in a three-dimensional space in which the inspection target exists. An inspection method according to one aspect of the present disclosure includes an image acquisition step, a display step, an inspection range setting step, an inspection step, a marker image setting step, and a marker processing step. The image acquisition step is an image of a three-dimensional inspection object, and data of the captured image including three-dimensional position information is acquired. The display step displays the captured image on a screen. The inspection range setting step sets an inspection range indicating the range in which the inspection object is captured in the captured image through a human operation. The inspection step inspects the inspection object by performing image recognition processing on the inspection range. The marker image setting step generates a marker image indicating the inspection range in the captured image. The marker image setting step displays the marker image on the screen, superimposed on the captured image. The marker processing step performs at least one of processing, including movement, rotation, enlargement, and reduction, on the marker image displayed on the screen in response to the human operation. The inspection range setting step modifies the inspection range in response to the at least one processing performed on the marker image.

[0009] A program according to one aspect of the present disclosure causes a computer system to execute the above-described inspection method. [Effects of the Invention]

[0010] As described above, the present disclosure has the effect of enabling the inspection range in a captured image to be set more accurately. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram showing the configuration of an inspection system according to an embodiment. [Figure 2] FIG. 2 is a perspective view showing an example of use of the inspection system. [Figure 3] FIG. 3 is a plan view showing an imaging unit of a tablet terminal constituting the above inspection system. [Figure 4] FIG. 4 is a diagram showing an image captured by the inspection system. [Figure 5] FIG. 5 is a diagram showing an inspection target designated in an image captured by the inspection system. [Figure 6] FIG. 6 is a diagram showing an inspection range set in an image captured by the inspection system. [Figure 7] FIG. 7 is a perspective view showing a three-dimensional model of the above inspection system. [Figure 8] FIG. 8 is a diagram showing a marker image set on an image captured by the inspection system. [Figure 9] FIG. 9 is a flowchart showing an inspection method of the above inspection system. [Figure 10] FIG. 10 is a diagram showing an inspection target designated in an image captured by the inspection system of the second modified example. [Figure 11] FIG. 11 is a diagram showing an inspection range set in an image captured by the inspection system. [Figure 12] FIG. 12 is a perspective view showing a three-dimensional model of the above inspection system. [Figure 13] FIG. 13 is a diagram showing a marker image set on an image captured by the inspection system. [Figure 14] FIG. 14 is a diagram showing an inspection target designated in an image captured by the inspection system of the third and fourth modified examples. [Figure 15] FIG. 15 is a block diagram showing the configuration of an inspection system according to the fifth modified example. [Figure 16] FIG. 16 is a block diagram showing the configuration of another inspection system according to the fifth modified example. DETAILED DESCRIPTION OF THE INVENTION

[0012] The following embodiments generally relate to an inspection system, an inspection method, and a program. More specifically, the following embodiments relate to an inspection system, an inspection method, and a program that inspect an object by performing image recognition processing on a captured image.

[0013] Hereinafter, an inspection system, an inspection method, and a program according to an embodiment will be described in detail with reference to the drawings. However, each drawing described in the following embodiment is a schematic drawing, and the ratio of the size and thickness of each component does not necessarily reflect the actual dimensional ratio.

[0014] Furthermore, the embodiment described below is merely an example of an embodiment of the present disclosure. The present disclosure is not limited to the following embodiment, and various modifications are possible depending on the design, etc., as long as the effects of the present disclosure can be achieved.

[0015] (1) Overview In the construction of a building, various inspections are performed at each process. For example, components such as reinforcement, wallpaper, wiring fixtures, wiring, or piping are inspected to determine the arrangement, shape, and size of the inspection object. In addition, the building itself is inspected to determine the structure, arrangement, and shape of the inspection object. In such inspections, image recognition processing is performed on captured images of the inspection object to evaluate inspection items such as the arrangement, shape, size, and structure of the inspection object. Note that the construction of a building may be any work related to the construction of a building, and includes, for example, construction work, electrical work, water supply and drainage work, and civil engineering work.

[0016] In recent years, it has become easier to generate captured images that include three-dimensional position information, rather than two-dimensional captured images. Therefore, the inspection system of this embodiment shown in Fig. 1 inspects the inspection object 9 using captured images that include three-dimensional position information.

[0017] The inspection system 1 comprises an image acquisition unit 1b, a display control unit 1c, an inspection range setting unit 1e, and an inspection unit 1h. The image acquisition unit 1b is an image of a three-dimensional inspection object 9, and acquires data of the captured image including three-dimensional position information. The display control unit 1c displays the captured image on a screen D1. The inspection range setting unit 1e sets an inspection range indicating the range in which the inspection object 9 appears in the captured image through human operation. The inspection unit 1h inspects the inspection object 9 by performing image recognition processing on the inspection range.

[0018] Therefore, the inspection system 1 sets the inspection range in the captured image by human operation, which makes it possible to prevent structures other than the inspection target from being included in the inspection range, and to set the inspection range in the captured image more accurately.

[0019] (2) Details The inspection system 1 of this embodiment will be described in detail below.

[0020] (2.1) Inspection target In this embodiment, the inspection object 9 is the reinforcement of columns, beams, walls, slabs, foundations, etc. of a building. Fig. 2 shows reinforcement 91, 92. The reinforcement 91, 92 exist in a three-dimensional space W1 inside a building under construction. The reinforcement 91, 92 are reinforcement of beams. The outline of the reinforcement 91 and the outline of the reinforcement 92 are rectangular.

[0021] Each of the reinforcement bars 91, 92 is made up of multiple reinforcing bars 9a arranged in a lattice pattern, and is a component with a three-dimensional shape. The reinforcement inspection is carried out by individually determining whether the arrangement and arrangement of the multiple reinforcing bars 9a, as well as the size of the reinforcing bars 9a, for each of the reinforcement bars 91, 92, meets predetermined specifications.

[0022] In this embodiment, it is assumed that the rectangular outline of the reinforcement is made up of eight sides extending along the vertical and horizontal directions in the three-dimensional space W1.

[0023] (2.2) Inspection system The inspection system 1 preferably includes a computer system. The computer system executes a program to realize part or all of the inspection system 1. The computer system's main hardware component is a processor that operates according to the program. The processor can be of any type, as long as it can realize its functions by executing the program. The processor may be composed of one or more electronic circuits, including a semiconductor integrated circuit (IC) or large-scale integration (LSI). While ICs and LSIs are used here, the names may vary depending on the degree of integration, and may be called system LSIs, very large-scale integration (VLSIs), or ultra large-scale integration (ULSIs). Field-programmable gate arrays (FPGAs), which are programmed after the LSI is manufactured, or reconfigurable logic devices that can reconfigure the connections within the LSI or set up circuit partitions within the LSI, can also be used for the same purpose. Multiple electronic circuits may be integrated on a single chip or on multiple chips. Multiple chips may be integrated into a single device or on multiple devices. The program is recorded on a non-transitory recording medium, such as a ROM, optical disk, or hard disk drive, that can be read by a computer system. The program may be stored in advance on a non-transitory recording medium, or may be supplied to the non-transitory recording medium via a wide area communication network including the Internet.

[0024] The computer system is not limited to a single computer device, but may be realized by multiple computers linked together. The computer system may also be constructed as a cloud computing system.

[0025] As shown in FIG. 1, the inspection system 1 of this embodiment includes an imaging unit 1a, an image acquisition unit 1b, a display control unit 1c, an operation unit 1d, an inspection range setting unit 1e, a marker generation unit 1f, a marker processing unit 1g, an inspection unit 1h, and a communication unit 1i. Preferably, the inspection system 1 further includes a screen D1. As shown in FIGS. 1 and 2, the inspection system 1 of this embodiment is an inspection device configured with a tablet terminal T1 that can be carried by an operator H1. The operator H1 carries the tablet terminal T1 and moves within a three-dimensional space W1. The operator H1 then captures an image of the reinforcement bar that is the inspection target 9 using the tablet terminal T1, and the inspection system 1 inspects the reinforcement bar that is the inspection target 9. In this embodiment, the operator H1 corresponds to a person who specifies the inspection range within the captured image.

[0026] (2.2.1) Imaging unit As shown in FIG. 3, the imaging unit 1a is a three-lens stereo camera having three lenses 101. The imaging unit 1a generates captured image data including three-dimensional position information based on image data captured by each of the three lenses 101. The three-dimensional position information is, for example, three-dimensional coordinates (e.g., world coordinates) of an object included in the imaging range of the imaging unit 1a. In other words, the imaging unit 1a generates three-dimensional image data as captured image data.

[0027] The imaging unit 1a captures an image of a three-dimensional space W1 including a three-dimensional inspection object 9, and generates data of the captured image. In this embodiment, the reinforcement 91 of the reinforcement bars 91, 92 is the inspection object 9. Therefore, the worker H1 operates the tablet terminal T1 so that the reinforcement bar 91 appears in the captured image. However, depending on the arrangement of the reinforcement bars 91, 92 and the position of the worker H1, not only the reinforcement bar 91 but also the reinforcement bar 92 may appear in the captured image.

[0028] The imaging unit 1a then outputs the captured image data as real-time video data. That is, the imaging unit 1a generates, as the captured image data, data of a live view image captured by the movable imaging unit 1a.

[0029] (2.2.2) Image acquisition unit The image acquisition unit 1b acquires live view image data (captured image data) from the imaging unit 1a.

[0030] In this embodiment, the imaging unit 1a and the image acquisition unit 1b are provided in the same device, that is, the tablet terminal T1, but the imaging unit 1a may be provided separately from the tablet terminal T1.

[0031] (2.2.3) Display control section The display control unit 1c displays the captured image acquired by the image acquisition unit 1b on a screen D1. The screen D1 is, for example, a liquid crystal display or an organic EL display. The screen D1 in this embodiment is a display device provided in the tablet terminal T1.

[0032] The display control unit 1c receives data of the live view image acquired by the image acquisition unit 1b and displays the live view image on the screen D1.

[0033] A worker H1 carrying the tablet terminal T1 can check the captured image by looking at the screen D1. Since the captured image in this embodiment is a live view image, the worker H1 can monitor the captured image in real time.

[0034] 4 shows captured image G1 displayed on screen D1. Captured image G1 not only captures reinforcement 91, which is the inspection target 9, but also reinforcement 92, which is not the inspection target 9. In captured image G1, the area capturing reinforcement 91 is designated Q91, and the area capturing reinforcement 92 is designated Q92. The data of captured image G1 includes three-dimensional position information for area Q91 and area Q92.

[0035] (2.2.4) Operation section The operation unit 1d is a touch sensor placed on top of the screen D1, and the screen D1 and the operation unit 1d form a touch panel. The worker H1 can specify any position in the captured image G1 on the screen D1 by touching the operation unit 1d, which is a touch sensor, with a finger or a touch pen. The any position in the captured image G1 may be, for example, any point, line, or area in the captured image G1.

[0036] For example, the worker H1 specifies at least one point within the area Q91 in which the reinforcement bars 91 appear in the captured image G1, or at least one point near the area Q91.

[0037] In this embodiment, the outline of the reinforcement 91 is rectangular and has six faces. The worker H1 captures an image of the reinforcement 91 from an oblique angle so that at least one of the six faces constituting the rectangular outline of the reinforcement 91 is captured in the area Q91 (in FIG. 4, the reinforcement 91 is captured from an oblique angle so that three faces are captured). The worker H1 then specifies four points P1-P4, which correspond to the four vertices of the outline of the rectangular inspection range R1 (see FIG. 6), in the area Q91 where the reinforcement 91 is captured, as shown in FIG. 5. Point P1 corresponds to a vertex common to three mutually orthogonal faces of the rectangular outline. Points P2-P4 correspond to the ends of three sides extending from a vertex common to the three mutually orthogonal faces of the rectangle. That is, by specifying four points P1-P4 in the captured image G1, the worker H1 can specify within the captured image G1 an area corresponding to three mutually perpendicular faces out of the six rectangular faces that form the outline of the inspection range R1 (see Figure 6).

[0038] In Fig. 5, by specifying four points P1-P4 in area Q91, which shows the entire reinforcement 91, the entire reinforcement 91 is shown in inspection area R1 (see Fig. 6). However, there are cases where area Q91 does not show the entire reinforcement 91, but only shows a part of the reinforcement 91. Even in such cases, as described above, by specifying four points P1-P4 in area Q91 that correspond to the four vertices of the outline of the rectangular inspection area R1 (see Fig. 6), the area showing a part of the reinforcement 91 can be set as the inspection area.

[0039] The operation unit 1d may be a mouse, a keyboard, or the like.

[0040] (2.2.5) Inspection range setting section 6 within the captured image G1 based on the position specified by the worker H1 within the captured image G1. The inspection range R1 includes an area Q91 within the captured image G1 in which the reinforcement bars 91 to be inspected are captured.

[0041] The inspection area setting unit 1e sets an inspection area R1 within the captured image G1 based on the four points P1-P4 set in the captured image G1.

[0042] Specifically, the inspection range setting unit 1e can recognize the vertical and horizontal directions in the captured image G1 based on the detection results of a motion sensor, such as an acceleration sensor, a gyro sensor, or a tilt sensor, built into the tablet terminal T1. The inspection range setting unit 1e can also recognize the position and orientation of the tablet terminal T1 using at least one (or both) of the detection results of the motion sensor and changes in the captured image G1 (image frame differences). Therefore, the inspection range setting unit 1e sets an imaging area in the captured image G1, in which a rectangular object having four points P1-P4 as its four vertices, as the inspection range R1 (see FIG. 6 ). In this embodiment, it is assumed that the rectangular outline of the reinforcement is composed of eight sides extending vertically and horizontally in the three-dimensional space W1. Based on this assumption, the inspection range setting unit 1e estimates the imaging area in which a rectangular object having four points P1-P4 as its four vertices is imaged.

[0043] Therefore, the inspection system 1 can distinguish the area Q91 containing the reinforcement 91 from the area Q92 containing the reinforcement 92, and set the area Q91 as the inspection area R1 without including the area Q92 in the inspection area. That is, the inspection system 1 can more accurately set the inspection area R1 in the captured image G1.

[0044] (2.2.6) Marker Generation Unit The marker generating unit 1f generates a marker image M1 (see FIG. 8) that indicates the inspection range R1 in the captured image G1.

[0045] In this embodiment, the marker generation unit 1f generates a rectangular three-dimensional model K1 (see FIG. 7) including three mutually orthogonal faces with four points P1-P4 as vertices. Given the symmetry of the rectangular body, if three mutually orthogonal faces among the six faces constituting the outer surface of the rectangular body are known, it is possible to estimate the remaining three faces constituting the outer surface of the rectangular body. Therefore, the marker generation unit 1f generates a rectangular three-dimensional model K1 in which the four points P1-P4 correspond to the four vertices. That is, the marker generation unit 1f generates the shape of the three-dimensional model K1 as a rectangular body including three mutually orthogonal faces specified by the four points P1-P4 in the captured image G1.

[0046] The marker generation unit 1f then generates a marker image M1 shown in FIG. 8 by perspectively projecting the three-dimensional model K1 onto the captured image G1. This marker image M1 is generated in an area where the three-dimensional model K1 is perspectively projected into the captured image G1, and becomes a three-dimensional graphic image in which the three-dimensional model K1 is displayed within the captured image G1. As a result, the display control unit 1c displays the marker image M1 on the screen D1, superimposed on the captured image G1, as shown in FIG. 8. The marker image M1 is an image having a color, pattern, and design so that it can be distinguished from the captured image G1. The range of the marker image M1 in the captured image G1 is the same as the inspection range R1. In other words, the range of the marker image M1 in the captured image G1 includes the area Q91 in which the reinforcement bars 91 are displayed.

[0047] Therefore, the operator H1 can confirm the inspection range R1 in the captured image G1 by looking at the marker image M1 in the captured image G1 displayed on the screen D1. As a result, the inspection system 1 can set the inspection range R1 in the captured image G1 more accurately.

[0048] (2.2.7) Marker processing unit After setting the inspection range R1 and the marker image M1 in the captured image G1 as described above, the inspection system 1 links the subsequent captured image G1 with the marker image M1. That is, the range of the marker image M1 can be regarded as the inspection range R1, and the inspection range R1 follows the change in the marker image M1 and changes in the same way as the marker image M1.

[0049] In this embodiment, the marker processing unit 1g performs at least one of the following processes on the marker image M1 displayed on the screen D1: movement, rotation, enlargement, and reduction.

[0050] In this embodiment, the marker processing unit 1g performs processing such as movement, rotation, enlargement, and reduction on the marker image M1 displayed on the screen D1 in response to an operation by the operator H1 on the operation unit 1d.

[0051] Specifically, the worker H1 operates the operation unit 1d, which is a touch sensor overlaid on the screen D1, with a finger or a touch pen. The worker H1 performs a slide operation on the marker image M1 displayed on the screen D1 to move or rotate the marker image M1. The worker H1 performs a pinch-out operation on the marker image M1 displayed on the screen D1 to enlarge the marker image M1. The worker H1 performs a pinch-in operation on the marker image M1 displayed on the screen D1 to reduce the marker image M1.

[0052] The marker processing unit 1g reflects the movement, rotation, enlargement, and reduction processes performed on the marker image M1 on the three-dimensional model K1, and performs correction processes for the movement of the three-dimensional model K1 relative to the projection surface, and the rotation, enlargement, and reduction of the three-dimensional model K1.

[0053] The inspection range setting unit 1e performs correction processes of movement, rotation, enlargement, and reduction on the inspection range R1 in the same manner as the processes on the marker image M1, in accordance with the processes of movement, rotation, enlargement, and reduction on the marker image M1. That is, the inspection range setting unit 1e corrects the inspection range R1 in accordance with the processes performed on the marker image M1.

[0054] Therefore, the inspection system 1 can correct the inspection range R1 by correcting the display area of ​​the marker image M1. As a result, the inspection system 1 can set the inspection range R1 in the captured image G1 more accurately.

[0055] (2.2.8) Inspection Department The inspection unit 1h inspects the reinforcement bars 91, which are the inspection object 9, by performing image recognition processing on the inspection range R1 of the captured image G1.

[0056] Specifically, the inspection unit 1h extracts feature quantities of the inspection range R1 by performing binarization, filtering, edge extraction, and other processes on the inspection range R1 of the captured image G1. The inspection unit 1h also acquires design information from the design information storage unit 2a of the server device 2 via the communication unit 1i. The design information includes information such as the arrangement and assembly of multiple reinforcing bars 9a and the size of the reinforcing bars 9a as design specifications for the reinforcement 91. The inspection unit 1h then compares the feature quantities of the inspection range R1 with the design information to determine whether the reinforcement 91 satisfies the design specifications. The inspection unit 1h regards the determination result as the inspection result of the reinforcement 91 and generates inspection notification data as image data for notifying the inspection result of the reinforcement 91.

[0057] The display control unit 1c displays on the screen D1 the inspection notification data of the reinforcement 91. Therefore, the worker H1 can check the inspection results of the reinforcement 91 by looking at the inspection notification data displayed on the screen D1.

[0058] (2.2.9) Communications Department The communication unit 1i is connected to a network NT1 including the Internet, communicates with devices on the network NT1, and has a communication interface function for sending and receiving signals. A server device 2 is connected to the network NT1, and the communication unit 1i can communicate with the server device 2 via the network NT1.

[0059] The server device 2 is managed by, for example, a contractor who has been contracted to design or construct a building, and is equipped with a design information storage unit 2a. The design information storage unit 2a stores various information related to the specifications, design, and construction of the building. The information stored in the design information storage unit 2a also includes design information for reinforcement 91, 92.

[0060] Therefore, the inspection system 1 can acquire design information of the reinforcement bars 91 and 92 and the like from the server device 2 via the communication unit 1i.

[0061] (2.2.10) Linking captured images with marker images As described above, the inspection system 1 inspects the reinforcement 91 by setting the inspection range R1 and the marker image M1 in the captured image G1 of the reinforcement 91.

[0062] After setting the inspection area R1 and the marker image M1 in the captured image G1, the inspection system 1 links the subsequent captured image G1 with the marker image M1 and the inspection area R1. When the worker H1 moves within the three-dimensional space W1 while capturing an image of the reinforcement 91 with the tablet terminal T1, the captured image G1 displayed on the screen D1 changes, and the area Q91 in which the reinforcement 91 appears in the captured image G1 also changes. When the tablet terminal T1 moves within the three-dimensional space W1, the positions and shapes of the marker image M1 and the inspection area R1 in the captured image G1 also change.

[0063] Specifically, the captured image G1 includes three-dimensional position information. Therefore, the marker generation unit 1f acquires three-dimensional coordinate information (e.g., world coordinates) as coordinate information indicating the position of the reinforcement 91 (inspection target 9) in the three-dimensional space W1 using the three-dimensional position information in the inspection range R1 (i.e., area Q91) of the captured image G1. The marker generation unit 1f can also detect the movement distance and movement direction of the tablet terminal T1 based on at least one (or both) of the detection results of a motion sensor, such as an acceleration sensor, gyro sensor, or tilt sensor, built into the tablet terminal T1 and changes in the captured image G1 (difference between image frames). Therefore, the marker generation unit 1f corrects the three-dimensional coordinates indicating the position of the reinforcement 91 in the three-dimensional space W1 based on the movement distance and movement direction of the tablet terminal T1. That is, the marker generation unit 1f can detect the position of the reinforcement 91 relative to the imaging unit 1a even if the tablet terminal T1 moves.

[0064] Then, the marker generation unit 1f adjusts the projection distance and projection direction of the three-dimensional model K1 relative to the captured image G1 based on the three-dimensional coordinates indicating the position of the reinforcement 91, thereby causing the position and shape of the marker image M1 in the captured image G1 to follow changes in the inspection range R1 in the captured image G1.

[0065] Furthermore, the inspection range setting unit 1e changes the inspection range R1 in the same manner as the marker image M1.

[0066] Therefore, even if the captured image G1 is a live view image, the inspection system 1 can link the marker image M1 to the captured image G1. Also, even if the captured image G1 is a live view image, the inspection system 1 can link the inspection range R1 to the captured image G1.

[0067] (2.2.11) Inspection method The inspection method performed by the inspection system 1 described above can be summarized as shown in the flowchart of Fig. 9. For example, the inspection method is performed by a computer system included in the inspection system 1 executing a program.

[0068] The inspection method includes an image acquisition step S1, a display step S2, an inspection range setting step S3, and an inspection step S5. Preferably, the inspection method further includes a marker image setting step S4.

[0069] In the image acquisition step S1, the image acquisition unit 1b acquires data of a captured image G1. The captured image G1 is an image of an inspection object 9 having a three-dimensional shape, and includes three-dimensional position information.

[0070] In the display step S2, the display control unit 1c displays the captured image G1 on the screen D1.

[0071] In the inspection range setting step S3, the inspection range setting unit 1e sets an inspection range R1 indicating the range in which the inspection object 9 appears in the captured image G1, in response to an operation by the operator H1.

[0072] In the marker image setting step S4, the marker generating unit 1f generates a marker image M1 indicating the inspection range R1 in the captured image G1, and displays the marker image M1 on the screen D1 in a state where the marker image M1 is superimposed on the captured image G1.

[0073] In the inspection step S5, the inspection unit 1h inspects the inspection object 9 by performing image recognition processing on the inspection range R1.

[0074] The above-described inspection method allows the inspection range R1 in the captured image G1 to be set more accurately.

[0075] (3) First Modification The marker generation unit 1f may input the captured image G1 into a learning model constructed by machine learning using the captured image and the range of the marker image in the captured image as training data, and thereby obtain the captured image G1 with the marker image M1 superimposed thereon from the learning model.

[0076] Specifically, a learning model is constructed by learning using a large amount of training data. The learning model preferably uses a neural network constructed by machine learning such as deep learning. Alternatively, the learning model may use other algorithms such as a support vector machine.

[0077] The machine learning system that constructs the learning model may be either a configuration provided in the inspection system 1 or a configuration provided outside the inspection system 1.

[0078] (4) Second Modification The inspection range setting unit 1e may set the inspection range using information on the dimensions of the inspection object 9.

[0079] The following description will be given using the captured image G11 in FIG.

[0080] The captured image G11 displayed on the screen D1 includes areas Q93 and Q94. The reinforcement shown in area Q93 is the reinforcement of a column. The reinforcement shown in area Q94 is the reinforcement of a foundation. The outline of the reinforcement of the foundation is formed in the shape of a horizontally extending rectangular plate, while the outline of the reinforcement of the column is formed in the shape of a rectangular body that is long in the vertical direction. In this modified example, worker H1 takes an image of the reinforcement of the column from the front, with the reinforcement of the column shown in area Q93 as the inspection object 9.

[0081] By operating the tablet terminal T1, the worker H1 downloads in advance (pre-stores in the tablet terminal T1) design information for the reinforcement of the column corresponding to the area Q93 (the reinforcement that is the inspection target 9) from the design information storage unit 2a of the server device 2 via the communication unit 1i to the tablet terminal T1. The design information includes dimensional information for the reinforcement as the design specifications for the reinforcement of the column corresponding to the area Q93.

[0082] 10, worker H1 specifies one face of the rectangular outline of the column reinforcement in region Q93 by specifying three points P11-P13 in region Q93. Points P11-P13 define the front surface of the rectangular outline of the reinforcement. In other words, by specifying three points P11-P13 in the captured image G11, worker H1 can specify an area in the captured image G11 that corresponds to one of the six rectangular faces that make up the outline of the reinforcement.

[0083] The inspection range setting unit 1e can recognize the vertical and horizontal directions in the captured image G11 based on the detection results of motion sensors, such as an acceleration sensor, gyro sensor, or tilt sensor, built into the tablet terminal T1. The inspection range setting unit 1e can also recognize the position and orientation of the tablet terminal T1 using at least one (or both) of the detection results of the motion sensor and changes in the captured image G1 (image frame differences). Therefore, the inspection range setting unit 1e sets an inspection range R11 (see FIG. 11 ) by comparing three points P11-P13 specified in the captured image G11 with the reinforcement design information captured in area Q93. This inspection range R11 corresponds to the captured area of ​​a rectangular object including one surface defined by the three points P11-P13.

[0084] Therefore, the inspection system 1 can distinguish the area Q93, which shows the reinforcement of the inspection target, from the area Q94, which shows reinforcement that is not the inspection target 9, and set the area Q93 as the inspection area R11 without including the area Q94 in the inspection area. That is, the inspection system 1 can more accurately set the inspection area R11 in the captured image G11.

[0085] Furthermore, the inspection system 1 uses information on the dimensions of the reinforcement bars 93, thereby simplifying the operation of the worker H1.

[0086] Then, the marker generating unit 1f generates a marker image M11 (see FIG. 13) that indicates the inspection range R11 in the captured image G11.

[0087] In this embodiment, the marker generation unit 1f generates a rectangular three-dimensional model K11 (see FIG. 12) including one face specified in the captured image G11, using information on the dimensions of the reinforcement that is the inspection object 9. That is, the marker generation unit 1f generates the rectangular three-dimensional model K11 by comparing three points P11-P13 specified in the captured image G11 with design information on the reinforcement that is the inspection object 9. That is, the marker generation unit 1f sets the shape of the three-dimensional model K11 to the shape of a rectangle that includes one face specified by the three points P11-P13 in the captured image G11.

[0088] The marker generation unit 1f then generates a marker image M11 shown in FIG. 13 by perspectively projecting the three-dimensional model K11 onto the captured image G11. This marker image M11 is generated in an area where the three-dimensional model K11 is perspectively projected into the captured image G11, and becomes a three-dimensional graphic image in which the three-dimensional model K11 is displayed within the captured image G11. As a result, the display control unit 1c displays the marker image M11 on the screen D1, superimposed on the captured image G11, as shown in FIG. 13. The marker image M11 is an image having a color, pattern, or design so that it can be distinguished from the captured image G11. The range of the marker image M11 in the captured image G11 is the same as the inspection range R11. In other words, the range of the marker image M11 relative to the captured image G11 includes the area Q93 in which the reinforcement bar, which is the inspection target 9, is displayed.

[0089] Therefore, the operator H1 can confirm the inspection range R11 in the captured image G11 by looking at the marker image M11 in the captured image G11 displayed on the screen D1. As a result, the inspection system 1 can more accurately set the inspection range R11 in the captured image G11.

[0090] The worker H1 may specify multiple faces of the rectangular outline of the reinforcement in the area Q93 of the captured image G11. In this case, the inspection area setting unit 1e sets the inspection area R11 (see FIG. 11) by comparing the multiple faces specified in the captured image G11 with the design information of the reinforcement (shown in the area Q93) that is the inspection target 9. This inspection area R11 corresponds to the imaging area in which the rectangular body including the multiple faces specified in the captured image G11 is imaged. The marker generation unit 1f also sets the shape of the three-dimensional model K11 to the shape of the rectangular body including the multiple faces specified in the captured image G11.

[0091] (5) Third Modification As shown in Fig. 14, the worker H1 may specify one side of the rectangular outline of the reinforcement in the area Q93 by specifying one point P21 in the area Q93 of the captured image G11 displayed on the screen D1. In this modification, the point P21 defines the upper front side of the rectangular outline of the reinforcement. In other words, by specifying one point P21 in the captured image G11, the worker H1 can specify one side of the eight sides that make up the outline of the reinforcement in the captured image G11.

[0092] In this case, the inspection range setting unit 1e sets an inspection range R11 (see FIG. 11) by comparing one point P21 specified in the captured image G11 with the design information of the reinforcement (reinforcement shown in area Q93) that is the inspection target 9. This inspection range R11 corresponds to an imaging area in which an image of a rectangle including one side defined by one point P21 is captured.

[0093] Then, the marker generation unit 1f generates a marker image M11 (see FIG. 13) indicating the inspection range R11 in the captured image G11. Here, the marker generation unit 1f sets the shape of the three-dimensional model K11 to the shape of a rectangle having one side specified by one point P21 in the captured image G11.

[0094] The worker H1 may specify multiple sides of the rectangular outline of the reinforcement in the area Q93 of the captured image G11. In this case, the inspection area setting unit 1e sets the inspection area R11 (see FIG. 11) by comparing the multiple sides specified in the captured image G11 with the design information of the reinforcement (the reinforcement shown in the area Q93) that is the inspection target 9. This inspection area R11 corresponds to the imaging area in which the rectangular body including the multiple sides specified in the captured image G11 is imaged. Furthermore, the marker generation unit 1f sets the shape of the three-dimensional model K11 to the shape of the rectangular body including the multiple sides specified in the captured image G11.

[0095] (6) Fourth Modification 14, the worker H1 may specify a point on the rectangular outline of the reinforcement in the area Q93 by specifying a point P21 in the area Q93 of the captured image G11 displayed on the screen D1. In this modification, the point P31 defines a point included in the upper front side of the rectangular outline of the reinforcement.

[0096] In this case, the inspection range setting unit 1e sets an inspection range R11 (see FIG. 11) by comparing one point P21 specified in the captured image G11 with the design information of the reinforcement (reinforcement shown in area Q93) that is the inspection object 9. This inspection range R11 corresponds to the imaging area in which an image of a rectangular body including one point P21 is captured.

[0097] Then, the marker generation unit 1f generates a marker image M11 (see FIG. 13) indicating the inspection range R11 in the captured image G11. Here, the marker generation unit 1f sets the shape of the three-dimensional model K11 to the shape of a rectangle that includes one point P21 in the captured image G11.

[0098] The worker H1 may specify multiple points on the rectangular outline of the reinforcement in the area Q93 of the captured image G11. In this case, the inspection area setting unit 1e sets the inspection area R11 (see FIG. 11) by comparing the multiple points specified in the captured image G11 with the design information of the reinforcement (the reinforcement shown in the area Q93) that is the inspection target 9. This inspection area R11 corresponds to the imaging area in which a rectangular body including the multiple points specified in the captured image G11 is imaged. The marker generation unit 1f also sets the shape of the three-dimensional model K11 to the shape of a rectangular body including the multiple points specified in the captured image G11.

[0099] (7) Fifth Modification FIG. 15 shows an inspection system 1A as a modified example of the inspection system 1. As shown in FIG.

[0100] In the inspection system 1A, an inspection unit 1h is provided in a server device 2. The tablet terminal T1 and the server device 2 are configured to be able to communicate with each other via a network NT1, and the inspection unit 1h inspects an inspection object 9 by performing image recognition processing on an inspection range of a captured image.

[0101] FIG. 16 shows an inspection system 1B as another modification of the inspection system 1. As shown in FIG.

[0102] In the inspection system 1B, in addition to the inspection unit 1h, an inspection range setting unit 1e, a marker generation unit 1f, and a marker processing unit 1g are also provided in the server device 2. The tablet terminal T1 and the server device 2 are configured to be able to communicate with each other via a network NT1, and the inspection range setting unit 1e, the marker generation unit 1f, and the marker processing unit 1g also have the same functions as those described above.

[0103] The components of the inspection system 1 are not limited to being integrated into one device as in the inspection system 1, or being distributed across two devices as in the inspection systems 1A and 1B, but may also be distributed across three or more devices.

[0104] (8) Sixth Modification The inspection system 1 does not need to have the screen D1. For example, the screen D1 may be provided in an external terminal such as a tablet terminal, a smartphone, or a personal computer that is configured to be able to communicate with the inspection system 1.

[0105] The inspection object 9 may be a component other than reinforcement, such as wallpaper, wiring fixtures, wiring, or piping. In other words, the inspection object 9 is not limited to a specific component, and may be any component having a three-dimensional shape.

[0106] Furthermore, the operator H1 may operate the tablet terminal T1 to change the live view image to a still image at a desired timing and set the inspection range for the still image. In this case, while monitoring the live view image, the operator H1 may change the live view image to a still image when the inspection object 9 appears in the live view image, thereby displaying the still image containing the inspection object 9 on the screen D1. Note that both the live view image and the still image correspond to captured image data.

[0107] The imaging unit 1a is not limited to a stereo camera having three lenses, but may be a stereo camera having two or four or more lenses. Furthermore, the imaging unit 1a may be a monocular camera, in which case captured image data including three-dimensional position information can be generated from a series of captured images. Furthermore, the imaging unit 1a may be configured using a laser, such as a three-dimensional laser scanner or LiDAR (Light Detection and Ranging).

[0108] The inspection object 9 is not limited to the reinforcement bars 91 and 93, but may be reinforcement bars 92 and 94. The inspection object 9 may also be something other than reinforcement bars, such as wallpaper, wiring devices, wiring, or piping. The inspection object 9 may also be a building.

[0109] In the above-described embodiment, the inspection range setting unit 1e and the marker generation unit 1f set the inspection range and generate the marker image under the assumption that the rectangular outline of the reinforcement is composed of eight sides extending vertically and horizontally in the three-dimensional space W1. However, the assumptions used by the inspection system 1 may be set appropriately depending on the shape, structure, arrangement, etc. of the inspection target 9, and are not limited to any specific assumptions.

[0110] (9) Summary The inspection system (1, 1A, 1B) of the first aspect according to the above-described embodiment includes an image acquisition unit (1b), a display control unit (1c), an inspection area setting unit (1e), and an inspection unit (1h). The image acquisition unit (1b) acquires data of captured images (G1, G11) that are images of a three-dimensional inspection object (9) and include three-dimensional position information. The display control unit (1c) displays the captured images (G1, G11) on a screen (D1). The inspection area setting unit (1e) sets inspection areas (R1, R11) that indicate the area in which the inspection object (9) appears in the captured images (G1, G11) through operation by a person (H1). The inspection unit (1h) inspects the inspection object (9) by performing image recognition processing on the inspection areas (R1, R11).

[0111] The above-described inspection system (1, 1A, 1B) can set the inspection range (R1, R11) in the captured image (G1, G11) more accurately.

[0112] In the inspection system (1, 1A, 1B) of the second aspect according to the above-described embodiment, in the first aspect, it is preferable that the inspection range setting unit (1e) sets the inspection range (R1, R11) based on a position specified by the operation of a person (H1) in the captured image (G1, G11) displayed on the screen (D1).

[0113] The above-described inspection system (1, 1A, 1B) can set the inspection range (R1, R11) in the captured image (G1, G11) more accurately.

[0114] In the inspection system (1, 1A, 1B) of the third aspect according to the above-described embodiment, in the second aspect, it is preferable that the inspection range setting unit (1e) sets an inspection range (R1, R11) that includes at least one point (P1-P4, P11-P13, P21) specified in the captured image (G1, G11).

[0115] The above-described inspection system (1, 1A, 1B) can set the inspection range (R1, R11) in the captured image (G1, G11) more accurately.

[0116] In the inspection system (1, 1A, 1B) of the fourth aspect according to the above-described embodiment, in the second aspect, it is preferable that the inspection range setting unit (1e) sets an inspection range (R1, R11) that includes at least one range specified in the captured image (G1, G11).

[0117] The above-described inspection system (1, 1A, 1B) can set the inspection range (R1, R11) in the captured image (G1, G11) more accurately.

[0118] In the inspection system (1, 1A, 1B) of the fifth aspect according to the above-described embodiment, in any one of the first to fourth aspects, it is preferable that the inspection range setting unit (1e) sets the inspection range (R11) using information on the dimensions of the inspection object (9).

[0119] The above-described inspection system (1, 1A, 1B) can simplify the operation of the person (H1) by using the information on the dimensions of the inspection object (9).

[0120] The inspection system (1, 1A, 1B) of the sixth aspect according to the above-described embodiment, in any one of the first to fifth aspects, preferably further includes a marker generation unit (1f) that generates marker images (M1, M11) that indicate the inspection ranges (R1, R11) in the captured images (G1, G11). The display control unit (1c) displays the marker images (M1, M11) on the screen (D1) by superimposing them on the captured images (G1, G11).

[0121] In the above-described inspection system (1, 1A, 1B), a person (H1) can check the inspection range (R1, R11) in the captured image (G1, G11) by looking at the marker images (M1, M11) in the captured image (G1, G11) displayed on the screen (D1). As a result, the inspection system (1, 1A, 1B) can more accurately set the inspection range (R1, R11) in the captured image (G1, G11).

[0122] In the inspection system (1, 1A, 1B) of the seventh aspect according to the above-described embodiment, in the sixth aspect, it is preferable that the marker generation unit (1f) generates a three-dimensional model (K1, K11) of the inspection range (R1, R11) based on three-dimensional position information included in the inspection range (R1, R11). The marker generation unit (1f) sets a projected image of the three-dimensional model (K1, K11) as a marker image (M1, M11).

[0123] In the above-described inspection system (1, 1A, 1B), a person (H1) can easily check the inspection ranges (R1, R11) in the captured images (G1, G11).

[0124] In the inspection system (1, 1A, 1B) of the eighth aspect of the above-mentioned embodiment, in the seventh aspect, it is preferable that the marker generation unit (1f) sets the shape of the three-dimensional model (K1, K11) to a rectangular shape including three mutually perpendicular faces specified in the captured image (G1, G11).

[0125] The above-mentioned inspection systems (1, 1A, 1B) can easily generate marker images (M1, M11).

[0126] In the inspection system (1, 1A, 1B) of the ninth aspect according to the above-mentioned embodiment, in the seventh aspect, it is preferable that the marker generation unit (1f) uses information on the dimensions of the inspection object (9) to set the shape of the three-dimensional model (K11) to a three-dimensional shape that includes at least one surface specified in the captured image (G11).

[0127] The above-mentioned inspection systems (1, 1A, 1B) can easily generate marker images (M1, M11).

[0128] In the inspection system (1, 1A, 1B) of the tenth aspect according to the above-mentioned embodiment, in the seventh aspect, it is preferable that the marker generation unit (1f) uses information on the dimensions of the inspection object (9) to set the shape of the three-dimensional model (K11) to a three-dimensional shape that includes at least one side specified in the captured image (G11).

[0129] The above-mentioned inspection systems (1, 1A, 1B) can easily generate marker images (M1, M11).

[0130] In the inspection system (1, 1A, 1B) of the eleventh aspect according to the above-mentioned embodiment, in the seventh aspect, it is preferable that the marker generation unit (1f) uses information on the dimensions of the inspection object (9) to make the shape of the three-dimensional model (K11) a three-dimensional shape that includes at least one point specified in the captured image (G11).

[0131] The above-mentioned inspection systems (1, 1A, 1B) can easily generate marker images (M1, M11).

[0132] In the inspection system (1, 1A, 1B) of the 12th aspect according to the above-described embodiment, in the 6th aspect, it is preferable that the marker generation unit (1f) inputs the captured image (G1, G11) into a learning model constructed by machine learning using the captured image (G1, G11) and the range of the marker image (M1, M11) in the captured image (G1, G11) as training data, thereby obtaining the captured image (G1, G11) on which the marker image (M1, M11) is superimposed from the learning model.

[0133] The above-mentioned inspection systems (1, 1A, 1B) can easily generate marker images (M1, M11).

[0134] In the inspection system (1, 1A, 1B) of the thirteenth aspect according to the above-described embodiment, in any one of the sixth to twelfth aspects, the marker images (M1, M11) are projection images of the three-dimensional model (K1, K11). It is preferable that the marker generation unit (1f) displays the marker images (M1, M11) on the screen (D1) by converting three-dimensional coordinates of the three-dimensional model (K1, K11) into two-dimensional coordinates based on coordinate information indicating the position of the inspection object (9) in the three-dimensional space (W1) in which the inspection object (9) exists.

[0135] The above-described inspection system (1, 1A, 1B) can link the marker images (M1, M11) to the captured images (G1, G11).

[0136] The inspection system (1, 1A, 1B) of the 14th aspect according to the above-described embodiment, in any one of the 6th to 13th aspects, preferably further includes a marker processing unit (1g) that performs at least one of processing of movement, rotation, enlargement, and reduction on the marker image (M1, M11) displayed on the screen (D1) in response to an operation by a person (H1). The inspection range setting unit (1e) corrects the inspection range (R1, R11) in response to at least one of processing performed on the marker image (M1, M11).

[0137] The above-described inspection systems (1, 1A, 1B) can correct the inspection ranges (R1, R11) by correcting the marker images (M1, M11).

[0138] In the inspection system (1, 1A, 1B) of the 15th aspect of the above-mentioned embodiment, in any one of the 1st to 14th aspects, it is preferable that the captured image (G1, G11) is a live view image captured by a movable imaging unit (1a).

[0139] The above-mentioned inspection system (1, 1A, 1B) can monitor the captured images (G1, G11) in real time.

[0140] In the inspection system (1, 1A, 1B) of the sixteenth aspect according to the above-described embodiment, in any one of the first to fifteenth aspects, the inspection object (9) is preferably reinforcement (91-94).

[0141] The above-described inspection system (1, 1A, 1B) can inspect the reinforcement (91-94).

[0142] In the inspection system (1, 1A, 1B) of the seventeenth aspect according to the above-described embodiment, in any one of the first to fifteenth aspects, the inspection object (9) is preferably a building.

[0143] The inspection system (1, 1A, 1B) described above is capable of inspecting buildings.

[0144] The inspection method of the eighteenth aspect according to the above-described embodiment includes an image acquisition step (S1), a display step (S2), an inspection area setting step (S3), and an inspection step (S5). The image acquisition step (S1) acquires data of captured images (G1, G11) that are images of a three-dimensional inspection object (9) and include three-dimensional position information. The display step (S2) displays the captured images (G1, G11) on a screen (D1). The inspection area setting step (S3) sets inspection areas (R1, R11) that indicate the area in which the inspection object (9) appears in the captured images (G1, G11) through the operation of a person (H1). The inspection step (S5) inspects the inspection object (9) by performing image recognition processing on the inspection areas (R1, R11).

[0145] The above-described inspection method allows for more accurate setting of the inspection range (R1, R11) within the captured image (G1, G11).

[0146] A program according to a nineteenth aspect of the above-described embodiment causes a computer system to execute the inspection method according to the eighteenth aspect.

[0147] The above-described program can set the inspection range (R1, R11) in the captured image (G1, G11) more accurately. [Explanation of symbols]

[0148] 1, 1A, 1B Inspection System 1a Imaging unit 1b Image acquisition unit 1c Display control unit 1e Inspection range setting section 1f Marker generation section 1g Marker processing section 1h Inspection Department 9. Inspection Subjects 91-94 Reinforcement D1 screen W1 3D space H1 Operator (person) G1, G11 captured images R1, R11 inspection range M1, M11 marker images K1, K11 3D models P1-P4, P11-P13, P21 points S1 Image acquisition step S2 Display Step S3 Inspection range setting step S5 Inspection step

Claims

1. an image acquisition unit that acquires data of an image of an inspection object having a three-dimensional shape, the image including three-dimensional position information; a display control unit that displays the captured image on a screen; an inspection range setting unit that sets, by human operation, an inspection range indicating a range in which the inspection object is captured in the captured image; an inspection unit that inspects the inspection object by performing image recognition processing on the inspection range, the inspection range setting unit sets, as the inspection range, an imaging region in which an image of a rectangular body is captured, the four points specified by the human operation in the captured image displayed on the screen corresponding to the four vertices of the rectangular body; one of the four points corresponds to a vertex common to three mutually orthogonal faces of the outline of the rectangular body, among the four vertices; The remaining three points of the four points correspond to the ends of three sides extending from a vertex common to the three faces among the four vertices. Inspection system.

2. An image acquisition unit that acquires data of an image of a three-dimensional inspection object, the image including three-dimensional position information; a display control unit that displays the captured image on a screen; an inspection range setting unit that sets, by human operation, an inspection range indicating a range in which the inspection object is captured in the captured image; an inspection unit that inspects the inspection object by performing image recognition processing on the inspection range, The inspection range setting unit sets the inspection range using information on the dimensions of the inspection object. Inspection system.

3. further comprising a marker generation unit that generates a marker image indicating the inspection range in the captured image; The display control unit displays the marker image on the screen by superimposing the marker image on the captured image.

3. The inspection system of claim 1 or 2.

4. An image acquisition unit that acquires data of an image of a three-dimensional inspection object, the image including three-dimensional position information; a display control unit that displays the captured image on a screen; an inspection range setting unit that sets, by human operation, an inspection range indicating a range in which the inspection object is captured in the captured image; an inspection unit that inspects the inspection object by performing image recognition processing on the inspection range; a marker generation unit that generates a marker image that indicates the inspection range in the captured image, the display control unit displays the marker image on the screen by superimposing the marker image on the captured image; The marker generation unit generating a three-dimensional model of the inspection range based on the three-dimensional position information included in the inspection range; The projected image of the three-dimensional model is used as the marker image. Inspection system.

5. The marker generation unit generates a shape of the three-dimensional model as a rectangular body shape including three mutually orthogonal faces specified in the captured image. The inspection system of claim 4.

6. The marker generation unit uses information on the dimensions of the inspection object to generate a shape of the three-dimensional model that includes at least one surface specified in the captured image. The inspection system of claim 4.

7. The marker generation unit uses information on the dimensions of the inspection object to generate a shape of the three-dimensional model that includes at least one side specified in the captured image. The inspection system of claim 4.

8. The marker generation unit uses information on the dimensions of the inspection object to generate a shape of the three-dimensional model that includes at least one point specified in the captured image. The inspection system of claim 4.

9. The marker generation unit inputs the captured image into a learning model constructed by machine learning using the captured image and the range of the marker image in the captured image as training data, and acquires the captured image with the marker image superimposed from the learning model. The inspection system of claim 3.

10. An image acquisition unit that acquires data of an image of a three-dimensional inspection object, the image including three-dimensional position information; a display control unit that displays the captured image on a screen; an inspection range setting unit that sets, by human operation, an inspection range indicating a range in which the inspection object is captured in the captured image; an inspection unit that inspects the inspection object by performing image recognition processing on the inspection range; a marker generation unit that generates a marker image that indicates the inspection range in the captured image, the display control unit displays the marker image on the screen by superimposing the marker image on the captured image; the marker image is a projection image of a three-dimensional model, The marker generation unit converts three-dimensional coordinates of the three-dimensional model into two-dimensional coordinates based on coordinate information indicating a position of the inspection object in a three-dimensional space in which the inspection object exists, thereby displaying the marker image on the screen. Inspection system.

11. An image acquisition unit that acquires data of an image of a three-dimensional inspection object, the image including three-dimensional position information; a display control unit that displays the captured image on a screen; an inspection range setting unit that sets, by human operation, an inspection range indicating a range in which the inspection object is captured in the captured image; an inspection unit that inspects the inspection object by performing image recognition processing on the inspection range; a marker generation unit that generates a marker image that indicates the inspection range in the captured image, the display control unit displays the marker image on the screen by superimposing the marker image on the captured image; a marker processing unit that performs at least one of a process of moving, rotating, enlarging, and reducing the marker image displayed on the screen in response to the operation by the person; The inspection range setting unit corrects the inspection range in accordance with the at least one process performed on the marker image. Inspection system.

12. The captured image is a live view image captured by a movable imaging unit.

12. An inspection system according to any one of claims 1 to 11.

13. The inspection object is reinforcement.

13. An inspection system according to any one of claims 1 to 12.

14. The inspection object is a building.

13. An inspection system according to any one of claims 1 to 12.

15. An image acquisition step of acquiring data of an image of an inspection object having a three-dimensional shape, the captured image including three-dimensional position information; a display step of displaying the captured image on a screen; an inspection range setting step of setting, by a human operation, an inspection range indicating a range in which the inspection object is captured in the captured image; an inspection step of inspecting the inspection object by performing image recognition processing on the inspection range, In the inspection range setting step, an imaging area in which an image of a rectangular body in which the four points specified by the human operation in the captured image displayed on the screen correspond to four vertices is set as the inspection range, one of the four points corresponds to a vertex common to three mutually orthogonal faces of the outline of the rectangular body, among the four vertices; The remaining three points of the four points correspond to the ends of three sides extending from a vertex common to the three faces among the four vertices. Testing method.

16. An image acquisition step of acquiring data of an image of an inspection object having a three-dimensional shape, the captured image including three-dimensional position information; a display step of displaying the captured image on a screen; an inspection range setting step of setting, by a human operation, an inspection range indicating a range in which the inspection object is captured in the captured image; an inspection step of inspecting the inspection object by performing image recognition processing on the inspection range. In the inspection range setting step, the inspection range is set using information on the dimensions of the inspection object. Testing method.

17. An image acquisition step of acquiring data of an image of an inspection object having a three-dimensional shape, the captured image including three-dimensional position information; a display step of displaying the captured image on a screen; an inspection range setting step of setting, by a human operation, an inspection range indicating a range in which the inspection object is captured in the captured image; an inspection step of inspecting the inspection object by performing image recognition processing on the inspection range; a marker image setting step of generating a marker image indicating the inspection range in the captured image, In the marker image setting step, The marker image is displayed on the screen by being superimposed on the captured image; generating a three-dimensional model of the inspection range based on the three-dimensional position information included in the inspection range; The projected image of the three-dimensional model is used as the marker image. Testing method.

18. an image acquiring step of acquiring data of an image of an inspection object having a three-dimensional shape, the image including three-dimensional position information; a display step of displaying the captured image on a screen; an inspection range setting step of setting, by a human operation, an inspection range indicating a range in which the inspection object is captured in the captured image; an inspection step of inspecting the inspection object by performing image recognition processing on the inspection range; a marker image setting step of generating a marker image indicating the inspection range in the captured image, In the marker image setting step, the marker image is displayed on the screen in a state where it is superimposed on the captured image; the marker image is a projection image of a three-dimensional model, In the marker image setting step, the marker image is displayed on the screen by converting three-dimensional coordinates of the three-dimensional model into two-dimensional coordinates based on coordinate information indicating the position of the inspection object in the three-dimensional space in which the inspection object exists. Testing method.

19. An image acquisition step of acquiring data of an image of an inspection object having a three-dimensional shape, the captured image including three-dimensional position information; a display step of displaying the captured image on a screen; an inspection range setting step of setting, by a human operation, an inspection range indicating a range in which the inspection object is captured in the captured image; an inspection step of inspecting the inspection object by performing image recognition processing on the inspection range; a marker image setting step of generating a marker image indicating the inspection range in the captured image, In the marker image setting step, the marker image is displayed on the screen in a state where it is superimposed on the captured image; a marker processing step of performing at least one of a movement, a rotation, an enlargement, and a reduction on the marker image displayed on the screen in response to the operation by the person; In the inspection range setting step, the inspection range is corrected in accordance with the at least one process performed on the marker image. Testing method.

20. A computer system that executes the inspection method according to any one of claims 15 to 19. program.

Images