Information processing device, imaging system, imaging control method, and program

Abstract

To acquire appropriate images when performing an inspection work.SOLUTION: An information processing device controls an imaging device 101 to change the imaging direction with respect to a wall surface 403 being the inspection target and acquire a preview image for each section of the wall surface 403, analyzes the preview image to determine whether the proportion of a prescribed area that occupies the region of the section of the wall surface 403 satisfies conditions permitted in an infrastructure inspection of the wall surface 403 for each section of the wall surface 403, and controls whether or not the imaging device 101 captures images for inspection in the section of the wall surface 403 according to the result of determination on whether the conditions permitted in the infrastructure inspection of the wall surface 403 are satisfied.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present disclosure relates to techniques for obtaining images for inspecting an inspection object. [Background technology]

[0002] For example, as part of the inspection of architectural structures such as buildings, dams, and various infrastructure facilities, high-resolution images are taken and analyzed to understand the presence or absence of deformation such as cracks and rust and the transition from the past. At this time, in order to detect the presence or absence of minute deformation in millimeters on the wall surface, for example, the wall surface is divided into small sections, and high-resolution images are taken for each section. In addition, since it is necessary to take a large number of images covering the entire wall surface for all of these sections, an imaging system is used that combines a camera platform device that can automatically perform pan and tilt drive with an imaging device. The camera platform device moves the imaging direction of the imaging device in sequence, and repeats an operation such that it automatically stops when the imaging direction of the imaging device is in a direction in which the target section can be photographed. Then, the imaging device takes an image every time the camera platform device stops. In addition, in order to obtain a high-resolution image, it is necessary to set the resolution to high. Patent Document 1 describes a technology that analyzes the captured image of a structure to identify damaged parts, applies a high compression rate to the parts of the captured image other than the identified parts, and saves the image data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2019 / 130391 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the following problems may occur in automatic photography using a camera-head device. For example, when the camera-head device moves the shooting direction for each section so as to cover the entire wall surface, backgrounds other than the wall surface or obstacles such as rocks and trees may appear in the section. In such cases, images that are not suitable for inspection may be captured, and unnecessary images are stored in the memory area. In Patent Document 1, it is not possible to reduce the number of captured images or to skip capturing unnecessary images. As described above, in order to check for minute changes in architectural structures, etc., it is necessary to set the resolution and continue shooting, which creates a problem of the amount of data for the captured images becoming large and putting pressure on the capacity of the memory area.

[0005] Therefore, an object of the present disclosure is to acquire appropriate images when performing inspection work. [Means for solving the problem]

[0006] The present disclosure provides an information processing device for acquiring inspection images for performing inspection work on a subject, the information processing device having an imaging control means for controlling an imaging means to change the shooting direction with respect to the subject to acquire a preview image for each section of the subject, and an image analysis means for analyzing the preview image for each section of the subject and determining whether a proportion of a specified area in the area of ​​the section of the subject satisfies the conditions permitted for inspection work of the subject, and the imaging control means controls whether or not to cause the imaging means to capture the inspection image of the section of the subject depending on the result of the determination by the image analysis means. Effect of the Invention

[0007] According to the present disclosure, appropriate images can be acquired when performing inspection work. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of the overall configuration of an imaging system. [Diagram 2] FIG. 2 is a diagram illustrating an example of a functional configuration of the imaging system. [Diagram 3] FIG. 2 illustrates an example of a hardware configuration of each device. [Figure 4] FIG. 1 is a diagram illustrating the photographing of an inspection target by the photographing system. [Diagram 5] FIG. 2 is a diagram illustrating an image captured by the imaging system. [Figure 6] 4 is a flowchart showing the overall operation of the imaging system. [Figure 7] 5 is a flowchart showing a shooting process according to the first embodiment. [Figure 8] FIG. 1 is a diagram showing each shooting direction. [Figure 9] 11A to 11C are diagrams showing results of the shooting process in each shooting direction. [Figure 10] 10 is a flowchart showing a shooting process according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, preferred embodiments of the present disclosure will be described with reference to the accompanying drawings. Note that the configurations shown in the following embodiments are merely examples, and the present disclosure is not limited to these configurations.

[0010] <Embodiment 1> FIG. 1 shows the overall configuration of an imaging system according to this embodiment. 1 includes an imaging device 101, a camera platform device 102, a computing device 103, and a tripod 104. The imaging system of this embodiment is a system for capturing high-definition images for checking minute changes in the wall surfaces of architectural structures, for example, for the purpose of inspecting infrastructure structures such as architectural structures such as buildings and dams.

[0011] The imaging device 101 is mounted on a pan head device 102. The pan head device 102 is placed on a tripod 104. The pan head device 102 is an electric pan head device having a pan drive unit 105 that can be rotated in a pan direction, and a tilt drive unit 106 that can be rotated in a tilt direction. The pan head device 102 and the imaging device 101 are connected by a communication cable 107. The pan head device 102 and the arithmetic device 103 are connected by a communication cable 108. The imaging device 101 and the arithmetic device 103 are connected by the communication cable 107, the pan head device 102, and the communication cable 108. The arithmetic device 103 performs imaging control of the imaging device 101, image analysis of images captured by the imaging device 101, and operation control of the pan head device 102.

[0012] FIG. 2 shows a schematic functional configuration of the imaging system according to the first embodiment. First, a description will be given of the functional configuration of the imaging device 101. The imaging device 101 has the functions of an imaging unit 202, an imaging control unit 203, an image recording unit 204, a focus driving unit 205, an AF distance measuring unit 206, and a communication unit 207. The imaging unit 202 captures images. The imaging control unit 203 controls the entire imaging device 101. The image recording unit 204 records images captured by the imaging unit 202. The focus driving unit 205 drives the focus of an optical lens device (not shown) attached to the imaging device 101. The optical lens device may be an interchangeable lens unit that is detachable from the imaging device 101, or may be a lens unit fixed to the imaging device 101. The AF distance measuring unit 206 measures the distance from the imaging device 101 to a subject for an autofocus function (hereinafter referred to as AF). The communication unit 207 communicates between the camera platform device 102 and the arithmetic device 103.

[0013] Next, a description will be given of the functional configuration of the pan head device 102. The pan head device 102 has the functions of a pan head control unit 209, a pan drive unit 210, a tilt drive unit 211, and a communication unit 212. The camera-head control unit 209 controls the entire camera-head device 102. The pan drive unit 210 drives the pan drive unit 105 in Fig. 1. The tilt drive unit 211 drives the tilt drive unit 106 in Fig. 1. The communication unit 212 performs communication between the imaging device 101 and the arithmetic device 103.

[0014] Next, a description will be given of the functional configuration of the arithmetic device 103. The arithmetic device 103 has the functions of a communication unit 214, a camera head control unit 215, an image holding unit 216, an imaging control unit 217, and an image analysis unit 218. The communication unit 214 performs communication between the pan head device 102 and the imaging device 101. The pan head control unit 215 controls a drive unit for the pan / tilt angle of the pan head device 102. The image holding unit 216 holds the captured image acquired from the imaging device 101 together with additional information. The imaging control unit 217 controls imaging in the imaging device 101. The image analysis unit 218 analyzes whether or not an obstacle is captured in the image acquired from the imaging device 101. The detailed operation and processing of each functional block in this arithmetic unit 103 will be described later.

[0015] FIG. 3 shows the hardware configuration of each device in the imaging system according to the first embodiment. 3(a) shows the hardware configuration of the imaging device 101. The imaging device 101 is configured to include a CPU 301, a ROM 302, a RAM 303, an imaging module 304, an AF distance measurement module 305, a lens control module 306, an input / output I / F module 309, a communication module 310, and a recording device 311. These modules are connected to each other via a bus 312.

[0016] The CPU 301 is a central processing unit and is responsible for the overall control of the imaging device 101. The CPU 301 executes a control program stored in the ROM 302 or the like to realize each functional configuration of the imaging device 101 and the processing of a flowchart described below. The ROM 302 stores the control program that defines the operation processing procedure of the CPU 301, as well as basic software (OS) that is a system program that controls the imaging device 101, various data, and the like. The RAM 303 functions as a work memory required for executing the control program.

[0017] The imaging module 304 includes an imaging element that receives subject light incident through an optical lens device, converts the received light into an imaging signal, and outputs the image signal. The imaging module 304 also directly or indirectly writes the image data output from the A / D converter to a recording device 311, which will be described later. The AF distance measurement module 305 measures the distance to a subject through the optical lens device of the imaging module 304 and determines a focus control position. The lens control module 306 controls the optical lens device. The focus drive module 307 and the iris drive module 308 drive the focus and the iris (aperture), respectively, based on instructions from the lens control module 306.

[0018] The input / output I / F module 309 is a module that allows the user to perform various settings, such as shooting settings, for the imaging device 101. The communication module 310 is a module that allows the imaging device 101 to transmit and receive various information and control signals between the camera head device 102 and the arithmetic device 103. The recording device 311 is a storage device such as a hard disk fixedly installed in the imaging device 101 or a memory card that is detachable from the imaging device 101, and stores images captured by the imaging module 304.

[0019] 3(b) shows the hardware configuration of the pan head device 102. The pan head device 102 is configured to include a CPU 313, a ROM 314, a RAM 315, an imaging device communication module 316, a pan drive module 317, a tilt drive module 318, an input / output I / F module 319, and an arithmetic device communication module 320. These modules are connected to each other by a bus 321.

[0020] The CPU 313 is responsible for controlling the entire pan / tilt head device 102. The CPU 313, ROM 314, and RAM 315 of the pan / tilt head device 102 can be realized with the same configuration as the CPU 301, ROM 302, and RAM 303 of the imaging device 101, and therefore a description thereof will be omitted.

[0021] 1 of the pan head device 102. The tilt drive module 318 drives the tilt drive unit 106 of the pan head device 102 of FIG. 1. The input / output I / F module 319 is a module that allows a user to perform various settings for the pan head device 102. The imaging device communication module 316 is a module that allows the pan head device 102 to transmit and receive various information and control signals to and from the imaging device 101. The arithmetic device communication module 320 is a module that allows the pan head device 102 to transmit and receive various information and control signals to and from the arithmetic device 103.

[0022] 3(c) shows the hardware configuration of the arithmetic device 103. The arithmetic device 103 is configured to include a CPU 322, a ROM 323, a RAM 324, a display device 325, an input device 326, a communication module 327, and a HDD 328. These modules are connected to each other via a bus 329.

[0023] The CPU 322 is a central processing unit and is responsible for the overall control of the arithmetic device 103. The CPU 322 executes a control program stored in the ROM 323 or the like to realize each functional configuration of the arithmetic device 103 and the processing of a flowchart described below. The ROM 323 stores the control program that defines the operation processing procedure of the CPU 322, as well as basic software (OS) which is a system program that controls the equipment of the arithmetic device 103, various data, etc. The RAM 324 functions as a work memory required for executing the control program.

[0024] The display device 325 displays various types of information in the arithmetic device 103 to present to the user. The input device 326 is a device such as a mouse, keyboard, or touch panel that allows the user to input various types of information to the arithmetic device 103. The communication module 327 is a module that allows the arithmetic device 103 to transmit and receive various types of information and control signals between the imaging device 101 and the camera platform device 102. The HDD 328 is a hard disk device that serves as an auxiliary storage device, and stores captured images acquired from the imaging device 101, etc.

[0025] As described above, communication between the imaging device 101, the camera platform device 102, and the computing device 103 passes through the communication units and communication modules described above, but from here on, in order to simplify the explanation, the description of passing through each of these communication units, etc. will be omitted. Similarly, communication between the imaging device 101 and the computing device 103 passes through the camera platform device 102, but the description of this will also be omitted.

[0026] Next, photographing an inspection target by the photographing system will be described with reference to Fig. 4. Hereinafter, taking a wall surface 403 of a building 402 as an example of an inspection target, the operation of photographing the wall surface 403 by the photographing system of this embodiment will be described. In the photographing system of this embodiment, the imaging device 101 is mounted on a pan head device 102, which sequentially repeats an operation of panning at a certain angle and then stopping the panning, and the imaging device 101 photographs each time the panning of the pan head device 102 stops. As a result, photographing by the imaging device 101 is performed for each of the sections into which a wall surface 403 of a building 402 to be inspected is divided.

[0027] FIG. 4(a) is a simplified conceptual diagram showing the state in which the imaging device 101 is imaging the first section of a wall surface 403 of a building 402 in the imaging system of this embodiment, from a viewpoint directly above the building 402. In FIG. 4(a), an installation position 401 indicates a position where the tripod 104 is installed when imaging the wall surface 403 of the building 402 to be inspected. Note that the tripod 104 is mounted with the pan head device 102, and the imaging device 101 is mounted on the pan head device 102, so the installation position 401 in FIG. 4(a) actually indicates the position where the imaging device 101 is arranged. At this time, it is desirable that the imaging device 101 is arranged on a perpendicular line to the center in the width direction of the wall surface 403 to be inspected. However, in an actual imaging site, other buildings or the like become obstacles, and there are cases where the imaging device 101 cannot be arranged on the perpendicular line to the center in the width direction of the wall surface 403. For this reason, the installation position 401 shown in FIG. 4(a) is a position slightly shifted from the center in the width direction of the wall surface 403.

[0028] Moreover, the shooting direction 404 (optical axis direction of the optical lens device) of the imaging device 101 when shooting the first section of the wall surface 403 to be inspected is set to a direction in which the left end in the width direction of the wall surface 403 to be inspected falls within the angle of view 419, as shown in FIG. 4(a). When shooting the first section, the imaging device 101 performs focusing by AF on the wall surface 403 at the position of the intersection 410 where the shooting direction 404 and the wall surface 403 intersect. Here, when focusing is performed at the intersection 410, the focal plane 416 on which the point image can be focused on the smallest circle (circle of confusion) on the imaging surface of the imaging element of the imaging device 101 becomes a plane. Moreover, even if the distance from the imaging surface to the subject is out of the distance from the imaging surface to the plane of the focal plane 416 and the size of the circle on which the point image is focused is not the smallest, if the size of the circle is within the pixel size of the imaging element. The distance range on the subject side when the focus shift cannot be detected is called the depth of field. Note that the depth of field can be calculated based on the pixel size of the image sensor, the focal length and aperture value of the optical lens device, and the distance to the subject.

[0029] In the example of Fig. 4(a), the front depth of field on the side closer to the imaging device 101 with respect to the in-focus plane 416 can be represented by a plane 417. Also, the rear depth of field on the side farther from the imaging device 101 can be represented by a plane 418. Therefore, the wall surface 403 within the distance range from the plane 417 to the plane 418 is considered to be in focus. Therefore, in the imaging system of this embodiment, when imaging the wall surface 403 shown in Fig. 4(a), the installation position 401 of the imaging device 101 and the focal length and aperture value of the optical lens device are set in advance taking into account the depth of field.

[0030] FIG. 4B is a simplified conceptual diagram showing, from a viewpoint directly above a building 402, how the camera platform device 102 sequentially repeats panning and stopping at a fixed angle, and the imaging device 101 captures images each time the panning of the camera platform device 102 stops. Shooting directions 405, 406, 407, 408, and 409 in FIG. 4B indicate the center direction (optical axis direction) of the direction in which the imaging device 101 captures images, similar to the shooting direction 404 in FIG. 4A. Shooting direction 405 indicates the shooting direction of the imaging device 101 when the camera platform device 102 stops after being panned at a fixed angle after capturing images in the shooting direction 404. Shooting directions 406, 407, 408, and 409 also indicate the shooting directions when the camera platform device 102 stops after being panned at a fixed angle. 4B indicate the angles of view when the imaging device 101 is in each of the imaging directions 405, 406, 407, 408, and 409, similar to the angle of view 419 when the imaging direction is 404 in FIG.

[0031] In addition, if the wall surface 403 continues further to the right than in the example shown in Fig. 4(b), and the angle between the shooting direction and the wall surface when shooting the wall surface 403 becomes even more acute, a high-definition captured image suitable for inspection cannot be obtained. In such a case, the installation position of the imaging device 101 is moved to the right, and in the new installation position after the movement, the operation of panning at a certain angle and stopping is repeated in sequence as described above to capture images. In addition, the installation position of the imaging device 101 may be moved multiple times as necessary.

[0032] For the sake of simplicity, the following description will be given of an example in which a wall surface 403 to be inspected is divided into six sections as shown in Fig. 4(b) and photographed for each section by only driving in the pan direction, without driving in the height direction of the building 402, i.e., the tilt direction by the camera platform device 102. Intersections 411, 412, 413, 414, and 415 in Fig. 4(b) are intersections between each shooting direction 405, 406, 407, 408, and 409 and the wall surface 403, similar to the case of intersection 410 in Fig. 4(a). Therefore, when shooting each image, the imaging device 101 performs focusing by AF on the wall surface at the positions of intersections 411, 412, 413, 414, and 415, respectively.

[0033] In FIG. 4(b), similarly to FIG. 4(a), the focal plane when AF is performed at the intersections 411 to 415, the plane representing the front depth of field, and the plane representing the rear depth of field are indicated by solid lines, dotted lines, and dashed lines, respectively. It is known that the range of the rear depth of field is wider than the range of the front depth of field. It is also known that the range of the depth of field becomes wider as the distance to the subject becomes longer, as shown in FIG. 4(b).

[0034] Furthermore, when photographing the wall surface 403 shown in Fig. 4(b), as shown by the respective angles of view 419 to 424, the photographing ranges for the wall surface 403 are made to overlap in large portions. This is to reliably cover the wall surface 403 to be inspected, and to obtain a high-quality inspection image with less aberration by utilizing the central portion of the image circle of the optical lens device used. Note that in Fig. 4(b), the photographing ranges are made to overlap by aligning the edge of the angle of view when photographing the previous frame with the center of the angle of view when photographing the next frame, but the overlapping ratio may be different from this example.

[0035] FIG. 4C shows an example of a finder screen 425 of the imaging device 101. When a high-definition image of an inspection target is captured as in this embodiment, a distance measurement point mark 426 indicating a distance measurement point used for AF during the capture is displayed on the finder screen 425. In the imaging device 101, distance measurement points for AF are arranged over a wide range corresponding to substantially the entire surface of the finder screen 425, but when a high-definition image of an inspection target is captured, a more accurate distance measurement point in the center is used. Therefore, when a high-definition image of an inspection target is captured, only the distance measurement point mark 426 indicating each high-precision distance measurement point in the center is displayed on the finder screen 425. Since the above is an explanation of the procedure for capturing an image of a wall surface 403 of an inspection target by the imaging device 101 mounted on the pan head device 102, it is assumed that focusing by AF at each intersection 410 to 415 in FIG. 4(b) is successful.

[0036] Figures 5(a) and 5(b) are figures explaining the captured images obtained by photographing the wall surface 403 to be inspected in multiple sections in the photographing system of this embodiment as described above, and the processing of each captured image. Here, each of the images 501 to 506 in FIG. 5(a) and FIG. 5(b) shows an image of each section acquired by panning the camera platform device 102 as described above and moving the shooting direction of the imaging device 101 from the upper left end of the wall surface 403 to the right in sequence. In FIG. 5(b), the wall surface 403 continues further to the right than in the example shown in FIG. 4(b). The image 507 in FIG. 5(b) is acquired by shifting the installation position of the imaging device 101 to the right from the installation position shown in FIG. 4(b) and moving the shooting direction of the imaging device 101 to the upper right end of the wall surface 403 to be inspected. Note that in the example of FIG. 5(a), the images 501 to 506 are arranged with a slight vertical shift to make them easy to distinguish from one another, but in reality there is no vertical shift. After acquiring image 507 of the upper rightmost section of wall surface 403 to be inspected, panning is performed in camera head device 102 to move the shooting direction of imaging device 101 down in the tilt direction by an amount equivalent to one section (including the overlapping area) and to move it to the left end of the wall surface. Thereafter, in the shooting system, panning and shooting are performed sequentially in the same manner as described above, so that shooting is performed from the leftmost section of the wall surface to the rightmost section.

[0037] By performing such pan driving, tilt driving, and photographing, the imaging device 101 acquires each image 501 to 509 covering all sections of the wall surface 403 to be inspected. FIG. 5(b) shows a collection of images photographed by performing a series of pan driving and tilt driving of the camera platform device 102 as described above. Then, all of these captured images are integrated into one image by accurately aligning the positions of the respective overlapping areas, and an image is generated as if the wall surface 403 to be inspected was photographed in high resolution. In the inspection of infrastructure structures such as bridges and tunnels, an inspector uses the integrated image to check the presence or absence of minute abnormalities in the inspection object. Note that a detailed description of the contents of the inspection is omitted. In the following, the inspection work using such images of infrastructure structures is called infrastructure inspection.

[0038] Fig. 6 is a flowchart showing the flow of the operation of the entire system in the imaging system of embodiment 1. Before the operation shown in the flowchart of Fig. 6 is started, the installation position 401 of the imaging device 101 and the selection of the aperture value and focal length based on the calculation of the depth of field in the optical lens device to be used, as explained in Fig. 4(a) and Fig. 4(b), are assumed to have been completed. In the following explanation, each process (step) is represented by adding an S to the beginning, and the notation of the process (step) is omitted.

[0039] In S601, the camera platform device 102 sets the shooting range and the overlap rate in the overlapping area for the wall surface or the like to be inspected, based on an instruction input by the user via the input / output I / F module 319. The wall surface to be inspected is an example of a subject to be inspected. In S602, the camera platform device 102 performs settings related to pan driving and tilt driving to enable shooting of the wall surface, etc., to be inspected, for each of the above-mentioned sections, based on the information set in S601. The calculation device 103 calculates the total number of times shooting has been performed, based on the settings related to pan driving and tilt driving of the camera platform device 102. The total number of times shooting has been performed corresponds to the total number of sections on the wall surface, etc., to be inspected.

[0040] In S603, the imaging device 101 performs appropriate settings for imaging, such as aperture value and shutter speed corresponding thereto, for imaging each section preselected in S601. Thereafter, the imaging device 101 waits for an instruction to start imaging from the user via the input / output I / F module 309. In S604, when the imaging device 101 receives an input of an instruction to start imaging from the user, the imaging device 101 notifies the camera platform device 102 and the arithmetic device 103 of the instruction to start imaging.

[0041] In S605, when the arithmetic device 103 receives an instruction to start shooting from the imaging device 101, it controls the camera platform device 102 to perform pan driving and tilt driving for shooting each section by the imaging device 101 as described above, and sequentially moves the shooting direction of the imaging device 101. Then, every time the camera platform device 102 completes pan driving or the like at a certain angle and stops moving (changing) the shooting direction, it sends an instruction to the arithmetic device 103 to execute shooting processing.

[0042] In S606, when the arithmetic device 103 receives an instruction to execute the photographing process from the camera-to-camera device 102, the arithmetic device 103 executes the photographing process by controlling the imaging device 101. The photographing process in S606 will be described in detail later. In S607, when the photographing process is completed, the arithmetic device 103 increments the count value of the number of photographs taken and records the incremented count value in the ROM 323, etc. The count value of the number of photographs taken indicates the photographing order of the sections of the wall surface, etc., to be inspected. The arithmetic device 103 adds the current count value stored in the ROM 323, etc. to the captured image acquired in the photographing process of S606 this time, and saves it in the HDD 328, etc.

[0043] Next, in S608, the arithmetic device 103 judges whether or not the photographing of all the inspection ranges for the inspection target (i.e., all the sections for the wall surface, etc., of the inspection target) has been completed. For example, the arithmetic device 103 judges whether or not the photographing has been completed by comparing the total number of photographing times with the current count value stored in the ROM 323, etc. If the arithmetic device 103 judges that the photographing has not been completed, it returns the process to S605. As a result, the camera platform device 102 performs pan driving or the like to enable the next section, for which an image for inspection has not yet been photographed, to be photographed. On the other hand, if it is judged that the photographing of all the inspection ranges (all sections) has been completed, the process of FIG. 6 in the imaging system ends.

[0044] Fig. 7 is a flowchart showing the photographing process executed by the arithmetic device 103 of the imaging system, and shows the detailed process flow of S606 in Fig. 6. This flowchart is executed mainly by the arithmetic device 103, more specifically, by the CPU 322 of the arithmetic device 103. The arithmetic device 103 is an example of an information processing device. In S701, the arithmetic device 103 sends an instruction to the imaging device 101 to capture a preview image, and the imaging device 101 captures a preview image in a section (hereinafter referred to as a target section) according to the current imaging direction. The target section here is a section for which a preview image has not yet been acquired. The preview image is an image used only for image analysis in S702 to S704 described below, and is different from an image for inspection. In other words, it does not need to be a high-definition image. Therefore, the imaging device 101 captures the preview image by setting a lower resolution than when capturing an image for inspection. The arithmetic device 103 acquires the preview image from the imaging device 101.

[0045] In S702, the arithmetic device 103 performs image analysis to check whether the image area of ​​the preview image acquired in S701 includes an area of ​​an object other than the inspection target or a background. The image analysis here is performed by object detection processing, area detection processing, etc. Objects other than the inspection target here refer to rocks, vegetation, signs, people, etc. that obstruct the inspection surface of an infrastructure structure. Hereinafter, objects other than the inspection target and the background will be collectively referred to as obstacles. Objects other than the inspection target are examples of objects other than the subject.

[0046] In S703, the arithmetic device 103 determines whether or not an obstacle is detected as a result of the image analysis in S702. If the arithmetic device 103 determines that an obstacle is detected, the process proceeds to S704, and if the arithmetic device 103 determines that an obstacle is not detected, the process proceeds to S706. In S704, the calculation device 103 performs image analysis on the preview image acquired in S701, and calculates the ratio of the area of ​​the obstacle to the area of ​​the target section.

[0047] In S705, the arithmetic device 103 judges whether the ratio of the obstacle area calculated in S704 is equal to or less than a threshold value. If the arithmetic device 103 judges that the ratio is equal to or less than a predetermined threshold value, the arithmetic device 103 advances the process to S706. If the arithmetic device 103 judges that the ratio exceeds the threshold value, the arithmetic device 103 ends the process of FIG. 7. This threshold value is a value set according to an allowable range effective for infrastructure inspection, and is set to 80% here. The threshold value may be changed according to the overlapping rate in the overlapping area set in advance in the camera platform device 102. An example of an allowable condition for inspection work is that the ratio of the obstacle area to the area of ​​the target section is equal to or less than 80%. If inspection images are taken for all sections specified in advance for inspection work, there are cases where an object such as a rock overlaps the wall surface of the inspection target, or the background is reflected, resulting in an image that is not suitable for inspection work. Therefore, in this step, if the arithmetic device 103 judges that the ratio of the obstacle area to the area of ​​the target section exceeds 80%, the arithmetic device 103 controls not to take inspection images for the target section.

[0048] In S706, the arithmetic device 103 sends an instruction to the imaging device 101 to capture an image for inspection, and the imaging device 101 captures an image of the target section with a high resolution setting. The arithmetic device 103 acquires the captured image from the imaging device 101. The arithmetic device 103 stores the acquired captured image in the HDD 328 or the like. The processing in Fig. 7 then ends. That is, when the arithmetic device 103 determines that the ratio of the area of ​​the obstacle to the area of ​​the target section is 80% or less, it controls to capture an image for inspection of the target section.

[0049] FIG. 8 is a diagram showing the imaging directions when the wall surface to be inspected is imaged section by section in sequence in the imaging system of this embodiment. In Fig. 8, position 801 indicates installation position 401 of pan head device 102 on which imaging device 101 described above is mounted, and wall surface 802 indicates wall surface 403 of building 402, which is the inspection target described above. Directions 803, 804, ..., 817 each indicate the shooting direction when pan driving of pan head device 102 is stopped after pan driving at a certain angle and the section is photographed. Note that convex portion 818 is an obstacle portion overlapping wall surface 802, which is the inspection target. Note that the pan driving direction is from left to right in the figure. Also, dotted line 819 on wall surface 802 in Fig. 8 indicates the position of the surface to be focused on as wall surface 802, which is the inspection target.

[0050] FIG. 9 is a table showing the presence or absence of obstacles, the ratio of obstacles, the effectiveness of inspection, and whether an image for inspection has been captured in each section when image capturing processing is performed in order for each of the 15 image capturing directions 803 to 817 in FIG. 8. In FIG. 9, "image capturing direction number" is a list of each number from the image capturing directions 803 to 817 in FIG. 8. "Presence or absence of obstacles" indicates whether an obstacle has been detected by image analysis in each image capturing direction. "Ratio of obstacles" indicates the ratio of the obstacle to the section when an obstacle is detected, expressed as a percentage. "Effective for inspection" indicates whether the ratio of obstacles is within a permissible range that is effective for infrastructure inspection. "Image capturing performed" indicates whether an image for inspection has been captured by the imaging device 101.

[0051] Hereinafter, with reference to FIGS. 8 and 9, the process flow of the flowcharts in FIGS. 6 and 7 in the imaging system, and the operations of each device in the imaging system and the user will be explained in a simplified manner that is easier to understand, based on the actual chronological operations.

[0052] When imaging of the wall surface 802 shown in Fig. 8 as an inspection target is planned and the flow chart of Fig. 6 is started in the imaging system of this embodiment, in S601, a user inputs settings to the camera-mounted device 102. Also, in S602, preparations are made for panning the camera-mounted device 102 from imaging direction 803 to imaging direction 817. Then, in S604, a user issues an instruction to start imaging. Then, in S605, the camera-mounted device 102 performs panning to the initial imaging direction 803. Furthermore, in S606, imaging processing is started in the imaging device 101 and the computing device 103.

[0053] In the photographing process of S606 in FIG. 6, as shown in the flowchart of FIG. 7, first, in S701, the arithmetic device 103 acquires a preview image of the current photographing direction from the imaging device 101. In the next S702, the arithmetic device 103 performs an object detection process for detecting objects such as rocks other than the wall surface 403 to be inspected, and an area detection process for detecting an area of ​​the background other than the wall surface 403 to be inspected, thereby performing image analysis of the preview image. In the next S703, the arithmetic device 103 determines whether an obstacle has been detected based on the image analysis result. Here, in the image analysis in the photographing direction 803, it is determined that no obstacle has been detected, and the process proceeds to S706, where an image for inspection is photographed.

[0054] Thereafter, as shown in Fig. 9, the loop process from S605 to S608 in Fig. 6 is repeated from the image capturing direction 804 to the image capturing direction 808 in sequence, and images for inspection are captured in sequence. Next, in the image capturing process for the image capturing direction 809, the arithmetic device 103 determines in S703 that an obstacle has been detected by the image analysis in S702 in Fig. 7, and proceeds to the process of S704. The obstacle ratio "30" is calculated by the process of S704. Here, since the threshold value of the obstacle ratio set according to the allowable range effective for infrastructure inspection is "80", the arithmetic device 103 determines that the allowable condition for the inspection work is satisfied, proceeds to the process of S706, and controls the imaging device 101 to capture images for inspection.

[0055] Next, in the photographing process in the photographing direction 810, the arithmetic device 103 determines in S703 that an obstacle has been detected by the image analysis in S702 in Fig. 7, and proceeds to the process in S704. The obstacle ratio "85" is calculated by the process in S704. Here, since the threshold value of the obstacle ratio set according to the allowable range effective for infrastructure inspection is "80", the arithmetic device 103 determines that the allowable condition for the inspection work is not satisfied, does not photograph an image for inspection, and returns the process to S608 in Fig. 6.

[0056] Thereafter, as shown in FIG. 9, the loop process from S605 to S608 in FIG. 6 is repeated in sequence from the imaging direction 811 to the imaging direction 817, and imaging for inspection of the wall surface image is completed.

[0057] According to the present embodiment as described above, when photographing each section while changing the photographing direction with respect to the inspection object, it is possible to comprehensively photograph the inspection object while skipping photographing unnecessary sections, thereby obtaining only images suitable for the inspection work. This makes it possible to reduce the amount of image data. In other words, it is possible to prevent unnecessary data from taking up space in the recording medium, and to prevent unnecessary transfer time and analysis processing time from occurring when transferring images to an analysis server. In addition, by recording the photographing order of the sections and adding this photographing order to the photographed images for inspection, the images of each section can be synthesized without misalignment during stitching work after photographing is completed.

[0058] As a modified example of this embodiment, in S705, the arithmetic device 103 may be configured to inquire of the user whether or not to take images for inspection of the target section when it is determined that the conditions for permitting the inspection work are not satisfied. In this case, the arithmetic device 103 may display a UI on the display device 325 for selecting whether or not to take images for inspection of the target section, and may determine whether or not to take images for inspection of the target section based on a user input from the input device 326.

[0059] In addition, the imaging system of this embodiment is configured as a system composed of multiple devices, but any one of the devices may be configured to include the other devices, or may be configured as a single imaging device.

[0060] <Embodiment 2> The photography system of the first embodiment is configured to perform image analysis of a preview image of each section, and to determine whether to photograph an image for inspection of each section depending on whether the image analysis results satisfy the conditions permitted by the inspection work. The photography system of the present embodiment is configured to perform image analysis of the image after photographing an image for inspection of each section, and to determine whether to delete the image for inspection of each section depending on whether the image analysis results satisfy the conditions permitted by the inspection work. Below, the description of the same parts as in the first embodiment will be omitted, and the description will focus on the parts that differ from the first embodiment.

[0061] FIG. 10 is a flowchart showing the photographing process executed by the arithmetic unit 103 of the imaging system, and shows the detailed process flow of S606 in FIG. In S1001, the arithmetic device 103 sends an instruction to the imaging device 101 to capture an image for inspection, and the imaging device 101 captures an image of a section (hereinafter, referred to as a target section) according to the current imaging direction with a high resolution setting. The arithmetic device 103 acquires a captured image from the imaging device 101. This captured image is an image for inspection. The arithmetic device 103 stores the acquired captured image in the HDD 328 or the like.

[0062] In S1002, the arithmetic unit 103 performs image analysis to check whether an obstacle is included in the image area of ​​the captured image acquired in S1001. The image analysis here is performed by object detection processing, area detection processing, etc. In S1003, the calculation device 103 determines whether or not an obstacle is detected as a result of the image analysis in S1002. If the calculation device 103 determines that an obstacle is detected, the process proceeds to S1004, and if the calculation device 103 determines that an obstacle is not detected, the process in FIG. 10 ends. In S1004, the calculation device 103 performs image analysis on the captured image acquired in S1001, and calculates the ratio of the area of ​​the obstacle to the target section.

[0063] In S1005, the arithmetic device 103 judges whether the ratio of the obstacle area calculated in S1004 is equal to or less than a threshold. If the arithmetic device 103 judges that the ratio is equal to or less than a predetermined threshold, it ends the process of Fig. 10, and if it judges that the ratio is greater than the threshold, it proceeds to S1006. This threshold is a value set according to an allowable range effective for infrastructure inspection, and is set to 80% here. In S1006, the arithmetic device 103 deletes the captured image acquired in S1001 from the HDD 328 etc. Then, the processing in Fig. 10 ends. That is, when the arithmetic device 103 determines that the ratio of the obstacle area to the target section exceeds 80%, it controls to delete the inspection image in the target section.

[0064] According to the present embodiment described above, when photographing each section while changing the photographing direction with respect to the inspection target, it is possible to comprehensively photograph the inspection target while deleting images of unnecessary sections, thereby saving only images suitable for the inspection work, thereby reducing the amount of image data.

[0065] In this embodiment, image analysis is performed each time an image is captured for each section, and it is determined whether the image is unnecessary, and if so, the image is deleted. However, as a modified example, it may be determined whether an image is unnecessary after all images of the sections to be inspected have been captured. In this case, the calculation device 103 may execute the processes of S1002 to S1006 for the images of all sections, and delete the images.

[0066] <Other embodiments> Although the present disclosure has been described above with reference to the embodiments, the above embodiments are merely illustrative of the specific examples of the present disclosure, and the technical scope of the present disclosure should not be interpreted as being limited by these. In other words, the present disclosure can be implemented in various forms without departing from its technical concept or main features.

[0067] The present disclosure can also be realized by a process in which a program for implementing one or more functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more functions.

[0068] The disclosure of each of the above-described embodiments includes the following configurations, methods, and programs. (Configuration 1) An information processing device for acquiring an image for inspection to perform an inspection work on a subject, an imaging control means for controlling the imaging means to change an imaging direction with respect to the subject and obtain a preview image for each section of the subject; an image analysis means for analyzing the preview image for each section of the subject and determining whether a ratio of a predetermined area to the area of ​​the section of the subject satisfies an acceptable condition for inspection work of the subject; having The information processing device is characterized in that the imaging control means controls whether or not to cause the imaging means to capture the inspection image of the subject section depending on the result of the judgment by the image analysis means. (Configuration 2) The information processing device described in configuration 1, characterized in that the imaging control means controls the imaging of inspection images for a section determined by the image analysis means to satisfy the condition, and controls the imaging of inspection images for a section determined not to satisfy the condition. (Configuration 3) 3. The information processing device according to configuration 1 or 2, wherein the predetermined area is an area of ​​an object other than the subject or a background area. (Configuration 4) the image analysis means executes a detection process for detecting an object or background other than the subject on the preview image of the target section, and when an object or background other than the subject is detected by the detection process, determines whether or not the proportion of the area of ​​the object or background other than the subject in the area of ​​the target section satisfies an allowable condition for inspection work of the subject; The information processing device according to any one of configurations 1 to 3, characterized in that the imaging control means controls the imaging of an inspection image of the target area when no object or background other than the subject is detected by the detection process. (Configuration 5) The information processing device according to any one of configurations 1 to 4, wherein the image analysis means determines whether a ratio of an area of ​​objects other than the subject or a background to an area of ​​the subject's section is equal to or less than a threshold representing an acceptable range for inspection work of the subject. (Configuration 6) The information processing device described in any one of configurations 1 to 5, characterized in that the imaging control means changes the shooting direction in sequence to cover the subject, targets each section of the subject in sequence, and if the image analysis means determines that the condition is not satisfied for the target section, does not cause the imaging means to capture the inspection image for the target section, but targets the next section. (Configuration 7) 7. The information processing device according to any one of configurations 1 to 6, wherein the preview image has a lower resolution than the image for inspection. (Configuration 8) 8. The information processing device according to any one of configurations 1 to 7, wherein the subject is a wall surface of an infrastructure structure, and the inspection work is a work of checking for abnormalities in the wall surface of the infrastructure structure. (Configuration 9) An information processing device for acquiring an image for inspection to perform an inspection work on a subject, an imaging control means for controlling the imaging means to capture the inspection images for each section of the subject by changing an imaging direction with respect to the subject; an image analysis means for analyzing the inspection image for each section of the subject and determining whether a ratio of a predetermined area to the area of ​​the section of the subject satisfies an allowable condition for the inspection work of the subject; a deletion means for deleting the inspection images in the sections determined by the image analysis means to not satisfy the conditions; 13. An information processing device comprising: (Configuration 10) An imaging system for acquiring an image for inspection to perform an inspection work on a subject, An imaging device having an imaging means, an information processing device that controls the imaging device, and a camera platform device having a means for changing the imaging direction of the imaging means, The information processing device includes: an imaging control means for controlling the imaging means to change an imaging direction with respect to the subject and obtain a preview image for each section of the subject; an image analysis means for analyzing the preview image for each section of the subject and determining whether a ratio of a predetermined area to the area of ​​the section of the subject satisfies an allowable condition for an inspection work of the subject; having The imaging system is characterized in that the imaging control means controls whether or not to cause the imaging means to capture the inspection image of the subject area, depending on the result of the judgment by the image analysis means. (Configuration 11) 11. The imaging system according to configuration 10, wherein the means for changing the imaging direction of the imaging means performs pan driving and tilt driving based on a predetermined setting. (method) An imaging control method for acquiring an inspection image for inspecting a subject, comprising: an imaging control step of controlling an imaging means to change an imaging direction with respect to the subject and acquire a preview image for each section of the subject; an image analysis step of analyzing the preview image for each section of the subject and determining whether a ratio of a predetermined area to the area of ​​the section of the subject satisfies an acceptable condition for an inspection work of the subject; Including, The imaging control method is characterized in that the imaging control step controls whether or not to cause the imaging means to capture the inspection image of the subject area, depending on the result of the judgment in the image analysis step. (program) A computer of an information processing device for acquiring an image for inspection to perform an inspection work on a subject, an imaging control means for controlling the imaging means to change an imaging direction with respect to the subject and obtain a preview image for each section of the subject; an image analysis means for analyzing the preview image for each section of the subject and determining whether a ratio of a predetermined area to the area of ​​the section of the subject satisfies an allowable condition for an inspection work of the subject; A program that functions as The imaging control means controls whether or not to capture the inspection image of the subject section depending on the result of the judgment by the image analysis means. [Explanation of symbols]

[0069] 101: imaging device, 102: camera head device, 103: computing device

Claims

1. An information processing device for acquiring an image for inspection to perform an inspection work on a subject, an imaging control means for controlling the imaging means to change an imaging direction with respect to the subject and obtain a preview image for each section of the subject; an image analysis means for analyzing the preview image for each section of the subject and determining whether a ratio of a predetermined area to the area of ​​the section of the subject satisfies an acceptable condition for inspection work of the subject; having The information processing device is characterized in that the imaging control means controls whether or not to cause the imaging means to capture the inspection image of the subject section depending on the result of the judgment by the image analysis means.

2. The information processing device described in claim 1, characterized in that the imaging control means controls the imaging of inspection images for a section determined by the image analysis means to satisfy the condition, and controls the imaging of inspection images for a section determined not to satisfy the condition.

3. 2. The information processing apparatus according to claim 1, wherein the predetermined area is an area of ​​an object other than the subject or a background area.

4. the image analysis means executes a detection process for detecting an object or background other than the subject on the preview image of the target section, and when an object or background other than the subject is detected by the detection process, determines whether or not the proportion of the area of ​​the object or background other than the subject in the area of ​​the target section satisfies an allowable condition for inspection work of the subject; The information processing device according to claim 1, characterized in that the imaging control means controls the imaging of an inspection image of the target area when no object or background other than the subject is detected by the detection process.

5. The information processing device according to claim 1, characterized in that the image analysis means determines whether the proportion of the area of ​​the subject's section occupied by objects other than the subject or the background is equal to or less than a threshold value representing an acceptable range for inspection work of the subject.

6. The information processing device described in claim 1, characterized in that the imaging control means changes the shooting direction in sequence to cover the subject, targeting each section of the subject in sequence, and if the image analysis means determines that the condition is not satisfied for the target section, does not cause the imaging means to capture the inspection image for the target section, and targets the next section.

7. The information processing apparatus according to claim 1 , wherein the preview image has a lower resolution than the inspection image.

8. 2. The information processing apparatus according to claim 1, wherein the subject is a wall surface of an infrastructure structure, and the inspection work is a work of checking for abnormalities in the wall surface of the infrastructure structure.

9. An information processing device for acquiring an image for inspection to perform an inspection work on a subject, an imaging control means for controlling the imaging means to capture the inspection images for each section of the subject by changing an imaging direction with respect to the subject; an image analysis means for analyzing the inspection image for each section of the subject and determining whether a ratio of a predetermined area to the area of ​​the section of the subject satisfies an acceptable condition for inspection work of the subject; a deletion means for deleting the inspection images in the sections determined by the image analysis means to not satisfy the conditions; 13. An information processing device comprising:

10. An imaging system for acquiring an image for inspection to perform an inspection work on a subject, An imaging device having an imaging means, an information processing device that controls the imaging device, and a camera platform device having a means for changing the imaging direction of the imaging means, The information processing device includes: an imaging control means for controlling the imaging means to change an imaging direction with respect to the subject and obtain a preview image for each section of the subject; an image analysis means for analyzing the preview image for each section of the subject and determining whether a ratio of a predetermined area to the area of ​​the section of the subject satisfies an acceptable condition for inspection work of the subject; having The imaging system is characterized in that the imaging control means controls whether or not to cause the imaging means to capture the inspection image of the subject area, depending on the result of the judgment by the image analysis means.

11. 11. The imaging system according to claim 10, wherein the means for changing the imaging direction of the imaging means performs pan driving and tilt driving based on a predetermined setting.

12. An imaging control method for acquiring an inspection image for inspecting a subject, comprising: an imaging control step of controlling an imaging means to change an imaging direction with respect to the subject and acquire a preview image for each section of the subject; an image analysis step of analyzing the preview image for each section of the subject and determining whether a ratio of a predetermined area to the area of ​​the section of the subject satisfies an acceptable condition for an inspection work of the subject; Including, The imaging control method is characterized in that the imaging control step controls whether or not to cause the imaging means to capture the inspection image of the subject area, depending on the result of the judgment in the image analysis step.

13. A computer of an information processing device for acquiring an image for inspection to perform an inspection work on a subject, an imaging control means for controlling the imaging means to change an imaging direction with respect to the subject and obtain a preview image for each section of the subject; an image analysis means for analyzing the preview image for each section of the subject and determining whether a ratio of a predetermined area to the area of ​​the section of the subject satisfies an allowable condition for an inspection work of the subject; A program that functions as The imaging control means controls whether or not to capture the inspection image of the subject section depending on the result of the judgment by the image analysis means.

Images