Information processing apparatus, information processing system, information processing method, and program

Abstract

To measure the dimensions of an inspection target from a photographed image without using auxiliary information.SOLUTION: A detection unit detects an area representing a predetermined phenomenon occurring in a subject from a first image photographed by a first camera. A shape estimation unit estimates the three-dimensional shape of the subject on the basis of the parallax between the first image and a second image photographed by a second camera. A measurement unit determines the dimensions of the area where the predetermined phenomenon occurs in the three-dimensional shape of the subject. The present embodiment may be any form such as an information processing apparatus, an information processing system, an information processing method, and a program.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] FIELD Embodiments of the present application relate to an information processing device, an information processing system, an information processing method, and a program. The embodiments of the present application relate to, for example, inspecting a structure using an image captured by a camera and creating an inspection record. [Background technology]

[0002] Infrastructure is an abbreviation for infrastructure, and is a general term for facilities and equipment that form the foundation of life, the economy, and industry. Inspection and investigation of infrastructure is essential to maintain its functionality. When inspecting or investigating infrastructure, it is common to record damage information using images taken with a camera. In order to record damage information, it is important to identify the dimensions of the object being inspected. For example, for cracks (i.e., fissures) in concrete, their width and length are important indicators. Information about the dimensions of the object being inspected cannot be immediately obtained from the captured image alone. To do this, dimensional information must be added.

[0003] When determining the dimensions of an object to be inspected, it is common to set the distance between the camera and the object and determine the size of the imaging range through geometric calculations. For example, the camera is held facing a flat surface such as a wall, and an image is captured. The size of the imaging range can be determined from the relationship between the distance between the object to be inspected and the camera and the focal length of the camera lens. The size of the imaging range can be used to obtain the length in three-dimensional space per pixel, which in turn determines the width and length of the detected crack. Another proposed method involves measuring the distance to the object to be inspected using an auxiliary component (e.g., a sensor, a light source, etc.), using the measured distance and an image to measure the distance between the camera and the object to be inspected, and then combining the measured distance and the captured image to determine the size of the object to be inspected as it appears in the image (see Patent Document 1).

[0004] Other methods have been proposed, including capturing a reference of known size and restoring the image size from the size of the reference shown in the captured image. For example, by capturing an object of known size as a reference, the size of the image can be estimated from the size of the reference in the image. If the dimensions of the object are known, multiple images can be integrated to generate a wide-area image, and the size can be measured on the wide-area image. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 3996946 Summary of the Invention [Problem to be solved by the invention]

[0006] When taking images using a camera mounted on a robot such as a drone, it is difficult to control the robot and maintain a constant distance from the subject. It is even more difficult to measure the dimensions of the object while maintaining a fixed distance from the subject. Methods that use auxiliary sensors or create combined images to determine the dimensions of the object based on known dimensions assume that the surface of the structure being measured is flat. Therefore, they are not applicable to curved surfaces. When photographing the object to be inspected, including a reference, it is necessary to directly access the structure, install a reference target, and measure its dimensions in advance. This is inconvenient for remote image inspection. [Means for solving the problem]

[0007] An information processing device according to a first aspect includes a detection unit that detects an area representing a predetermined phenomenon occurring in a subject from a first image captured by a first camera, a shape estimation unit that estimates the three-dimensional shape of the subject based on the parallax between a second image captured by a second camera and the first image, and a measurement unit that determines the dimensions of the area in the three-dimensional shape of the subject in which the predetermined phenomenon appears.

[0008] An information processing method according to a second aspect is a method in an information processing device, in which the information processing device detects an area representing a predetermined phenomenon occurring in a subject from a first image taken by a first camera, estimates the three-dimensional shape of the subject based on the parallax between a second image taken by a second camera and the first image, and determines the dimensions of the area in the three-dimensional shape of the subject in which the predetermined phenomenon appears. [Effects of the Invention]

[0009] According to this embodiment, the dimensions of the inspection object can be measured from the captured image without using auxiliary information. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 2 is a schematic block diagram illustrating an example of the functional configuration of the information processing system according to the present embodiment. [Figure 2] 1 is a schematic block diagram illustrating an example of a functional configuration of an information processing device according to an embodiment of the present invention. [Figure 3] FIG. 1 is a schematic block diagram illustrating an example of a hardware configuration of an information processing device according to an embodiment of the present invention. [Figure 4] FIG. 1 is an explanatory diagram illustrating stereo reconstruction. [Figure 5] FIG. 1 is an explanatory diagram illustrating an example of estimating a three-dimensional shape of a subject. [Figure 6] FIG. 1 is an explanatory diagram illustrating a three-dimensional damaged region. [Figure 7] 10 is a flowchart showing an example of a procedure for measuring a damaged area according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present application will be described with reference to the drawings. Fig. 1 is a schematic block diagram for explaining an example of the functional configuration of an information processing system S1 according to this embodiment. The information processing system S1 includes an information processing device 10, an imaging unit 20, and a display unit 30.

[0012] The information processing device 10 receives a first image and a second image from a first camera 20-1 and a second camera 20-2, respectively. The information processing device 10 detects an area from the first image that represents a predetermined phenomenon that may occur in the object St to be inspected. In the example of FIG. 1, the object St is a structure made of concrete blocks. The predetermined phenomenon is a crack Cr. The information processing device 10 estimates the three-dimensional shape of the object appearing in the first image, i.e., the stereoscopic shape, based on the parallax between the first image and the second image. The information processing device 10 determines the dimensions of the portion of the object where the predetermined phenomenon appears, based on the detected area and the estimated three-dimensional shape.

[0013] The photographing unit 20 includes two cameras: a first camera 20-1 and a second camera 20-2. The first image refers to an image photographed by the first camera 20-1. The second image refers to an image photographed by the second camera 20-2. The photographing unit 20 may be configured as a stereo camera system in which the cameras 20-1 and 20-2 are integrally installed, or the cameras 20-1 and 20-2 may be separate and their individual locations may be variable. The focal length, field of view, and baseline length of each of the first camera 20-1 and the second camera 20-2 are adjusted in advance so that their respective fields of view include a common object St. The baseline length corresponds to the distance between the first camera 20-1 and the second camera 20-2. The focal length, field of view, and baseline length depend on the distance from the photographing unit 20 to the inspection target (sometimes referred to herein as the "photography distance") and the size of the area in which a predetermined phenomenon occurs in the inspection target. The reference point for the shooting distance is, for example, the midpoint between the viewpoints (optical centers) of the first camera 20-1 and the second camera 20-2. The focal length, field of view, and baseline length are set so that the proportion of the area representing the common subject in the field of view of each of the first camera 20-1 and the second camera 20-2 is maximized.

[0014] The display unit 30 displays a display screen based on display data input from the information processing device 10. The display unit 30 may be, for example, a liquid crystal display (LCD) or an organic light emitting diode (OLED) display.

[0015] Next, an example of the functional configuration of the information processing device 10 according to this embodiment will be described. Fig. 2 is a schematic block diagram showing an example of the functional configuration of the information processing device 10 according to this embodiment. The information processing device 10 includes a control unit 110, a storage unit 130, and an input / output unit 140. The control unit 110 performs processing for implementing and controlling various functions of the information processing device 10. An example of the functional configuration of the control unit 110 will be described later.

[0016] The storage unit 130 stores various data referenced by the control unit 110 and various data acquired by the control unit 110. The input / output unit 140 connects the imaging unit 20 and the display unit 30 so that various data can be input and output. The input / output unit 140 may be connected to other devices via a wired or wireless connection to input and output various data. The input / output unit 140 may also be connected to other devices via a network.

[0017] Next, a description will be given of an example functional configuration of the control unit 110. The control unit 110 includes an image acquisition unit 112, a detection unit 114, a reconstruction unit 116, a shape estimation unit 118, a measurement unit 120, an output processing unit 122, and a calibration unit 124.

[0018] The image acquisition unit 112 acquires an image representing the shape of the inspection target, which is the subject, from the photographing unit 20. The inspection target is, for example, mainly a concrete block. The concrete block becomes part of a building, a wall, or other structure. The inspection target may have various types of damage, such as cracks. When an operation signal instructing photography is input from the input unit 158 ​​(described later) (for example, by pressing a button), the image acquisition unit 112 instructs the photographing unit 20 to take a photograph. The first camera 20-1 and the second camera 20-2 of the photographing unit 20 take a first image and a second image, respectively, and output them to the image acquisition unit 112. The image acquisition unit 112 stores the acquired first image and second image in the memory unit 130.

[0019] The detection unit 114 detects an area where damage appears (sometimes referred to herein as a "damaged area") from either the first image or the second image (for example, the first image) stored in the storage unit 130. When detecting the damaged area, the detection unit 114 may use, for example, a known image recognition technique. The detection unit 114 associates the detected damaged area with the first image and the second image and stores them in the storage unit 130.

[0020] The reconstruction unit 116 detects the disparity between the first image and the second image stored in the storage unit 130, and estimates the three-dimensional shape of the subject based on the detected disparity (stereo reconstruction). When detecting the disparity, the reconstruction unit 116 performs, for example, block matching. Block matching is a technique for searching, for each reference block that forms part of one of the first and second images (e.g., the first image), for a corresponding block in the other image that exhibits a pattern that most closely resembles the pattern that appears in the reference block. The reference block and the corresponding block are each areas that include multiple images and are equal in size. The deviation between the position of the reference block in one image and the position of the corresponding block in the other image corresponds to the disparity.

[0021] An example of parallax and shooting distance will now be described with reference to FIG. 4. FIG. 4 illustrates an example in which first camera 20-1 and second camera 20-2 capture images of a subject representing a common target point X in three-dimensional space. The x, y, and z axes are coordinate axes that are orthogonal to each other. The xz plane, defined by the x-axis and z-axis directions, is a horizontal plane, and the y-axis direction is a vertical direction perpendicular to the horizontal plane. The shooting directions of first camera 20-1 and second camera 20-2 are the z-axis direction. The optical center of second camera 20-2 is installed in the x-axis direction at a distance equivalent to baseline length b from the optical center of first camera 20-1.

[0022] Im1 and Im2 indicate the imaging planes of the first camera 20-1 and the second camera 20-2, respectively. Im1 and Im2 are positioned perpendicular to the imaging direction at a focal length f away from the optical centers of the first camera 20-1 and the second camera 20-2 in the imaging direction. The centers of the imaging planes Im1 and Im2 correspond to the intersections with the optical axes of the first camera 20-1 and the second camera 20-2. Points p and p' indicate the center of gravity of the reference block and the corresponding block, respectively, representing target point X. The displacement (deviation) between point p of the first image appearing on imaging plane Im1 and point p' of the second image appearing on imaging plane Im2 corresponds to the parallax d. Point p is located in the upper right corner of the first image, while point p' is located in the upper left corner of the second image. Generally, the larger the parallax d, the shorter the imaging distance from the imaging unit 20 to a common portion (e.g., target point X) appearing in the reference block and the corresponding block.

[0023] The reconstruction unit 116 performs a similar procedure for the other reference blocks to determine the parallax from the corresponding blocks, and can identify target points X in three-dimensional space that represent a pattern common to the corresponding blocks based on the position and parallax of the reference block in the first image. The reconstruction unit 116 stores in the storage unit 130 a point cloud made up of the target points X identified for each reference block.

[0024] The shape estimation unit 118 reads out the point cloud in the three-dimensional space stored in the storage unit 130, and estimates the plane spanned by the read point cloud as the surface of the subject. The estimated surface represents the three-dimensional shape of the subject. As illustrated in FIG. 5, the shape estimation unit 118 can estimate a surface Sf that interpolates the point cloud made up of the target points X identified for each reference block in the three-dimensional space as the surface of the subject. The determined surface represents the three-dimensional shape of the subject. The three-dimensional shape of the subject may be represented by parameters of a curved surface that interpolates the point cloud as illustrated in FIG. 5, or may be represented by a polygonal surface including, as elements, triangles with three adjacent points as vertices. The shape estimation unit 118 stores shape data representing the estimated three-dimensional shape in the storage unit 130.

[0025] The measurement unit 120 reads the shape data stored in the storage unit 130 and the damage area identified from the two-dimensional image, which is one of the first and second images. The measurement unit 120 identifies an area in three-dimensional space corresponding to the damage area on the surface of the subject shown in the shape data (hereinafter, this area may be referred to as a "three-dimensional damage area" to distinguish it from the damage area in the two-dimensional image). As illustrated in FIG. 6, the measurement unit 120 identifies a projection point Y projected from the optical center of the first camera 20-1 onto the surface of the subject in three-dimensional space for each pixel q in the damage area identified from the first image shown on the imaging plane Im1. The measurement unit 120 can determine the area occupied by a point cloud consisting of the projection points Y identified for each pixel q as the three-dimensional damage area. Then, the measurement unit 120 determines the dimensions of the three-dimensional damage area in three-dimensional space.

[0026] For a three-dimensional damage region having an elongated shape, such as a crack as illustrated in FIG. 6, the measurement unit 120 measures either the width or the length, or both, as dimensional indicators. The measurement unit 120 can determine the length of the three-dimensional damage region as the sum of the distances between projection points Y and Y' in three-dimensional space corresponding to adjacent pixels q and q' along a path from one end to the other of the damage region represented in the first image, which is a two-dimensional image. The measurement unit 120 can determine the width of the three-dimensional damage region as the average value of the distances between projection points Y in three-dimensional space corresponding to one end and the other end of the damage region in the intersecting direction that intersects the longitudinal direction in the two-dimensional image (however, the average between one end and the other end in the longitudinal direction in the two-dimensional image). The measurement unit 120 stores the measured dimensions of the three-dimensional damage region in the memory unit 130.

[0027] 2, the output processing unit 122 executes processing for outputting various pieces of information obtained by the control unit 110. The output processing unit 122, for example, constructs a display image showing the dimensions of the three-dimensional damaged area measured by the measurement unit 120. The output processing unit 122 outputs display data showing the constructed display image to the display unit 30 via the input / output unit 140. The output processing unit 122 may include in the display image an inspection object image obtained by superimposing the three-dimensional damaged area identified by the measurement unit 120 on the three-dimensional shape of the inspection object, which is the subject estimated by the shape estimation unit 118. The output processing unit 122 may output report data (report) indicating the three-dimensional shape, three-dimensional damaged area, and dimensions of the inspection target to another device separate from the information processing device 10 via the input / output unit 140.

[0028] The calibration unit 124 causes the imaging unit 20 to capture an image of a target whose shape and dimensions are known, and calibrates parameters related to the imaging using a first target image, which is a first image, and a second target image, which is a second image, obtained by the imaging (calibration). The calibration unit 124 applies the parameters obtained by calibration to the imaging unit 20 before performing the above-mentioned inspection. The calibration of the parameters may be performed offline at a time different from the time of the above-mentioned inspection.

[0029] The calibration unit 124 causes the first camera 20-1 and the second camera 20-2 constituting the imaging unit 20 to capture first and second target images representing the target for each of two or more patterns. Depending on the individual pattern, the calibration unit 124 changes one or both of the direction from the imaging unit 20 to the target (hereinafter, sometimes referred to as the "target direction") and the distance between the imaging unit 20 and the target (hereinafter, sometimes referred to as the "target shooting distance"). The target direction and the target shooting distance do not necessarily need to be known. The calibration unit 124 acquires correction parameters for correcting the lens distortion of the first camera 20-1 and the second camera 20-2 based on the obtained first and second target images. The calibration unit 124 changes the distance or direction between the imaging unit 20 and the target between the multiple images. Lens distortion mainly includes radial distortion and circumferential distortion. Radial distortion is a phenomenon in which the refraction of incident light becomes more pronounced as the distance from the optical center of the lens increases. The effect of radial distortion on an image is that the displacement of coordinates in the image due to the distortion is an even power r of the radius r from the center. 2 , r 4 , ...The proportionality constant multiplied by each multiplier term corresponds to the radial distortion coefficient. Circumferential distortion is a phenomenon in which an image is distorted in the circumferential direction around the optical center. Circumferential distortion is expressed as the cross component xy and r 2 +2x 2 , ... and for the y-coordinate, the cross component with the x-coordinate is xy and r 2 +2y 2 , ...The proportionality constant multiplied by each cross component term corresponds to the circumferential distortion coefficient.

[0030] The calibration unit 124 specifies, as actual measurement values, the coordinates of predetermined feature points on the target appearing in the first target image and the second target image captured by the first camera 20-1 and the second camera 20-2. For example, the calibration unit 124 can search for distortion coefficients in the radial and circumferential directions so that the difference between the coordinates of the feature points specified from each of the multiple first target images and the multiple second target images and the estimated coordinates of the feature points estimated by adding displacement using the above model from the coordinates of the feature points obtained under the assumption that no distortion occurs based on the known shape and dimensions of the target is minimized among multiple patterns.

[0031] The calibration unit 124 may analytically calculate distortion coefficients in the radial and circumferential directions that provide estimated values ​​equal to the actual measurement values ​​determined from each of a plurality of first and second target images. However, the number of patterns of target directions and target shooting distances is set in advance so that the number of independent proportionality constants, which are parameters of the above model, is equal to or less than the number of equations that indicate the relationship between the actual measurement values ​​and estimated values ​​determined from each of the first and second target images.

[0032] Calibration unit 124 calculates correction coefficients for correcting images distorted by the distortion coefficients obtained for first camera 20-1 and second camera 20-2 to an undistorted state. For example, if an inverse function of the radial distortion model and an inverse function of the circumferential distortion model are used as models for calculating the correction coefficients, parameters for these inverse functions as correction coefficients are determined by the radial distortion coefficient and the circumferential distortion coefficient. Calibration unit 124 stores the calculated correction coefficients in storage unit 130.

[0033] The image acquisition unit 112 corrects the first image and the second image using the correction coefficients calculated for each of the first camera 20-1 and the second camera 20-2. The corrected first image and the second image can be used for subsequent processing, such as detection of the damaged area, stereo reconstruction, and measurement of the dimensions of the three-dimensional damaged area. This correction removes or suppresses the effects of lens distortion from the first image and the second image. Note that the calibration related to distortion can be performed independently between the first camera 20-1 and the second camera 20-2. For example, calibration may be performed for the first camera 20-1 but not for the second camera 20-2, or calibration may be performed for the second camera 20-2 but not for the first camera 20-1.

[0034] The calibration unit 124 may cause the imaging unit 20 to capture a first image and a second image representing a target whose shape and dimensions are known. The calibration unit 124 may determine the relative positional relationship between the first camera 20-1 and the second camera 20-2, i.e., the baseline length, based on the captured first and second images. The calibration unit 124 may determine the distance from the imaging unit 20 to the target as the target shooting distance based on the area in which the target image appears in the first or second image, the dimensions of the target, and the viewing angle of the first camera 20-1 or the second camera 20-2. The calibration unit 124 may then determine the parallax between the first and second images and calculate the baseline length based on the shooting distance, parallax, and the known focal length, based on the relationship that the ratio of the target shooting distance to the baseline length is equal to the ratio of the focal length to the parallax. The calibration unit 124 may acquire the focal length set in the imaging unit 20 from the imaging unit 20, or may acquire it from setting information for the imaging unit 20 indicated in an operation signal input from the input unit 158 ​​(described later). The calibration unit 124 stores the calculated baseline length in the storage unit 130.

[0035] The calculated baseline length can be used for stereo reconstruction in the reconstruction unit 116, measurement of the dimensions of the two-dimensional damaged area in the measurement unit 120, etc. As a result, even if the baseline length changes due to changes in the shooting distance to the object to be inspected, the focal length, or the object, the baseline length can be obtained based on an image obtained by photographing a target with known dimensions. The target used for calibrating various parameters may be a known object to be inspected, or another known object located in the same space as the typical object to be inspected.

[0036] Next, an example of the hardware configuration of the information processing device 10 according to this embodiment will be described. The information processing device 10 may be configured to include dedicated components (e.g., integrated circuits) that perform the functions of each block or multiple blocks shown in Figure 2, or some or all of them may be configured as a computer system.

[0037] 3 is a schematic block diagram showing an example of the hardware configuration of information processing device 10 according to this embodiment. Information processing device 10 includes a processor 152, a drive unit 156, an input unit 158, an output unit 160, a ROM (Read Only Memory) 162, a RAM (Random Access Memory) 164, an auxiliary storage unit 166, and an interface unit 168. Processor 152, drive unit 156, input unit 158, output unit 160, ROM 162, RAM 164, auxiliary storage unit 166, and interface unit 168 are connected to one another using a bus BS (base line).

[0038] The processor 152, for example, reads out programs and various data stored in the ROM 162, executes the programs, and controls the operation of the information processing device 10. The processor 152 includes, for example, one or more central processing units (CPUs). In this application, "executing a program" or "executing a program" refers to executing processing instructed by commands written in the program.

[0039] The processor 152 may execute a predetermined program to realize some or all of the functions of the above-mentioned functional units, i.e., the image acquisition unit 112, the detection unit 114, the reconstruction unit 116, the shape estimation unit 118, the measurement unit 120, the output processing unit 122, and the calibration unit 124 of the control unit 110. The processor 152 may also realize the functions of the storage unit 130 in cooperation with one or a combination of the ROM 162, the RAM 164, and the auxiliary storage unit 166. The processor 152 may also realize the functions of the input / output unit 140 in cooperation with one or a combination of the input unit 158, the output unit 160, and the interface unit 168. Note that "executing a program" means executing processing instructed by various commands written in the program.

[0040] The storage medium 154 stores various types of data and is, for example, a portable storage medium such as a magneto-optical disk, a flexible disk, or a flash memory. The drive unit 156 is, for example, a device that reads various data from the storage medium 154 and / or writes various data to the storage medium 154 .

[0041] The input unit 158 ​​is an input device that receives a user operation, generates an operation signal in accordance with the received operation, and outputs the generated operation signal to the processor 152. Examples of the input unit 158 ​​include a mouse, a keyboard, and a pointing device such as a touch sensor. The output unit 160 includes, for example, a display unit such as a display, and a playback unit such as a speaker. The display unit 30 may be configured as the output unit 160, or may be connected to the interface unit 168 as a device separate from the output unit 160.

[0042] The ROM 162 stores, for example, a program to be executed by the processor 152 . The RAM 164 functions as a work area for temporarily storing various data and programs used by the processor 152, for example. The auxiliary storage unit 166 is a storage medium such as a hard disk drive (HDD) or a flash memory.

[0043] The interface unit 168 is connected to other devices and allows various data to be input and output. The interface unit 168 includes, for example, a communication module that connects to a network via wire or wirelessly. The input / output unit 140 may be configured as the interface unit 168.

[0044] Next, a procedure for measuring a damaged area according to this embodiment will be described below. Fig. 7 is a flowchart showing an example of the procedure for measuring a damaged area according to this embodiment. (Step S102) The calibration unit 124 causes the imaging unit 20 to capture a first image and a second image representing a known target. The calibration unit 124 determines the imaging distance from the imaging unit 20 to the target based on the size of the image of the target appearing in the captured first or second image. The calibration unit 124 calculates the baseline length based on the imaging distance and the parallax of the first or second image.

[0045] (Step S104) The image acquisition unit 112 causes the photographing unit 20 to photograph a first image and a second image representing the shape of the object to be inspected. (Step S106) The detection unit 114 detects a damaged area in either the first image or the second image using an image recognition technique.

[0046] (Step S108) The reconstruction unit 106 detects the parallax between the first image and the second image, performs stereo reconstruction based on the detected parallax, and identifies a target point in the three-dimensional space. (Step S110) Shape estimation unit 118 estimates the three-dimensional shape of the subject using a surface that interpolates a point group made up of the identified target points. (Step S112) The measurement unit 120 identifies, as the three-dimensional damaged area, an area where a point cloud consisting of projection points projected onto the estimated three-dimensional shape from each point in the detected damaged area is distributed. In step S112, the dimensions of the identified three-dimensional damaged area are measured. Then, the processing of FIG. 7 is terminated.

[0047] In the above description, the subject is a concrete block and the predetermined phenomenon to be inspected is mainly a crack, but this is not limiting. Objects other than concrete blocks may also be used as the subject. The predetermined phenomenon may be damage other than a crack, such as scratches, breakage, or stains. Furthermore, the predetermined phenomenon is not limited to damage, and may also be a sign of damage or other phenomena that appear externally, such as the shape, color, or shade of the subject.

[0048] The information processing device 10 may be configured as a single electronic device that integrally includes one or both of the imaging unit 20 and the display unit 30, or may not be configured as an integrated unit. The display unit 30 may be integrated with a touch sensor that serves as the input unit 158 ​​and configured as a touch panel. The imaging unit 20 may be mounted on a moving object such as a cart, a robot, or a drone.

[0049] As described above, the information processing device 10 according to this embodiment includes a detection unit 114 that detects an area representing a predetermined phenomenon occurring in the subject from the first image captured by the first camera 20-1, a shape estimation unit 118 that estimates the three-dimensional shape of the subject based on the parallax between the first image and the second image captured by the second camera 20-2, and a measurement unit 120 that determines the dimensions of an area in the three-dimensional shape of the subject where the predetermined phenomenon appears (e.g., a three-dimensional damaged area). According to this configuration, a region where a predetermined phenomenon detected in the first image appears is identified in the three-dimensional shape of the subject estimated from the parallax between the first and second images. Then, the dimensions of the identified region in three-dimensional space are obtained. Therefore, the dimensions of the region where the predetermined phenomenon appears in three-dimensional space can be measured from any distance without using auxiliary information such as detection information from a sensor, and without approaching the subject. By reducing the effort required to obtain and record the measurement results, the productivity of investigating or monitoring inspection targets of any shape can be improved.

[0050] The information processing device 10 may also include a calibration unit 124 that determines the distance (i.e., baseline length) between the first camera 20-1 and the second camera 20-2 based on the parallax between a first target image representing a target (i.e., a target) photographed by the first camera 20-1 and a second target image representing the target photographed by the second camera 20-2, and the dimensions of the target. With this configuration, the distance between the first camera 20-1 and the second camera 20-2 is determined based on the parallax between the first target image and the second target image, and the target object has known dimensions. Therefore, even when the positional relationship between the first camera 20-1 and the second camera 20-2 is variable, the distance between the first camera 20-1 and the second camera 20-2 can be measured without using auxiliary information. The measured distance is used to estimate the three-dimensional shape based on the parallax between the first image and the second image during inspection, and to measure the dimensions of the area where a predetermined phenomenon appears.

[0051] This embodiment may also be implemented as follows. The predetermined phenomenon may be damage to the subject, and the dimension may be the length or width of the damage in three-dimensional space. The subject may be a concrete block, and the predetermined phenomenon may be a crack in the concrete block. The information processing device 10 may include an output processing unit 122 that outputs report information indicating the measured dimensions and three-dimensional shape.

[0052] Another aspect of this embodiment may be a program for causing a computer to function as the information processing device 10. Another aspect of this embodiment may be an information processing system S1 including a first camera 20-1, a second camera 20-2, and an information processing device 10. Another aspect of this embodiment may be an information processing method in which information processing device 10 detects an area representing a predetermined phenomenon occurring in a subject from a first image captured by first camera 20-1, estimates the three-dimensional shape of the subject based on the parallax between a second image captured by second camera 20-2 and the first image, and determines the dimensions of the area in the three-dimensional shape of the subject where the predetermined phenomenon appears.

[0053] Although the present embodiment has been described above in detail with reference to the drawings, the specific configuration is not limited to the above-described configurations, and includes designs within the scope of the present embodiment. The above-described configurations can be combined in any manner, and some of them can be omitted. [Explanation of symbols]

[0054] S1...information processing system, 10...information processing device, 20...imaging unit, 20-1...first camera, 20-2...second camera, 30...display unit, 110...control unit, 112...image acquisition unit, 114...detection unit, 116...reconstruction unit, 118...shape estimation unit, 120...measurement unit, 122...output processing unit, 124...calibration unit, 130...storage unit, 140...input / output unit, 152...processor, 156...drive unit, 158...input unit, 160...output unit, 162...ROM, 164...RAM, 166...auxiliary storage unit, 168...interface unit

Claims

1. a detection unit that detects an area representing a predetermined phenomenon occurring in a subject from a first image captured by the first camera; a shape estimation unit that estimates a three-dimensional shape of the subject based on a parallax between a second image captured by a second camera and the first image; a measurement unit that determines the dimensions of an area in which the predetermined phenomenon appears in the three-dimensional shape of the subject. Information processing device.

2. a calibration unit that determines a distance between the first camera and the second camera based on a parallax between a first target image representing a target photographed by the first camera and a second target image representing the target photographed by the second camera, and a size of the target. The information processing device according to claim 1 .

3. the predetermined phenomenon is damage to the subject; The dimension is the length or width of the lesion in three-dimensional space. The information processing device according to claim 1 .

4. the subject is a concrete block, The predetermined phenomenon is a crack in the concrete block. The information processing device according to claim 3 .

5. an output processing unit that outputs report information indicating the dimensions and the three-dimensional shape; The information processing device according to claim 1 .

6. On the computer, A method for making the information processing device function as claimed in claim 1 program.

7. the first camera and the second camera; The information processing device according to claim 1 Information processing system.

8. A method in an information processing device, comprising: The information processing device, detecting an area representing a predetermined phenomenon occurring in the subject from a first image captured by the first camera; estimating a three-dimensional shape of the subject based on a parallax between a second image captured by a second camera and the first image; determining the dimensions of a region in the three-dimensional shape of the subject in which the predetermined phenomenon appears; Information processing methods.

Images