Depth estimation apparatus, calibration method, and calibration program

The depth estimation device automatically calculates calibration values for monocular cameras by analyzing image planes and partial regions, enhancing accuracy without manual effort or extra equipment.

JP2024166913AActive Publication Date: 2024-11-29TOYOTA JIDOSHA KK
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
JP2023083343
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-19
Publication Date
2024-11-29
Estimated Expiration
2043-05-19

AI Technical Summary

Technical Problem

Existing depth estimation methods using monocular cameras require manual work and additional equipment for calibration, which is costly and inefficient.

Method used

A depth estimation device and method that calculates calibration values automatically by analyzing the installation position of the camera relative to horizontal or vertical planes, identifying plane areas in the image, and setting partial regions to adjust depth estimates based on regression planes and calibration values without manual intervention.

Benefits of technology

Achieves highly accurate depth estimation without manual work or additional equipment, by calculating appropriate calibration values for each partial region in the image.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024166913000001_ABST
    Figure 2024166913000001_ABST
Patent Text Reader

Abstract

To provide a technique capable of realizing highly accurate depth estimation without requiring manual work or an additional apparatus, in a technique of estimating a depth in an image captured by a camera.SOLUTION: A depth estimation apparatus according to the present disclosure executes the processing of: estimating a depth in an image; acquiring an installation position of a camera with respect to a horizontal plane or a vertical plane; specifying a plane region in the image in which the horizontal plane or the vertical plane is reflected; setting a plurality of partial regions in the image; calculating a regression plane based on the estimated depth in the plane region for each partial region; calculating a calibration value for each partial region by comparing a position of the camera with respect to the regression plane with the installation position of the camera with respect to the horizontal plane or the vertical plane; and performing calibration based on the corresponding calibration value in each partial region.SELECTED DRAWING: Figure 6
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a technique for estimating depth in an image by analyzing an image captured by a camera. [Background technology]

[0002] Patent Document 1 discloses a calibration method for estimating camera parameters representing the characteristics of each imaging system in a distance measuring device equipped with multiple imaging systems. The calibration method disclosed in Patent Document 1 estimates the camera parameters using information acquired by capturing an image of a reference chart that is arranged so as to have a predetermined positional relationship with the distance measuring device.

[0003] Other documents that demonstrate the technical level of this technical field include Patent Document 2 and Patent Document 3 below. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2009 / 141998 [Patent Document 2] JP 2021-117130 A [Patent Document 3] JP 2013-187822 A Summary of the Invention [Problem to be solved by the invention]

[0005] The technology disclosed in Patent Document 1 is applied to a device (for example, a device using a stereo camera) that recognizes depth in an image by detecting parallax between a plurality of images.

[0006] On the other hand, a technology is being considered that estimates depth in an image by analyzing a single image captured by a monocular camera, etc. Such a technology has the advantage of being low cost and space-saving, since it requires less equipment. It also has the advantage that it can be applied to cameras used for other purposes. In particular, in recent years, it has been considered to analyze images using a trained machine learning model.

[0007] However, the appearance of the image to be analyzed varies depending on the characteristics of the camera used to capture the image. Therefore, in order to ensure estimation accuracy, it is necessary to perform calibration according to the appearance of the image and the characteristics of the camera.

[0008] However, conventionally, calibration requires manual work or additional equipment, resulting in considerable effort and cost. For example, when analyzing images using a trained machine learning model, calibration must be performed by re-learning parameters.

[0009] One objective of the present disclosure is to provide a technology for analyzing an image and estimating depth in the image that enables highly accurate depth estimation without requiring manual work or additional devices. [Means for solving the problem]

[0010] A first aspect of the present disclosure relates to a depth estimation device.

[0011] A depth estimation device according to a first aspect includes one or more processors that analyze an image captured by a camera to estimate a depth in the image, and a storage device that stores information on an installation position of the camera relative to a horizontal or vertical plane. The one or more processors are further configured to execute a first process of calculating a calibration value related to the estimated depth, and a second process of calibrating the estimated depth based on the calculated calibration value, the first process including: identifying a planar area in the image in which the horizontal or vertical plane is reflected; setting a plurality of partial areas in the image; calculating a regression plane representing the horizontal or vertical plane for each of the plurality of partial areas based on the depth estimated in the planar area included in each partial area; and calculating a calibration value for each partial area by comparing the position of the camera relative to the regression plane with the installation position, and the second process calibrates the estimated depth in each partial area based on the corresponding calibration value.

[0012] A second aspect of the present disclosure relates to a calibration method for performing, by a computer, calibration of a depth estimated in an image captured by a camera.

[0013] A calibration method according to a second aspect includes a process of acquiring information on an installation position of a camera relative to a horizontal or vertical plane, a process of identifying a planar area in an image in which the horizontal or vertical plane is reflected, a process of setting a plurality of partial areas in the image, a process of calculating, for each of the plurality of partial areas, a regression plane representing the horizontal or vertical plane based on a depth estimated in a planar area included in each partial area, a process of calculating a calibration value for each partial area by comparing a position of the camera relative to the regression plane with the installation position, and a process of calibrating the depth estimated in each partial area based on the corresponding calibration value, the process being performed by a computer.

[0014] A third aspect of the present disclosure relates to a calibration program that causes a computer to calibrate a depth estimated in an image captured by a camera.

[0015] A calibration program according to a third aspect causes a computer to execute the following processes: acquiring information on the installation position of a camera with respect to a horizontal or vertical plane; identifying a planar area in an image in which the horizontal or vertical plane is reflected; setting a plurality of partial areas in the image; calculating, for each of the plurality of partial areas, a regression plane representing the horizontal or vertical plane based on a depth estimated in a planar area included in each partial area; calculating a calibration value for each partial area by comparing the position of the camera with the installation position with respect to the regression plane; and calibrating the depth estimated in each partial area based on the corresponding calibration value. Effect of the Invention

[0016] According to the present disclosure, it is possible to calculate an appropriate calibration value for each partial region set in an image without requiring manual work or additional devices. Then, in each partial region, calibration of the estimated depth is performed based on the corresponding calibration value. This makes it possible to realize highly accurate depth estimation without requiring manual work or additional devices. [Brief description of the drawings]

[0017] [Figure 1] FIG. 2 is a diagram for explaining functions of the depth estimation device according to the present embodiment. [Diagram 2] FIG. 11 is a diagram for explaining a process of calculating a calibration value. [Diagram 3] 11A to 11C are diagrams illustrating examples of execution results of various processes. [Figure 4] FIG. 11 is a diagram for explaining execution results of various processes. [Diagram 5]FIG. 1 is a diagram illustrating an example of the configuration of a depth estimation device according to an embodiment of the present invention. [Figure 6] 11 is a diagram illustrating an example of a process performed by the depth estimation device according to the present embodiment for calibrating an estimated depth. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Hereinafter, the present embodiment will be described with reference to the drawings.

[0019] 1.Depth estimation device 1 is a diagram for explaining basic functions of a depth estimation device 10 according to this embodiment. The depth estimation device 10 analyzes an image captured by a camera 200 and estimates the depth in the image.

[0020] The camera 200 is installed so as to capture an image of an arbitrary area for measuring depth. The camera 200 outputs captured image data. The camera 200 may output video data composed of images captured continuously. In this embodiment, the configuration of the camera 200 is not particularly limited.

[0021] The depth estimation device 10 performs a depth estimation process P10 and a calibration process P20 on an image captured by the camera 200.

[0022] The depth estimation process P10 analyzes an image to estimate the depth in the image. The depth in the image is given, for example, to each pixel in the image. In this case, when each pixel is represented by coordinates (M, N) on the image, the depth in the image can be represented by D(M, N). D(M, N) indicates the magnitude of the depth given to the pixel corresponding to the coordinates (M, N) on the image. D(M, N) can also be called a "depth map."

[0023] Fig. 3(A) and Fig. 3(B) respectively show an example of an image captured by the camera 200 and an example of a depth estimated by the depth estimation process P10. In Fig. 3(B), the magnitude of the depth estimated for the image shown in Fig. 3(A) is shown in shading. In particular, the greater the estimated depth, the darker the shade shown.

[0024] The depth estimation process P10 may be performed using any suitable known technique. For example, the depth estimation process P10 may be configured to estimate the depth in an image using a trained machine learning model that uses an image as input data.

[0025] Incidentally, in the depth estimation device 10, it is considered that the camera 200 having suitable characteristics is selected according to the object and environment to be imaged. If the characteristics of the camera 200 differ, the appearance of the image changes. On the other hand, the depth estimation process P10 is usually optimized for an image captured by a specific camera. For example, when a trained machine learning model is used, the depth estimation process P10 is optimized for an image that is used as training data. For this reason, the depth estimated by the depth estimation process P10 may have an error according to the appearance of the image.

[0026] Depth estimation is particularly affected by distortion, which distorts the appearance of an image. In the image shown in Figure 3(A), barrel distortion occurs. As shown in Figure 3(A), distortion becomes greater the further away from the center of the image. For this reason, it is thought that errors associated with distortion differ in each part of the image and change depending on the magnitude of the distortion.

[0027] Referring again to Fig. 1, the calibration process P20 calibrates the estimated depth in order to reduce such errors. In the calibration process P20, the calibration is performed based on a calibration value managed in the depth estimation device 10. The depth estimation device 10 outputs the result of the calibration process P20 as an estimation result.

[0028] The depth estimation device 10 according to the present embodiment realizes highly accurate depth estimation by calculating an appropriate calibration value according to the appearance of the image. In particular, the depth estimation device 10 according to the present embodiment is capable of calculating the calibration value without requiring manual work or additional devices. The following describes the details of the process of calculating the calibration value and the calibration process P20 based on the calculated calibration value.

[0029] 2. Calculation of calibration value The depth estimation device 10 executes a process of calculating a calibration value at a predetermined timing. The predetermined timing may be suitably determined. For example, the predetermined timing is when the camera 200 is started. Alternatively, the predetermined timing is set periodically at regular intervals. Alternatively, the predetermined timing is when an instruction to update the calibration value is received from a user.

[0030] In the process of calculating the calibration value, the calibration value is calculated from an image captured by the camera 200 at a predetermined timing, a depth estimated by the depth estimation process P10, and the installation position of the camera 200 relative to a predetermined horizontal or vertical plane. Information on the installation position of the camera 200 relative to the horizontal or vertical plane may be managed in the depth estimation device 10. The horizontal or vertical plane for defining the installation position may be suitably selected according to the environment to which the present embodiment is applied. Typically, the horizontal plane is a floor or the ground. The vertical plane is a wall. The horizontal or vertical plane may be one surface of an object placed around the camera 200. For example, the horizontal plane may be the top surface of a table. However, it is desirable that the horizontal or vertical plane has a certain degree of spread. The installation position of the camera 200 relative to the horizontal plane is, for example, the height of the camera 200 relative to the horizontal plane. The installation position of the camera 200 relative to the vertical plane is, for example, the horizontal distance of the camera 200 relative to the vertical plane. The depth estimation device 10 may be configured to manage information on a plurality of installation positions with respect to each of a plurality of horizontal or vertical planes.

[0031] The process of calculating the calibration value will be described below with reference to Fig. 2. The process of calculating the calibration value includes a planar area identification process P31, a partial area setting process P32, a depth extraction process P33, a regression plane calculation process P34, and a calibration value calculation process P35.

[0032] The planar area identification process P31 identifies a planar area in an image in which a horizontal or vertical plane is reflected. FIG. 3C shows an example of a planar area 21 identified by the planar area identification process P31. In FIG. 3C, the planar area 21 is an area in which a floor is reflected in the image shown in FIG. 3A. The planar area identification process P31 can be realized, for example, by performing semantic segmentation on the image. In this case, the area of ​​pixels labeled as a horizontal or vertical plane may be set as the planar area 21. The planar area identification process P31 may identify multiple planar areas 21 for multiple horizontal or vertical planes. For example, the planar area identification process P31 may identify an area in which a wall is further reflected as another planar area 21 in the image shown in FIG. 3A.

[0033] The partial region setting process P32 sets a plurality of partial regions in the image. FIG. 3D shows an example of a plurality of partial regions 22 set by the partial region setting process P32. As described above, it is considered that the error due to distortion differs in each portion of the image and changes depending on the magnitude of the distortion. Therefore, by setting a plurality of partial regions 22 in this way, it can be expected that the error due to distortion will be about the same in each partial region 22.

[0034] There are various possible patterns for setting the multiple partial regions 22. For example, the multiple partial regions 22 may be set by dividing an image into a grid. In addition, the multiple partial regions 22 may be set by arranging regions of a predetermined shape on the image.

[0035] 3D, each partial region 22 is a region surrounded by sides having a curvature. In particular, the curvature of a partial region 22 is greater the farther it is from the center of the image. Also, the size of each partial region 22 is smaller the farther it is from the center of the image.

[0036] As described above, the distortion becomes larger the farther away from the center of the image. Therefore, by making the curvature of the side of the partial region 22 larger for the partial region 22 farther from the center of the image, each partial region 22 can be shaped according to the change in distortion. Furthermore, by making the size of each partial region 22 smaller for the partial region 22 farther from the center of the image, the partial regions 22 can be set finer in the part where the distortion becomes large. In other words, by giving the curvature of the side and the size of each partial region 22 in this way, it is possible to set a plurality of partial regions 22 more appropriate for making the error associated with the distortion aberration in each partial region 22 the same. In this case, each partial region 22 can be specified by, for example, the curvature of the side and the width and height of the region. In this case, the curvature of the side and the width and height of the region may be optimally determined by testing.

[0037] Referring again to Fig. 2, the depth extraction process P33 extracts, for each partial region 22, an estimated depth in the planar region 21 contained in the partial region 22. The depth extracted for each partial region 22 is linked to the corresponding partial region 22. The depth extraction process P33 can be realized, for example, by calculating an area of ​​a common portion between the target partial region 22 and the planar region 21, and referring to the estimated depth using the calculated area of ​​the common portion.

[0038] Fig. 4A shows an overlapping view of the planar region 21 and the partial regions 22 shown in Fig. 3C and Fig. 3D, respectively. Fig. 4B shows an example of the depth extracted by the depth extraction process P33 for the partial region 22A shown in Fig. 4A. Fig. 4B shows that, of the depths shown in Fig. 3C, the depths in the area of ​​the intersection between the partial region 22A and the planar region 21 are extracted.

[0039] Incidentally, partial region 22B shown in Fig. 4(A) has no common portion with planar region 21. Such partial region 22 can be dealt with by executing the above-mentioned processing on other planar region 21 reflecting other horizontal or vertical surfaces. For example, for partial region 22B, the above-mentioned processing can be executed on other planar region 21 identified as an area reflecting a wall.

[0040] 2 again. The regression plane calculation process P34 calculates a regression plane representing a horizontal plane or a vertical plane based on the extracted depth for each partial region 22. The regression plane calculated for each partial region 22 is associated with the corresponding partial region 22.

[0041] The regression plane calculation process P34 is executed, for example, as follows.

[0042] First, the regression plane calculation process P34 acquires the position in world coordinates (X, Y, Z) of the planar region 21 included in the target partial region 22 from the internal parameters of the camera 200 and the depth extracted for the target partial region 22. That is, when the extracted depth is represented as D(M, N), the coordinates (M, N) on the image of the planar region 21 included in the target partial region 22 are converted to world coordinates (X, Y, Z) using the internal parameters of the camera 200 and the extracted depth D(M, N).

[0043] Then, the regression plane calculation process P34 performs regression analysis using the position of the planar region 21 in the acquired world coordinates (X, Y, Z) as an explanatory variable, to calculate a regression plane that represents a horizontal plane or a vertical plane for the target partial region 22. In this case, the regression plane is expressed by an equation for a plane in the world coordinates (X, Y, Z) with the position of the camera 200 as the origin.

[0044] The calibration value calculation process P35 calculates a calibration value for each partial region 22 by comparing the position of the camera 200 with respect to the calculated regression plane with the installation position of the camera 200 with respect to the horizontal or vertical plane managed in the depth estimation device 10. That is, in the depth estimation device 10 according to this embodiment, the calibration value is managed for each partial region 22.

[0045] As described above, the regression plane is calculated for each partial region 22 from the estimated depth. Furthermore, it is expected that the errors occurring in each partial region 22 are of the same order of magnitude. For this reason, it is considered that the difference between the position of the camera 200 with respect to the regression plane and the installation position of the camera 200 with respect to the horizontal or vertical plane is an amount proportional to the error occurring in each partial region 22. Therefore, by comparing the position of the camera 200 with respect to the regression plane and the installation position of the camera 200 with respect to the horizontal or vertical plane, it is possible to calculate an appropriate calibration value for each partial region 22.

[0046] For example, the calibration value calculation process P35 can calculate the calibration value as follows. Now, as shown in Fig. 4C, it is assumed that the installation position of the camera 200 with respect to the horizontal plane 1 is given by the height h of the camera 200 with respect to the horizontal plane 1. It is also assumed that the regression plane 2 has been calculated for a certain partial region 22 by the regression plane calculation process P34. Then, it is assumed that the height of the camera 200 with respect to the regression plane 2 is expressed as c from the equation of the calculated regression plane 2. At this time, the calibration value calculation process P35 can calculate the calibration value for this partial region 22 by h / c, which is a scale factor for the estimated depth.

[0047] Depending on the pattern of the set partial regions 22, an image may have a region that is not included in any of the partial regions 22. For example, in the pattern shown in FIG. 4A, the regions at the four corners are not included in any of the partial regions 22. For such a region, a calibration value may be given by referring to a calibration value calculated for a neighboring partial region 22. For example, the calibration value for the region at the lower right corner shown in FIG. 4A may be the average value of the calibration value calculated for the partial region 22C and the calibration value calculated for the partial region 22D. In the neighboring partial regions 22, the degree of error associated with distortion aberration is considered to be similar. Therefore, by performing the process in this manner, it is possible to give an appropriate calibration value to a region that is not included in any of the partial regions 22.

[0048] 3. Calibration process Next, the calibration process P20 will be described in detail. As described above, the depth estimation device 10 according to the present embodiment manages a calibration value for each partial region 22. Therefore, the calibration process P20 calibrates the depth estimated in each partial region 22 based on the corresponding calibration value.

[0049] For example, the calibration process P20 executes the process as follows. Now, assume that the depth estimated by the depth estimation process P10 is D(M,N). Also assume that a scale factor α is managed as a calibration value for each partial region 22. In this case, the calibration process P20 performs calibration by calculating α×D(M,N) for each partial region 22 using the corresponding scale factor α. In this case, the depth estimation device 10 outputs α×D(M,N) as the estimation result.

[0050] 4. Configuration The configuration of the depth estimation device 10 according to this embodiment will be described below. Fig. 5 is a diagram showing an example of the configuration of the depth estimation device 10 according to this embodiment.

[0051] The depth estimation device 10 includes a processing unit 100, a camera 200, and a user interface 300.

[0052] The processing unit 100 is a computer including one or more processors 110 (hereinafter simply referred to as “processors 110”) and one or more storage devices 120 (hereinafter simply referred to as “storage devices 120”). The processing unit 100 is configured to be able to communicate with the camera 200 and the user interface 300.

[0053] The processor 110 executes various processes. The processor 110 can be configured, for example, by a CPU (Central Processing Unit) including an arithmetic unit, a register, and the like. The storage device 120 is connected to the processor 110 and stores various information required for the processor 110 to execute the processes. The storage device 120 can be configured, for example, by a recording medium such as a ROM (Read Only Memory), a RAM (Random Access Memory), a HDD (Hard Disk Drive), or a SSD (Solid State Drive).

[0054] The storage device 120 stores a computer program 121, calibration values ​​122, and installation position information 123.

[0055] The computer program 121 is composed of a plurality of instructions that cause the processor 110 to execute various processes. The computer program 121 may be stored in a computer-readable recording medium included in the storage device 120. The processor 110 operates in accordance with the computer program 121, thereby realizing the execution of various processes by the processor 110. In particular, the processor 110 executes the process of calculating a calibration value (first process), the depth estimation process P10, and the calibration process P20 (second process) as described above.

[0056] The calibration value 122 is provided for each partial region 22. The processor 110 stores in the storage device 120 the calibration value 122 calculated for each partial region 22 by executing the process.

[0057] The installation position information 123 gives the installation position of the camera 200 with respect to a horizontal or vertical plane. The installation position information 123 may be stored in advance in the storage device 120. Alternatively, the installation position information 123 may be set by the user via the user interface 300.

[0058] The user interface 300 is provided to allow a user to utilize the functions of the depth estimation device 10. The user interface 300 is configured with, for example, input devices such as a keyboard and a touch panel, and output devices such as a display and a speaker. Through the user interface 300, the user inputs setting information, checks the estimation result, obtains the data of the estimation result, and so on.

[0059] As described above, the depth estimation device 10 according to the present embodiment is configured. Furthermore, the processor 110 executes processing to realize the calibration method according to the present embodiment.

[0060] FIG. 6 is a flowchart illustrating an example of a process performed by the processor 110 for calibrating the estimated depth.

[0061] In step S100, the processor 110 refers to the installation position information 123 to acquire the installation position of the camera 200 relative to a horizontal or vertical plane.

[0062] Next, in step S110, the processor 110 identifies a planar area 21 in the image that shows a horizontal or vertical surface.

[0063] Next, in step S120, the processor 110 sets a plurality of partial regions 22 in the image.

[0064] Next, in step S130, the processor 110 calculates a regression plane for each partial region 22 based on the depth estimated in the planar region 21 included in each partial region 22.

[0065] Next, in step S140, the processor 110 calculates a calibration value for each partial region 22 by comparing the position of the camera 200 with respect to the regression plane and the installation position acquired in step S100.

[0066] Next, in step S150, the processor 110 performs a calibration of the estimated depth in each sub-region 22 based on the corresponding calibration value.

[0067] In this manner, the calibration method according to the present embodiment is realized. Furthermore, the calibration program according to the present embodiment is realized by the computer program 121 that causes the processor 110 to execute the processes in this manner.

[0068] 5.Effects As described above, according to this embodiment, for each partial region 22 set in an image, the regression plane 2 representing a horizontal plane or a vertical plane is calculated based on the depth estimated in the planar region 21 included in each partial region 22. Then, a calibration value is calculated for each partial region 22 by comparing the position of the camera 200 with respect to the regression plane with the installation position of the camera 200 with respect to the horizontal plane or the vertical plane. This makes it possible to calculate an appropriate calibration value for each partial region 22. Then, in each partial region 22, calibration of the estimated depth is performed based on the corresponding calibration value. This makes it possible to realize highly accurate depth estimation without requiring manual work or additional devices. [Explanation of symbols]

[0069] 10 Depth estimation device 21 Planar area 22 Partial area 100 Processing section 110 Processor 120 Storage device 121 Computer Programs 122 Calibration Values 123 Installation location information 200 Cameras 300 User Interface

Claims

1. one or more processors for analysing images captured by the camera to estimate depth in the images; a storage device for storing information on the installation position of the camera relative to a horizontal or vertical plane; Equipped with The one or more processors further include: A first process for calculating a calibration value related to the estimated depth; a second process of calibrating the estimated depth based on the calibration value; configured to run The first process includes: Identifying a planar area in the image in which the horizontal or vertical surface is reflected; setting a plurality of partial regions in the image; calculating, for each of the plurality of partial regions, a regression plane representing the horizontal plane or the vertical plane based on the estimated depth of the planar region included in the each partial region; calculating the calibration value for each of the partial regions by comparing the position of the camera with the installation position relative to the regression plane; Including, The second process calibrates the estimated depth in each of the partial regions based on the corresponding calibration value. Depth estimation device.

2. The depth estimation device according to claim 1 , Each of the partial regions is a region surrounded by sides having a curvature, The curvature is larger in a partial region farther from the center of the image. Depth estimation device.

3. The depth estimation device according to claim 1 or 2, The size of each partial region is smaller the further the partial region is from the center of the image.

4. 1. A calibration method for performing a calibration of a depth estimated in an image captured by a camera by a computer, comprising: A process of acquiring information on the installation position of the camera relative to a horizontal or vertical plane; A process of identifying a planar area in the image in which the horizontal or vertical surface is reflected; A process of setting a plurality of partial regions in the image; A process of calculating a regression plane representing the horizontal plane or the vertical plane for each of the plurality of partial regions based on the estimated depth of the planar region included in each of the partial regions; A process of calculating a calibration value for each of the partial regions by comparing a position of the camera with the installation position with respect to the regression plane; performing a calibration of the estimated depth in each of the partial regions based on the corresponding calibration value; is executed by the computer. Calibration method.

5. A calibration program that causes a computer to calibrate a depth estimated in an image captured by a camera, A process of acquiring information on the installation position of the camera relative to a horizontal or vertical plane; A process of identifying a planar area in the image in which the horizontal or vertical surface is reflected; A process of setting a plurality of partial regions in the image; A process of calculating a regression plane representing the horizontal plane or the vertical plane for each of the plurality of partial regions based on the estimated depth of the planar region included in each of the partial regions; A process of calculating a calibration value for each of the partial regions by comparing a position of the camera with the installation position with respect to the regression plane; performing a calibration of the estimated depth in each of the partial regions based on the corresponding calibration value; causing the computer to execute Calibration program.

Citation Information

Patent Citations

  • Vehicle-mounted three-dimensional laser radar calibration method and system

    CN109696663A

  • Method and apparatus for calibrating range finder

    JP2007256091A

  • Object distance detector

    JP2019027882A

  • Depth sensor calibration and per-pixel correction

    US20150302570A1

  • Correction formula calculation method, correction method, correction apparatus, and imaging apparatus

    JP2013187822A