Information processing device, information processing method, and program

Abstract

This information processing device has a RAW processing unit and a ranging calculation unit. The RAW processing unit acquires a normalized luminance image in which the luminance of respective colors is normalized from the RAW signal. The ranging calculation unit acquires depth information of a subject from the normalized luminance image by using a monocular depth estimation model trained with regard to correlations between depth and luminance.

Description

Information processing device, information processing method, and program

[0001] The present invention relates to an information processing device, an information processing method, and a program.

[0002] Endoscopic images are often taken with a monocular camera attached to the tip of the endoscope. However, it is difficult to obtain depth information from monocular camera images. It is difficult to measure size calculated from depth information and two-dimensional information (such as RGB images), which hinders diagnosis and surgery.

[0003] Japanese Patent Application Laid-Open No. 2022-172654

[0004] Currently, methods include inserting a tape measure through the forceps opening to directly measure the size of the object, or projecting images using lasers or mesh structures, but these tasks require a high level of skill. While artificial intelligence (AI) is also used to estimate depth information from endoscopic images, subtle changes in color can easily cause errors, making it difficult to achieve high inference accuracy.

[0005] Therefore, the present disclosure proposes an information processing device, an information processing method, and a program that are capable of acquiring depth information with high accuracy.

[0006] According to the present disclosure, there is provided an information processing device including a RAW processing unit that acquires a normalized luminance image in which the luminance of each color is normalized from a RAW signal, and a ranging calculation unit that acquires depth information of a subject from the normalized luminance image using a monocular depth estimation model that has learned about the correlation between depth and luminance. Also, according to the present disclosure, there is provided an information processing method in which information processing of the information processing device is executed by a computer, and a program that causes a computer to realize the information processing of the information processing device.

[0007] 1 is a diagram illustrating an overview of the distance measurement method of the present disclosure. FIG. 1 is a diagram illustrating an overview of the distance measurement method of the present disclosure. FIG. 2 is a diagram illustrating an overview of the distance measurement method of the present disclosure. FIG. 3 is a diagram illustrating an inference result of depth information obtained by applying the depth estimation method of the present disclosure to a monocular image. FIG. 4 is a diagram illustrating an inference result of depth information obtained by applying the depth estimation method of the present disclosure to a monocular image. FIG. 5 is a diagram illustrating an example of capturing an image of a rectangular plate with a monocular camera of an endoscope. FIG. 6 is a diagram illustrating an example of capturing an image of an incision in implant surgery. FIG. 7 is a diagram illustrating an example of an information processing system that implements the depth estimation method of the present disclosure. FIG. 8 is a diagram illustrating an example of the configuration of an image distance acquisition unit. FIG. 9 is a diagram illustrating switching of learning models according to lighting conditions. FIG. 10 is a diagram illustrating variations of an information processing system. FIG. 11 is a diagram illustrating an example of the hardware configuration of an information processing device.

[0008] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the following embodiments, the same components are designated by the same reference numerals, and redundant description will be omitted.

[0009] The description will be given in the following order: [1. Overview of the distance measurement method of the present disclosure] [2. Configuration of information processing device] [3. Switching of learning model according to lighting conditions] [4. System variations] [5. Hardware configuration example] [6. Effects]

[0010] 1. Overview of the Distance Measuring Method of the Present Disclosure FIGS. 1 to 3 are diagrams illustrating an overview of the distance measuring method of the present disclosure.

[0011] There is a growing demand for depth estimation technology for devices and applications such as surgical robots and endoscopy robots. Well-known methods for acquiring depth information include those using structured light and triangulation technology, and those using lidar sensors.

[0012] While these methods can obtain highly accurate depth information, they require special optical systems and dedicated sensors separate from the main system, resulting in large-scale equipment. This not only increases costs but also creates the problem of eliminating system configurations such as existing endoscope-based endoscopy robots. Depth estimation using stereo image matching, which does not require special lighting, can also be used, but this method is severely restricted to stereo endoscopes.

[0013] The present disclosure has been made in consideration of the above-mentioned problems. This disclosure proposes a depth estimation method capable of acquiring highly accurate depth information without using a special device configuration. The example in FIG. 1 shows lighting conditions using a point light source LS, which is common in the field of monocular endoscopes. There is a correlation between image brightness and depth. For example, in a lighting environment using a point light source LS, brightness attenuates in proportion to the square of distance based on the inverse square of illuminance. The correlation between brightness and depth may vary depending on the lighting conditions. In this disclosure, depth information is estimated from a RAW signal based on a correlation between brightness and depth that has been learned in advance.

[0014] In the example of Fig. 2, an image of an object OB is captured under an illumination environment using a point light source LS. The object OB has colors such as red, green, and blue. Although the luminance of each color varies depending on wavelength characteristics (such as the light source spectrum, the transmission characteristics of the color filter, and the spectral reflectance characteristics of the object OB), if the luminance is corrected (normalized) to offset the wavelength characteristics, depth information can be obtained from the corrected luminance.

[0015] Normalization refers to the process of adjusting the scaling coefficient for each color to equalize the luminance range of each color obtained under the same lighting conditions. Normalization is performed on the image signal before development processing (the RAW signal, or a signal that has been subjected to corrections such as denoising and distance detection enhancement on the RAW signal). Normalization is performed on the image signal before development processing to prevent significant distortion of the original color and luminance information during development processing.

[0016] In the example of FIG. 3 , the luminance of each color of object OB is converted to a white-equivalent luminance by normalization. The white-equivalent luminance refers to the luminance that would be expected to be obtained if the color of the object were white. In this disclosure, the process of converting the luminance of each color to a white-equivalent luminance is referred to as white-equivalent luminance conversion. The left side of FIG. 3 shows a luminance distribution (a luminance image before normalization) included in the RAW signal. The right side of FIG. 3 shows a luminance distribution (a normalized luminance image) after the white-equivalent luminance conversion has been performed on the RAW signal.

[0017] 4 and 5 are diagrams showing inference results of depth information obtained by applying the depth estimation method of the present disclosure to monocular images. The example of Fig. 4 shows an endoscopic image of a lesion. The example of Fig. 5 shows monocular images of multiple color panels. In both examples, accurate depth information reflecting the luminance information contained in the RAW signal is obtained.

[0018] Figure 6 shows an example of capturing an image of a rectangular plate with an endoscope's monocular camera. The plate is tilted obliquely with respect to the optical axis of the monocular camera. The shape of the plate captured by the monocular camera is trapezoidal. The position of the plate in the depth direction is estimated based on the normalized brightness image. The rectangular planar shape is acquired by restoring the shape taking the depth into account.

[0019] Figure 7 shows an example of an incision taken during implant surgery. During implant surgery, a portion of the temporal bone is removed and an electrode is inserted into the cochlea. The incision, which is oblique in the depth direction, is covered with a thin, flat plate (plane plate) made of cartilage or other material. The endoscope observes the area where the plane plate will be placed. Depth information is acquired using normalized intensity images to measure the three-dimensional shape of the object and the contour shape (two-dimensional shape) of the area where the plane plate will be placed.

[0020] By taking depth into account, the accurate shape of the observed object can be restored. For example, it becomes possible to obtain the outer shape of the observed object projected onto a two-dimensional plane from a specified direction. Conventional methods cannot obtain accurate depth information from monocular images. Therefore, obtaining an accurate shape that takes depth into account requires the experience of a doctor and the effort of actually creating an object, inserting it into the affected area, and then reshaping it. The method disclosed herein can derive an accurate shape without requiring the experience and effort of a doctor.

[0021] 8 is a diagram showing an example of an information processing system that implements the depth estimation method of the present disclosure. This system acquires depth information from, for example, a monocular endoscopic image. The information processing system includes an information processing device 1, an endoscopic camera unit 2, and an external input device 3.

[0022] The endoscopic camera unit 2 has a monocular camera and a light source device. Light output from the light source device is projected from the tip of a light-guiding fiber toward the affected area. The tip of the fiber, which serves as the light source, serves as the illumination light source. The endoscopic camera unit 2 uses the monocular camera to capture an image of the subject SB illuminated by the illumination light. The light source device can project visible light as illumination light, or can project special light that can improve the visibility of the affected area. The endoscopic camera unit 2 may be a flexible endoscope or a rigid endoscope.

[0023] The information processing device 1 acquires a RAW signal RD of the subject SB from the endoscopic camera unit 2. The information processing device 1 analyzes the RAW signal RD to acquire information such as the depth, size, shape, and distance between points of the subject SB. For example, the information processing device 1 includes an image acquisition unit 11, an image distance acquisition unit 12, an image development unit 13, and a measurement processing unit 14.

[0024] The image acquisition unit 11 acquires a RAW signal RD from the endoscopic camera unit 2 and outputs it to the image distance acquisition unit 12 and the image development unit 13. The RAW signal RD is an image signal that records information received from the monocular camera (image sensor) in almost its original state. The RAW signal RD has not undergone significant adjustments to color or brightness, as is done in development processing.

[0025] The image distance acquisition unit 12 analyzes the RAW signal RD to acquire depth information DI of the subject SB. Fig. 9 is a diagram showing an example of the configuration of the image distance acquisition unit 12. For example, the image distance acquisition unit 12 includes a RAW processing unit 15 and a distance measurement calculation unit 16.

[0026] The RAW processing unit 15 obtains a normalized luminance image in which the luminance of each color is normalized from the RAW signal RD. Normalization refers to a process of adjusting the scaling coefficient for each color to equalize the luminance range of each color obtained under the same lighting conditions. Normalization does not involve significant adjustments to color or brightness, as is done in development processing. Therefore, the luminance information after normalization includes highly accurate depth information DI based on the correlation between depth and luminance.

[0027] The RAW processing unit 15 can perform various correction processes on the RAW signal RD within a range that does not involve significant changes in color or brightness that would interfere with the calculation of the depth information DI. In the example of Fig. 9, the correction processes include noise suppression, tone curve correction, and frequency correction.

[0028] Noise suppression processing removes noise that causes errors in distance measurement calculations. Tone curve correction adjusts the image sensor output to match the characteristics of distance measurement calculations, obtaining a signal with good distance detection properties. Frequency correction enhances the contours of the subject SB by increasing (boosting, multiplying) attenuated high-frequency signals. By highlighting the contours, errors in measuring the size of affected areas, etc., can be reduced.

[0029] The ranging calculation unit 16 acquires depth information DI of the subject SB from the normalized luminance image using a learning model (monocular depth estimation model 18: see FIG. 10 ) that has learned about the correlation between depth and luminance. As described above, the RAW signal RD is not subjected to significant adjustments of color and brightness, as is done in development processing. Therefore, accurate depth information DI can be obtained by normalizing the luminance of each color contained in the RAW signal RD and applying the normalized luminance distribution (normalized luminance image) to the monocular depth estimation model 18.

[0030] The monocular depth estimation model 18 is trained using a group of images acquired under lighting conditions consistent with the actual shooting environment as training data. The training may be supervised learning or self-supervised learning. When the RAW signal RD that has undergone correction processing such as tone curve correction or frequency correction is used as the input to the monocular depth estimation model 18, a group of images that have undergone the same correction processing can be used as training data.

[0031] In the example of FIG. 9 , the depth information DI is acquired through a two-stage process: normalization and depth estimation. However, these processes may be performed collectively by a single AI. For example, it is difficult to explicitly determine the formula for the white-equivalent luminance conversion performed in normalization. Therefore, it is preferable to perform normalization using a learning model (normalization model). By assigning the role of the normalization model to the monocular depth estimation model MD, the depth information DI is acquired directly from the RAW signal RD. In this case, the ranging calculation unit 16 also functions as the RAW processing unit 15.

[0032] It is preferable that the lighting conditions used in the endoscopic camera unit 2 match the lighting conditions of the training data for the monocular depth estimation model 18. For example, the image acquisition unit 11 acquires, as a RAW signal RD, an image signal acquired under specific lighting conditions that are the same as the lighting conditions used for the training data for the monocular depth estimation model 18. The monocular depth estimation model 18 is a training model trained using a group of images that have a correlation between depth and brightness under specific lighting conditions.

[0033] The lighting conditions may be those commonly used in actual imaging environments. For example, in an endoscope, the light source is the tip of a thin fiber that guides light from a light source device. In this case, the specific lighting conditions indicate an illumination environment using a single point light source LS. The monocular depth estimation model 18 is a learning model that learns the correlation between depth and brightness based on the inverse square of illuminance.

[0034] The image developing unit 13 develops the RAW signal RD to generate an RGB image CI. The development process may include exposure adjustment, contrast adjustment, white balance adjustment, saturation adjustment, cropping, perspective adjustment, etc. The measurement processing unit 14 acquires the position of the observation target depicted in the RGB image CI. The measurement processing unit 14 estimates various information about the observation target based on the position information and depth information DI of the observation target, and outputs the estimation result ES to an external device.

[0035] For example, the measurement processing unit 14 can obtain the size of the observation target based on the depth information DI of the observation target. The size can be calculated based on the image sensor size, the size of the corresponding part projected onto the pixel, the focal length of the lens, etc. For example, the measurement processing unit 14 obtains the size of the observation target based on the magnification ratio used during imaging. The measurement processing unit 14 measures the size of the target in the endoscopic image and calculates the size by multiplying it by a magnification ratio corresponding to the depth information to the target.

[0036] The measurement processing unit 14 can reconstruct the three-dimensional shape of the observation target based on the depth information DI of the observation target. The measurement processing unit 14 can acquire the outer shape of the observation target projected onto a two-dimensional plane from a predetermined direction based on the depth information DI of the observation target. The measurement processing unit 14 can acquire two points included in the observation target as target points, and acquire the distance between the target points based on the depth information DI of each target point.

[0037] The external input device 3 inputs position information of points or ranges to be measured to the measurement processing unit 14. The input may be performed manually by a user, or information extracted from the RGB image CI by image analysis may be input automatically.

[0038] 3. Switching of Learning Models According to Illumination Conditions FIG. 10 is a diagram illustrating switching of learning models according to illumination conditions.

[0039] A user can switch between multiple endoscopic camera units 2 depending on the application. Each endoscopic camera unit 2 has different lighting, optical, and imaging conditions. Therefore, a different monocular depth estimation model 18 is required depending on the individual conditions. The image distance acquisition unit 12 recognizes the type of endoscopic camera unit 2 and selects the appropriate monocular depth estimation model 18, thereby enabling accurate distance measurement.

[0040] For example, the image distance acquisition unit 12 includes a memory 17. The memory 17 stores a plurality of monocular depth estimation models 18 that have been trained for each lighting environment. The ranging calculation unit 16 acquires the monocular depth estimation model 18 to be used from the memory 17 by switching between them depending on the lighting conditions under which the image is captured.

[0041] 10, "Endoscope 1," "Endoscope 2," and "Endoscope 3" are prepared as available endoscopic camera units 2A, 2B, and 2C. Memory 17 stores "Learning Model 1," "Learning Model 2," and "Learning Model 3" as monocular depth estimation models 18A, 18B, and 18C corresponding to "Endoscope 1," "Endoscope 2," and "Endoscope 3."

[0042] 4. System Variations FIG. 11 shows variations of the information processing system.

[0043] 8, the image acquisition unit 11, the image distance acquisition unit 12, the image development unit 13, and the measurement processing unit 14 are implemented in a single processor. However, the image acquisition unit 11, the image distance acquisition unit 12, the image development unit 13, and the measurement processing unit 14 may be distributed across multiple processors.

[0044] 11 , the image acquisition unit 11 and the image development unit 13 are implemented in the image processing device 4, and the image distance acquisition unit 12 and the measurement processing unit 14 are implemented in the external PC or the server 5. By assigning the processing of each unit to an appropriate device depending on the processing load, the capabilities of the device can be effectively utilized.

[0045] 5. Example of Hardware Configuration FIG. 12 is a diagram illustrating an example of the hardware configuration of the information processing device 1. As shown in FIG.

[0046] The information processing of the information processing device 1 is realized by, for example, a computer 1000. The computer 1000 has a CPU (Central Processing Unit) 1100, a RAM (Random Access Memory) 1200, a ROM (Read Only Memory) 1300, a HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. The components of the computer 1000 are connected by a bus 1050.

[0047] The CPU 1100 operates and controls each component based on a program (program data 1450) stored in the ROM 1300 or the HDD 1400. For example, the CPU 1100 loads the program stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processing corresponding to the various programs.

[0048] The ROM 1300 stores boot programs such as a Basic Input Output System (BIOS) that is executed by the CPU 1100 when the computer 1000 starts up, as well as programs that depend on the hardware of the computer 1000 .

[0049] The HDD 1400 is a non-transitory computer-readable recording medium that non-temporarily records programs executed by the CPU 1100 and data used by such programs. Specifically, the HDD 1400 is a recording medium that records an information processing program according to an embodiment as an example of program data 1450.

[0050] The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (e.g., the Internet). For example, the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

[0051] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input device such as a keyboard or a mouse via the input / output interface 1600. The CPU 1100 also transmits data to an output device such as a display device, a speaker, or a printer via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading programs recorded on a predetermined recording medium. Examples of media include optical recording media such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disc), magneto-optical recording media such as an MO (Magneto-Optical Disk), tape media, magnetic recording media, and semiconductor memories.

[0052] For example, when the computer 1000 functions as the information processing device 1 according to the embodiment, the CPU 1100 of the computer 1000 executes an information processing program loaded onto the RAM 1200 to realize the functions of the aforementioned components. The HDD 1400 stores the information processing program, various models, and various data according to the present disclosure. The CPU 1100 reads and executes program data 1450 from the HDD 1400. Alternatively, the CPU 1100 may acquire these programs from another device via an external network 1550. The processing of the present disclosure may be performed by a processor other than the CPU 1100, such as a GPU (Graphics Processing Unit).

[0053] [6. Effects] The information processing device 1 has a RAW processing unit 15 and a distance measurement calculation unit 16. The RAW processing unit 15 acquires a normalized luminance image in which the luminance of each color is normalized from the RAW signal RD. The distance measurement calculation unit 16 acquires depth information DI of the subject SB from the normalized luminance image using a monocular depth estimation model 18 that has learned about the correlation between depth and luminance. In the information processing method disclosed herein, the processing of the information processing device 1 is executed by a computer 1000. A program disclosed herein causes the computer 1000 to realize the processing of the information processing device 1.

[0054] According to this configuration, by using normalized luminance, variations between colors based on wavelength characteristics are suppressed, and depth information DI can be obtained with high accuracy.

[0055] The information processing device 1 includes an image acquisition unit 11. The image acquisition unit 11 acquires an image signal acquired under specific lighting conditions that are the same as the lighting conditions used for the training data of the monocular depth estimation model 18 as a RAW signal RD.

[0056] According to this configuration, the learning results are reflected in the measurement results with high accuracy.

[0057] The specific lighting conditions indicate a lighting environment using a single point light source LS. The monocular depth estimation model 18 is a learning model that learns the correlation between depth and luminance based on the inverse square of illuminance.

[0058] According to this configuration, the depth information DI can be obtained with high accuracy based on the inverse square of the illuminance.

[0059] The monocular depth estimation model 18 is a learning model that is trained using a group of images that have a correlation between depth and brightness under specific lighting conditions.

[0060] This configuration allows for highly accurate measurements.

[0061] The information processing device 1 has a memory 17. The memory 17 stores a plurality of monocular depth estimation models 18 that have been trained for each lighting environment. The ranging calculation unit 16 acquires the monocular depth estimation model 18 to be used from the memory 17 by switching between them depending on the lighting conditions under which the image is captured.

[0062] This configuration enables accurate measurements in a variety of lighting environments.

[0063] The RAW processing unit 15 performs processing on the RAW signal RD to enhance the contours of the subject SB.

[0064] This configuration reduces errors in size measurements and the like.

[0065] The information processing device 1 has an image developing unit 13 and a measurement processing unit 14. The image developing unit 13 develops the RAW signal RD to generate an RGB image CI. The measurement processing unit 14 acquires the position of an observation target appearing in the RGB image CI.

[0066] According to this configuration, the position of the observation target can be determined with high accuracy based on the RGB image CI.

[0067] The measurement processing unit 14 acquires the size of the object to be observed based on the depth information DI of the object to be observed.

[0068] According to this configuration, the size of the object to be observed can be determined with high accuracy.

[0069] The measurement processing unit 14 reconstructs the two-dimensional shape and three-dimensional shape of the object taking the depth into account based on the depth information DI of the object.

[0070] According to this configuration, the two-dimensional and three-dimensional shapes of the object to be observed, taking into account the depth, can be determined with high accuracy.

[0071] The measurement processing unit 14 acquires two points included in the observation target as target points, and acquires the distance between the target points based on the depth information DI of each target point.

[0072] According to this configuration, the three-dimensional distance between the target points can be determined with high accuracy.

[0073] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0074] [Additional Notes] The present technology may also be configured as follows. (1) An information processing device including: a RAW processing unit that acquires a normalized luminance image in which the luminance of each color is normalized from a RAW signal; and a ranging calculation unit that acquires depth information of a subject from the normalized luminance image using a monocular depth estimation model that has learned about the correlation between depth and luminance. (2) The information processing device according to (1), including an image acquisition unit that acquires, as the RAW signal, an image signal acquired under specific lighting conditions that are the same as lighting conditions used for training data of the monocular depth estimation model. (3) The information processing device according to (2), wherein the specific lighting conditions indicate a lighting environment using a single point light source, and the monocular depth estimation model is a training model that has learned about the correlation between the depth and the luminance based on an inverse-squared measurement of illuminance. (4) The information processing device according to (2) or (3), wherein the monocular depth estimation model is a training model that has learned about the correlation between depth and luminance under the specific lighting conditions. (5) The information processing device according to any one of (2) to (4), further comprising a memory that stores multiple monocular depth estimation models trained for each lighting environment, and the ranging calculation unit switches and acquires the monocular depth estimation model to be used from the memory according to the lighting conditions under which image capture is performed. (6) The information processing device according to any one of (1) to (5), further comprising: a raw signal processing unit that performs processing to emphasize the contour of the subject on the raw signal; (7) The information processing device according to any one of (1) to (6), further comprising: an image development unit that develops the raw signal to generate an RGB image; and a measurement processing unit that acquires the position of an observation target appearing in the RGB image. (8) The information processing device according to (7), further comprising: a measurement processing unit that acquires the size of the observation target based on the depth information of the observation target; and (9) The information processing device according to (8), further comprising: a measurement processing unit that acquires the size of the observation target based on a magnification ratio used during image capture. (10) The information processing device according to (7), wherein the measurement processing unit reconstructs a three-dimensional shape of the observation target based on the depth information of the observation target.(11) The information processing device according to (7), wherein the measurement processing unit acquires an outer shape of the observation target projected onto a two-dimensional plane from a predetermined direction based on the depth information of the observation target. (12) The information processing device according to (7), wherein the measurement processing unit acquires two points included in the observation target as target points, and acquires a distance between the target points based on the depth information of each target point. (13) An information processing method executed by a computer, comprising: acquiring a normalized luminance image in which the luminance of each color is normalized from a RAW signal; and acquiring depth information of the subject from the normalized luminance image using a monocular depth estimation model that has learned about the correlation between depth and luminance. (14) A program that causes a computer to perform the steps of: acquiring a normalized luminance image in which the luminance of each color is normalized from a RAW signal; and acquiring depth information of the subject from the normalized luminance image using a monocular depth estimation model that has learned about the correlation between depth and luminance.

[0075] REFERENCE SIGNS LIST 1 Information processing device 11 Image acquisition unit 13 Image development unit 14 Measurement processing unit 15 RAW processing unit 16 Distance measurement calculation unit 17 Memory 18 Monocular depth estimation model CI RGB image DI Depth information RD RAW signal

Claims

1. An information processing device having: a RAW processing unit that acquires a normalized luminance image in which the luminance of each color is normalized from a RAW signal; and a ranging calculation unit that acquires depth information of a subject from the normalized luminance image using a monocular depth estimation model that has learned about the correlation between depth and luminance.

2. The information processing device according to claim 1, further comprising an image acquisition unit that acquires, as the RAW signal, an image signal acquired under specific lighting conditions that are the same as lighting conditions used for training data for the monocular depth estimation model.

3. The information processing device described in claim 2, wherein the specific lighting conditions indicate a lighting environment using a single point light source, and the monocular depth estimation model is a learning model that learns the correlation between the depth and the brightness based on the inverse square of illuminance.

4. The information processing device according to claim 2, wherein the monocular depth estimation model is a learning model trained using a group of images having a correlation between depth and luminance under the specific lighting conditions.

5. An information processing device as described in claim 2, having a memory that stores multiple monocular depth estimation models that have been trained for each lighting environment, and the ranging calculation unit switches and obtains the monocular depth estimation model to be used from the memory according to the lighting conditions under which the image is taken.

6. The information processing device according to claim 1, wherein the RAW processing section performs processing on the RAW signal to enhance the contours of the subject.

7. An information processing device according to claim 1, comprising: an image development unit that develops the RAW signal to generate an RGB image; and a measurement processing unit that acquires the position of an observation target that appears in the RGB image.

8. The information processing device according to claim 7, wherein the measurement processing unit acquires the size of the object to be observed based on the depth information of the object to be observed.

9. The information processing device according to claim 8, wherein the measurement processing unit acquires the size of the object to be observed based on a magnification ratio used during photography.

10. The information processing device according to claim 7, wherein the measurement processing unit reconstructs the three-dimensional shape of the object to be observed based on the depth information of the object to be observed.

11. The information processing device according to claim 7, wherein the measurement processing unit acquires the outer shape of the object of observation projected onto a two-dimensional plane from a predetermined direction based on the depth information of the object of observation.

12. The information processing device according to claim 7, wherein the measurement processing unit acquires two points included in the observation target as target points, and acquires the distance between the target points based on the depth information of each target point.

13. An information processing method executed by a computer, comprising: obtaining a normalized luminance image in which the luminance of each color is normalized from a RAW signal; and obtaining depth information of a subject from the normalized luminance image using a monocular depth estimation model that has learned about the correlation between depth and luminance.

14. A program that enables a computer to obtain a normalized luminance image in which the luminance of each color is normalized from a RAW signal, and obtain depth information of a subject from the normalized luminance image using a monocular depth estimation model that has learned about the correlation between depth and luminance.

Images