Information processing apparatus, information processing method, and program

Abstract

The information processing apparatus has a RAW processing unit and a range finding calculation unit. The RAW processing unit acquires a normalized luminance image in which luminance of each color is normalized from a RAW signal. The range finding calculation unit acquires depth information of a subject from the normalized luminance image by using a monocular depth estimation model trained for correlation between depth and luminance.

Description

Technical Field

[0001] This disclosure relates to an information processing apparatus, information processing method, and program. Background Technology

[0002] Endoscopic images are typically captured by a monocular camera mounted at the distal end of the endoscope. However, it is difficult to obtain depth information from images taken from a monocular camera. This makes it difficult to measure dimensions calculated based on depth and two-dimensional information (such as RGB images), and this hinders diagnosis and surgery.

[0003] List of citations

[0004] Patent documents

[0005] Patent Document 1: JP 2022-172654 A Summary of the Invention

[0006] Technical issues

[0007] Currently, attempts have been made to measure the size of objects directly by inserting a measuring tape through the jaws of clamps, or to project laser or grid-structured images, but such tasks require a high level of skill. Estimating depth information from endoscopic images using artificial intelligence (AI) has also been performed, but it is prone to errors due to subtle color variations and struggles to achieve high inference accuracy.

[0008] Therefore, this disclosure proposes an information processing device, information processing method and program capable of acquiring depth information with high precision.

[0009] Solution to the problem

[0010] According to this disclosure, an information processing apparatus is provided, comprising: a RAW processing unit configured to acquire a normalized luminance image of each color from a RAW signal, with the luminance normalized; and a distance measurement calculation unit configured to acquire depth information of a subject from the normalized luminance image using a monocular depth estimation model trained on the correlation between depth and luminance. Furthermore, according to this disclosure, an information processing method is provided, wherein the information processing of the information processing apparatus is performed by a computer, and a program for enabling the computer to perform the information processing of the information processing apparatus. Attached Figure Description

[0011] Figure 1 This is a diagram illustrating an overview of the distance measurement scheme of this disclosure.

[0012] Figure 2 This is a diagram illustrating an overview of the distance measurement scheme of this disclosure.

[0013] Figure 3This is a diagram illustrating an overview of the distance measurement scheme of this disclosure.

[0014] Figure 4 This is a diagram showing the inference results of depth information obtained by applying the depth estimation scheme of this disclosure to a monocular image.

[0015] Figure 5 This is a diagram showing the inference results of depth information obtained by applying the depth estimation scheme of this disclosure to a monocular image.

[0016] Figure 6 This is a diagram illustrating an example of imaging a rectangular plate using a monocular camera within an endoscope.

[0017] Figure 7 This is a diagram illustrating an example of imaging the incision site during implantation surgery.

[0018] Figure 8 This is a diagram illustrating an example of an information processing system used to implement the depth estimation scheme of this disclosure.

[0019] Figure 9 This is a diagram illustrating an example configuration of the image distance acquisition unit.

[0020] Figure 10 This diagram illustrates the switching of the learning model based on lighting conditions.

[0021] Figure 11 This is a diagram illustrating a variation of an information processing system.

[0022] Figure 12 This is a diagram illustrating an example of the hardware configuration of an information processing device. Detailed Implementation

[0023] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the following embodiments, the same components are indicated by the same reference numerals, and repeated descriptions thereof are omitted.

[0024] The description will proceed in the following order: 1. Overview of the distance measurement scheme disclosed herein 2. Configuration of information processing devices 3. Switching the learning model based on lighting conditions 4. System Variations 5. Hardware Configuration Example 6. Effects 1. Overview of the distance measurement scheme disclosed herein Figures 1 to 3 This is a diagram illustrating an overview of the distance measurement scheme of this disclosure.

[0025] The demand for depth estimation techniques is increasing for devices and applications such as surgical support robots and scopist robots. Methods for acquiring depth information, including those using structured light and triangulation techniques, and those using LiDAR sensors, are well-known.

[0026] These approaches can acquire high-precision depth information, but require specialized optical systems or dedicated sensors separate from the main optical system, leading to larger devices. This not only increases costs but also excludes system configurations such as endoscope-holding robots that utilize existing endoscopes. Depth estimation using stereo image matching, which does not require special illumination, is also an option, but it suffers from the significant limitation of being restricted to stereo endoscopes.

[0027] This disclosure is made in view of the above-mentioned problems. In this disclosure, a depth estimation scheme capable of acquiring high-precision depth information without using special device configurations is proposed. Figure 1 The example illustrates illumination conditions commonly used in the field of monocular endoscopy, employing a point light source (LS). A correlation exists between the brightness and depth of an image. For instance, in illumination environments using a point light source LS, a decrease in brightness proportional to the square of the distance is observed based on the inverse square law of illuminance. The correlation between brightness and depth can vary depending on the illumination conditions. In this disclosure, depth information is estimated from a RAW signal based on a pre-learned correlation between brightness and depth.

[0028] exist Figure 2 In the example, imaging of an object OB is performed under illumination using a point light source LS. The object OB has colors such as red, green, and blue. Although the brightness of each color is different due to wavelength characteristics (light source spectrum, color filter transmission characteristics, spectral reflectance characteristics of the object OB, etc.), depth information can be obtained from the corrected brightness when brightness correction (normalization) is performed to compensate for the wavelength characteristics.

[0029] Normalization means the process of uniformizing the brightness range of each color obtained under the same lighting conditions by adjusting the scaling factor of each color. Normalization is performed on the image signal (RAW signal, or signal obtained by performing noise suppression or correction on the RAW signal to improve distance detection) before development processing is performed. Normalization is performed on the image signal before development processing to avoid the original color and brightness information being greatly distorted by development processing.

[0030] exist Figure 3In the example, the brightness of each color of object OB is converted to white equivalent brightness through normalization. White equivalent brightness means the brightness expected when the object is white. In this disclosure, the process of converting the brightness of each color to white equivalent brightness is referred to as white equivalent brightness conversion. Figure 3 The left side shows the luminance distribution included in the RAW signal (luminance image before normalization). Figure 3 The right side shows the luminance distribution (normalized luminance image) after performing a white equivalent luminance conversion on the RAW signal.

[0031] Figure 4 and Figure 5 This is a diagram illustrating the inference results of depth information obtained by applying the depth estimation scheme of this disclosure to monocular images. Figure 4 The example shows an endoscopic image of the lesion site. Figure 5 The examples show monocular images from multiple color panels. In all examples, accurate depth information reflecting the luminance information included in the RAW signal was obtained.

[0032] Figure 6 This diagram illustrates an example of imaging a rectangular plate using a monocular camera within an endoscope. The plate is tilted relative to the optical axis of the monocular camera. The shape of the plate projected onto the monocular camera is trapezoidal. The position of the plate in the depth direction is estimated based on a normalized brightness image. By restoring the shape that takes depth into account, the planar shape of the rectangle is obtained.

[0033] Figure 7 This diagram illustrates an example of imaging the incision site during implantation surgery. In the procedure, a portion of the temporal bone is removed, and electrodes are inserted into the cochlea. The site, tilted and oriented along the depth of the incision, is covered by a thin plate (planar plate) made of cartilage or similar material with a planar shape. The site where the planar plate is to be installed is observed endoscopically. To measure the three-dimensional shape of the target and the contour shape (two-dimensional shape) of the site where the planar plate is to be installed, depth information is obtained using a normalized brightness image.

[0034] By taking depth into account, the accurate shape of the observed target is reconstructed. For example, the external shape of the observed target projected onto a two-dimensional plane from a predetermined direction can be obtained. In related art solutions, accurate depth information cannot be obtained from monocular images. Therefore, to obtain an accurate shape that takes depth into account, the experience of a doctor or the labor involved in actually manufacturing an object, inserting it into the affected area, and reshaping the affected area is required. According to the solution of this disclosure, an accurate shape can be derived without requiring such experience or labor from a doctor.

[0035] 2. Configuration of information processing devices

[0036] Figure 8 This is a diagram illustrating an example of an information processing system implementing the depth estimation scheme of this disclosure. The system acquires depth information, for example, from monocular endoscope images. The information processing system includes an information processing device 1, an endoscope camera unit 2, and an external input device 3.

[0037] The endoscope camera unit 2 includes a monocular camera and a light source device. Light output from the light source device is projected from the distal end of a light guide fiber toward the surgical site. The distal end of the fiber, which serves as the light projection part, acts as an illumination light source. The endoscope camera unit 2 uses the monocular camera to image the subject SB, which is illuminated by the illumination light. The light source device can project visible light as illumination light and can also project special light that can improve the visibility of the affected area. The endoscope camera unit 2 can be a flexible endoscope or a rigid endoscope.

[0038] Information processing device 1 acquires the RAW signal RD of the subject SB from endoscope camera unit 2. Information processing device 1 analyzes the RAW signal RD to obtain information such as the depth, size, shape, and distances between points of the subject SB. For example, information processing device 1 includes image acquisition unit 11, image distance acquisition unit 12, image development unit 13, and measurement processing unit 14.

[0039] Image acquisition unit 11 acquires RAW signal RD from endoscope camera unit 2 and outputs RAW signal RD to image distance acquisition unit 12 and image developing unit 13. RAW signal RD is an image signal in which information received from a single-lens reflex camera (image sensor) is recorded substantially as is. In RAW signal RD, large adjustments such as color and brightness that are performed in the developing process are not performed.

[0040] Image distance acquisition unit 12 analyzes RAW signal RD to obtain depth information DI of subject SB. Figure 9 This is a diagram illustrating an example configuration of the image distance acquisition unit 12. For example, the image distance acquisition unit 12 includes a RAW processing unit 15 and a distance measurement calculation unit 16.

[0041] RAW processing unit 15 acquires a normalized luminance image of each color from the RAW signal RD, with the luminance normalized. Normalization means a process that uniformizes the luminance range of each color obtained under the same lighting conditions by adjusting the scaling factor of each color. Normalization does not include large adjustments such as color or luminance performed during the development process. Therefore, the normalized luminance information includes highly reliable depth information DI based on the correlation between depth and luminance.

[0042] The RAW processing unit 15 can perform various correction processing operations on the RAW signal RD within a range that does not involve large changes in color or brightness that would interfere with the calculation of depth information DI. Figure 9 In the example, noise suppression, tone curve correction, and frequency correction are performed as correction operations.

[0043] Noise suppression removes noise that contributes to errors during distance measurement calculations. Tone curve correction matches the image sensor output to the characteristics of distance measurement calculations, resulting in a signal with good distance detection capabilities. Frequency correction enhances (boosts or multiplies) attenuated high-frequency signals, thereby emphasizing the contours of the subject (SB). By emphasizing the contours, errors in dimensional measurements of areas such as affected parts can be reduced.

[0044] Distance measurement calculation unit 16 uses a learning model trained on the correlation between depth and brightness (monocular depth estimation model 18: see...). Figure 10 Depth information (DI) of the subject (SB) is obtained from the normalized luminance image. As mentioned above, in the RAW signal RD, no significant adjustments such as color or luminance are performed as in the development process. Therefore, accurate depth information (DI) is obtained by normalizing the luminance of each color included in the RAW signal RD and applying the normalized luminance distribution (normalized luminance image) to the monocular depth estimation model 18.

[0045] The monocular depth estimation model 18 uses a set of images obtained under illumination conditions that match the actual imaging environment as learning data for learning. The learning can be supervised or self-supervised. When a RAW signal RD that has undergone correction processing, such as tone curve correction or frequency correction, is used as input to the monocular depth estimation model 18, a set of images that have undergone the same correction processing can be used as learning data.

[0046] exist Figure 9 In the example, depth information (DI) is acquired through a two-stage process of normalization and depth estimation. However, these processes can be performed centrally by a single AI. For example, it is difficult to explicitly obtain the equation for the white equivalent luminance conversion performed in normalization. Therefore, it is preferable that normalization is performed by a learned model (normalization model). By having the monocular depth estimation model 18 perform the action of the normalization model, depth information (DI) is directly acquired from the RAW signal RD. In this case, the distance measurement calculation unit 16 also functions as the RAW processing unit 15.

[0047] Preferably, the lighting conditions used in the endoscope camera unit 2 are consistent with the lighting conditions of the learning data used for the monocular depth estimation model 18. For example, the image acquisition unit 11 acquires an image signal as a RAW signal RD under specific lighting conditions that are the same as those used for the learning data of the monocular depth estimation model 18. The monocular depth estimation model 18 is a learning model trained using a set of images that have a correlation between depth and brightness under specific lighting conditions.

[0048] Illumination conditions can be those commonly used in actual imaging environments. For example, in an endoscope, the distal end of a thin fiber that guides light from a light source device serves as the light source. In this case, specific illumination conditions represent the illumination environment using a single point light source LS. The monocular depth estimation model18 is a learning model that learns the correlation between depth and luminance based on the inverse square law of illuminance.

[0049] The image developing unit 13 develops the RAW signal RD to generate an RGB image CI. Developing processes may include exposure adjustment, contrast adjustment, white balance adjustment, saturation adjustment, cropping, perspective adjustment, etc. The measurement processing unit 14 acquires the position of the observed target appearing in the RGB image CI. Based on the position information and depth information DI of the observed target, the measurement processing unit 14 estimates various types of information about the observed target and outputs the estimation results ES to an external device.

[0050] For example, the measurement processing unit 14 can obtain the size of the observed target based on the depth information DI of the observed target. The size calculation can be performed based on the size of the image sensor, the size of the corresponding portion projected onto the pixel, the focal length of the lens, etc. For example, the measurement processing unit 14 obtains the size of the observed target based on the magnification during imaging. The measurement processing unit 14 measures the size of the object in the endoscopic image and calculates the size by multiplying the measured size by the magnification corresponding to the depth information of the object.

[0051] The measurement processing unit 14 is capable of reconstructing the three-dimensional shape of the observed target based on the depth information DI of the observed target. The measurement processing unit 14 is also capable of acquiring the external shape of the observed target projected onto a two-dimensional plane from a predetermined direction based on the depth information DI of the observed target. Furthermore, the measurement processing unit 14 is capable of acquiring two points included in the observed target as target points, and of acquiring the distance between the target points based on the depth information DI of each target point.

[0052] External input device 3 inputs the location information of the point or range to be measured to measurement into measurement processing unit 14. The input can be performed manually by the user, or the information extracted from the RGB image CI through image analysis can be automatically input.

[0053] 3. Switching the learning model based on lighting conditions

[0054] Figure 10 This diagram illustrates the switching of the learning model based on lighting conditions.

[0055] Users can switch between and use multiple endoscope camera units 2 depending on the application. The conditions for light projection, optics, and imaging differ for each endoscope camera unit 2. Therefore, different monocular depth estimation models 18 are required depending on the specific conditions. The image distance acquisition unit 12 can perform accurate distance measurements by identifying the type of endoscope camera unit 2 and selecting the appropriate monocular depth estimation model 18.

[0056] For example, the image distance acquisition unit 12 includes a memory 17. The memory 17 stores multiple monocular depth estimation models 18 trained for various lighting environments. The distance measurement calculation unit 16 switches from the memory 17 and retrieves the monocular depth estimation model 18 to be used based on the lighting conditions during imaging.

[0057] exist Figure 10 In the example, "Endoscope 1", "Endoscope 2" and "Endoscope 3" are prepared as multiple available endoscope camera units 2A, 2B and 2C. In memory 17, "Learning Model 1", "Learning Model 2" and "Learning Model 3" are stored as monocular depth estimation models 18A, 18B and 18C corresponding to "Endoscope 1", "Endoscope 2" and "Endoscope 3".

[0058] 4. System Variations

[0059] Figure 11 This is a diagram illustrating a variation of an information processing system.

[0060] exist Figure 8 In the example, the image acquisition unit 11, the image distance acquisition unit 12, the image development unit 13, and the measurement processing unit 14 are implemented in a single processor. However, the image acquisition unit 11, the image distance acquisition unit 12, the image development unit 13, and the measurement processing unit 14 can be distributed across multiple processors.

[0061] For example, in Figure 11 In the example, the image acquisition unit 11 and the image development unit 13 are implemented in the image processing apparatus 4, while the image distance acquisition unit 12 and the measurement processing unit 14 are implemented by an external PC or server 5. By allocating the processing of each unit to the appropriate apparatus according to the processing load, the capabilities of the apparatus can be effectively utilized.

[0062] 5. Hardware Configuration Example

[0063] Figure 12This is a diagram illustrating an example of the hardware configuration of the information processing device 1.

[0064] Information processing in information processing device 1 is performed, for example, by computer 1000. Computer 1000 includes a central processing unit (CPU) 1100, random access memory (RAM) 1200, read-only memory (ROM) 1300, hard disk drive (HDD) 1400, communication interface 1500, and input / output interface 1600. The various units of computer 1000 are connected together via bus 1050.

[0065] The CPU 1100 operates based on the program (program data 1450) stored in the ROM 1300 or HDD 1400 and controls the various units. For example, the CPU 1100 expands the program stored in the ROM 1300 or HDD 1400 to the RAM 1200 and executes the processing corresponding to the various programs.

[0066] ROM 1300 stores boot programs such as the Basic Input / Output System (BIOS) executed by CPU 1100 when computer 1000 starts up, as well as programs that depend on the hardware of computer 1000.

[0067] HDD 1400 is a non-transitory computer-readable recording medium in which a program executed by CPU 1100 and the data used by the program are recorded non-transitory. For example, HDD 1400 is a recording medium that records an information processing program with an embodiment as an example of program data 1450.

[0068] Communication interface 1500 is an interface for connecting computer 1000 to an external network 1550 (e.g., the Internet). For example, CPU 1100 receives data from another device via communication interface 1500, or sends data generated by CPU 1100 to another device via communication interface 1500.

[0069] Input / output interface 1600 is an interface used to connect input / output device 1650 and computer 1000. For example, CPU 1100 receives data from input devices such as keyboard and mouse via input / output interface 1600. CPU 1100 also sends data to output devices such as monitor, speaker, and printer via input / output interface 1600. Input / output interface 1600 can be used as a media interface for reading programs recorded on a predetermined recording medium (medium). Medium examples include optical recording media such as digital multifunction discs (DVDs) or phase-change rewritable discs (PDs), magneto-optical recording media such as magneto-optical discs (MOs), magnetic tape media, magnetic recording media, semiconductor memory, etc.

[0070] For example, when the computer 1000 is used as an information processing device 1 according to the embodiment, the CPU 1100 of the computer 1000 executes an information processing program loaded onto the RAM 1200 to implement the functions of the aforementioned units. Furthermore, the HDD 1400 stores the information processing program of this disclosure, various models, and various types of data. Note that the CPU 1100 reads program data 1450 from the HDD 1400 and executes the program; however, in another example, the CPU 1100 may obtain these programs from another device via an external network 1550. The processing of this disclosure may be executed by a processor other than the CPU 1100 (such as a graphics processing unit (GPU)).

[0071] 6. Effects

[0072] Information processing apparatus 1 includes a RAW processing unit 15 and a distance measurement calculation unit 16. The RAW processing unit 15 acquires a normalized luminance image of each color from the RAW signal RD. The distance measurement calculation unit 16 uses a monocular depth estimation model 18 trained on the correlation between depth and luminance to acquire depth information DI of the subject SB from the normalized luminance image. In the information processing method of this disclosure, the processing of information processing apparatus 1 is performed by a computer 1000. The program of this disclosure enables computer 1000 to implement the processing of information processing apparatus 1.

[0073] According to this configuration, by using normalized brightness, the deviation of each color based on wavelength characteristics is suppressed, and depth information (DI) is obtained with high precision.

[0074] The information processing device 1 includes an image acquisition unit 11. The image acquisition unit 11 acquires an image signal as a RAW signal RD under specific lighting conditions that are the same as the lighting conditions used for the learning data of the monocular depth estimation model 18.

[0075] With this configuration, the learning results are accurately reflected in the measurement results.

[0076] Specific lighting conditions represent the lighting environment using a single point light source (LS). The monocular depth estimation model 18 is a learning model that learns the correlation between depth and luminance based on the inverse square law of illuminance.

[0077] Based on this configuration, depth information (DI) is obtained with high precision using the inverse square law of illumination.

[0078] The monocular depth estimation model18 is a learning model trained using a set of images that have a correlation between depth and brightness under specific lighting conditions.

[0079] Based on this configuration, high-precision measurements are performed.

[0080] The information processing device 1 includes a memory 17. The memory 17 stores multiple monocular depth estimation models 18 trained for various lighting environments. The distance measurement calculation unit 16 switches from the memory 17 and retrieves the monocular depth estimation model 18 to be used according to the lighting conditions during imaging.

[0081] This configuration enables high-precision measurements in various lighting environments.

[0082] RAW processing unit 15 performs processing on RAW signal RD to emphasize the outline of subject SB.

[0083] This configuration reduces errors in areas such as dimensional measurement.

[0084] The information processing apparatus 1 includes an image developing unit 13 and a measurement processing unit 14. The image developing unit 13 develops the RAW signal RD to generate an RGB image CI. The measurement processing unit 14 acquires the position of the observed target appearing in the RGB image CI.

[0085] Based on this configuration, the position of the observed target can be obtained with high precision based on the CI of the RGB image.

[0086] The measurement processing unit 14 obtains the size of the observed target based on the depth information DI of the observed target.

[0087] With this configuration, the size of the observed target can be obtained with high precision.

[0088] The measurement processing unit 14 restores the two-dimensional and three-dimensional shapes of the observed target based on the depth information of the observed target.

[0089] Based on this configuration, the two-dimensional and three-dimensional shapes of the observed target, taking into account depth, can be obtained with high precision.

[0090] The measurement processing unit 14 acquires two points included in the observed target as target points. The measurement processing unit 14 obtains the distance between the target points based on the depth information DI of each target point.

[0091] Based on this configuration, the three-dimensional distance between target points can be obtained with high precision.

[0092] The effects described in this specification are exemplary and not limiting, and other effects may also be obtained.

[0093] Replenish

[0094] This technology can also be configured as follows. (1)

[0096] An information processing device, comprising: The RAW processing unit is configured to acquire a normalized luminance image of each color from the RAW signal; and The distance measurement calculation unit is configured to obtain the depth information of the subject from the normalized brightness image using a monocular depth estimation model trained on the correlation between depth and brightness. (2)

[0098] The information processing apparatus according to (1) further includes: The image acquisition unit is configured to acquire an image signal under specific lighting conditions that are the same as those used for the learning data of the monocular depth estimation model, as the RAW signal. (3)

[0100] According to the information processing device described in (2), wherein, The specific lighting conditions refer to a lighting environment using a single point light source, and The monocular depth estimation model is a learning model that learns the correlation between depth and brightness based on the inverse square law of illumination. (4)

[0102] According to the information processing apparatus described in (2) or (3), wherein, The monocular depth estimation model is a learning model trained using a set of images that show the correlation between depth and brightness under the specific lighting conditions. (5)

[0104] The information processing apparatus according to any one of (2) to (4) further includes: The memory is configured to store multiple monocular depth estimation models learned for various lighting environments. The distance measurement calculation unit switches from the memory and retrieves the monocular depth estimation model to be used based on the illumination conditions during imaging. (6)

[0106] The information processing apparatus according to any one of (1) to (5), wherein, The RAW processing unit performs processing on the RAW signal to emphasize the outline of the subject. (7)

[0108] The information processing apparatus according to any one of (1) to (6) further includes: An image developing unit is configured to develop the RAW signal to generate an RGB image; and The measurement processing unit is configured to acquire the position of the observed target appearing in the RGB image. (8)

[0110] According to the information processing apparatus described in (7), wherein, The measurement and processing unit obtains the size of the observed target based on the depth information of the observed target. (9)

[0112] According to the information processing apparatus described in (8), wherein, The measurement and processing unit obtains the size of the observed target based on the magnification during imaging. (10)

[0114] According to the information processing apparatus described in (7), wherein, The measurement and processing unit reconstructs the three-dimensional shape of the observed target based on the depth information of the observed target. (11)

[0116] According to the information processing apparatus described in (7), wherein, The measurement and processing unit acquires the external shape of the observed target projected onto a two-dimensional plane from a predetermined direction based on the depth information of the observed target. (12)

[0118] According to the information processing apparatus described in (7), wherein, The measurement processing unit acquires two points in the observed target as target points, and obtains the distance between the target points based on the depth information of each target point. (13)

[0120] An information processing method executed by a computer, the method comprising: Obtain normalized brightness images of each color from the RAW signal; and Depth information of the subject is obtained from the normalized brightness image using a monocular depth estimation model trained on the correlation between depth and brightness. (14)

[0122] A program for enabling a computer to perform the following processes: Obtain normalized brightness images of each color from the RAW signal; and Depth information of the subject is obtained from the normalized brightness image using a monocular depth estimation model trained on the correlation between depth and brightness.

[0123] List of reference numerals

[0124] 1. Information processing device

[0125] 11 Image Acquisition Unit

[0126] 13 Image developing unit

[0127] 14 Measurement and Processing Unit

[0128] 15 RAW processing units

[0129] 16 Distance Measurement and Calculation Unit

[0130] 17. Memory

[0131] 18 Monocular Depth Estimation Model

[0132] CI RGB image

[0133] DI depth information

[0134] RD RAW signal

Claims

1. An information processing apparatus, comprising: The RAW processing unit is configured to acquire a normalized brightness image of each color from the RAW signal, with the brightness normalized. as well as The distance measurement calculation unit is configured to obtain the depth information of the subject from the normalized brightness image using a monocular depth estimation model trained on the correlation between depth and brightness.

2. The information processing apparatus according to claim 1, further comprising: The image acquisition unit is configured to acquire an image signal under specific lighting conditions that are the same as those used for the learning data of the monocular depth estimation model, as the RAW signal.

3. The information processing apparatus according to claim 2, wherein, The specific lighting conditions refer to a lighting environment using a single point light source, and The monocular depth estimation model is a learning model that learns the correlation between depth and brightness based on the inverse square law of illumination.

4. The information processing apparatus according to claim 2, wherein, The monocular depth estimation model is a learning model trained using a set of images that show the correlation between depth and brightness under the specific lighting conditions.

5. The information processing apparatus according to claim 2, further comprising: The memory is configured to store multiple monocular depth estimation models learned for various lighting environments. The distance measurement calculation unit switches from the memory and retrieves the monocular depth estimation model to be used based on the illumination conditions during imaging.

6. The information processing apparatus according to claim 1, wherein, The RAW processing unit performs processing on the RAW signal to emphasize the outline of the subject.

7. The information processing apparatus according to claim 1, further comprising: An image developing unit is configured to develop the RAW signal to generate an RGB image; as well as The measurement processing unit is configured to acquire the position of the observed target appearing in the RGB image.

8. The information processing apparatus according to claim 7, wherein, The measurement and processing unit obtains the size of the observed target based on the depth information of the observed target.

9. The information processing apparatus according to claim 8, wherein, The measurement and processing unit obtains the size of the observed target based on the magnification during imaging.

10. The information processing apparatus according to claim 7, wherein, The measurement and processing unit reconstructs the three-dimensional shape of the observed target based on the depth information of the observed target.

11. The information processing apparatus according to claim 7, wherein, The measurement and processing unit acquires the external shape of the observed target projected onto a two-dimensional plane from a predetermined direction based on the depth information of the observed target.

12. The information processing apparatus according to claim 7, wherein, The measurement processing unit acquires two points in the observed target as target points, and obtains the distance between the target points based on the depth information of each target point.

13. An information processing method executed by a computer, the method comprising: Obtain normalized brightness images of each color from the RAW signal; as well as Depth information of the subject is obtained from the normalized brightness image using a monocular depth estimation model trained on the correlation between depth and brightness.

14. A program for enabling a computer to perform the following processes: Obtain normalized brightness images of each color from the RAW signal; and Depth information of the subject is obtained from the normalized brightness image using a monocular depth estimation model trained on the correlation between depth and brightness.

Images