Information processing device, information processing method, and program

Abstract

To provide a technology for appropriately acquiring depth information of an image in which a plurality of virtual objects including semitransparent material are drawn.SOLUTION: An information processing device includes rendering means which acquires first depth information being depth information of a first image in which a plurality of virtual objects including semitransparent material are arranged, and generates the first image on the basis of the first depth information, an acquisition means which acquires second depth information indicating the depth of the semitransparent material in the first image, and correction means which corrects the first depth information on the basis of the second depth information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] In the metaverse, users can create avatars, which are their own avatars, and interact with others and shop in the virtual space. In the metaverse, there is a need to use translucent materials (semi-transparent virtual objects) to achieve more original expressions.

[0003] On the other hand, there are challenges in drawing translucent materials. In general CG rendering processes, in order to efficiently draw a large number of materials (virtual objects), the depth information of the foreground of each pixel is managed, and the drawing process of materials that exist behind the position indicated by the depth information is omitted. However, because translucent materials have transparency, materials that exist behind the translucent materials also need to be drawn. For this reason, the depth information acquired during the rendering process does not reflect the depth of the translucent materials. In addition, in the HMD, which is a device for experiencing the metaverse, there are multiple image processes that use the depth of CG images. If accurate depth information is not available, the quality of the results of these image processes may deteriorate, which may impair the user's experience.

[0004] In Patent Document 1, a threshold is set to determine whether a material is semi-transparent, and materials with transparency below the threshold are treated as opaque materials, and materials with transparency above the threshold are treated as semi-transparent materials. In this case, it is not possible to render a virtual object that exists behind a material with transparency below the threshold. On the other hand, it is possible to obtain the depth of a material with a transparency below the threshold. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2008-310377 A Summary of the Invention [Problem to be solved by the invention]

[0006] In Patent Document 1, a threshold is set for transparency, and materials with transparency equal to or lower than the threshold are treated as opaque materials. This makes it possible to treat materials with transparency equal to or lower than the threshold as materials from which depth can be obtained. However, the depth of materials with transparency higher than the threshold cannot be reflected in the depth information of a CG image.

[0007] Therefore, an object of the present invention is to provide a technique for appropriately acquiring depth information of an image in which multiple virtual objects including translucent materials are rendered. [Means for solving the problem]

[0008] One aspect of the present invention is a method for producing a composition comprising the steps of: a rendering means for acquiring first depth information, which is depth information of a first image in which a plurality of virtual objects including a translucent material are arranged, and generating the first image based on the first depth information; obtaining means for obtaining second depth information indicative of a depth of the translucent material in the first image; a correction means for correcting the first depth information based on the second depth information; The information processing device is characterized by having:

[0009] One aspect of the present invention is a method for producing a composition comprising the steps of: a rendering step of acquiring first depth information, which is depth information of a first image in which a plurality of virtual objects including a translucent material are arranged, and generating the first image based on the first depth information; acquiring second depth information indicative of a depth of the translucent material in the first image; a correcting step of correcting the first depth information based on the second depth information; The information processing method is characterized by having the following features. Effect of the Invention

[0010] According to the present invention, it is possible to appropriately acquire depth information of an image in which a plurality of virtual objects including semi-transparent materials are rendered. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a hardware configuration diagram of a correction system according to a first embodiment. [Diagram 2] FIG. 1 is a functional block diagram of a correction system according to a first embodiment. [Diagram 3] 4 is a flowchart of a process of a depth acquisition unit according to the first embodiment. [Figure 4] 5 is a flowchart of a process of a depth correction unit according to the first embodiment. [Diagram 5] FIG. 2 is a diagram showing the internal configuration of an image processing unit according to the first embodiment. [Figure 6] FIG. 11 is a functional block diagram of a correction system according to a second embodiment. [Figure 7] FIG. 11 is a diagram showing the internal configuration of a depth correction unit according to the second embodiment. [Figure 8] 11 is a flowchart of a process of a depth selection unit according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

[0013] <Embodiment 1> 1 is a hardware configuration diagram of a correction system 100. The correction system 100 is an information processing device such as a computer, a smartphone, a head mounted display (HMD), a digital camera (imaging device), or other home appliances. The correction system 100 has a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, and an interface unit 104. Each component is connected by a bus 105.

[0014] The CPU 101 controls the entire correction system 100 in accordance with a program stored in the ROM 102 or a program loaded into the RAM 103 .

[0015] The ROM 102 is a read-only memory that stores a boot program, firmware, various processing programs, and various data.

[0016] The RAM 103 is a work memory that temporarily stores programs and data for processing by the CPU 101. Various processing programs and data are loaded into the RAM 103 by the CPU 101.

[0017] The interface unit 104 is an interface for communicating with an external device via a network, and transmits and receives data via the network.

[0018] 2 is a functional block diagram of the correction system 100 in the first embodiment. In the first embodiment, the correction system 100 is a display system (HMD system) that displays a virtual reality space or an augmented reality space, but may be other systems. The correction system 100 includes an input information storage unit 200, a rendering processing unit 201, an image storage unit 202, a depth storage unit 203, a depth acquisition unit 204, a semi-transparent depth storage unit 205, a depth correction unit 206, a corrected depth storage unit 207, an image processing unit 208, and a composite image storage unit 209.

[0019] The input information storage unit 200 stores input information necessary for rendering a plurality of virtual objects (CG) including semi-transparent materials (virtual objects having transparency). The input information includes model data of the virtual objects, position information of the virtual objects, and viewpoint information (information such as position, angle of view, and resolution) when rendering the virtual objects.

[0020] The rendering processing unit 201 executes rendering processing based on the input information acquired from the input information storage unit 200. Specifically, first, the rendering processing unit 201 acquires (generates) depth information of a CG image, which is an image in which multiple virtual objects including semi-transparent materials (virtual objects having transparency) are arranged, based on the input information. After acquiring (generating) the depth information, the rendering processing unit 201 generates a CG image based on the input information and the depth information. The depth information is an image in which each pixel indicates the depth to a subject. Furthermore, the depth information of a CG image is information that indicates the depth of each pixel (each position) of the CG image.

[0021] Here, in order to generate CG images efficiently, the rendering processing unit 201 generates the CG image so that, for each pixel, only the virtual object at the depth indicated by the depth information and the virtual object located in front of the depth must be rendered. Therefore, since virtual objects that exist behind the semi-transparent material must also be rendered in the CG image, the depth information must indicate the depth of the virtual object. Therefore, the rendering processing unit 201 acquires the depth information while ignoring the presence of the semi-transparent material.

[0022] The image storage unit 202 stores the CG image (two-dimensional image) generated by the rendering processing unit 201.

[0023] The depth storage unit 203 stores depth information of the CG image acquired by the rendering processing unit 201 (hereinafter referred to as “CG depth information”).

[0024] The depth acquisition unit 204 acquires depth information of only the semi-transparent materials in the CG image (hereinafter referred to as "semi-transparent depth information") among a plurality of virtual objects rendered in the CG image based on the input information. For example, the depth acquisition unit 204 acquires the semi-transparent depth information by performing rendering while treating the semi-transparent materials as non-transparent materials based on the input information. Details of the processing by the depth acquisition unit 204 will be described later with reference to the flowchart of FIG. 3.

[0025] The semi-transparent depth storage unit 205 stores semi-transparent depth information.

[0026] The depth correction unit 206 corrects the CG depth information based on the semi-transparent depth information. Details of the process of the depth correction unit 206 will be described later with reference to the flowchart of FIG.

[0027] The corrected depth holding unit 207 holds the CG depth information corrected by the depth correcting unit 206 (hereinafter referred to as “corrected information”).

[0028] The image processing unit 208 synthesizes a specific image (for example, a real image captured by an imaging device of a real space) with a CG image based on the corrected information. generates a composite image. The image processing unit 208 may execute any processing based on the corrected information as long as it is processing related to a CG image. For example, the image processing unit 208 may estimate the position of each virtual object in the next frame based on the corrected information and the degree of movement of the user's head, and correct the CG image based on the estimation result. Alternatively, the image processing unit 208 may generate an image in which each pixel of the CG image is colored with a color according to the magnitude of the depth indicated by the depth information.

[0029] The composite image storage unit 209 stores the composite image generated by the image processing unit 208 .

[0030] The process of the depth acquisition unit 204 will be described in detail with reference to the flowchart of FIG.

[0031] In step S300, the depth acquisition unit 204 initializes a depth map, which is depth information indicating the depths of all semi-transparent materials in a CG image. Specifically, the depth acquisition unit 204 sets a value equivalent to infinity to each of all pixels in the depth map.

[0032] In step S301, the depth acquisition unit 204 generates a list of translucent materials indicating one or more translucent materials that have been rendered, based on the input information held by the input information holding unit 200.

[0033] In step S302, the depth acquisition unit 204 selects one translucent material from the list of translucent materials generated in step S301. The depth acquisition unit 204 executes the process of step S303 and subsequent steps for the selected translucent material (hereinafter referred to as the "selected material"). When the process of step S303 and subsequent steps has been executed for all translucent materials in the list of translucent materials, the process of this flowchart ends. Then, the depth acquisition unit 204 treats the depth map as translucent depth information.

[0034] In step S303, the depth acquisition unit 204 acquires depth information of the selected material based on the input information. The depth acquisition unit 204 acquires depth information only of pixels of the selected material that are drawn in the CG image. The depth acquisition unit 204 does not acquire depth information of pixels that are not drawn in the CG image.

[0035] In step S304, the depth acquisition unit 204 compares, for each pixel of the selected material in the CG image, the "depth indicated by the depth information of the selected material acquired in step S303" with the "depth indicated by the depth map" at the same pixel coordinates. If, for each pixel coordinate, the "depth indicated by the depth information of the selected material" is smaller (the distance is shorter) than the "depth indicated by the depth map," the depth acquisition unit 204 updates the "depth indicated by the depth map" to the "depth indicated by the depth information of the selected material."

[0036] The process of the depth correction unit 206 will be described in detail with reference to the flowchart of FIG.

[0037] In step S400, the depth correction unit 206 determines whether the processing from step S401 onwards has been performed for all pixels of the semi-transparent depth information. The screen resolution (aspect ratio and number of pixels) of the CG depth information and the semi-transparent depth information are the same. If it is determined that the processing from step S401 onwards has been performed for all pixels of the semi-transparent depth information, the processing of this flowchart ends. If it is determined that the processing from step S401 onwards has not been performed for at least one pixel of the semi-transparent depth information, the processing from step S401 onwards is performed for one pixel for which the processing from step S401 onwards has not yet been performed. Hereinafter, the pixel (pixel of the semi-transparent depth information) that is the target of the processing from step S401 onwards is referred to as the "selected pixel."

[0038] In step S401, the depth correction unit 206 performs the process of calculating the CG depth information at the coordinates of the selected pixel. It is determined whether the depth indicated by the selected pixel of the semi-transparent depth information (hereinafter referred to as "CG depth") is less than the depth indicated by the selected pixel of the semi-transparent depth information (hereinafter referred to as "selected depth"). If it is determined that the CG depth is less than the selected depth, the process proceeds to step S402. If it is determined that the CG depth is equal to or greater than the selected depth, the process proceeds to step S403.

[0039] In step S402, the depth correction unit 206 adopts the CG depth as the depth of the corrected information at the coordinates of the selected pixel. In step S403, the depth correction unit 206 adopts the selected depth as the depth of the corrected information at the coordinates of the selected pixel.

[0040] That is, in steps S401 to S403, the depth corrector 206 adopts the smaller of the CG depth and the selected depth (the one indicating the foreground) as the depth of the corrected information at the coordinates of the selected pixel.

[0041] 5 is a diagram for explaining a specific example of processing by the image processing unit 208. The image processing unit 208 has an actual image storage unit 500, an actual depth storage unit 501, and a synthesis unit 502.

[0042] The real-life image storage unit 500 stores images (hereinafter referred to as "real-life images") captured by an imaging device (an imaging device mounted in front of an HMD or the like) of a real space.

[0043] The real-image depth storage unit 501 stores depth information of a real-image. The depth information of a real-image can be acquired by a measurement value of a distance sensor, or by stereo matching using a plurality of images captured from a plurality of viewpoints.

[0044] The synthesis unit 502 synthesizes a real-life image and a CG image (an image in which multiple virtual objects including translucent materials are arranged) to generate a synthetic image. The synthesis unit 502 compares the "depth indicated by the depth information of the real-life image" with the "depth indicated by the corrected depth information" for each pixel. For pixels in the synthetic image where the "depth indicated by the depth information of the real-life image" is smaller, the synthesis unit 502 adopts the pixel value of the real-life image. For pixels in the synthetic image where the "depth indicated by the corrected depth information" is smaller, the synthesis unit 502 adopts the pixel value of the CG image. In this way, the synthesis unit 502 synthesizes the real-life image and the CG image while taking into consideration an appropriate depth.

[0045] As described above, in the first embodiment, the correction system 100 obtains depth information of a semi-transparent material by a process separate from the normal rendering process. Then, the correction system 100 corrects the depth information of the CG image obtained by the normal rendering process based on the depth information of the semi-transparent material. This makes it possible to obtain depth information of the CG image that takes the semi-transparent material into consideration. As a result, it becomes possible to prevent a decrease in the quality of the image processing.

[0046] <Embodiment 2> In the first embodiment, the correction system 100 obtains depth information of a translucent material by a process separate from a normal rendering process, and corrects the depth information obtained by the normal rendering process. This allows the correction system 100 to obtain depth information that takes the translucent material into consideration. However, in the first embodiment, the correction system 100 needs to refer to input information to obtain the depth information of the translucent material.

[0047] In the second embodiment, the correction system 100 acquires depth information of a semi-transparent material without referring to input information. Fig. 6 is a functional block diagram showing an example of the configuration of the correction system 100 in the second embodiment. The correction system 100 includes an input information storage unit 200, a rendering processing unit 201, an image storage unit 202, a depth storage unit 203, an image processing unit 208, a composite image storage unit 209, a depth acquisition unit 600, a storage unit 601, a depth correction unit 602, and a corrected depth storage unit 603.

[0048] The depth acquisition unit 600 acquires depth information of the CG image based on the CG image generated by the rendering processing unit 201. Specifically, the depth acquisition unit 600 acquires depth information of the CG image by stereo matching processing using a plurality of CG images in which a virtual object is viewed from a plurality of different viewpoints (for example, a CG image in which a virtual object is viewed from a left eye viewpoint and a CG image in which a virtual object is viewed from a right eye viewpoint). The depth acquisition unit 600 may acquire depth information of the CG image based on the CG image using a deep learning model (deep learning information) that has been learned in advance. For example, the depth acquisition unit 600 executes a depth estimation process based on a single image using deep learning. Hereinafter, the depth information acquired based on the CG image is referred to as "base information". The base information is acquired based on a CG image in which a semi-transparent material is drawn, and therefore includes depth information of the semi-transparent material. However, the accuracy (precision) of the depth of a virtual object other than a semi-transparent material is higher in the CG depth information acquired based on input information than in the base information.

[0049] The holding unit 601 holds the base information acquired by the depth acquisition unit 600 .

[0050] The depth correction unit 602 corrects the CG depth information based on the CG image and the base information, thereby obtaining corrected information.

[0051] The corrected depth storage unit 603 stores the corrected information.

[0052] 7 is a functional block diagram illustrating the processing of the depth correction unit 602. The depth correction unit 602 includes a segmentation processing unit 700, a region information holding unit 701, and a depth selection unit 702.

[0053] The segmentation processing unit 700 executes image segmentation processing on the CG image. As a result, the segmentation processing unit 700 divides the CG image in which multiple virtual objects are drawn into regions for each virtual object. Then, the segmentation processing unit 700 labels each divided region.

[0054] The region information storage unit 701 stores information on each region labeled by the segmentation processing unit 700 (hereinafter, referred to as a "mask region").

[0055] The depth selection unit 702 generates corrected information based on the CG depth information and the base information.

[0056] The processing of the depth selection unit 702 in the second embodiment will be described with reference to the flowchart of Fig. 8. The processing of the depth selection unit 702 is not limited to the processing of the flowchart of Fig. 8, and any processing may be adopted.

[0057] In step S800, the depth selection unit 702 determines whether or not the processing of step S801 and subsequent steps has already been performed on all mask regions held by the region information holding unit 701. If it is determined that the processing of step S801 and subsequent steps has already been performed on all mask regions, the processing of this flowchart ends. If it is determined that the processing of step S801 and subsequent steps has not been performed on at least one of all mask regions, the depth selection unit 702 selects one mask region from one or more mask regions determined that the processing of step S801 and subsequent steps has not been performed. Here, the mask region selected by the depth selection unit 702 is called a selected region. Then, the process proceeds to step S801.

[0058] In step S801, the depth selection unit 702 calculates an average depth A1 indicated by the CG depth information in the selected region. In step S802, the depth selection unit 702 calculates an average depth A1 indicated by the CG depth information in the selected region. Then, an average value A2 of the depths indicated by the base information in the area is calculated.

[0059] It should be noted that the "average depth value" in steps S801 and S802 may be any value that indicates the depth of the selected region, such as the "median depth value" or the "sum of depth values."

[0060] In step S803, the depth selection unit 702 determines whether the absolute value A3 of the difference between the average value A1 and the average value A2 is greater than a threshold value σ. If it is determined that the absolute value A3 is equal to or less than the threshold value σ, the process proceeds to step S804. If it is determined that the absolute value A3 is greater than the threshold value σ, the process proceeds to step S805. The threshold value σ may be a fixed value (for example, 1000 mm) or may be changeable by the image processing unit 208.

[0061] In step S804, the depth selection unit 702 substitutes (adopts) the depth of the selected area in the CG depth information for the depth of the selected area in the corrected information.

[0062] In step S805, the depth selection unit 702 substitutes (adopts) the depth of the selected region in the base information for the depth of the selected region in the corrected information. Here, if it is determined in step S803 that the absolute value A3 is greater than the threshold value σ, the selected region is likely to be a region of a translucent material. Therefore, in step S805, the depth selection unit 702 adopts the depth of the selected region in the base information, which includes the depth of the translucent material.

[0063] In this way, the correction system 100 corrects the CG depth information based on the CG image, making it possible to obtain depth information that takes into account the depth of the semi-transparent material.

[0064] Although the present invention has been described in detail based on the preferred embodiments, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Parts of the above-described embodiments may be combined as appropriate.

[0065] Also, in the above, "If A is equal to or greater than B, proceed to step S1, and if A is smaller (lower) than B, proceed to step S2" may be read as "If A is greater (higher) than B, proceed to step S1, and if A is equal to or less than B, proceed to step S2." Conversely, "If A is greater (higher) than B, proceed to step S1, and if A is equal to or less than B, proceed to step S2" may be read as "If A is greater (higher) than B, proceed to step S1, and if A is smaller (lower) than B, proceed to step S2." Therefore, unless a contradiction occurs, "equal to or greater than A" may be read as "equal to or greater than A (high; long; many)," and "equal to or less than A" may be read as "equal to or less than A (low; short; few)." And, "equal to or greater than A" may be read as "equal to or greater than A," and "equal to or less than A" may be read as "equal to or less than A."

[0066] Each functional unit in each of the above embodiments (variations) may or may not be individual hardware. The functions of two or more functional units may be realized by common hardware. Each of a plurality of functions of one functional unit may be realized by individual hardware. Two or more functions of one functional unit may be realized by common hardware. Furthermore, each functional unit may or may not be realized by hardware such as an ASIC, FPGA, or DSP. For example, the device may have a processor and a memory (storage medium) in which a control program is stored. Then, the functions of at least some of the functional units of the device may be realized by the processor reading and executing the control program from the memory.

[0067] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) for implementing one or more of the functions.

[0068] The disclosure of the above embodiments includes the following configurations, methods, and programs. (Configuration 1) a rendering means for acquiring first depth information, which is depth information of a first image in which a plurality of virtual objects including a translucent material are arranged, and generating the first image based on the first depth information; obtaining means for obtaining second depth information indicative of a depth of the translucent material in the first image; a correction means for correcting the first depth information based on the second depth information; 13. An information processing device comprising: (Configuration 2) the rendering means acquires the first depth information while ignoring the presence of the semi-transparent material in the plurality of virtual objects. The information processing apparatus according to Configuration 1, characterized in that... (Configuration 3) Each of the first depth information and the second depth information is information indicating the depth of a plurality of positions in the first image, For each of the plurality of positions in the first image, the correction means adopts, as the depth indicated by the corrected first depth information, the smaller one of the depth indicated by the first depth information and the depth indicated by the second depth information. The information processing apparatus according to Configuration 1 or 2, characterized in that... (Configuration 4) Each of the first depth information and the second depth information is information indicating the depth of the plurality of virtual objects in the first image, For each of the plurality of virtual objects, when the difference between the depth indicated by the first depth information and the depth indicated by the second depth information is greater than a threshold value, the correction means adopts, as the depth indicated by the corrected first depth information, the depth indicated by the second depth information. The information processing apparatus according to Configuration 1 or 2, characterized in that... (Configuration 5) The acquisition means acquires the second depth information based on the first image generated by the rendering means. The information processing apparatus according to any one of Configurations 1 to 4, characterized in that... (Configuration 6) The acquisition means acquires the second depth information based on a plurality of images of the plurality of virtual objects viewed from different viewpoints. The information processing apparatus according to any one of Configurations 1 to 5, characterized in that... (Configuration 7) The acquisition means acquires the second depth information by using a pre-trained deep learning model. The information processing apparatus according to any one of Configurations 1 to 6, characterized in that... (Configuration 8) The information processing apparatus further includes processing means for executing processing related to the first image based on the first depth information corrected by the correction means. 8. The information processing device according to any one of configurations 1 to 7. (Configuration 9) The processing means synthesizes the first image and the second image based on the first depth information. 9. The information processing device according to configuration 8. (method) a rendering step of acquiring first depth information, which is depth information of a first image in which a plurality of virtual objects including a translucent material are arranged, and generating the first image based on the first depth information; acquiring second depth information indicative of a depth of the translucent material in the first image; a correcting step of correcting the first depth information based on the second depth information; 13. An information processing method comprising: (program) A program for causing a computer to function as each of the means of the information processing device according to any one of configurations 1 to 9. [Explanation of symbols]

[0069] 100: correction system (information processing device), 201: rendering processing unit, 204: depth acquisition unit, 206: Depth correction section

Claims

1. a rendering means for acquiring first depth information, which is depth information of a first image in which a plurality of virtual objects including a translucent material are arranged, and generating the first image based on the first depth information; obtaining means for obtaining second depth information indicative of a depth of the translucent material in the first image; a correcting means for correcting the first depth information based on the second depth information; 13. An information processing device comprising:

2. the rendering means acquires the first depth information while ignoring the presence of the semi-transparent material in the plurality of virtual objects; 2. The information processing apparatus according to claim 1,

3. Each of the first depth information and the second depth information is information indicating depths at a plurality of positions of the first image, the correction means adopts, for each of a plurality of positions of the first image, a smaller one of a depth indicated by the first depth information and a depth indicated by the second depth information as a depth indicated by the corrected first depth information.

3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

4. Each of the first depth information and the second depth information is information indicating a depth of the plurality of virtual objects in the first image, the correction means, when a difference between a depth indicated by the first depth information and a depth indicated by the second depth information is greater than a threshold, adopts the depth indicated by the second depth information as the depth indicated by the corrected first depth information for each of the plurality of virtual objects.

3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

5. The acquisition means acquires the second depth information based on the first image generated by the rendering means.

3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

6. the acquiring means acquires the second depth information based on a plurality of images obtained by viewing the plurality of virtual objects from a plurality of different viewpoints.

3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

7. The acquisition means acquires the second depth information by utilizing a deep learning model that has been trained in advance.

3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

8. The method further includes a processing unit that executes a process on the first image based on the first depth information corrected by the correction unit.

3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

9. The processing means synthesizes the first image and the second image based on the first depth information.

9. The information processing apparatus according to claim 8,

10. A first image including a plurality of virtual objects including a semi-transparent material, which is depth information of the first image, and generating the first image based on the first depth information. acquiring second depth information indicative of a depth of the translucent material in the first image; a correcting step of correcting the first depth information based on the second depth information; 13. An information processing method comprising:

11. A program for causing a computer to function as each of the means of the information processing device according to claim 1 or 2.

Images