Information processing device, information processing method, and program

Abstract

To enable the generation of training data that can correctly estimate depth information. [Solution] A set of image data for training, a set of defocus map data from the image data set, and a set of defocus GT data from these are acquired. A random sign inversion process is performed on the defocus map data set and the defocus GT data set according to a specified probability of occurrence to generate a sign-inverted defocus map and a sign-inverted defocus GT.

Description

Technical Field

[0001] The present invention particularly relates to an information processing apparatus, an information processing method, and a program suitable for use in generating learning data.

Background Art

[0002] In recent years, research in the field of image recognition has been actively conducted, and many methods for detecting object regions in images using neural networks have been proposed. RGB color images are often input to neural networks for image recognition. In addition, when depth information (depth direction information) is also input, object recognition considering three-dimensional information can be realized.

[0003] Therefore, a method has also been proposed that uses machine learning to estimate depth information. Patent Document 1 proposes a method in which depth information is generated by machine learning from an image captured using a monocular camera mounted on a vehicle or the like, and more accurate distance information can be obtained by correcting the depth information using the distance estimation result from the self-position.

[0004] On the other hand, many cameras equipped with a phase difference AF function for directly calculating the deviation amount (defocus amount) from the imaging position to the imaging surface based on two images with parallax have become widespread. When estimating depth information, such a defocus amount from the imaging position to the imaging surface can also be considered as a kind of depth information. Therefore, it is also possible to estimate the depth information (defocus amount) of a subject using a neural network from an image and a depth map (defocus map).

[0005] In the field of image recognition, improving the performance of neural networks, particularly their generalization performance, generally requires a large amount of training data. Therefore, data augmentation (DA) techniques are often used as a method to improve generalization performance even with a small amount of training data. Here, DA techniques are methods that artificially process and augment training data. For example, in DA techniques, blurring, image compositing, rotation, translation, scaling, vertical / horizontal flipping, noise addition, color tone changes, and brightness changes can be applied to training image data. In addition to methods that process training image data, Patent Document 2 proposes a method that changes the drawing parameters in a three-dimensional model and generates training data (images and correct labels) for recognition based on the changed drawing parameters. [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2023-108723 [Patent Document 2] Japanese Patent Publication No. 2018-163554 [Overview of the project] [Problems that the invention aims to solve]

[0007] To estimate depth information, it is necessary to train a neural network to learn the spatial relationships of objects in the depth direction in various scenes. However, if the training data for the neural network is biased, the neural network that estimates depth information may not be properly trained, potentially leading to the estimation of incorrect depth information.

[0008] In particular, when collecting training data using an imaging device, depending on the characteristics of the subject and environmental conditions, the collected images tend to be biased towards blurred images that are in focus in front of the subject or blurred images that are in focus behind the subject. For example, when photographing a subject such as a flying bird, it is difficult to focus on the fast-moving bird, so the image tends to focus on the background scene behind the subject. On the other hand, when photographing skiers in snowy weather conditions, the image tends to focus on the snow that is in front of the skiers.

[0009] Since the scenes in which subjects are generally photographed are diverse, using a large amount of training data consisting of scenes with a bias towards the direction of focus shifts, as in this case, increases the likelihood of inaccurate depth information being estimated.

[0010] In view of the aforementioned problems, the present invention aims to enable the generation of training data that can correctly estimate depth information. [Means for solving the problem]

[0011] The information processing device according to the present invention is characterized by comprising: acquisition means for acquiring image data, a depth map showing depth information of the image data, and a correct label relating to the depth information; generation means for generating an inverted depth map obtained by inverting the depth information in the depth map, and an inverted correct label obtained by inverting the depth information in the correct label; and output means for outputting training data including the image data, the inverted depth map, and the inverted correct label. [Effects of the Invention]

[0012] According to the present invention, it is possible to generate training data that can correctly estimate depth information. [Brief explanation of the drawing]

[0013] [Figure 1] This is a block diagram showing an example of the hardware configuration of a training data generation device. [Figure 2]This diagram illustrates the relationship between the amount of defocus and the amount of image shift in an imaging optical system. [Figure 3] This is a block diagram showing an example of the functional configuration of a learning data generation device according to the first embodiment. [Figure 4] This is a diagram illustrating the sign inversion process in the first embodiment. [Figure 5] This flowchart shows an example of the procedure for generating training data in the first embodiment. [Figure 6] This is a block diagram showing an example of the functional configuration of a learning data generation device according to the second embodiment. [Figure 7] This is a diagram illustrating the sign inversion process in the second embodiment. [Figure 8] This flowchart shows an example of the procedure for generating training data in the second embodiment. [Modes for carrying out the invention]

[0014] Embodiments of the present invention will be described below with reference to the drawings. However, the present invention is not limited to the embodiments described below, and various forms that do not depart from the spirit of the invention are also included. Furthermore, each embodiment described below is merely one embodiment of the present invention, and it is possible to combine each embodiment as appropriate.

[0015] (First Embodiment) Figure 1 is a block diagram showing an example of the hardware configuration of a learning data generation device 300 that functions as an information processing device according to this embodiment. The learning data generation device 300 can be an information processing device such as a PC (personal computer), smartphone, or tablet terminal.

[0016] The CPU (Central Processing Unit) 100 executes various processes using computer programs and data stored in the RAM (Random Access Memory) 120. As a result, the CPU 100 controls the operation of the entire learning data generation device 300, and executes or controls various processes, which will be described later, as processes performed by the learning data generation device 300.

[0017] The ROM (Read-Only-Memory) 110 stores setting data of the learning data generation device 300, computer programs and data related to the startup and basic operations of the learning data generation device 300, and the like.

[0018] The RAM 120 has an area for storing computer programs and data loaded from the ROM 110 or the HDD (Hard Disk Drive) 130. Further, the RAM 120 has an area for storing computer programs and data received from the outside by the communication unit 160. Further, the RAM 120 has a work area used when the CPU 100 executes various processes. Thus, the RAM 120 can appropriately provide various areas.

[0019] The HDD 130 stores an OS (Operating System), computer programs and data for causing the CPU 100 to execute or control various processes, which will be described later, as processes performed by the learning data generation device 300, and the like. The computer programs and data stored in the HDD 130 are appropriately loaded into the RAM 120 according to the control by the CPU 100 and become processing targets by the CPU 100.

[0020] The input unit 140 is a user interface such as a keyboard, a mouse, a touch panel, etc., and various information and instructions can be input to the learning data generation device 300 by the user's operation. The display unit 150 is a display device having a liquid crystal screen or a touch panel, and can display the processing results of the CPU 100 as images, characters, etc. The display unit 150 may also be a projection device such as a projector that projects images and characters. The communication unit 160 is a communication interface for performing data communication with the outside world via wired and / or wireless networks such as LANs and the Internet. The CPU 100, ROM 110, RAM 120, HDD 130, input unit 140, display unit 150, and communication unit 160 are connected to the system bus 170.

[0021] The configuration shown in Figure 1 is an example of a hardware configuration applicable to the learning data generation device 300, and can be modified or altered as appropriate. For example, an external storage device may be used instead of or in addition to the HDD 130. The external storage device can be implemented, for example, by a media (recording medium) and an external storage drive for accessing the media. Examples of such media include flexible disks (FD), CD-ROMs, DVDs, USB memory, MOs, and flash memory. The external storage device may also be a server device connected to the learning data generation device 300 via a network.

[0022] Next, the amount of defocus used as depth information for the image in this embodiment will be described. Figure 2 is a diagram illustrating the relationship between the amount of defocus of the imaging optical system and the phase difference (image shift amount) between the first focus detection signal and the second focus detection signal acquired from the image sensor. An image sensor (not shown) is placed on the imaging surface 200, and the exit pupil of the imaging optical system is divided into two parts: a first pupil region 211 and a second pupil region 212.

[0023] Here, let |d| be the distance (magnitude) from the imaging position C of the light beam from the subject to the image sensor 200, where d is the amount of defocus. In a front-focus state where the imaging position C is on the subject side of the image sensor 200, the amount of defocus d is represented as negative (d<0), and in a back-focus state where the imaging position C is on the opposite side of the subject from the image sensor 200, the amount of defocus d is represented as positive (d>0). In a focused state where the imaging position C is on the image sensor 200, d=0. In the imaging optical system shown in Figure 2, the image sensor is in focus (d=0) for subject 221 and in front-focus state (d<0) for subject 222. Hereafter, the front-focus state (d<0) and the back-focus state (d>0) are collectively referred to as a defocus state (|d|>0).

[0024] In the front-focused state (d<0), the light beam from the subject 222 that passes through the first pupil region 211 (second pupil region 212) is focused and then spreads out with a width Γ1 (Γ2) centered on the centroid position G1 (G2) of the light beam, forming a blurred image on the imaging surface 200. This blurred image is received by the respective first focus detection pixels (second focus detection pixels) on the image sensor, and a first focus detection signal (second focus detection signal) is generated. In other words, the first focus detection signal (second focus detection signal) is a signal that represents the subject image in which the subject 222 is blurred by a blur width Γ1 (Γ2) at the centroid position G1 (G2) of the light beam on the imaging surface 200.

[0025] The blur width Γ1 (Γ2) of the subject image is roughly proportional to the magnitude of the defocus amount d |d|. Also, the image shift amount p between the first focus detection signal and the second focus detection signal is the difference in the centroid positions of the light beam (G1-G2), and the magnitude of the image shift amount p |p| is also roughly proportional to the magnitude of the defocus amount d |d|. Even in a back-focus state (d>0), the direction of image shift between the first focus detection signal and the second focus detection signal is the opposite of that in a front-focus state, but the same principle applies.

[0026] Thus, as the magnitude of the defocus amount |d| increases, the magnitude of the image shift amount |p| between the first focus detection signal and the second focus detection signal also increases. In this embodiment, focus detection is performed by an image plane phase difference detection method, which calculates the defocus amount d from the image shift amount p between the first focus detection signal and the second focus detection signal obtained using the image sensor. In this embodiment, the distribution of defocus amounts assigned to a region of a certain number of pixels on the image plane is called a defocus map. The unit of defocus amount is the product of the aperture value F and the allowable circle of confusion diameter δ in the optical system of the imaging device during image acquisition [Fδ]. In the following, the correct label may be referred to as GT (abbreviation for Ground Truth).

[0027] Figure 3 is a block diagram showing an example of the functional configuration of the learning data generation device 300 according to this embodiment. The CPU 100 executes the computer program corresponding to each part in Figure 3, thereby realizing the functions of each part. In this embodiment, an example is described in which all the parts shown in Figure 3 are realized by software (computer programs), but one or more of the functional parts shown in Figure 3 may be implemented by hardware such as dedicated circuits.

[0028] Figure 5 is a flowchart showing an example of the training data generation process by the training data generation device 300. The processing details of each part shown in Figure 3 will be explained below, following the processing procedure in Figure 5.

[0029] First, in step S501, the acquisition unit 310 acquires a group of learning image data 311, a group of defocus map data 312 of the image data group, and a group of defocus GT data 313 of these. The acquisition unit 310 may acquire these data groups from the HDD 130 or from an external source via the communication unit 160. In addition, the acquisition unit 310 may determine the amount of defocus in each image of the image data group 311 by the focus detection method of the imaging plane phase difference detection method described above, and acquire a defocus map and defocus GT based on the determined amount of defocus.

[0030] Figure 4 is a diagram illustrating specific examples of the image data group 311, the defocus map data group 312, and the defocus GT data group 313. The image 400 shown in Figure 4(a) shows a bird 401, which is the subject, and a background object 402. In this image 400, the subject bird failed to be in focus, and the area around the background object 402 is in focus, resulting in the subject bird 401 being out of focus.

[0031] When an image is captured, a defocus map 410 is generated centered on the subject area, as shown in Figure 4(b). Each rectangle in the defocus map 410 shown in Figure 4(b) represents a cell in the defocus map 410, and the defocus map 410 holds the amount of defocus corresponding to each cell. Of the defocus map 410, region 411 consists of cells showing the amount of defocus of the bird 401, and region 412 consists of cells showing the amount of defocus of the background object 402. The amount of defocus in each cell of region 412 is a value near focus (0[Fδ]), for example, a value of -0.3 to 0.4[Fδ]. In addition, the amount of defocus of the bird 401 is the amount of defocus corresponding to a back-focus state, for example, a value of 7.5 to 12.0[Fδ].

[0032] Figure 4(c) shows the defocus GT420, which indicates the amount of defocus of the subject bird 401, based on the defocus map 410 in Figure 4(b). The defocus GT420 holds information as a range value for the amount of defocus for each part of the bird, namely the pupil, face, and body. In the example in Figure 4(c), the defocus range is held as follows: pupil 9.3 [Fδ] to 9.5 [Fδ], face 8.9 to 10.0 [Fδ], and body 8.0 to 10.8 [Fδ]. Note that the information in the defocus GT420 and the defocus GT data group 313 is not limited to the above configuration; for example, the amount of defocus may be a single value rather than a range, or it may be a value limited to a specific part (such as the pupil).

[0033] Next, in step S502 of Figure 5, the DA (Data Augmentation) unit 320 specifies the probability of performing a sign inversion operation on the defocus amount in the data set acquired in step S501. The probability may be set by the user via the input unit 140, or a default value may be set in advance. The probability can be any value, but in this embodiment, setting the probability to 10-20% improves the estimation accuracy of the neural network.

[0034] Next, in step S503 of Figure 5, the DA unit 320 randomly generates sign-inverted data from the defocus map data group 312 and the defocus GT data group 313 according to the occurrence probability set in step S502. Of the DA unit 320, the sign-inverting process for the defocus map data group 312 is performed by the defocus map sign-inverting unit 330, and the sign-inverting process for the defocus GT data group 313 is performed by the defocus GT sign-inverting unit 340. As an example of sign-inverting processing of the defocus amount, Figures 4(d) and 4(e) show examples of the defocus map 410 and defocus GT 420 in Figure 4 being sign-inverted.

[0035] The sign-inverted defocus map 430 shown in Figure 4(d) is an inverted depth map obtained by reversing the sign of the values ​​in each cell of the defocus map 410 shown in Figure 4(b). As a result, in the sign-inverted defocus map 430, in the region 431 of the bird 401, the range of the defocus amount changes from the aforementioned 7.5 to 12.0 [Fδ] to the range of -12.0 to -7.5 [Fδ]. Also, in the region 432 of the background object 402, the range of the defocus amount changes from the aforementioned -0.3 to 0.4 [Fδ] to the range of -0.4 to 0.3 [Fδ].

[0036] The sign-reversed defocus GT440 shown in Figure 4(e) is the inverted ground truth label of the defocus GT420 shown in Figure 4(c), with the signs of the defocus range for each part reversed. In the sign-reversed defocus GT440, the pupil is -9.5[Fδ] to -9.3[Fδ], the face is -10.0 to -8.9[Fδ], and the torso is -10.8 to -8.0[Fδ].

[0037] This sign inversion operation of the defocus amount generates sign-inverted data where the defocus amount is reversed in relation to the positional relationship between the subject bird 401 and the background object 402. As a result, the defocus amount of the background object 402, which is located near the image formation position, does not change much, but the defocus amount of bird 401 changes from back focus to front focus blur. Therefore, the sign inversion operation of the defocus amount causes the correspondence between the real space in the image at the time of shooting and the defocus map and defocus GT to become inconsistent. However, in estimating the defocus amount, it is necessary to learn the positional relationships in the depth direction in various shooting scenes. For this reason, it has a greater impact on the neural network's learning if the training data is biased towards front focus or back focus images, rather than ensuring that the correspondence between the training data images and the defocus amount in real space is consistent.

[0038] Next, in step S504 of Figure 5, the output unit 350 outputs the training data, including the sign-inverted data generated in step S503. In the example of Figure 4, the image 400, the sign-inverted defocus map 430, and the sign-inverted defocus GT 440 are output as the original training data that has been DA'd. Note that this DA by sign-inverting the defocus amount may be performed in advance before training, or it may be performed dynamically during training. In this embodiment, the timing of performing DA is not limited.

[0039] Furthermore, while this embodiment shows an example where the probability p of sign inversion processing is set uniformly for all training data, it is not limited to this. For example, the probability of occurrence may be set separately for front-pin images and back-pin images in the training data. For instance, if the distribution of defocus amounts in the training data is biased towards either front-pin or back-pin images, the probability of occurrence should be set higher for the side with more data. By performing sign inversion processing with a probability corresponding to the distribution of defocus amounts in this way, the data on the side with fewer front-pin or back-pin images can be compensated for by the sign inversion-based DA, thereby further improving estimation accuracy.

[0040] Furthermore, while this embodiment demonstrates an example of performing DA by sign inversion processing on all training data according to the probability of occurrence, it is not limited to this. For example, sign inversion processing may be performed only on data in the training data where significant blurring occurs due to front focus or back focus. Training data of the amount of defocus near focus is easily collected when training data is collected, for example, by taking photographs, and the change in the value of the amount of defocus after sign inversion processing is small. Therefore, by setting a threshold (predetermined value) above which the absolute value of the amount of defocus is greater than a certain value, and performing sign inversion processing on data greater than the threshold, the estimation accuracy of conditions where significant blurring occurs due to front focus or back focus can be improved.

[0041] As described above, according to this embodiment, by performing a digital array (DA) that reverses the sign of the defocus amount and inverts the positional relationship in the depth direction in the depth information, it is possible to suppress bias in depth estimation and improve the generalization performance of the neural network for depth estimation.

[0042] (Second embodiment) In this embodiment, a method of performing digital articulation (DA) by limiting the sign inversion process of the defocus amount to the subject area will be described. The following description will omit explanations of aspects similar to those in the first embodiment, and will focus on the differences from the first embodiment.

[0043] Figure 6 is a block diagram showing an example of the functional configuration of the learning data generation device 301 in this embodiment. Figure 8 is a flowchart showing an example of the learning data generation process by the learning data generation device 301 in this embodiment. The processing contents of each part shown in Figure 6 will be described below in accordance with the processing procedure in Figure 8.

[0044] In step S801 of Figure 8, the subject detection unit 610 detects the subject area in each image in the image data group 311. The detection result of the subject area in each image is output as a bounding box (hereinafter referred to as BB), which is information about the width, height, and center coordinates in the image, and the subject detection unit 610 saves this information as a subject BB data group 611 in the HDD 130.

[0045] Next, in step S802, the acquisition unit 310 acquires the training image data group 311, the defocus map data group 312 of the image data group, the defocus GT data group 313, and the subject BB data group 611 generated in step S801. Here, the training image data group 311, the defocus map data group 312 of the image data group, and the defocus GT data group 313 are acquired by the same procedure as in the first embodiment. The next step S803 is the same process as in step S502 in Figure 5.

[0046] Next, in step S804, the DA unit 320 generates random sign-inverted data from the defocus map data group 312 and the defocus GT data group 313 according to the occurrence probability p set in S803, similar to step S503 in Figure 5. In this embodiment, the defocus map sign-inverting unit 330 performs the sign-inverting process only on the subject area in the defocus map. An example of the sign-inverting process of the defocus amount limited to the subject area is shown in Figure 7.

[0047] Image 700, shown in Figure 7(a), shows a bird 701, which is the subject, and a background object 702. Note that in this image 700, the focus on the bird failed, resulting in an image being formed between the bird 701 and the background object 702. Therefore, the bird 701 is out of focus behind the subject, and the background object 702 is out of focus in front of the subject. BB703 indicates the subject area of ​​the bird 701.

[0048] During image capture, a defocus map 710 centered on the subject area is generated, as shown in Figure 7(b). Of the defocus map 710, area 711 overlaps with the area of ​​BB703 and consists of cells indicating the amount of defocus of the bird 401. On the other hand, area 713 consists of cells indicating the amount of defocus of the background object 702. The boundary area 712 also overlaps with the area of ​​BB703, but is composed of cells indicating the amount of defocus near the boundary between the bird 701 and the background object 702. As mentioned above, the subject bird 701 is in a back-focused blur state, and the defocus amount indicating back focus is assumed to be a value of, for example, 2.0 to 5.0 [Fδ]. Area 713 is in a front-focused blur state of the background object 702, and the defocus amount indicating front focus is assumed to be a value of, for example, -10.0 to -20.0 [Fδ]. Furthermore, the boundary region 712 is assumed to have a value where the amount of defocus is slightly pulled towards the background object 702 compared to region 711, which shows the amount of defocus of the bird 701, and for example, it shows a value of -1.0 to -3.0 [Fδ].

[0049] Figure 7(c) shows the defocus GT720, which indicates the amount of defocus of the subject bird 701, based on the defocus map 710 in Figure 7(b). Similar to the example in Figure 4, the defocus GT720 maintains a range of defocus for each part of the bird: the pupil, face, and body. In the example in Figure 7(c), the pupil maintains a range of 3.0[Fδ]~3.2[Fδ], the face maintains a range of 2.8~3.8[Fδ], and the body maintains a range of 2.5~4.5[Fδ].

[0050] The sign-inverted defocus map 730 shown in Figure 7(d) is an inverted depth map obtained by reversing the sign of the values ​​of the cells included in the subject area in the defocus map 710 shown in Figure 7(b). Region 731 and boundary region 732 represent the regions where the sign-inverted processing was performed in regions 711 and boundary region 712, respectively, while region 713 is outside the subject area and therefore does not undergo the sign-inverted processing. As a result, in the sign-inverted defocus map 730, the range of the defocus amount in region 731 changes from the aforementioned 2.0 to 5.0 [Fδ] to -5.0 to -2.0 [Fδ]. Also, in boundary region 732, the range of the defocus amount changes from the aforementioned -1.0 to -3.0 [Fδ] to 1.0 to 3.0 [Fδ].

[0051] The sign-reversed defocus GT740 shown in Figure 7(e) is the inverted ground truth label of the defocus GT720 shown in Figure 7(c), with the signs of the defocus range for each part reversed. In the sign-reversed defocus GT740, the pupil is -3.2[Fδ] to -3.0[Fδ], the face is -3.8 to -2.8[Fδ], and the torso is -4.5 to -2.5[Fδ].

[0052] Next, in step S805 of Figure 8, the output unit 350 outputs training data including the sign-inverted data and the subject BB data group 611. In the example of Figure 7, the training data output includes the image 700, the sign-inverted defocus map 730, the sign-inverted defocus GT 740, and the subject BB data group 611.

[0053] The sign inversion process in this embodiment is particularly useful in situations where the background object and the subject overlap, and the background object has high contrast, resulting in a large number of cells in the defocus map showing defocus on the background side. Defocus on the background object with high contrast often indicates a front-focus blur state, and if these are sign-inverted, the cells on the background side will be inverted to show a back-focus blur state. In terms of a shooting scene in an actual imaging device, this situation is equivalent to a situation where there are many obstructions blocking the subject close to the imaging device, and the subject is being photographed behind the obstructions. In real space, it is impossible to obtain depth information of a subject behind an obstruction, making this situation unrealistic. Therefore, even if the probability p of DA occurrence to be sign-inverted in the first embodiment is set to a small value, there is a concern that such unrealistic data will occur, potentially confusing the neural network's learning.

[0054] Therefore, in this embodiment, the defocus amount sign inversion process is performed only on the subject area, specifically, as shown in the example in Figure 7, the defocus amount inversion is limited to the subject and its boundary area. This eliminates the influence of the defocus amount inversion on high-contrast background objects, enabling realistic DA processing and stable learning without disrupting the neural network's learning process.

[0055] In this embodiment, an example was shown in which BB information, which is the subject area, is acquired using a detector such as the subject detection unit 610, but the embodiment is not limited to this. For example, the user may input BB information in advance via the input unit 140.

[0056] (Other embodiments) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0057] This embodiment includes the following configurations, methods, and programs.

[0058] (Composition 1) An acquisition means for acquiring image data, a depth map showing depth information of the image data, and a ground truth label relating to the depth information, A generation means for generating an inverted depth map obtained by inverting the depth information in the aforementioned depth map, and an inverted ground truth label obtained by inverting the depth information in the aforementioned ground truth label, An output means for outputting training data including the image data, the inverted depth map, and the inverted ground truth labels, An information processing device characterized by comprising:

[0059] (Configuration 2) The acquisition means further acquires information about the subject area in the image data, The information processing device according to Configuration 1, characterized in that the generation means generates an inverted depth map by inverting the depth information in the subject region based on the acquired information of the subject region. (Composition 3) The system further comprises detection means for detecting a subject area from the aforementioned image data, The information processing apparatus according to configuration 2, characterized in that the acquisition means acquires information about the subject area detected by the detection means. (Composition 4) The information processing device according to any one of configurations 1 to 3, characterized in that the depth information is information on the amount of defocus. (Composition 5) The information processing apparatus according to configuration 4, characterized in that the generation means generates the inverted depth map and the inverted ground truth label by inverting the sign of the defocus amount.

[0060] (Composition 6) The information processing apparatus according to configuration 4 or 5, characterized in that the generation means generates the inverted depth map and the inverted ground truth label according to a probability corresponding to the distribution of the amount of defocus in the image data. (Composition 7) The information processing apparatus according to configuration 5 or 6, characterized in that the generation means generates the inverted depth map and the inverted ground truth label for image data where the absolute value of the defocus amount is greater than a predetermined value. (Composition 8) The information processing device according to any one of configurations 1 to 5, characterized in that the generation means generates the inverted depth map and the inverted correct label according to a predetermined probability.

[0061] (method) An acquisition process for acquiring image data, a depth map showing depth information of the image data, and a ground truth label related to the depth information, A generation process that generates an inverted depth map obtained by inverting the depth information in the aforementioned depth map, and an inverted ground truth label obtained by inverting the depth information in the aforementioned ground truth label, An output step that outputs training data including the image data, the inverted depth map, and the inverted ground truth labels, An information processing method characterized by comprising:

[0062] (program) An acquisition process for acquiring image data, a depth map showing depth information of the image data, and a ground truth label related to the depth information, A generation process that generates an inverted depth map obtained by inverting the depth information in the aforementioned depth map, and an inverted ground truth label obtained by inverting the depth information in the aforementioned ground truth label, An output step that outputs training data including the image data, the inverted depth map, and the inverted ground truth labels, A program that causes a computer to execute something. [Explanation of symbols]

[0063] 310 Acquisition unit, 320 DA unit, 350 Output unit

Claims

1. An acquisition means for acquiring image data, a depth map showing depth information of the image data, and a ground truth label relating to the depth information, A generation means for generating an inverted depth map obtained by inverting the depth information in the aforementioned depth map, and an inverted ground truth label obtained by inverting the depth information in the aforementioned ground truth label, An output means for outputting training data including the image data, the inverted depth map, and the inverted ground truth labels, An information processing device characterized by comprising:

2. The acquisition means further acquires information about the subject area in the image data, The information processing apparatus according to claim 1, characterized in that the generation means generates an inverted depth map by inverting the depth information in the subject region based on the acquired subject region information.

3. The system further comprises detection means for detecting a subject area from the aforementioned image data, The information processing apparatus according to claim 2, characterized in that the acquisition means acquires information of the subject area detected by the detection means.

4. The information processing apparatus according to claim 1, characterized in that the depth information is information on the amount of defocus.

5. The information processing apparatus according to claim 4, characterized in that the generation means generates the inverted depth map and the inverted ground truth label by inverting the sign of the defocus amount.

6. The information processing apparatus according to claim 5, characterized in that the generation means generates the inverted depth map and the inverted ground truth label according to a probability corresponding to the distribution of the amount of defocus in the image data.

7. The information processing apparatus according to claim 5, characterized in that the generation means generates the inverted depth map and the inverted ground truth label for image data where the absolute value of the defocus amount is greater than a predetermined value.

8. The information processing apparatus according to claim 1, characterized in that the generation means generates the inverted depth map and the inverted correct label according to a predetermined probability.

9. An acquisition process for acquiring image data, a depth map showing depth information of the image data, and a ground truth label related to the depth information, A generation process that generates an inverted depth map obtained by inverting the depth information in the aforementioned depth map, and an inverted ground truth label obtained by inverting the depth information in the aforementioned ground truth label, An output step that outputs training data including the image data, the inverted depth map, and the inverted ground truth labels, An information processing method characterized by comprising:

10. An acquisition process for acquiring image data, a depth map showing depth information of the image data, and a ground truth label related to the depth information, A generation process that generates an inverted depth map obtained by inverting the depth information in the aforementioned depth map, and an inverted ground truth label obtained by inverting the depth information in the aforementioned ground truth label, An output step that outputs training data including the image data, the inverted depth map, and the inverted ground truth labels, A program that causes a computer to execute something.

Images