Information processing device, information processing method, and program

Abstract

A technique is provided for improving the clarity of a reconstructed three-dimensional model when the three-dimensional model is reconstructed from a plurality of two-dimensional images using the three-dimensional reconstruction model. [Solution] An information processing device according to one aspect of the present disclosure acquires an image group consisting of multiple 2D images generated by capturing images of an object while changing the imaging angle, inputs the acquired image group into a 3D reconstruction model, reconstructs a 3D model of the object using the 3D reconstruction model, and outputs the reconstructed 3D model. The multiple 2D images include one or more partial attention images that capture a partial area of ​​the object that is to be focused on.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Conventionally, technological development of 3D computer vision for restoring a 3D model from multiple 2D images has been progressing. In recent years, technological advances in artificial intelligence have made it possible to easily restore a 3D model from a 2D image by using a trained machine learning model (3D restoration model). For example, Non-Patent Documents 1 and 2 propose a 3D base model for restoring a 3D model from a 2D image. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Shuzhe Wang, et al. “DUSt3R: Geometric 3D Vision Made Easy”, [online], [Retrieved August 29, 2025], Internet <URL:https: / / openaccess.thecvf.com / content / CVPR2024 / papers / Wang_DUSt3R_Geometric_3D_Vision_Made_Easy_CVPR_2024_paper.pdf> [Non-patent document 2] Jianyuan Wang, et al. “VGGT: Visual Geometry Grounded Transformer”, [online], [Retrieved August 29, 2025], Internet <URL:https: / / openaccess.thecvf.com / content / CVPR2025 / papers / Wang_VGGT_Visual_Geometry_Grounded_Transformer_CVPR_2025_paper.pdf> Summary of the Invention [Problem to be solved by the invention]

[0004] The present inventors have found that conventional methods using a 3D reconstruction model including a 3D base model have the following problems. Due to limitations on computational resources (such as memory capacity), there are cases where a resolution limit is imposed on the 2D image input to the 3D reconstruction model (e.g., the resolution of the input image is set to a fixed value). Furthermore, in an effort to efficiently reconstruct an object, there is a tendency to use a 2D image capturing a wider area of ​​the object, such as an overall image, for input. In a 2D image capturing a wider area, each part of the object may be blurred (i.e., each part has low resolution). If the resolution of the input 2D image is reduced in accordance with the resolution limit for inputting the 3D reconstruction model, each part of the object may become even blurrier. Blurred parts of the object captured in the 2D image used for reconstruction limit the ability to reconstruct a 3D model with clear parts.

[0005] In one aspect, the present disclosure has been made in view of the above circumstances, and an object of the present disclosure is to provide a technique for improving the clarity of a reconstructed 3D model when the 3D model is reconstructed from multiple 2D images using a 3D reconstruction model. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, the present disclosure employs the following configurations. Note that the following configurations can be combined as appropriate.

[0007] An information processing device according to one aspect of the present disclosure includes a control unit. The control unit is configured to acquire an image group consisting of multiple 2D images generated by capturing images of an object while changing the imaging angle, input the acquired image group to a 3D reconstruction model, cause the 3D reconstruction model to reconstruct a 3D model of the object, and output the reconstructed 3D model. The multiple 2D images include one or more partial attention images that depict a partial region of the object that is to be focused on.

[0008] In this configuration, at least one partial image of interest is used to reconstruct a 3D model of the object. The partial image of interest narrows the imaging range to a partial region on which the focus is placed. Therefore, even if a resolution limit is imposed on the input of the 3D reconstruction model, the resolution of this partial region can be ensured. Therefore, with this configuration, when a 3D model is reconstructed from multiple 2D images using the 3D reconstruction model, the clarity of the reconstructed 3D model can be expected to be improved.

[0009] In the information processing device according to the above aspect, the plurality of 2D images may further include one or more wide-area images capturing a wider range of the object than the focal partial region. In this configuration, not only the focal partial image but also the wide-area image capturing a wider range is used to reconstruct a 3D model of the object. The wide-area image can capture the macroscopic characteristics of the object. Therefore, in the reconstructed 3D model, the macroscopic characteristics of the object can be maintained while improving the clarity of the partial region.

[0010] In the information processing device according to the above aspect, the one or more partial attention images may include a partial attention image of the object that captures a partial region of the object that is focused on in the object. The one or more wide-area images may include a wide-area image of the object that captures a range of the object that includes the partial region of the object. With this configuration, the partial attention image of the object can be expected to improve the clarity of the partial region of the object within the range of the object, while the wide-area image of the object can maintain the macroscopic characteristics of the range of the object.

[0011] In the information processing device according to the above aspect, the partial attention image of the target may be generated by cropping the wide-area image of the target. With this configuration, it is not necessary to capture the partial attention image of the target and the wide-area image of the target separately, and the partial attention image of the target can be obtained from the wide-area image of the target by simple image processing such as cropping. This is expected to reduce the effort required to obtain the partial attention image of the target.

[0012] In the information processing device according to the above aspect, the wide range in at least one of the one or more wide-area images may include the entire object from the imaging angle. With this configuration, in a scene where an image capturing the entire object (overall image) is used as at least a part of the wide-area image, it is possible to expect improved clarity of partial regions in the restored 3D model while maintaining the macroscopic characteristics of the object.

[0013] In the information processing device according to the above aspect, the wide area in at least one of the one or more wide-area images may be a local area of ​​the object. With this configuration, in a scene where an image capturing a local area of ​​the object is used as at least a part of the wide-area image, it is possible to expect improved clarity of the local area in the restored 3D model while maintaining the macroscopic characteristics of the object.

[0014] Note that the embodiments of the present disclosure may not be limited to the above-described information processing device. As another aspect of the information processing device according to each of the above aspects, one aspect of the present disclosure may be an information processing method that realizes all or part of each of the above configurations, a program, or a machine-readable storage medium storing such a program. Here, the machine-readable storage medium may be a non-transitory medium that stores information such as a program by electrical, magnetic, optical, mechanical, or chemical action. The non-transitory storage medium may include a storage medium (e.g., a CD, a DVD, a semiconductor memory), an auxiliary storage device of a computer, an external storage device connected to a computer, etc.

[0015] For example, the information processing method according to one aspect of the present disclosure may be executed by a computer. The information processing method may include acquiring an image group consisting of a plurality of two-dimensional images generated by imaging an object while changing an imaging angle, inputting the acquired image group into a three-dimensional reconstruction model to reconstruct a three-dimensional model of the object using the three-dimensional reconstruction model, and outputting the reconstructed three-dimensional model. The plurality of two-dimensional images may include one or more partial attention images capturing a partial region of the object to be focused on.

[0016] Furthermore, for example, a program according to an aspect of the present disclosure may be a program for causing a computer to execute an information processing method. The information processing method may include acquiring an image group consisting of multiple 2D images generated by capturing images of an object while changing the imaging angle, inputting the acquired image group into a 3D reconstruction model, restoring a 3D model of the object using the 3D reconstruction model, and outputting the restored 3D model. The multiple 2D images may include one or more partial attention images capturing a partial region of the object to be focused on. [Effects of the Invention]

[0017] According to one aspect of the present disclosure, a technique can be provided for improving the clarity of a reconstructed 3D model when the 3D model is reconstructed from multiple 2D images using a 3D reconstruction model. [Brief explanation of the drawings]

[0018] [Figure 1] FIG. 1 schematically illustrates an example of a situation to which the present disclosure is applied. [Figure 2] FIG. 2 is a schematic diagram showing an example of an imaging range captured in a wide-area image. [Figure 3] FIG. 3 is a diagram showing an example of the relationship between a partial image of interest and a wide-area image. [Figure 4] FIG. 4 is a diagram illustrating an example of a hardware configuration of an information processing device. [Figure 5] FIG. 5 is a diagram illustrating an example of the software configuration of the information processing device. [Figure 6]FIG. 6 is a flowchart illustrating an example of a processing procedure of the information processing device. [Figure 7] FIG. 7 shows a three-dimensional model reconstructed in the comparative example. [Figure 8] Figure 8 shows the 3D model reconstructed in the experimental example. DETAILED DESCRIPTION OF THE INVENTION

[0019] An embodiment according to one aspect of the present disclosure will be described below with reference to the drawings. However, the embodiment described below is merely an example of the present disclosure in all respects. Various improvements or modifications may be made without departing from the scope of the present disclosure. In implementing the present disclosure, specific configurations according to the embodiment may be appropriately adopted. Note that while data appearing in the present embodiment is described in natural language, more specifically, it may be specified using pseudo-language, commands, parameters, machine language, electrical signals, etc. that can be recognized by a machine such as a computer.

[0020] §1 Application Examples 1 schematically illustrates an example of a situation to which the present disclosure is applied. An information processing device 1 according to this embodiment is one or more computers configured to execute information processing for reconstructing a 3D model 4 from an image group 2 composed of a plurality of 2D images 20.

[0021] In this embodiment, the information processing device 1 acquires an image group 2 consisting of multiple 2D images 20 generated by capturing images of the object TO while changing the imaging angle. The multiple 2D images 20 include one or more partial images of interest 21 that capture a partial area PA on which the object TO is focused. The information processing device 1 inputs the acquired image group 2 to a 3D reconstruction model 3, and causes the 3D reconstruction model 3 to reconstruct a 3D model 4 of the object TO. The information processing device 1 outputs the reconstructed 3D model 4.

[0022] In this embodiment, at least one partial image of interest 21 is a three-dimensional model of the object TO. The partial image of interest 21 is used to restore the partial area PA on which the focus is placed. Therefore, even if a resolution limit is imposed on the input of the 3D restoration model 3, the resolution of this partial area PA can be ensured. Therefore, according to this embodiment, when the 3D restoration model 3 is used to restore the 3D model 4 from multiple 2D images 20 (image group 2), the clarity of the restored 3D model 4 can be expected to be improved.

[0023] [Object] The object TO may include any type of object that can be the subject of reconstruction of a 3D model 4. The object TO may consist of a single object or multiple objects. For example, the object TO may be an item that is easy to photograph in its entirety at once, such as everyday items, tools, toys, furniture, clothing, decorations, food, electrical appliances, crafts, or other products. Toys may include stuffed toys, miniature models of large objects, etc. In another example, the object TO may be a large object that is difficult to photograph in its entirety at once, such as a building, structure, structure, or landscape. Buildings may include, for example, houses, apartment buildings, buildings, schools, hospitals, hotels, etc. Structures may include any man-made object fixed to the ground. Structures may include any artificially constructed object. Structures may include, for example, civil engineering structures such as roads, bridges, tunnels, dams, and levees. Buildings, structures, or structures may include ruins. Landscapes may include natural terrain.

[0024] [Images] Each of the two-dimensional images 20 (including the partial image of interest 21) constituting the image group 2 may be obtained by capturing an image of the object TO with the imaging device CA. In one example, the captured image (raw image) of the imaging device CA may be used as the two-dimensional image 20 as is. The two-dimensional image 20 may also be a processed image obtained by applying image processing to the captured image of the imaging device CA. The image processing may include any type of processing such as enlargement, reduction, cropping, adjustment, filtering, etc.

[0025] In one example, changing the imaging angle may mean changing the imaging conditions (position, orientation, etc.) of the imaging device CA to change the range of the appearance of the object TO captured in the captured image. The imaging angle of each 2D image 20 (including the partial image of interest 21) may be determined appropriately depending on the embodiment. Furthermore, the number of 2D images 20 constituting the image group 2 may be determined appropriately depending on the embodiment. The number of partial images of interest 21 included in the image group 2 may also be determined appropriately depending on the embodiment. The imaging angle and number of each 2D image 20 may be determined appropriately so as to cover the range of the object TO from which the 3D model 4 is to be reconstructed.

[0026] The type of imaging device CA is not particularly limited as long as it can generate a two-dimensional image, and may be appropriately selected depending on the embodiment. The imaging device CA may include any type of sensor that acquires data in the form of an image or an image representation. For example, the imaging device CA may be a general RGB camera.

[0027] The route for acquiring the image group 2 (plurality of two-dimensional images 20) is not particularly limited and may be determined appropriately depending on the embodiment. In one example, the information processing device 1 may be directly or indirectly connected to the imaging device CA, and may acquire at least a portion of the image group 2 (plurality of two-dimensional images 20) from the imaging device CA. An indirect connection may be a connection via another computer. The information processing device 1 may acquire at least a portion of the image group 2 (plurality of two-dimensional images 20) by performing image processing on the captured images acquired from the imaging device CA. The information processing device 1 may indirectly acquire at least a portion of the image group 2 (plurality of two-dimensional images 20) via another computer, a storage medium, an external storage device, etc.

[0028] The two-dimensional images 20 may be collected at any time. In one example, at least some of the two-dimensional images 20 may be collected in advance and stored in any storage area. The memory area may be configured, for example, by memory resources of the information processing device 1, memory resources of another computer, an external storage device (including a storage medium), etc. The external storage device may include a data server such as a network attached storage (NAS). At least some of the multiple 2D images 20 may be appropriately acquired from any memory area. In addition, in one example, at least some of the multiple 2D images 20 may be collected by the imaging device CA immediately before restoring the 3D model 4. At least some of the multiple 2D images 20 may be appropriately acquired from the imaging device CA.

[0029] (Partial featured image) In one example, capturing the partial area PA on which focus is placed may mean that the imaging range captured in the partial image of interest 21 is narrowed down to the partial area PA. For example, the partial image of interest 21 may be an image that captures only the partial area PA.

[0030] The partial area PA may be arbitrarily determined from the object TO. In one example, the image group 2 (plurality of two-dimensional images 20) may include a plurality of partial images of interest 21. When the image group 2 includes a plurality of partial images of interest 21, the partial areas PA depicted in each partial image of interest 21 may not overlap. Alternatively, the partial areas PA may overlap in some of the plurality of partial images of interest 21.

[0031] As long as the imaging range is limited to the partial area PA, the method for acquiring the partial image of interest 21 is not particularly limited and may be determined appropriately depending on the embodiment. In one example, the partial image of interest 21 may be generated by capturing an image of the object TO with the imaging device CA and extracting a portion from the obtained captured image (raw image, processed image). A known method such as cropping may be used to extract a portion (partial image of interest 21) from the original captured image. Cropping may be a process of cutting out the target portion and removing the remaining portion. In another example, the partial image of interest 21 may be obtained without relying on image processing. For example, the partial image of interest 21 may be generated by capturing an image by bringing the imaging device CA close to the partial area PA. The partial image of interest 21 may be generated by zooming in on the partial area PA with the imaging device CA and capturing an image.

[0032] (Image composition) In one example, as shown in FIG. 1 , the multiple 2D images 20 (image group 2) may further include one or more wide-area images 22 capturing a wider range of the object TO than the focal partial area PA. For example, some of the multiple 2D images 20 may be partial images of interest 21, and the rest may be wide-area images 22. As a result, not only the partial images of interest 21 but also the wide-area images 22 capturing a wider range are used to reconstruct a 3D model 4 of the object TO. The wide-area images 22 can capture macroscopic characteristics of the object TO. Therefore, according to one example of the present embodiment, the reconstructed 3D model 4 can be expected to have improved clarity of the partial area PA while maintaining the macroscopic characteristics of the object TO. However, the configuration of the image group 2 is not limited to this example. In another example, the multiple 2D images 20 (image group 2) may be composed of only the partial images of interest 21.

[0033] (Wide-area image capture range) 2 schematically shows an example of an imaging range captured in a wide-area image 22 according to this embodiment. The range captured in the wide-area image 22 may be appropriately defined to be relatively wider than the partial area PA captured in the partial image of interest 21. For example, the range captured in the wide-area image 22 being wider than the focused partial area PA may mean that the average value of the ranges captured in one or more wide-area images 22 is greater than the average value of the partial areas PA captured in one or more partial images of interest 21.

[0034] In one example, the wide area of ​​at least one of the one or more wide area images 22 may include the entire object TO from the imaging angle. That is, at least one wide area image 228 of the one or more wide area images 22 is configured to capture the entire image of the object TO from the imaging angle. In one example, when the object TO is an item that can be easily photographed in its entirety at once, the wide-area image 228 capturing the entire image may be actively used as the wide-area image 22. For example, when the object TO is an item, one or more wide-area images 22 may all be configured to capture the entire image of the object TO from each imaging angle. According to one example of the present embodiment, in a situation where an image capturing the entire object TO (the entire image) is used as at least one of the wide-area images 22 (wide-area image 228), it is possible to expect improved clarity of the partial area PA in the reconstructed 3D model 4 while maintaining the macroscopic features of the object TO.

[0035] Also, in one example, the wide area in at least one of the one or more wide area images 22 may be a local area of ​​the object TO. That is, at least one wide area image 229 of the one or more wide area images 22 may be configured to capture a local area of ​​the object TO. The local area need not be particularly limited as long as it is a portion of the object TO and is wider than the partial area PA (a range that is not the entire object TO), and may be defined appropriately depending on the embodiment. In one example, when the object TO is a large object that is difficult to capture in its entirety at once, the wide area image 229 capturing a local area may be actively used as the wide area image 22. For example, when the object TO is a large object, all of the one or more wide area images 22 may be configured to capture local areas of the object TO from their respective imaging angles. According to one example of this embodiment, in a situation where an image capturing a local area of ​​the object TO is used as at least one of the wide-area images 22 (wide-area image 229), it is possible to expect improved clarity of the partial area PA in the restored three-dimensional model 4 while maintaining the macroscopic characteristics of the object TO.

[0036] In one example, the one or more wide area images 22 may be composed of only wide area images 228 that capture the entire image, or may be composed of wide area images 229 that capture a local area. Furthermore, when the image group 2 (plurality of two-dimensional images 20) includes multiple wide area images 22, the multiple wide area images 22 may include a mixture of wide area images 228 that capture the entire image and wide area images 229 that capture a local area.

[0037] (Relationship between partial attention image and wide area image) 3 schematically shows an example of the relationship between the partial image of interest 21 and the wide-area image 22 according to this embodiment. The imaging range (partial area PA) of the partial image of interest 21 and the imaging range of the wide-area image 22 may be unrelated or may have a relationship. Having a relationship may mean overlapping or including one another.

[0038] In one example, the one or more partial images of interest 21 may include a partial image of interest 211 of an object that captures a partial region of the object TO that is focused on. The one or more wide-area images 22 may include a wide-area image 221 of an object that captures a range of the object that encompasses the partial region of the object. That is, at least one of the one or more partial images of interest 21 (partial image of interest 211 of the object) may be captured for the same range as at least one of the one or more wide-area images 22 (corresponding wide-area image 221 of the object). Capturing the same range may include capturing images at the same imaging angle. The wide-area image 22 (wide-area image 221 of the object) and the partial image of interest 21 (partial image of interest 211 of the object) may have a corresponding relationship in that they capture the same range. The wide-area image 221 of the object may be a wide-area image 228 that captures the entire image or a wide-area image 229 that captures a local range. FIG. 3 illustrates an example of a scene in which the wide-area image 228 is selected as the wide-area image 221 of the object. According to one example of this embodiment, the wide-area image 221 of the object can be expected to maintain the macroscopic characteristics of the range of the object, while the partial focus image 211 of the object can be expected to improve the clarity of a partial area of ​​the object within the range of the object.

[0039] As long as the partial image of interest 211 of the target and the wide-area image of the target 221 can be acquired for the same range, the method of acquiring the partial image of interest 211 of the target and the wide-area image of the target 221 may not be particularly limited and may be appropriately selected depending on the embodiment. In one example, the partial image of interest 211 of the target may be generated by cropping the wide-area image of the target 221. In other words, the partial image of interest 211 of the target may be generated by cropping the corresponding wide-area image of the target 221. The target wide-area image 221 may be derived from the target wide-area image 221. The target wide-area image 221 may be a raw image or a processed image. According to an example of this embodiment, it is not necessary to capture the target partial image of interest 211 and the target wide-area image 221 separately. Furthermore, the target partial image of interest 211 can be acquired from the target wide-area image 221 by a simple image processing called cropping. Therefore, it is expected that the effort required to obtain the target partial image of interest 211 can be reduced.

[0040] [3D reconstruction model] As long as the 3D reconstruction model 3 has the ability to reconstruct a 3D model from multiple 2D images, the configuration of the 3D reconstruction model 3 is not particularly limited and may be determined appropriately depending on the embodiment. In one example, the ability to reconstruct a 3D model may be acquired through machine learning, and the 3D reconstruction model 3 may be configured by a trained machine learning model. The configuration (type, structure, etc.) of the machine learning model may be determined arbitrarily. The 3D reconstruction model 3 (trained machine learning model) may use a known model such as the 3D foundation model described in Non-Patent Documents (1, 2).

[0041] In one example, a resolution limit may be imposed on the input of the 3D reconstruction model 3. For example, the resolution of the image (2D image 20) input to the 3D reconstruction model 3 may be set to a fixed value. When the multiple 2D images 20 include a partial image of interest 21 and a wide-area image 22, the partial image of interest 21 and the wide-area image 22 may be subjected to image processing at any time (e.g., in preprocessing) to set the resolution to a fixed value. The imaging range (partial area PA) of the partial image of interest 21 is narrower than the imaging range of the wide-area image 22. Therefore, after the resolutions of the partial image of interest 21 and the wide-area image 22 are converted to a uniform fixed value by image processing, the partial image of interest 21 may become an image in which the object TO is relatively enlarged compared to the wide-area image 22 (i.e., an image with a higher resolution for the partial area PA). Therefore, by using this partial image of interest 21 to restore the 3D reconstruction model 3, improved clarity of the restored 3D model 4 can be expected, even when the resolution limit as described above is imposed.

[0042] (3D model data format) As long as the appearance of the object TO can be expressed in three dimensions, the data format of the restored three-dimensional model 4 is not particularly limited and may be appropriately selected depending on the embodiment. The data format of the three-dimensional model 4 may be any known data format, such as point cloud data, mesh data (including polygons), volume data, or voxel data.

[0043] (Executing entity) The 3D reconstruction model 3 may be installed in any computer. In one example, the 3D reconstruction model 3 may be installed in the information processing device 1. Restoring the 3D model 4 of the object TO using the 3D reconstruction model 3 may be achieved by the information processing device 1 executing arithmetic processing of the 3D reconstruction model 3 to obtain the restored 3D model 4 (the restoration result of the 3D model 4). In another example, the 3D reconstruction model 3 may be installed in a computer other than the information processing device 1. The other computer may input the image group 2 (plurality of 2D images 20) into the 3D reconstruction model 3 and execute arithmetic processing of the 3D reconstruction model 3 to restore the 3D model 4 of the object TO. The information processing device 1 may issue a command to the other computer to instruct the arithmetic processing of the 3D reconstruction model 3, and the other computer may execute the arithmetic processing of the 3D reconstruction model 3 in response to the command from the information processing device 1. The other computer may execute the arithmetic processing of the 3D reconstruction model 3 autonomously. The other computer may acquire the image group 2 used for restoration from the information processing device 1, or may acquire it from an information source other than the information processing device 1. Acquiring the image group 2 may be configured by having the other computer acquire the image group 2. The information processing device 1 may acquire the restored 3D model 4 (the restoration result of the 3D model 4) directly or indirectly from the other computer. Indirect acquisition may be acquisition via yet another computer, a storage medium, an external storage device, or the like. That is, the 3D restored model Having the computer 3 reconstruct a three-dimensional model 4 of the object TO may be accomplished by having another computer execute the computational processing of the three-dimensional reconstruction model 3 and obtaining the reconstructed three-dimensional model 4 from the other computer.

[0044] §2 Configuration example [Hardware configuration] 4 is a schematic diagram illustrating an example of a hardware configuration of the information processing device 1 according to this embodiment. In one example, the information processing device 1 may be configured as a computer to which a control unit 11, a storage unit 12, an external interface 13, an input device 14, and an output device 15 are electrically connected.

[0045] The control unit 11 is configured to execute information processing based on programs and various data. For example, the control unit 11 includes a hardware processor such as a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read Only Memory). The control unit 11 (CPU) is an example of a processor resource.

[0046] The storage unit 12 is configured to hold any data. For example, the storage unit 12 may include a hard disk drive, a solid state drive, a semiconductor memory, etc. The storage unit 12, RAM, and ROM are examples of memory resources. In one example of this embodiment, the storage unit 12 may store various information such as a program 81. The program 81 is a program for causing the information processing device 1 to execute information processing (see FIG. 6, which will be described later) related to the restoration of the three-dimensional model 4. The program 81 includes a series of instructions for the information processing.

[0047] In one example, when the 3D reconstruction model 3 is installed in the information processing device 1, the storage unit 12 may store model data 30 representing the 3D reconstruction model 3. As long as the calculation of the 3D reconstruction model 3 can be reproduced, the configuration of the model data 30 is not particularly limited and may be determined appropriately depending on the embodiment. For example, the model data 30 may include values ​​of calculation parameters adjusted by machine learning, a model structure (e.g., a neural network structure), etc. The model data 30 may be incorporated into the program 81. The model data 30 may be omitted from the storage unit 12.

[0048] In one example, the program 81 may be stored in a storage medium 91 instead of or together with the storage unit 12. The storage medium 91 is configured to store various information (such as the stored program) by electrical, magnetic, optical, mechanical, or chemical action so that a machine such as a computer can read the information. The storage unit 12 and the storage medium 91 are examples of non-transitory storage media. The information processing device 1 may acquire the program 81 from the storage medium 91. The storage medium 91 may be a disk-type storage medium (such as a CD or DVD) or a non-disk-type storage medium such as a semiconductor memory (such as a flash memory). Any drive device may be used to read information stored in the storage medium 91. The type of drive device may be selected depending on the storage medium 91. The drive device may be connected to the information processing device 1 in any manner. The storage medium 91 may include an external storage device. In one example, the model data 30 may be stored in the storage medium 91 instead of or together with the storage unit 12.

[0049] The external interface 13 is configured to connect to an external device via a wired or wireless connection. The external interface 13 may include, for example, a USB (Universal Serial Bus) port, a dedicated port, a communication port (communication module), etc. The type and number of external interfaces 13 may be determined appropriately depending on the embodiment. When the external interface 13 includes a communication port (communication module), the standard of the communication network may be selected arbitrarily. The communication standard may be selected appropriately from, for example, the Internet, a wireless communication network, a mobile communication network, a telephone network, a dedicated network, etc. In one example, the information processing device 1 communicates with an imaging device via the external interface 13. It may be connected to an external device such as a CA, another computer, or an external storage device.

[0050] The input device 14 is configured to accept input of information. The input device 14 may be configured, for example, by an imaging device, a microphone, a mouse, a keyboard, a touch panel, an operator, etc. The imaging device of the input device 14 may be the same as or different from that of the imaging device CA. The output device 15 is configured to output information. The output device 15 may be configured, for example, by a display, a speaker, etc. The information processing device 1 may be operated using the input device 14 and the output device 15. The input device 14 and the output device 15 may be directly connected to the information processing device 1 or indirectly connected via the external interface 13. The input device 14 and the output device 15 may be at least partially integrated with a touch panel display, etc.

[0051] It should be noted that, with regard to the specific hardware configuration of the information processing device 1, components may be omitted, replaced, or added as appropriate depending on the embodiment. For example, the control unit 11 may include multiple hardware processors. The hardware processors may be a microprocessor, a field-programmable gate array (FPGA), a digital signal processor (DSP), a GP The external interface 13, the input device 14, and the output device 15 may be configured by a graphics processing unit (U), an application specific integrated circuit (ASIC), etc. At least one of the above may be omitted. At least one of the program 81 and the model data 30 may be stored in an external storage device such as a NAS. An external storage device is also an example of a non-transitory storage medium. When at least some of the multiple 2D images 20 to be used for restoration are collected in advance, the collected 2D images 20 may be stored in at least one of the storage unit 12, the storage medium 91, and the external storage device. The information processing device 1 may be composed of multiple computers. In this case, the hardware configurations of the computers may or may not be the same. The information processing device 1 may be a computer designed specifically for the service to be provided, as well as a general-purpose server device, a general-purpose PC, a laptop PC, a terminal device, etc. The terminal device may include a smartphone, a tablet terminal, etc.

[0052] [Software configuration] 5 schematically shows an example of the software configuration of the information processing device 1 according to this embodiment. The control unit 11 of the information processing device 1 executes instructions included in the program 81 stored in the storage unit 12 using the CPU. As a result, the information processing device 1 operates as a computer including an acquisition unit 111, a restoration unit 112, and an output processing unit 113 as software modules. That is, in one example, each software module of the information processing device 1 may be realized by the control unit 11 (CPU).

[0053] The acquisition unit 111 is configured to acquire an image group 2 consisting of a plurality of two-dimensional images 20 generated by capturing images of the object TO while changing the imaging angle. The restoration unit 112 is configured to input the acquired image group 2 to a three-dimensional restoration model 3 and cause the three-dimensional restoration model 3 to restore a three-dimensional model 4 of the object TO. The output processing unit 113 is configured to output the restored three-dimensional model 4.

[0054] In one example of this embodiment, each software module of the information processing device 1 is implemented by a general-purpose CPU. However, the method of implementing each of the above modules is not limited to this example and may be changed as appropriate depending on the embodiment. Some or all of the above software modules may be implemented by one or more dedicated processors or chipsets. Each of the above modules may be implemented as a hardware module. With regard to the software configuration of the information processing device 1, modules may be omitted, replaced, or added as appropriate depending on the embodiment.

[0055] §3 Example of operation FIG. 6 is a flowchart showing an example of a processing procedure for restoring a three-dimensional model 4 by the information processing device 1 according to this embodiment. The information processing device 1 (controller 11) is configured to execute the processing of each of the following steps in accordance with instructions included in the program 81. The following processing procedure is an example of an information processing method (image processing method) executed by a computer. The following processing procedure is merely an example, and each step may be changed as much as possible. Furthermore, steps in the following processing procedure may be omitted, replaced, or added as appropriate depending on the embodiment.

[0056] (Step S101) In step S101, the control unit 11 operates as the acquisition unit 111. That is, the control unit 11 acquires an image group 2 consisting of a plurality of two-dimensional images 20 generated by capturing images of the object TO while changing the imaging angle. The plurality of two-dimensional images 20 includes one or more partial images of interest 21 that capture a partial area PA on which the object TO is focused.

[0057] In one example, the plurality of two-dimensional images 20 (image group 2) may further include one or more wide-area images 22 that capture a range of the object TO that is wider than the focal partial area PA.

[0058] In one example, the wide range in at least one of the one or more wide range images 22 (wide range image 228) may include the entire object TO from the imaging angle, and in another example, the wide range in at least one of the one or more wide range images 22 (wide range image 229) may be a local range of the object TO.

[0059] In one example, the one or more partial images of interest 21 may include a partial image of interest 211 of the object that captures a partial region of the object TO that is focused on. The one or more wide-area images 22 may include a wide-area image 221 of the object that captures a range of the object that includes the partial region of the object. The wide-area image 221 of the object may be a wide-area image 228 that captures the entire image, or a wide-area image 229 that captures a local range. In another example, the partial image of interest 211 of the object may be generated by cropping the wide-area image 221 of the object. After acquiring the image group 2, the control unit 11 proceeds to the next step S102.

[0060] (Step S102) In step S102, the control unit 11 operates as the restoration unit 112. That is, the control unit 11 inputs the acquired image group 2 to the three-dimensional restoration model 3, and causes the three-dimensional restoration model 3 to restore a three-dimensional model 4 of the object TO.

[0061] In one example, the arithmetic processing of the 3D reconstruction model 3 may be executed on the information processing device 1 or on another computer. In another example, a resolution limit may be set for the input of the 3D reconstruction model 3. For example, the resolution of the image (2D image 20) input to the 3D reconstruction model 3 may be set to a fixed value. Accordingly, at any timing before executing step S102, each 2D image 20 (partial image of interest 21, wide-area image 22) may be subjected to image processing to adapt to the resolution limit (for example, to set the resolution to a fixed value). The image processing for each 2D image 20 may be executed on the information processing device 1 or on any computer other than the information processing device 1. After the 3D model 4 is restored, the control unit 11 proceeds to the next step S103.

[0062] (Step S103) In step S103, the control unit 11 operates as the output processing unit 113. That is, the control unit 11 outputs the restored three-dimensional model 4.

[0063] The output contents and output destination may be determined appropriately depending on the embodiment. In one example, the control unit 11 may output the restored 3D model 4 as is. In another example, the control unit 11 may perform predetermined information processing (model processing, etc.) on the restored 3D model 4. The control unit 11 may output the results of the information processing. In another example, the output destination may be RAM, the storage unit 12, the output device 15, the storage medium 91, another computer, an external storage device, etc.

[0064] After outputting the restored three-dimensional model 4, the control unit 11 ends the processing procedure according to this operation example. The control unit 11 may repeatedly execute the processes of steps S101 to S103 at any timing. In one example, after the two-dimensional images 20 constituting the image group 2 are collected, the control unit 11 may execute the processes of steps S101 to S103 afterward to restore the three-dimensional model 4 from the image group 2 (plurality of two-dimensional images 20).

[0065] (Features) In this embodiment, in the process of step S102, at least one or more partial images of interest 21 are used to reconstruct a 3D model 4 of the object TO. The partial images of interest 21 narrow the imaging range to the partial area PA on which the focus is placed. Therefore, even if a resolution limit is imposed on the input of the 3D reconstruction model 3, the resolution of this partial area PA can be ensured. Therefore, according to this embodiment, when the 3D reconstruction model 3 is used to reconstruct the 3D model 4 from multiple 2D images 20 (image group 2), an improvement in the clarity of the reconstructed 3D model 4 can be expected.

[0066] §4 Variations Although the embodiments of the present disclosure have been described in detail above, the above description is merely an example of the present disclosure in every respect. The processes and means described in the present disclosure can be freely combined and implemented as long as no technical contradiction occurs. Furthermore, various improvements or modifications may be made to the above embodiments as appropriate.

[0067] §5 Experimental Examples The following experiments were conducted to verify the effects of the present disclosure, but the present disclosure is not limited to the following experimental examples.

[0068] A stuffed toy was used as the object for which a 3D model was to be reconstructed. In the comparative example, two wide-area images (2D images) were collected by capturing the entire image of the stuffed toy using a commercially available imaging device. Each collected wide-area image was preprocessed to have a resolution of 336 x 518. The two preprocessed wide-area images were input into a 3D reconstruction model (VGGT) to obtain a 3D model composed of point cloud data. Meanwhile, in the experimental example, two partial attention images were obtained by cropping each of the two wide-area images to extract partial regions of the stuffed toy. Each wide-area image and each partial attention image was preprocessed to have a resolution of 336 x 518. The two preprocessed wide-area images and two partial attention images were input into a 3D reconstruction model (VGGT) to obtain a 3D model composed of point cloud data.

[0069] Figure 7 shows a 3D model restored in the comparative example. Figure 8 shows a 3D model restored in the experimental example. The restored point cloud data was accompanied by a reliability calculated by the 3D restoration model (VGGT). In the comparative example and experimental example, points with a reliability of 0.7 or higher were extracted. In the comparative example, 121,834 points were restored with a reliability of 0.7 or higher, while in the experimental example, 243,667 points were restored with a reliability of 0.7 or higher. These results verified the effectiveness of using partial attention images to improve the clarity of the restored 3D model. Furthermore, even though the imaging angle variation of the object was not increased, the restored 3D model was improved simply by adding partial attention images captured from the same range. This demonstrates that it is possible to improve the clarity of a 3D model by using a wide-area image of the object while maintaining the macroscopic characteristics of the object's range. [Explanation of symbols]

[0070] 1...information processing device, 11...control unit, 12...storage unit, 2...image group, 20...2D image, TO...object, 21...partial image of interest, PA...partial area, 22...wide-area image, 3...3D reconstruction model, 4...3D model

Claims

1. An information processing device including a control unit, The control unit Acquiring an image group consisting of a plurality of two-dimensional images generated by imaging the object while changing the imaging angle; inputting the acquired image group into a three-dimensional reconstruction model, and causing the three-dimensional reconstruction model to reconstruct a three-dimensional model of the object; Output the restored 3D model. It is configured as follows: the plurality of two-dimensional images include one or more partial attention images capturing a partial region to be focused on in the object, and one or more wide-area images capturing a range of the object that is wider than the partial region to be focused on, the one or more partial attention images include a partial attention image of an object that depicts a partial area of ​​the object that is to be focused on in the object; the one or more wide-area images include a wide-area image of the object depicting a range of the object that encompasses a subregion of the object; The partial attention image of the object has a narrower imaging range than the wide-area image of the object, and is an image in which a partial area of ​​the object is enlarged compared to the wide-area image of the object. Information processing device.

2. The partial attention image of the object has the same resolution as the wide-area image of the object, and has a narrower imaging range than the wide-area image of the object, so that the resolution of the partial area of ​​the object is higher than that of the wide-area image of the object. The information processing device according to claim 1 .

3. The partial attention image of the object is generated by cropping a wide-area image of the object. The information processing device according to claim 1 .

4. The wide range in at least one of the one or more wide-area images is The entire object includes the above objects. The information processing device according to claim 1 .

5. The wide area in at least one of the one or more wide-area images is a local area of ​​the object. The information processing device according to claim 1 .

6. 1. A computer-implemented information processing method, comprising: acquiring an image group consisting of a plurality of two-dimensional images generated by capturing images of an object while changing the imaging angle; Inputting the acquired images into a three-dimensional reconstruction model and restoring a three-dimensional model of the object using the three-dimensional reconstruction model; and outputting the reconstructed three-dimensional model; Including, the plurality of two-dimensional images include one or more partial attention images capturing a partial region to be focused on in the object, and one or more wide-area images capturing a range of the object that is wider than the partial region to be focused on, the one or more partial attention images include a partial attention image of an object that depicts a partial area of ​​the object that is to be focused on in the object; the one or more wide-area images include a wide-area image of the object depicting a range of the object that encompasses a subregion of the object; The partial attention image of the object has a narrower imaging range than the wide-area image of the object, and is an image in which a partial area of ​​the object is enlarged compared to the wide-area image of the object. Information processing methods.

7. The partial attention image of the object has the same resolution as the wide-area image of the object, and has a narrower imaging range than the wide-area image of the object, so that the resolution of the partial area of ​​the object is higher than that of the wide-area image of the object. The information processing method according to claim 6.

8. The partial attention image of the object is generated by cropping a wide-area image of the object. The information processing method according to claim 6.

9. A program for causing a computer to execute an information processing method, The information processing method includes: acquiring an image group consisting of a plurality of two-dimensional images generated by capturing images of an object while changing the imaging angle; Inputting the acquired images into a three-dimensional reconstruction model and restoring a three-dimensional model of the object using the three-dimensional reconstruction model; and outputting the reconstructed three-dimensional model; Including, the plurality of two-dimensional images include one or more partial attention images capturing a partial region to be focused on in the object, and one or more wide-area images capturing a range of the object that is wider than the partial region to be focused on, the one or more partial attention images include a partial attention image of an object that depicts a partial area of ​​the object that is to be focused on in the object; the one or more wide-area images include a wide-area image of the object depicting a range of the object that encompasses a subregion of the object; The partial attention image of the object has a narrower imaging range than the wide-area image of the object. , an image in which a partial region of the object is enlarged compared to a wide-area image of the object, program.

10. The partial attention image of the object has the same resolution as the wide-area image of the object, and has a narrower imaging range than the wide-area image of the object, so that the resolution of the partial area of ​​the object is higher than that of the wide-area image of the object. The program according to claim 9.

11. The partial attention image of the object is generated by cropping a wide-area image of the object. The program according to claim 9.

Images