Three-dimensional restoration image manufacture program, three-dimensional restoration image manufacture device, three-dimensional restoration image manufacture method, and recording medium

Abstract

To provide a three-dimensional restoration image manufacture program which can easily manufacture an image for three-dimensional restoration.SOLUTION: A three-dimensional restoration image manufacture program includes an image acquisition procedure, a reference designation procedure, an image extraction procedure, and an output procedure. The image acquisition procedure acquires a plurality of area images in which the inside of a target area is captured from a plurality of spots. The reference designation procedure designates restoration reference in the target area, from the area image. The image extraction procedure extracts an image in which at least a part of the restoration reference is included, from the plurality of area images. The output procedure outputs the extracted image in which at least a part of the restoration reference is included, as a three-dimensional restoration image of the restoration reference.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a program for producing an image for three-dimensional restoration, an apparatus for producing an image for three-dimensional restoration, a method for producing an image for three-dimensional restoration, and a recording medium. [Background technology]

[0002] There is a three-dimensional restoration method for generating a three-dimensional shape model of an object based on a plurality of images of the object captured from different viewpoints. For example, Patent Document 1 discloses a three-dimensional shape model generation device for generating a three-dimensional shape model of an object, the device including: a blur map generation unit that generates a blur map that indicates the amount of blur for each pixel in each of two or more multi-viewpoint images of the object captured from different viewpoints; and a three-dimensional restoration processing unit that uses the multi-viewpoint images and the blur map to weight matching scores by window matching according to the amount of blur for each pixel in the blur map, thereby generating a three-dimensional shape model that indicates the three-dimensional shape of the object. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-009255 Summary of the Invention [Problem to be solved by the invention]

[0004] However, to realize the 3D reconstruction method, it is necessary to prepare multi-viewpoint images with the center of the object to be restored as the focus point, which poses a problem that preparing multi-viewpoint images for 3D reconstruction is extremely time-consuming.

[0005] Therefore, the present disclosure aims to provide a program for producing an image for three-dimensional restoration, an apparatus for producing an image for three-dimensional restoration, a method for producing an image for three-dimensional restoration, and a recording medium that can easily produce an image for three-dimensional restoration. [Means for solving the problem]

[0006] In order to achieve the above object, the three-dimensional reconstruction image production program of the present disclosure includes: It includes an image acquisition procedure, a reference specification procedure, an image extraction procedure, and an output procedure; The image acquisition step acquires a plurality of area images captured from a plurality of points within a target area, The criterion designation step designates a restoration criterion for the target area from the area image; The image extraction step extracts an image including at least a part of the restoration criterion from the plurality of area images; the output step outputs an image including at least a part of the extracted restoration reference as an image for three-dimensional restoration of the restoration reference; This is a 3D restoration image production program for causing a computer to execute each of the above procedures.

[0007] The three-dimensional restoration image production device of the present disclosure includes: An image acquisition unit, a reference specification unit, an image extraction unit, and an output unit, the image acquisition unit acquires a plurality of area images captured from a plurality of points within a target area; the criterion designation unit designates a restoration criterion for the target area from the area image; the image extraction unit extracts an image including at least a part of the restoration criterion from the plurality of area images; The output unit outputs an image including at least a part of the extracted restoration reference as an image for three-dimensional restoration of the restoration reference.

[0008] The method for producing a three-dimensional reconstruction image according to the present disclosure includes: The method includes an image acquisition step, a reference specification step, an image extraction step, and an output step, The image acquisition step acquires a plurality of area images captured from a plurality of points within a target area, The reference designation step designates a restoration reference for the target area from the area image, The image extraction step extracts an image including at least a part of the restoration criterion from the plurality of area images, The output step outputs an image including at least a part of the extracted restoration reference as an image for three-dimensional restoration of the restoration reference; This is a method for producing an image for three-dimensional reconstruction, in which each of the steps is executed by a computer.

[0009] The recording medium of the present disclosure includes: It includes an image acquisition procedure, a reference specification procedure, an image extraction procedure, and an output procedure; The image acquisition step acquires a plurality of area images captured from a plurality of points within a target area, The criterion designation step designates a restoration criterion for the target area from the area image; The image extraction step extracts an image including at least a part of the restoration criterion from the plurality of area images; the output step outputs an image including at least a part of the extracted restoration reference as an image for three-dimensional restoration of the restoration reference; A computer-readable recording medium storing a program for producing an image for three-dimensional reconstruction for causing a computer to execute each of the above procedures. [Effects of the Invention]

[0010] According to the present disclosure, images for three-dimensional reconstruction can be easily produced. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram showing the configuration of an example of a three-dimensional restoration image production device according to the present disclosure. [Figure 2] FIG. 2 is a block diagram showing an example of the hardware configuration of the three-dimensional restoration image production device of the present disclosure. [Figure 3] FIG. 3 is a flowchart showing an example of processing in the three-dimensional restoration image production device of the present disclosure. [Figure 4]FIG. 4 is a schematic diagram for explaining the processing of the three-dimensional restoration image production device of the present disclosure. [Figure 5] FIG. 5 is a photograph for explaining the processing of the 3D restoration image production device of the present disclosure. [Figure 6] FIG. 6 is a photograph for explaining the processing of the 3D restoration image production device of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The present disclosure is not limited to the following embodiments. In the following drawings, the same parts are denoted by the same reference numerals. Furthermore, the descriptions of the embodiments can be mutually incorporated unless otherwise specified, and the configurations of the embodiments can be combined unless otherwise specified.

[0013] [Embodiment 1] The image production program for three-dimensional restoration disclosed herein is a program for causing a computer to execute an image acquisition procedure, a reference designation procedure, an image extraction procedure, and an output procedure. The image production program for three-dimensional restoration disclosed herein can also be said to be a program for causing a computer to function as the image acquisition procedure, the reference designation procedure, the image extraction procedure, and the output procedure. Furthermore, the image production program for three-dimensional restoration disclosed herein can also be said to be a program for causing a computer to execute, for example, each step of the image production method for three-dimensional restoration described below.

[0014] The image acquisition procedure acquires multiple area images taken from multiple points within a target area, the criterion designation procedure designates a restoration criterion for the target area from the area images, the image extraction procedure extracts images containing at least a portion of the restoration criterion from the multiple area images, and the output procedure outputs the extracted images containing at least a portion of the restoration criterion as images for three-dimensional restoration of the restoration criterion.

[0015] For example, the "procedure" in each step can be read as "processing." The 3D reconstruction image production program of the present disclosure may be recorded on a computer-readable recording medium, for example. The recording medium may be, for example, a non-transitory computer-readable storage medium. The recording medium is not particularly limited, and examples thereof include random access memory (RAM), read-only memory (ROM), hard disk (HD), flash memory (e.g., solid state drive (SSD), USB flash memory, SD / SDHC card, etc.), optical disk (e.g., CD-R / CD-RW, DVD-R / DVD-RW, BD-R / BD-RE, etc.), magneto-optical disk (MO), and floppy disk (FD). The 3D reconstruction image production program of the present disclosure (also referred to as a programming product or program product) may be distributed from an external computer, for example. The "distribution" may be, for example, distribution via a communication network or via a device connected via a wired connection. The image production program for three-dimensional reconstruction of the present disclosure may be installed and executed on the device to which it is distributed, or may be executed without being installed. An information processing device capable of executing the image production program for three-dimensional reconstruction of the present disclosure can be referred to as, for example, the image production device for three-dimensional reconstruction of the present disclosure.

[0016] Next, the configuration of an example of an apparatus for producing an image for three-dimensional reconstruction according to the present disclosure will be described with reference to FIG. 1. FIG. 1 is a block diagram showing the configuration of an example of an apparatus for producing an image for three-dimensional reconstruction 10 according to the present embodiment (hereinafter also referred to as "the apparatus"). As shown in FIG. 1, the apparatus 10 includes an image acquisition unit 11, a reference designation unit 12, an image extraction unit 13, and an output unit 14. The apparatus 10 may also include, for example, an input unit, an output unit, a display unit, and / or a storage unit, although these are not shown. The image acquisition unit 11, the reference designation unit 12, the image extraction unit 13, and the output unit 14 can respectively execute, for example, an image acquisition procedure, a reference designation procedure, an image extraction procedure, and an output procedure in the program for producing an image for three-dimensional reconstruction according to the present disclosure.

[0017] The device 10 may be, for example, a single device including the above-described units, or a device in which the units can be connected via a communication network. The device 10 can also be connected to an external device (described later) via the communication network. The communication network is not particularly limited and any known network can be used, for example, a wired or wireless network. Examples of the communication network include the Internet, the World Wide Web (WWW), a telephone line, a Local Area Network (LAN), a Storage Area Network (SAN), a Delay Tolerant Networking (DTN), a Low Power Wide Area Network (LPWA), and a Local 5G (L5G). Examples of wireless communication include Wi-Fi (registered trademark), Bluetooth (registered trademark), Local 5G, and LPWA. Examples of the wireless communication include direct communication between devices (Ad Hoc communication), infrastructure communication, and indirect communication via an access point. The device 10 may be incorporated into a server as a system. Furthermore, the present device 10 may be, for example, a personal computer (PC, for example, desktop or notebook type) on which the program of the present disclosure is installed, a smartphone, a tablet terminal, etc. The present device 10 may be in the form of cloud computing or edge computing, for example, in which at least one of the above-mentioned units is located on a server and the other units are located on a terminal.

[0018] 2 shows a block diagram of the hardware configuration of the device 10. The device 10 includes, for example, a central processing unit (CPU, GPU, etc.) 101, a memory 102, a bus 103, a storage device 104, an input device 105, an output device 106, and a communication device 107. The components of the device 10 are connected to each other via the bus 103 and their respective interfaces (I / F).

[0019] The central processing unit 101 cooperates with other components via a controller (such as a system controller or an I / O controller) and is responsible for overall control of the device 10. In the device 10, the central processing unit 101 executes, for example, the program of the present disclosure and other programs, and also reads and writes various types of information. Specifically, for example, the central processing unit 101 functions as an image acquisition unit 11, a reference designation unit 12, an image extraction unit 13, and an output unit 14. The device 10 may include other arithmetic units such as a CPU, a GPU (Graphics Processing Unit), an APU (Accelerated Processing Unit), or a combination thereof as the arithmetic unit.

[0020] The bus 103 can also be connected to, for example, external devices. Examples of the external devices include a user terminal, an external storage device (such as an external database), a printer, an external input device, an external display device, and an external imaging device. The device 10 can be connected to an external network (the communication line network) by, for example, a communication device 107 connected to the bus 103, and can also be connected to other devices via the external network.

[0021] The memory 102 may be, for example, a main memory (primary storage device). When the central processing unit 101 performs processing, the memory 102 reads various operating programs, such as the program of the present disclosure, stored in the storage device 104 (described later), and the central processing unit 101 receives data from the memory 102 and executes the programs. The main memory may be, for example, a RAM (random access memory). The memory 102 may also be, for example, a ROM (read only memory).

[0022] The storage device 104 is also referred to as an auxiliary storage device, for example, in contrast to the main memory (primary storage device). As described above, the storage device 104 stores an operating program including the program of the present disclosure. The storage device 104 may be, for example, a combination of a recording medium and a drive that reads and writes data from and to the recording medium. The recording medium is not particularly limited and may be, for example, an internal or external type, such as a hard disk (HD), CD-ROM, CD-R, CD-RW, MO, DVD, flash memory, or memory card. The storage device 104 may be, for example, a hard disk drive (HDD) or a solid state drive (SSD) in which the recording medium and drive are integrated.

[0023] In the present device 10, the memory 102 and the storage device 104 can also store various information such as log information, information acquired from an external database (not shown) or an external device, information generated by the present device 10, and information used by the present device 10 when executing processing. In this case, the memory 102 and the storage device 104 may store, for example, the above-mentioned information on users of the present device. Note that at least a portion of the information may be stored, for example, in an external server other than the memory 102 and the storage device 104, or may be stored in a distributed manner across multiple terminals using blockchain technology or the like. Furthermore, when the present device 10 includes the storage unit, for example, the memory 102 and the storage device 104 function as the storage unit.

[0024] The device 10 further includes, for example, an input device 105 and an output device 106. Examples of the input device 105 include pointing devices such as a touch panel, track pad, and mouse; a keyboard; imaging means such as a camera and scanner; card readers such as an IC card reader and a magnetic card reader; and audio input means such as a microphone. Examples of the output device 106 include display devices such as an LED display and a liquid crystal display; audio output devices such as a speaker; a printer; and the like. In the first embodiment, the input device 105 and the output device 106 are configured separately, but the input device 105 and the output device 106 may be configured as an integrated device, such as a touch panel display.

[0025] An example of processing by the three-dimensional restoration image production program of the present disclosure will be described in more detail with reference to Fig. 3. Fig. 3 is a flowchart showing an example of each procedure of the three-dimensional restoration image production program of the present disclosure. The three-dimensional restoration image production program of the present disclosure is implemented as follows, for example, using the device 10 of Fig. 1 or Fig. 2 in which the three-dimensional restoration image production program of the present disclosure is installed. Note that implementation of the three-dimensional restoration image production program of this embodiment is not limited to use of the device 10 of Fig. 1 or Fig. 2.

[0026] The image acquisition unit 11 acquires a plurality of area images captured from a plurality of points within a target area (S11, image acquisition step). The target area is not particularly limited as long as it is an area for performing three-dimensional reconstruction, and may be indoors or outdoors. The area images are not particularly limited as long as they are images of the target area captured from a plurality of points, and may be videos or still images. The area images may be, for example, spherical images captured within the target area. The spherical images are, for example, images captured by a spherical camera (also called a 360° camera). The spherical images are captured so that the exclusion criterion is included in the image combination area. The image combination area is an area where an image captured by one wide-angle camera included in the spherical camera and an image captured by the other wide-angle camera are combined when the spherical image is captured. The format of the spherical image is not particularly limited. The spherical image may be, for example, a still image or a video. The area image may be recorded, for example, in the storage unit of the device 10, or may be recorded in a recording medium external to the device 10. The image acquisition unit 11 may acquire, for example, the area image recorded in the storage unit or an external recording medium, or may acquire the area image from an imaging device (for example, the omnidirectional camera) that captured the area image.

[0027] The criterion designation unit 12 designates a restoration criterion for the target area from the area image (S12, criterion designation step). The restoration criterion is, for example, a target for performing three-dimensional restoration, and may be an object in the target area or a spatial coordinate. The criterion designation unit 12 may, for example, designate a restoration criterion designated by a user in the area image, or may designate the restoration criterion from restoration criterion candidates included in each area image. The criterion designation unit 12 may, for example, estimate feature quantities of a plurality of area images and propose restoration criterion candidates included in each area image based on the feature quantities.

[0028] The image extraction unit 13 extracts images containing at least a portion of the restoration criteria from the multiple area images (S13, image extraction procedure). The image extraction unit 13 can extract images containing at least a portion of the restoration criteria by, for example, detecting the restoration criteria from the area images using a known image processing method for detecting and tracking a specified object included in an image and selecting images containing at least a portion of the restoration criteria. Examples of the known image processing method include, but are not limited to, R-CNN (Regional CNN), YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), DETR (End-to-End Object Detection with Transformers), Semantic Segmentation, and Instance Segmentation. An image containing at least a portion of the restoration criteria may be, for example, an extracted image containing the restoration criteria. The extracted image may be, for example, an image containing a portion of the restoration criteria or an image containing the entire restoration criteria. Furthermore, the image extraction unit 13 may extract images from the plurality of area images so that the restoration reference is included and the restoration reference is positioned at the center of the extracted image. For example, the image extraction unit 13 preferably extracts images so that an overlapping area exists in at least one set of images in the extracted image group. Furthermore, the resolution of the images extracted by the image extraction unit 13 is not particularly limited, and the images may be cut out at any resolution, for example.

[0029] Specifically, the image extraction unit 13, for example, detects a restoration reference from the area image and draws a bounding box (rectangular area) so as to include the detected restoration reference. Then, the image extraction unit 13, for example, estimates the center point of the bounding box in each image and cuts out each image so that the estimated center point is at the center, thereby extracting an image that includes at least a part of the restoration reference. Furthermore, the processing of the image extraction unit 13 is not limited to this, and, for example, the image extraction unit 13 may detect a restoration reference from the area image and cut out the area of ​​the detected restoration reference in pixel units to extract an image that includes at least a part of the restoration reference.

[0030] The output unit 14 outputs an image including at least a portion of the extracted restoration reference as an image for three-dimensional restoration of the restoration reference (S14, output step). The image for three-dimensional restoration is an image used for three-dimensional restoration, and its format is not particularly limited. The output unit 14 may output the image for three-dimensional restoration to, for example, the storage unit of the device 10, or may output the image for three-dimensional restoration to an external device of the device 10. In the latter case, the output unit 14 may output the image for three-dimensional restoration to, for example, a device capable of executing a program for performing three-dimensional restoration, or may output the image for three-dimensional restoration to a database (three-dimensional restoration database) that stores images for three-dimensional restoration. The output unit 14 may, for example, project the spherical coordinate system of the omnidirectional image onto planar coordinates of an image including at least a portion of the restoration reference.

[0031] The output unit 14 may, for example, associate various other information necessary for three-dimensional reconstruction with the image for three-dimensional reconstruction. Examples of the various information necessary for three-dimensional reconstruction include position information of the omnidirectional camera that captured the omnidirectional image, attitude information of the omnidirectional camera when capturing the omnidirectional image, camera parameters of the omnidirectional camera, and various metadata. The position information of the omnidirectional camera is, for example, information indicating the position of the omnidirectional camera when capturing the omnidirectional image and may be GPS (Global Positioning System) information. Examples of the camera parameters include focal length, lens distortion, and sensor size. The metadata is various information embedded in the omnidirectional image and may include identification information of the omnidirectional camera, capture date and time, and GPS coordinates. The various information may be acquired from metadata of the omnidirectional camera or image, or may be estimated from the area image. In the latter case, the output unit 14 may, for example, estimate the information using SfM (Structure from Motion) technology.

[0032] A specific example of processing by the device 10 will be described with reference to Fig. 4 and Fig. 5. Fig. 4 is a top view showing a shooting point of an area image in the target area 1. In Fig. 4, reference numeral 1 indicates a section of the target area 1 (for example, a wall in a room), reference numeral 2 indicates an object (an object to be three-dimensionally restored) placed in the target area 1 that serves as a restoration reference, and reference numeral 3 indicates the imaging point. Restoration reference 2 (a table 2A, a bottle 2B placed on the table 2A, and a monitor 2C) is placed in the target area 1. First, prior to processing by the device 10, a user captures a celestial sphere image at imaging point 3 (imaging points 3A to 3I) in the target area 1 using an omnidirectional camera.

[0033] FIG. 5 shows an example of a celestial sphere image captured at imaging point 3I. A user using device 10 selects bottle 2B from the celestial sphere image as restoration standard 2. Device 10 detects bottle 2B from multiple celestial sphere images captured at each imaging point 3 (3A to 3I), cuts out the image region including the detected bottle 2B, and outputs it as an image for three-dimensional restoration of bottle 2B. FIG. 6 shows an example of the cut-out image (image for three-dimensional restoration of bottle 2B). As shown in FIG. 6, device 10 can easily produce an image for three-dimensional restoration of the restoration standard by simply capturing celestial sphere images from multiple points for target area 1, in which restoration standard 2 included in target area 1 is captured from multiple directions.

[0034] The three-dimensional restoration image production method of the present disclosure is, for example, a method implemented by replacing the "procedures" in the three-dimensional restoration image production program of the present disclosure with "processes." Specifically, the three-dimensional restoration image production method of the present disclosure includes an image acquisition process, a reference designation process, an image extraction process, and an output process. The image acquisition process acquires multiple area images captured from multiple locations within a target area. The reference designation process designates a restoration reference for the target area from the area images. The image extraction process extracts images from the multiple area images that include at least a portion of the restoration reference. The output process outputs the extracted images that include at least a portion of the restoration reference as three-dimensional restoration images of the restoration reference. The three-dimensional restoration image production method of the present disclosure can be implemented, for example, using the three-dimensional restoration image production device 10 of the present disclosure shown in FIG. 1 or FIG. 2. Note that the three-dimensional restoration image production method of the present disclosure is not limited to methods using the three-dimensional restoration image production device 10. The method of producing an image for three-dimensional restoration according to the present disclosure can, for example, refer to the description of the program for producing an image for three-dimensional restoration and the apparatus for producing an image for three-dimensional restoration according to the present disclosure.

[0035] The present disclosure provides a program for producing an image for three-dimensional restoration, including an image acquisition step, a reference designation step, an image extraction step, and an output step. The image acquisition step acquires multiple area images captured from multiple locations within a target area. The reference designation step designates a restoration reference for the target area from the area images. The image extraction step extracts an image including at least a portion of the restoration reference from the multiple area images. The output step outputs the extracted image including at least a portion of the restoration reference as an image for three-dimensional restoration of the restoration reference. Therefore, the program for producing an image for three-dimensional restoration of the present disclosure can easily produce an image for three-dimensional restoration of the restoration reference, for example, without preparing a multi-viewpoint image with the center of the restoration reference as a gaze point. Furthermore, the present disclosure also provides a program for producing a three-dimensional restoration image of any restoration target (restoration reference) present in the target area by comprehensively photographing the target area using a spherical camera.

[0036] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0037] <Additional Notes> Some or all of the above embodiments can be described as, but are not limited to, the following supplementary notes. (Appendix 1) It includes an image acquisition procedure, a reference specification procedure, an image extraction procedure, and an output procedure; The image acquisition step acquires a plurality of area images captured from a plurality of points within a target area, The criterion designation step designates a restoration criterion for the target area from the area image; The image extraction step extracts an image including at least a part of the restoration criterion from the plurality of area images; the output step outputs an image including at least a part of the extracted restoration reference as an image for three-dimensional restoration of the restoration reference. A three-dimensional reconstruction image production program for causing a computer to execute each of the above procedures. (Appendix 2) The image production program for three-dimensional restoration described in Appendix 1, wherein the image extraction procedure extracts an image from the plurality of area images so that the restoration reference is included and the restoration reference is positioned in the center of the extracted image. (Appendix 3) The image production program for three-dimensional restoration described in Appendix 1 or 2, wherein the reference designation procedure estimates features of multiple area images and proposes restoration reference candidates contained in each area image based on the features. (Appendix 4) 4. The three-dimensional reconstruction image production program according to claim 1, wherein the area image is a celestial sphere image captured within the target area. (Appendix 5) 5. The three-dimensional reconstruction image production program according to claim 4, wherein the output step projects a spherical coordinate system of the celestial sphere image onto planar coordinates of an image that includes at least a part of the reconstruction reference. (Appendix 6) An image acquisition unit, a reference specification unit, an image extraction unit, and an output unit, the image acquisition unit acquires a plurality of area images captured from a plurality of points within a target area; the criterion designation unit designates a restoration criterion for the target area from the area image; the image extraction unit extracts an image including at least a part of the restoration criterion from the plurality of area images; The output unit outputs an image including at least a portion of the extracted restoration reference as an image for three-dimensional restoration of the restoration reference. (Appendix 7) The image extraction unit extracts an image from the plurality of area images so that the restoration reference is included and the restoration reference is positioned in the center of the extracted image. (Appendix 8) An image manufacturing device for three-dimensional restoration according to claim 6 or 7, wherein the reference designation unit estimates features of a plurality of area images and proposes restoration reference candidates contained in each area image based on the features. (Appendix 9) 9. The three-dimensional restoration image manufacturing device according to claim 6, wherein the area image is a celestial sphere image captured within the target area. (Appendix 10) 10. The image production device for three-dimensional restoration according to claim 9, wherein the output unit projects a spherical coordinate system of the celestial sphere image onto planar coordinates of an image including at least a part of the restoration reference. (Appendix 11) The method includes an image acquisition step, a reference specification step, an image extraction step, and an output step, The image acquisition step acquires a plurality of area images captured from a plurality of points within a target area, The reference designation step designates a restoration reference for the target area from the area image, The image extraction step extracts an image including at least a part of the restoration criterion from the plurality of area images, The output step outputs an image including at least a part of the extracted restoration reference as an image for three-dimensional restoration of the restoration reference; A method for producing an image for three-dimensional reconstruction, in which each of the steps is executed by a computer. (Appendix 12) 12. The method for producing an image for three-dimensional reconstruction according to claim 11, wherein the image extraction step extracts an image from the plurality of area images so that the restoration reference is included and the restoration reference is positioned at the center of the extracted image. (Appendix 13) 13. A method for producing an image for three-dimensional reconstruction according to claim 11, wherein the reference designation step estimates features of a plurality of area images and proposes candidate restoration references contained in each area image based on the features. (Appendix 14) 14. The method for producing an image for three-dimensional reconstruction according to claim 11, wherein the area image is a celestial sphere image captured within the target area. (Appendix 15) 15. The image production method for three-dimensional reconstruction according to claim 14, wherein the outputting step projects a spherical coordinate system of the celestial sphere image onto planar coordinates of an image including at least a part of the reconstruction reference. (Appendix 16) It includes an image acquisition procedure, a reference specification procedure, an image extraction procedure, and an output procedure; The image acquisition step acquires a plurality of area images captured from a plurality of points within a target area, The criterion designation step designates a restoration criterion for the target area from the area image; The image extraction step extracts an image including at least a part of the restoration criterion from the plurality of area images; the output step outputs an image including at least a part of the extracted restoration reference as an image for three-dimensional restoration of the restoration reference; A computer-readable recording medium storing a program for producing an image for three-dimensional reconstruction for causing a computer to execute each of the above procedures. (Appendix 17) 17. The recording medium according to claim 16, wherein the image extraction step extracts an image from the plurality of area images such that the restoration criterion is included and the restoration criterion is positioned at the center of the extracted image. (Appendix 18) 18. The recording medium according to claim 16, wherein the step of specifying the criteria includes estimating feature amounts of a plurality of area images and proposing restoration criteria candidates included in each area image based on the feature amounts. (Appendix 19) 19. The recording medium according to any one of appendices 16 to 18, wherein the area image is a celestial sphere image captured within the target area. (Appendix 20) 20. The recording medium according to claim 19, wherein the output step projects a spherical coordinate system of the celestial sphere image onto planar coordinates of an image that includes at least a part of the restoration reference. [Industrial Applicability]

[0038] According to the present disclosure, it is possible to easily create a restoration-reference 3D restoration image. Therefore, the present disclosure can be suitably used in fields requiring 3D restoration. [Explanation of symbols]

[0039] 10. 3D reconstruction image production device 11 Image acquisition unit 12 Standard specification section 13 Image extraction section 14 Output section 101 Central Processing Unit 102 memory 103 Bus 104 Storage device 105 Input Device 106 Output Device 107 Communication Devices

Claims

1. It includes an image acquisition procedure, a reference specification procedure, an image extraction procedure, and an output procedure; The image acquisition step acquires a plurality of area images captured from a plurality of points within a target area, The criterion designation step designates a restoration criterion for the target area from the area image; The image extraction step extracts an image including at least a part of the restoration criterion from the plurality of area images; the output step outputs an image including at least a part of the extracted restoration reference as an image for three-dimensional restoration of the restoration reference. A three-dimensional reconstruction image production program for causing a computer to execute each of the above procedures.

2. 2. The image production program for three-dimensional restoration according to claim 1, wherein the image extraction procedure extracts an image from the plurality of area images so that the restoration reference is included and the restoration reference is positioned at the center of the extracted image.

3. 3. The three-dimensional reconstruction image production program according to claim 1, wherein the reference designation step estimates feature amounts of the plurality of area images and proposes restoration reference candidates included in each area image based on the feature amounts.

4. 3. The three-dimensional restoration image production program according to claim 1, wherein the area image is a spherical image captured within the target area.

5. 5. The three-dimensional reconstruction image production program according to claim 4, wherein the output step projects a spherical coordinate system of the omnidirectional image onto planar coordinates of an image including at least a part of the reconstruction reference.

6. An image acquisition unit, a reference specification unit, an image extraction unit, and an output unit, the image acquisition unit acquires a plurality of area images captured from a plurality of points within a target area; the criterion designation unit designates a restoration criterion for the target area from the area image; the image extraction unit extracts an image including at least a part of the restoration criterion from the plurality of area images; The output unit outputs an image including at least a portion of the extracted restoration reference as an image for three-dimensional restoration of the restoration reference.

7. 7. The image production device for three-dimensional restoration according to claim 6, wherein the image extraction unit extracts an image from the plurality of area images so that the restoration reference is included and the restoration reference is positioned at the center of the extracted image.

8. 8. The image production device for three-dimensional restoration according to claim 6, wherein the reference designation unit estimates feature quantities of the plurality of area images and proposes restoration reference candidates included in each area image based on the feature quantities.

9. The method includes an image acquisition step, a reference specification step, an image extraction step, and an output step, The image acquisition step acquires a plurality of area images captured from a plurality of points within a target area, The reference designation step designates a restoration reference for the target area from the area image, The image extraction step extracts an image including at least a part of the restoration criterion from the plurality of area images, The output step outputs an image including at least a part of the extracted restoration reference as an image for three-dimensional restoration of the restoration reference; A method for producing an image for three-dimensional reconstruction, in which each of the steps is executed by a computer.

10. It includes an image acquisition procedure, a reference specification procedure, an image extraction procedure, and an output procedure; The image acquisition step acquires a plurality of area images captured from a plurality of points within a target area, The criterion designation step designates a restoration criterion for the target area from the area image; The image extraction step extracts an image including at least a part of the restoration criterion from the plurality of area images; the output step outputs an image including at least a part of the extracted restoration reference as an image for three-dimensional restoration of the restoration reference; A computer-readable recording medium storing a program for producing an image for three-dimensional reconstruction for causing a computer to execute each of the above procedures.

Images