Information processing method, information processing device, and program

Abstract

[Problem] To enable support of an imaging operation by appropriately presenting progress of imaging. [Solution] This information processing method includes: acquiring, by a processor, captured images obtained by imaging a common object using imaging devices, and information related to the position and orientation of the imaging devices and assigned to the captured images; and outputting, by the processor, information for displaying an image indicating a result of imaging of the object, the information being generated on the basis of the acquired information, in a display format that can be generated according to the progress stage of imaging of the object among a plurality of different display formats.

Description

Information Processing Method, Information Processing Apparatus, and Program 【0001】 The present disclosure relates to an information processing method, an information processing apparatus, and a program. 【0002】 In recent years, a technique for generating (modeling) the three-dimensional shape of a subject from captured images captured by each of a plurality of imaging devices has been known. 【0003】 In Patent Document 1 below, a technique for calibrating parameters of a plurality of imaging devices arranged at different positions and each imaging a common three-dimensional space is disclosed. 【0004】 International Publication No. 2019 / 225681 【0005】 However, when performing an operation of imaging an object from various locations and angles by a plurality of people for acquiring three-dimensional data of the object, it has been difficult to share the progress status such as the already photographed range. 【0006】 Therefore, the present disclosure proposes an information processing method, an information processing apparatus, and a program capable of appropriately presenting the progress status of imaging and supporting the imaging operation. 【0007】 According to the present disclosure, a processor acquires each captured image obtained by imaging an object common to each imaging device, and information on the position and orientation of the imaging device attached to each captured image, and information for displaying an image showing the result of imaging the object, which is generated based on the acquired information, is output in a display format that can be generated according to the progress stage of imaging the object among a plurality of different display formats. An information processing method is provided. 【0008】 Further, according to the present disclosure, a process of acquiring each captured image obtained by imaging an object common to each imaging device, and information on the position and orientation of the imaging device attached to each captured image, and information for displaying an image showing the result of imaging the object, which is generated based on the acquired information, is output in a display format that can be generated according to the progress stage of imaging the object among a plurality of different display formats. An information processing apparatus including a control unit that performs the process is provided. 【0009】 Furthermore, this disclosure provides a program for causing a computer to function as a control unit that performs the following processes: acquiring each captured image obtained by each imaging device capturing a common object, and position and orientation information of the imaging device attached to each captured image; and outputting information for displaying an image showing the result of imaging the object, generated based on the acquired information, in a display format that can be generated from among a plurality of different display formats according to the progress stage of imaging the object. 【0010】 This is a diagram illustrating the outline of an information processing system according to one embodiment of the present disclosure. This is a diagram showing the configuration of an information processing system 1 according to one embodiment of the present disclosure. This is a block diagram showing an example of the configuration of an imaging device 10 according to this embodiment. This is a block diagram showing an example of the configuration of an information processing device 20 according to this embodiment. This is a block diagram showing an example of the configuration of a display terminal 30 according to this embodiment. This is a diagram illustrating the stepwise display of the display format of the image capture result image according to this embodiment. This is a diagram showing a comparison table 400 of the display formats from phase 1 to phase 3 according to this embodiment. This is an example of virtual space display of the image capture result image in phase 1 of this embodiment. This is an example of AR display of the image capture result image in phase 1 of this embodiment. This is an example of virtual space display of the image capture result image in phase 2 of this embodiment. This is an example of AR display of the image capture result image in phase 2 of this embodiment. This is an example of virtual space display of the image capture result image in phase 3 of this embodiment. This is an example of AR display of the image capture result image in phase 3 of this embodiment. This is an example of virtual space display of the image capture result image combining phase 1 and phase 2 of this embodiment. This is a flowchart showing an example of the operation processing flow of the imaging device 10 according to this embodiment. This is a flowchart showing an example of the operation processing flow of the information processing device 20 according to this embodiment. This is a flowchart showing an example of the operation processing flow of the display terminal 30 according to this embodiment. This is a block diagram showing an example of the configuration of a display terminal 30A according to a modified example of this embodiment. This is a block diagram showing an example of the hardware configuration of an information processing device 900 according to one embodiment of the present disclosure. 【0011】Preferred embodiments of this disclosure will be described in detail below with reference to the attached drawings. In this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions will be omitted. 【0012】 Furthermore, the explanation will be given in the following order: 1. Overview 2. Configuration 2-1. System Configuration 2-2. Configuration of Imaging Device 10 2-3. Configuration of Information Processing Device 20 2-4. Configuration of Display Terminal 30 3. Operation Processing 4. Details of Display Format 5. Operation Processing 6. Modification Examples 7. Hardware Configuration 8. Supplementary Information 【0013】 <1. Overview> Figure 1 is a diagram illustrating the overview of an information processing system according to one embodiment of the present disclosure. In this embodiment, as shown in Figure 1, it is assumed that multiple users U (U1 to U3) operate their respective imaging devices 10 (10a to 10c) to image the object 50 (for example, a house) from various locations and angles in order to acquire three-dimensional data of the object 50. Such imaging work is assumed to be imaging work for generating a three-dimensional model of the object 50. 【0014】 In the example shown in Figure 1, users U1 and U2 operate the drone using controllers 12a and 12b, respectively, and capture images of the target object 50 from the air using imaging devices 10a and 10b attached to the drone. The drone and the imaging devices 10a and 10b attached to the drone are operated by controllers 12a and 12b. Note that the drone is not limited to a mobile body that moves in the air, but may also be a mobile body that moves in water. In other words, the task of acquiring three-dimensional data of an object in water is also envisioned. Furthermore, the controllers 12a and 12b display, for example, the through images captured by the camera or imaging devices 10a and 10b installed on the drone in real time. 【0015】 Additionally, from the ground, user U3 operates the imaging device 10c to image the target object 50. 【0016】 The number and form of the imaging device 10 are not particularly limited. 【0017】When the object is large, it is expected that multiple people will work in parallel to reduce the workload. Also, when working outdoors, it is expected that multiple people will work to complete the imaging work for acquiring 3D data before the sunlight or shooting environment changes. However, there is no way to share progress such as the areas that have already been captured, which has resulted in omissions and overlaps in imaging. If there are many overlapping areas between captured images, the storage capacity will increase, and the image processing load will also increase. On the other hand, if the imaging work is performed with the aim of minimizing overlap, the risk of omissions and misalignments in imaging increases. 【0018】 Therefore, this disclosure makes it possible to support the imaging work by appropriately displaying the progress of the imaging. 【0019】 When attempting to output information in real time based on captured images, processing load is incurred for high-resolution 3D modeling and estimation of uncaptured areas. Therefore, the information processing system according to this embodiment provides multiple display formats with different levels of processing load (different levels of detail, i.e., different granularity), and during the process, the system presents the user with the information that has been processed so far (i.e., the image showing the current imaging results) using one of the display formats. Regarding the generation of the image showing the current imaging results (hereinafter also referred to as the imaging result image), convenience is enhanced by making it possible to present it even in the early stages of the process when the number of captured images and processing time are small. Furthermore, as the imaging process progresses, the number of captured images increases and processing time becomes available, making it possible to present imaging result images with increasingly detailed representations in stages. 【0020】 The configuration of the information processing system according to this embodiment will be described in detail below. 【0021】 <2. Configuration> <<2-1. System Configuration>> Figure 2 is a diagram showing the configuration of an information processing system 1 according to one embodiment of the present disclosure. The information processing system 1 according to this embodiment includes an imaging device 10 (10a to 10c), an information processing device 20, and a display terminal 30 (an example of an information processing terminal). 【0022】Each imaging device 10 captures a common object 50 (Figure 1) and transmits the resulting image to the information processing device 20. Each image is associated with the position and orientation information of the imaging device 10 that captured the image. 【0023】 The information processing device 20 generates an image showing the current imaging result of the object 50 based on the acquired information (specifically, the captured image and the positional and orientation information attached to the captured image). At this time, the information processing device 20 generates the image of the imaging result in a display format that can be generated according to the progress stage of imaging, from among several different display formats (which can also be called generation formats). The information processing device 20 transmits the generated image of the imaging result to the display terminal 30. The information processing device 20 can be implemented as, for example, a PC (Personal Computer), a smartphone, a tablet terminal, an HMD (Head Mounted Display), or a server. 【0024】 The display terminal 30 displays the image capture result received from the information processing device 20. The display terminal 30 is an example of an information processing terminal used by the user. By visually inspecting the current image capture result, the user can intuitively grasp the areas that have not yet been captured and use this information to their advantage in subsequent imaging work. 【0025】 The display terminal 30 can be implemented as, for example, a PC (personal computer), smartphone, tablet, or HMD. There may also be multiple display terminals 30, allowing multiple users U (U1 to U3) to view the same image capture result on each display terminal 30. Furthermore, the functions of the display terminal 30 may be implemented by the drone's controller 12. That is, the image capture result may be displayed on the controller 12's display unit. The user U can intuitively grasp the uncaptured area and then operate the drone using the controller 12. 【0026】 <<2-2. Configuration of the Imaging Device 10>> Figure 3 is a block diagram showing an example of the configuration of the imaging device 10 according to this embodiment. As shown in Figure 3, the imaging device 10 includes a communication unit 110, an imaging unit 120, a control unit 130, a sensor unit 140, a storage unit 150, and an operation display unit 160. 【0027】 (Communication Unit 110) The communication unit 110 has a transmitting unit that transmits data to an external device and a receiving unit that receives data from an external device. The communication unit 110 according to this embodiment communicates with an external device or the Internet using, for example, a wired / wireless LAN (Local Area Network), Wi-Fi®, Bluetooth®, a mobile communication network, etc. 【0028】 For example, the communication unit 110 transmits the captured image to the information processing device 20 in accordance with the control of the control unit 130. 【0029】 (Imaging Unit 120) The imaging unit 120 has one or more lenses (optical system) and an image sensor consisting of a CCD image sensor or a CMOS image sensor, and performs imaging according to the control of the control unit 130. 【0030】 (Sensor unit 140) The sensor unit 140 is an acquisition unit that acquires various sensing data. Specifically, the sensor unit 140 includes a position sensor 141, an IMU (Inertial Measurement Unit) 142, and a geomagnetic sensor 143. 【0031】 The position sensor 141 is a receiver that receives data used to calculate position. For example, it can be implemented using a GNSS receiver that receives signals from navigation satellites such as GPS (Global Positioning System), Quasi-Zenith Satellite System, GLONASS, Galileo, and BeiDou. The signals received from the navigation satellites by the position sensor 141 are output to the control unit 130. 【0032】 The IMU 142 detects information on the three-dimensional inertial motion of the imaging device 10 (three-axis acceleration information and three-axis angular velocity information). More specifically, the IMU 142 has an acceleration sensor and an angular velocity sensor, and detects the acceleration and angular velocity of the imaging device 10 (output of three-axis acceleration information and three-axis angular velocity information). The information detected by the IMU 142 is output to the control unit 130. 【0033】The geomagnetic sensor 143 detects the Earth's magnetic field and determines direction. The geomagnetic sensor 143 is also called an electronic compass. The information detected by the geomagnetic sensor 143 is output to the control unit 130. 【0034】 The above describes a specific example of the sensor unit 140, but the sensor unit 140 is not limited to this and may have other sensors, or it may not have all of the sensors described above. 【0035】 (Control Unit 130) The control unit 130 functions as an arithmetic processing unit and control unit, and controls the overall operation of the imaging device 10 according to various programs. The control unit 130 is implemented by electronic circuits such as a CPU (Central Processing Unit) or a microprocessor. The control unit 130 may also include a ROM (Read Only Memory) for storing programs and calculation parameters to be used, and a RAM (Random Access Memory) for temporarily storing parameters that change as needed. 【0036】 The control unit 130 in this embodiment can function as a parameter setting unit 131, an imaging control unit 132, a position and orientation estimation unit 133, and an image transmission control unit 134. 【0037】 The parameter setting unit 131 sets various imaging parameters for the imaging unit 120. The parameter setting unit 131 may set various imaging parameters in response to user operation, or it may set various imaging parameters based on information input from the sensor unit 140. 【0038】The image capture control unit 132 controls the imaging performed by the imaging unit 120. The image capture control unit 132 may perform imaging control in response to user operation, or it may set various imaging parameters based on information acquired from the sensor unit 140. The image capture control unit 132 may store the captured image in the storage unit 150. The image capture control unit 132 also adds position information and orientation information of the imaging device 10 at the time of imaging (i.e., information on the imaging position and imaging direction) to the captured image. The image capture control unit 132 also adds information on various imaging parameters at the time of imaging (such as the focal length of the lens, aperture value (F number), shutter speed (SS), ISO sensitivity, and angle of view (field of view (FoV)) to the captured image. 【0039】 User operations may be input from the operation display unit 160, or from an external device (for example, a controller 12) via the communication unit 110. 【0040】 The position and attitude estimation unit 133 estimates the position and attitude of the imaging device 10 based on the information input from the sensor unit 140. For example, the position and attitude estimation unit 133 performs self-position estimation processing based on the signal received by the position sensor 141 from the navigation satellite. The position and attitude estimation unit 133 also performs attitude estimation processing based on the information detected by the IMU 142 and the geomagnetic sensor 143. The position and attitude estimation unit 133 continuously performs position and attitude estimation. 【0041】 The image transmission control unit 134 controls the transmission of the captured image captured by the imaging unit 120 from the communication unit 110 to the information processing device 20. The captured image transmitted to the information processing device 20 is pre-filled with information on the imaging position, imaging direction, and various imaging parameters at the time of imaging by the imaging control unit 132. The addition of various information to the captured image may be performed by the imaging unit 120. There is no particular limit to the interval at which the image transmission control unit 134 transmits the captured image to the information processing device 20. For example, the image transmission control unit 134 may transmit the captured image immediately, transmit it at regular intervals, or transmit it at an appropriate timing depending on the communication status. 【0042】(Memory Unit 150) The memory unit 150 is realized by a storage medium that stores programs used in the processing of the control unit 130, arithmetic parameters, parameters that change as appropriate, and the like. The memory unit 150 according to the present embodiment may store, for example, captured images, information on various imaging parameters, and the history of the position and orientation of the imaging device 10. 【0043】 (Operation Display Unit 160) The operation display unit 160 has a function as an operation unit that receives operation inputs from the user and a function as a display unit that displays various screens. For example, the operation display unit 160 may be realized by a touch panel display. Also, the function as an operation unit and the function as a display unit may be realized separately, or an operation unit or a display unit may be provided separately from the operation display unit 160. The operation unit is realized by, for example, a touch sensor, a switch, a button, or the like. The display unit may be realized by a display panel such as a liquid crystal display (LCD: Liquid Crystal Display) or an organic EL (Electro Luminescence) display. 【0044】 As described above, the configuration of the imaging device 10 has been specifically described, but the configuration of the imaging device 10 according to the present disclosure is not limited to the example shown in FIG. 3. For example, the imaging device 10 may not have the operation display unit 160, or may further have various sensors such as an illuminance sensor. 【0045】 <<2-3. Configuration of Information Processing Device 20>> FIG. 4 is a block diagram showing an example of the configuration of the information processing device 20 according to the present embodiment. As shown in FIG. 4, the information processing device 20 includes a communication unit 210, a control unit 220, and a memory unit 230. 【0046】 (Communication Unit 210) The communication unit 210 has a transmission unit that transmits data to an external device and a reception unit that receives data from an external device. The communication unit 210 according to the present embodiment may communicate and connect to an external device or the Internet using, for example, a wired or wireless LAN, Wi-Fi, Bluetooth, a mobile communication network, or the like. 【0047】The communication unit 210 according to this embodiment functions as an acquisition unit that acquires the captured image from the imaging device 10, and the position information and orientation information of the imaging device 10 attached to the captured image. Further, according to the control of the control unit 220, the communication unit 210 transmits, as display data for the captured result image, the data of the captured image to which the position information, the orientation information, and the imaging parameters are attached, and the three-dimensional processing data to the display terminal 30. 【0048】 (Control unit 220) The control unit 220 functions as an arithmetic processing device and a control device, and controls the overall operation within the information processing device 20 according to various programs. The control unit 220 is realized by an electronic circuit such as a CPU or a microprocessor, for example. Further, the control unit 220 may include a ROM that stores programs and arithmetic parameters to be used, etc., and a RAM that temporarily stores parameters that change as appropriate. 【0049】 Further, the control unit 220 also functions as a three-dimensional processing unit 221. The three-dimensional processing unit 221 generates an object to be arranged in a three-dimensional space based on each captured image, and the position information, orientation information, and imaging parameters attached to each captured image. The generated object can be displayed as a captured result image on the display terminal 30. 【0050】 A plurality of display formats (generation formats) of such an object (captured result image) are prepared. The three-dimensional processing unit 221 outputs (that is, generates and displays and outputs) in a display format that can be generated according to the progress stage of imaging of the object 50 among the plurality of different display formats. As the progress stage of imaging for generating the three-dimensional model progresses, the processing time for generating the image and the number of captured images increase, so that it becomes possible to generate an image in a display format with a more detailed expression. 【0051】Multiple different display formats represent display formats with different levels of processing load. Display formats that provide more detailed representations have a higher level of processing load. The three-dimensional processing unit 221 generates an object to clearly indicate the current progress of imaging (i.e., the imaging results at the present time) using a display format that can be generated at the time of generation (the present time). This makes it possible for this system to generate and present objects that help the user understand the progress of imaging, even in the early stages of the work when the number of captured images and processing time are small, as well as the number of captured images and processing time increase as the imaging work progresses, making it possible to present imaging result images using objects with increasingly detailed representations in stages. 【0052】 More specifically, the multiple different display formats include at least one of a first display format that places a predefined object indicating the imaging position and orientation of each captured image, and a second display format that presents an object of the target 50 modeled based on each captured image. It is assumed that generating the object of the target corresponding to the second display format requires more processing time and a larger number of captured images than generating the object corresponding to the first display format. 【0053】 In the first display format, a predetermined object indicating the position and direction is placed in three-dimensional space according to the imaging position and direction of each image captured of the object 50. The user can intuitively grasp the position and direction of the captured images. If information on the predetermined objects (3D model data) is already available on the display terminal 30, the three-dimensional processing unit 221 may perform three-dimensional processing on the placement information of each predetermined object (for example, converting position information and orientation information to global coordinates) and output the placement information as three-dimensional processing data. 【0054】In the second display format, a 3D model of the object 50 is generated (modeled) based on each captured image of the object 50, and the object 50 is reconstructed in three dimensions. The three-dimensional reconstruction of the object 50 can be performed using a mesh object to reconstruct the shape of the object 50, or using a 3D object of the object 50 (for example, an object that has data such as color information, texture, and material). Thus, the second display format may be further divided into several stages. 【0055】 When using the second display format, the three-dimensional processing unit 221 generates (models) a 3D model of the object 50 to the extent possible at the time of generation. Therefore, depending on the progress of imaging, there may be insufficient three-dimensional data (data obtained from the captured image) of the object 50, resulting in the generation of a 3D model with some parts missing. When the object (mesh object or 3D object) of the object 50 is displayed with some parts missing, the user can intuitively grasp the area of ​​the object 50 that has been captured. The user can determine that there is insufficient three-dimensional data for the missing parts of the object 50 and proceed with the subsequent imaging work efficiently. 【0056】 Specific examples of the image capture results generated and displayed using each display format will be described later. 【0057】 (Storage Unit 240) The storage unit 240 is implemented by a storage medium that stores programs used in the processing of the control unit 220, calculation parameters, parameters that change as needed, etc. 【0058】 The storage unit 240 in this embodiment stores, for example, data for each captured image and three-dimensional processing data. 【0059】Although the configuration of the information processing device 20 has been described in detail above, the configuration of the information processing device 20 according to this disclosure is not limited to the example shown in Figure 4. For example, the information processing device 20 does not necessarily have all the configurations shown in Figure 4. Also, the information processing device 20 may be implemented by multiple devices. Furthermore, at least some of the functions of the information processing device 20 may be provided on a server on a network. For example, the information processing device 20 may further have an operation display unit, an audio input unit, an audio output unit, etc. 【0060】 <<3-3. Configuration of the Display Terminal 30>> Figure 5 is a block diagram showing an example of the configuration of the display terminal 30 according to this embodiment. As shown in Figure 5, the display terminal 30 includes a communication unit 310, an operation display unit 320, a control unit 330, a sensor unit 340, an imaging unit 350, and a storage unit 360. 【0061】 (Communication Unit 310) The communication unit 310 includes a transmitting unit that transmits data to an external device and a receiving unit that receives data from the external device. The communication unit 310 according to this embodiment may communicate with an external device or the Internet using, for example, a wired or wireless LAN, Wi-Fi, Bluetooth, a mobile communication network, etc. 【0062】 For example, the communication unit 310 receives each captured image, position information, orientation information, imaging parameters, and three-dimensional processing data from the information processing device 20. 【0063】 (Operation Display Unit 320) The operation display unit 320 has the function of an operation unit that accepts operation input from the user and the function of a display unit that displays various screens. For example, the operation display unit 320 may be implemented by a touch panel display. Alternatively, the operation unit function and the display unit function may be implemented separately, or an operation unit or display unit may be provided separately from the operation display unit 320. The operation unit may be implemented by, for example, a touch sensor, a switch, a button, etc. The display unit may be implemented by, for example, a display panel such as a liquid crystal display (LCD) or an organic EL (Electro Luminescence) display. 【0064】 (Sensor unit 340 and imaging unit 350) The sensor unit 340 is an acquisition unit that acquires various sensing data. Specifically, the sensor unit 340 includes a position sensor 341, an IMU (Inertial Measurement Unit) 342, and a geomagnetic sensor 343. The display terminal 30 is also provided with an imaging unit 350 facing outwards, which is the opposite direction when the display screen is oriented inwards. The basic functions of each sensor and the imaging unit 350 are the same as described above with reference to Figure 3, so a detailed explanation is omitted here. 【0065】 (Control Unit 330) The control unit 330 functions as an arithmetic processing unit and control unit, and controls the overall operation of the display terminal 30 according to various programs. The control unit 330 is implemented by an electronic circuit such as a CPU or microprocessor. The control unit 330 may also include a ROM for storing the programs and calculation parameters to be used, and a RAM for temporarily storing parameters that change as needed. 【0066】 In this embodiment, the control unit 330 also functions as an image generation unit 331 and a display control unit 332. 【0067】 The image generation unit 331 generates an image showing the imaging results based on the information received from the information processing device 20 (specifically, each captured image, position information, orientation information, imaging parameters, and three-dimensional processing data). The display control unit 332 controls the display of the image generated by the image generation unit 331 on the operation display unit 320. 【0068】 More specifically, the image generation unit 331, based on the three-dimensional processing data, uses each captured image, position information, orientation information, and imaging parameters as needed to render the resulting image. 【0069】 Specifically, for example, the image generation unit 331 may display the imaging results by superimposing objects onto a real-time image of real space input from the imaging unit 350 using AR (Augmented Reality) display. 【0070】The user, for example, points the display terminal 30 (its imaging unit 350) towards the object 50 and displays the captured image of the object 50 on the operation display unit 320. The image generation unit 331 controls the superimposition of objects indicating the position and orientation of each captured image, or objects of the object 50 generated based on each captured image, onto the captured image of the object 50 displayed on the operation display unit 320. Data such as the three-dimensional position of the displayed object and the 3D model of the object may be included in the three-dimensional processing data. Furthermore, when superimposing objects onto the captured image in real space, the image generation unit 331 appropriately uses various sensing data input from the sensor unit 340 (specifically, the position, orientation, and orientation of the display terminal 30, etc.) as needed. The user can intuitively grasp the captured position and range relative to the object 50. Specific display examples will be described later with reference to the drawings. 【0071】 Furthermore, the image generation unit 331 may display the imaging results using VR (Virtual Reality) display, which places objects in a virtual space. Data such as the three-dimensional position of the objects placed in the virtual space and the 3D model of the objects may be included in the three-dimensional processing data. Specific display examples will be described later with reference to the drawings. 【0072】 The three-dimensional processing data may also be drawing data for the captured image (i.e., display screen information). It is also conceivable that the captured image is generated on the information processing device 20 side, and the display screen information is transmitted to the display terminal 30 as three-dimensional processing data. In this case, the display control unit 332 outputs such three-dimensional processing data (display screen information) to the operation display unit 320 to display the captured image. 【0073】 (Storage Unit 360) The storage unit 360 is implemented by a storage medium that stores programs used in the processing of the control unit 330, calculation parameters, parameters that change as needed, etc. 【0074】 The storage unit 360 in this embodiment may store each captured image, position information, orientation information, imaging parameters, and three-dimensional processing data received from the information processing device 20. 【0075】Although the configuration of the display terminal 30 has been described in detail above, the configuration of the display terminal 30 according to this disclosure is not limited to the example shown in Figure 5. For example, the display terminal 30 does not necessarily have all the configurations shown in Figure 5. Also, the display terminal 30 may be implemented by multiple devices. Furthermore, at least some of the functions of the display terminal 30 may be provided in the information processing device 20. 【0076】 <4. Details of Display Format> Next, the display format of the image acquisition result according to this embodiment will be described in detail. 【0077】 <<4-1. Types of Display Formats>> Figure 6 is a diagram illustrating the stepwise display format of the image acquisition result according to this embodiment. As shown in Figure 6, for example, three stages are assumed as stepwise display: Phase 1 (Step S1), Phase 2 (Step S2), and Phase 3 (Step S3). Phase 1 is an example of the first display format, and Phases 2 and 3 are examples of the second display format. 【0078】 Phase 1 involves displaying the position and orientation of each captured image in three-dimensional space using a defined object that indicates the position and orientation. Phase 2 involves displaying the shape of the object modeled based on each captured image as a mesh (display of mesh objects). Phase 3 involves displaying the 3D object of the object modeled based on each captured image. 【0079】 Figure 7 shows a comparison table 400 of the display formats for phases 1 to 3 according to this embodiment. Phases 1, 2, and 3 according to this embodiment provide increasingly detailed display formats. Therefore, as shown in the comparison table 400 in Figure 7, the number of required captured images increases in the order of phase 1, phase 2, and phase 3. Also, the processing time increases in the order of phase 1, phase 2, and phase 3. 【0080】 Furthermore, as shown in the comparison table 400 in Figure 7, the display processing content of Phase 1 is the arrangement of the position and orientation of the imaging device acquired by the imaging device's sensor (arrangement of a defined object indicating the position and orientation). 【0081】 Furthermore, the display processing in Phase 2 involves creating a mesh (mesh object) that shows the shape of the object from multiple captured images. The display processing in Phase 3 involves creating a 3D object of the object from multiple captured images. 【0082】 The mesh objects used in Phase 2 are assumed to be shapes with low processing load and low information content (such as meshes with low resolution, no color information, and large, simple shapes). The 3D objects of the target objects used in Phase 3 are assumed to be shapes with higher processing load and more information content than those in Phase 2 (objects with high resolution and color information, etc.). 【0083】 Furthermore, the 3D objects used in Phase 3 may be of varying processing loads. For example, point cloud objects (colorless point clouds, colored point clouds), 3D objects without textures, and 3D objects with textures can be used. The processing load increases in the order listed, but visibility improves. In Phase 3 as well, the format of the 3D objects displayed may be changed in stages according to the progress of imaging. 【0084】 As shown in the comparison table 400 in Figure 7, the effect of Phase 1 is that the user can determine the current imaging position and direction at the early stages of the work. The effect of Phase 2 is that the user can determine the approximate imaging range (and simultaneously identify the un-imaged areas) at a stage where the work has progressed somewhat. The effect of Phase 3 is that the user can determine the approximate imaging range (and simultaneously identify the un-imaged areas) with better visibility. 【0085】 The types of display formats described above are merely examples, and this embodiment is not limited thereto. 【0086】 <<4-2. Examples of Display Using Each Display Format>> Next, we will explain specific examples of how the captured image results are displayed using each display format. The images in each of the specific examples described below are assumed to be displayed on the operation display unit 320 of the display terminal 30. 【0087】(1) Phase 1 Figure 8 is an example of a virtual space display of the captured image in Phase 1 of this embodiment. As shown in image 510 of Figure 8, a defined object 512 indicating the imaging position and orientation (imaging direction) of each captured image is placed in the virtual space V. Each defined object 512 has the shape of a pyramid. The captured image is displayed on the base of the pyramid, the vertex corresponds to the imaging position (3D position) of the captured image, and the base is oriented in the imaging direction. The display of the defined object 512 may refer to the field of view information of the corresponding captured image. Note that the shape of the defined object 512 shown in Figure 8 is just an example, and this disclosure is not limited thereto; it is sufficient if the imaging position and orientation (imaging direction) of each captured image can be generally understood. 【0088】 By viewing the image 510, the user can intuitively grasp the current imaging situation (imaging position, orientation) from the arrangement of each specified object 512, and take images from locations where imaging is insufficient. 【0089】 Figure 9 shows an example of AR display of the image acquisition result in Phase 1 of this embodiment. Image 520 in Figure 9 is a real-space image (through image) captured by the imaging unit 350 of the display terminal 30. The object 50 is shown in image 520. A defined object 522 indicating the imaging position and orientation (imaging direction) of each image is superimposed on it. The shape of the defined object 522 is the same as that of the defined object 512 described with reference to Figure 8. 【0090】 By viewing the image 520, the user can more intuitively grasp the current imaging situation (imaging position, orientation) from the arrangement of each specified object 522 superimposed on the captured image in real space, and take images from locations where imaging is insufficient. 【0091】 The above describes examples of virtual space display and AR display of the image acquisition results in Phase 1. The choice between virtual space display and AR display can be pre-configured, or a toggle button can be provided to allow the user to switch between them as needed. 【0092】(2) Phase 2 Figure 10 is an example of a virtual space display of the image acquisition result in Phase 2 of this embodiment. In image 530 of Figure 10, a mesh object 532 showing the shape of the object 50 modeled based on each acquired image is placed in the virtual space V. The mesh object 532 consists of, for example, a large number of triangular meshes. Since the mesh object 532 is based on a 3D model of the object 50 that has been generated to the extent that it is possible to generate (model) at the generation time (modeling time), a part of it may be missing depending on the progress of the imaging. 【0093】 By viewing the image 530, the user can intuitively grasp the current imaging range from the shape of the mesh object 532 and proceed to image the missing parts. 【0094】 Figure 11 shows an example of AR display of the captured image in Phase 2 of this embodiment. Image 540 in Figure 11 is a real-space image (through image) captured by the imaging unit 350 of the display terminal 30. The object 50 is shown in image 540. A mesh object 542 representing the shape of the object 50, modeled based on each captured image, is superimposed on this image. 【0095】 By viewing image 540, the user can more intuitively grasp the current imaging range from the shape of the mesh object 542 superimposed on the real-space image, and then proceed to image the missing parts. 【0096】 (3) Phase 3 Figure 12 is an example of a virtual space display of the image acquisition result in Phase 3 of this embodiment. In image 550 of Figure 12, 3D objects 552 of the object 50 modeled based on each acquired image are placed in the virtual space V. The 3D object 552 is, for example, an object with a texture. Since the 3D object 552 is based on the 3D model of the object 50 that has been generated to the extent possible at the time of generation, a part of it may be missing depending on the progress of the imaging. 【0097】By viewing image 550, the user can intuitively grasp the current imaging range from the state of the 3D object 552 and perform imaging of the missing parts. 【0098】 Figure 13 shows an example of AR display of the captured image in Phase 3 of this embodiment. Image 560 in Figure 13 is a real-space image (through image) captured by the imaging unit 350 of the display terminal 30. The object 50 is shown in image 560. A 3D object 562 of the object 50, modeled based on each captured image, is superimposed on it. 【0099】 By viewing image 560, the user can more intuitively grasp the current imaging range from the state of the 3D object 562 and perform imaging of the missing parts. 【0100】 (4) Combined Example The information processing system 1 according to this embodiment may generate and display an image capture result image by arbitrarily combining the display contents of each of the above-described phases 1 to 3. 【0101】 Figure 14 shows an example of a virtual space display of the image acquisition result combining Phase 1 and Phase 2 of this embodiment. For example, when it is possible to display up to Phase 2, it is possible to combine it with Phase 1. Also, when it is possible to display up to Phase 3, it is possible to combine it with Phase 1 or with Phase 2. 【0102】 In image 570 of Figure 14, a defined object 572 indicating the imaging position and orientation (imaging direction) of each image, and a mesh object 574 indicating the shape of the object 50 modeled based on each image, are placed in the virtual space V. 【0103】 By viewing image 570, the user can intuitively grasp the current imaging situation (imaging position and orientation), and at the same time, intuitively grasp the current imaging range from the state of the mesh object 574, enabling them to more accurately image the missing parts. 【0104】The content of the image acquisition results in Phases 1 to 3 has been explained in detail above. Note that Phases 1 to 3 may be automatically changed by the information processing system 1 when transitions become possible. Furthermore, the information processing system 1 may prioritize transitioning to the phase corresponding to the most detailed display format among the transitionable Phases 1 to 3. Additionally, a phase switching button may be provided to allow the user to switch phases at will. 【0105】 <5. Operation Processing> Figure 15 is a flowchart showing an example of the operation processing flow of the imaging device 10 in this embodiment. 【0106】 As shown in Figure 15, first, the imaging device 10 performs imaging (step S103) and detects the position and orientation of the imaging device 10 (step S106). 【0107】 Next, the imaging device 10 performs control to transmit the captured image and imaging information (position information, orientation information, imaging parameters) to the information processing device 20 (step S109). 【0108】 The imaging device 10 repeats steps S103 to S109 until imaging is completed (step S112). 【0109】 Figure 16 is a flowchart showing an example of the operation flow of the information processing device 20 in this embodiment. 【0110】 As shown in Figure 16, first, the information processing device 20 receives captured images and imaging information from one or more imaging devices 10 (step S203). 【0111】 Next, the information processing device 20 performs each of the processes in phases 1 to 3. 【0112】 Specifically, as part of the Phase 1 processing, the information processing device 20 organizes the position and orientation information of each captured image (step S206). For example, the information processing device 20 may perform arrangement processing of each captured image in three-dimensional space. 【0113】Furthermore, as part of the Phase 2 processing, the information processing device 20 generates a mesh indicating the shape of the object using each captured image (step S209). Also, as part of the Phase 3 processing, the information processing device 20 generates an object of the object using each captured image (step S212). In Phases 2 and 3, for example, the information processing device 20 generates a 3D model of the object 50 based on each captured image. Steps S206 to S212 may be performed in parallel. 【0114】 Next, if Phase 2 can be generated after processing Phase 1 (step S215 / Yes), or if Phase 3 can be generated (step S218 / Yes), the information processing device 20 controls the transmission of information for each phase to the display terminal 30 (step S221). The information for each phase refers to the processing results for each phase, and more specifically, includes captured images, imaging information, and three-dimensional processing data (3D model of the object 50, mesh object, 3D object). 【0115】 The information processing device 20 repeats steps S203 to S221 until imaging is completed (step S224). 【0116】 Figure 17 is a flowchart showing an example of the operation process flow of the display terminal 30 in this embodiment. 【0117】 As shown in Figure 17, first, the display terminal 30 receives information for each phase (processing results for each phase) from the information processing device 20 (step S303). 【0118】 Next, if the display terminal 30 has information for Phase 3 in the received information (step S306 / Yes), it enables the display of Phase 3 (display of 3D objects) (step S309). 【0119】 Next, if the display terminal 30 receives information about Phase 2 (step S312 / Yes), it enables the display of Phase 2 (display of mesh objects) (step S315). 【0120】Next, if the display terminal 30 has information for Phase 1 in the received information (step S318 / Yes), it activates the display for Phase 1 (display of imaging location) (step S321). 【0121】 Next, if AR display is ON (step S324 / Yes), the display terminal 30 acquires a through-image of the real space and position and orientation information of the camera (or the display terminal 30 if the through-image was acquired by the imaging unit 350 of the display terminal 30) (step S327). 【0122】 Then, the display terminal 30 superimposes the activated display information (each specified object indicating the imaging position and orientation, a mesh object indicating the shape of the object 50, or a 3D object of the object 50) onto the through image (step S330). The display terminal 30 may also prioritize displaying the more detailed phases among the activated phases 1 to 3. 【0123】 On the other hand, if the AR display is not turned ON (step S324 / No), the display terminal 30 displays the virtual space (step S336). The display terminal 30 may have already constructed the virtual space. 【0124】 The display terminal 30 places the activated display information within the virtual space (step S339). The display terminal 30 may also prioritize displaying the more detailed phases among the activated phases 1 to 3. 【0125】 The display terminal 30 repeats steps S303 to S339 until imaging is completed (step S342). As a result, for example, in the initial stages of the imaging process, only phase 1 is enabled because the amount of captured images is insufficient, and phase 1 is displayed. As the imaging process progresses and the amount of captured images increases, phases 2 and 3 are sequentially enabled, and phases 2 and 3 are displayed. In this embodiment, as the imaging process progresses, it becomes possible to clearly indicate the imaging results with a more detailed representation. 【0126】The operation process flow according to this embodiment has been described above. Note that the operation process shown in Figures 15 to 17 is just an example, and this disclosure is not limited thereto. For example, not all of the processes shown in Figures 15 to 17 are performed, and the order of some of the processes shown in Figures 15 to 17 may be different. 【0127】 <6. Modification> <<6-1. First Modification>> The configuration of the information processing system 1 is not limited to the example described above. For example, the functions of the information processing device 20 may be realized in the display terminal 30. The following explanation will be given with reference to Figure 18. 【0128】 Figure 18 is a block diagram showing an example of the configuration of a display terminal 30A according to a modified example of this embodiment. As shown in Figure 18, the display terminal 30A includes a communication unit 310, an operation display unit 320, a control unit 330A, a sensor unit 340, an imaging unit 350, and a storage unit 360. 【0129】 In this modified configuration, the control unit 330A functions as an image generation unit 331, a display control unit 332, and a three-dimensional processing unit 333. The function of the three-dimensional processing unit 333 is the same as that of the three-dimensional processing unit 221 of the information processing device 20 described with reference to Figure 4. 【0130】 Thus, in the information processing system according to the first modification, three-dimensional processing may be performed at the display terminal 30. Captured images can be transmitted to the display terminal 30 directly from each imaging device 10 or via a server or the like. 【0131】<<6-2. Second Modification>> In the above-described embodiment, the imaging device 10 is described as estimating the position and orientation of the image and assigning position and orientation information to each image. However, depending on the performance of the imaging device 10, it may not be possible to assign position and orientation information. In this case, the information processing system 1 may, for example, use the three-dimensional processing unit 221 of the information processing device 20 to perform a process to estimate the position and orientation of each image based on a large number of images. In this case, the display format that generates a mesh object of the object 50 is designated as Phase 1, the display format that shows the position and orientation of each image estimated based on a large number of images using a defined object is designated as Phase 2, and the display format that generates a 3D object of the object 50 is designated as Phase 3. 【0132】 <<6-3. Third Modification>> In the embodiments described above, the case in which the captured image (for example, the images in Figures 8 to 14) is displayed on the display terminal 30 has been explained, but the disclosure is not limited thereto, and for example the captured image may be displayed on the controller 12. Also, the captured image may be displayed in common on one or more display terminals 30 and one or more controllers 12 used by each user U. 【0133】 When the captured image is displayed in AR on the controller 12, the controller 12 displays the through-image from the drone's camera, and objects (predefined objects, mesh objects, or 3D objects) may be superimposed on this through-image as appropriate. 【0134】 <7. Hardware Configuration> An embodiment of the present disclosure has been described above. Next, with reference to Figure 19, an example of a hardware configuration used in an imaging device 10, an information processing device 20, or a display terminal 30 according to an embodiment of the present disclosure will be described. 【0135】Figure 19 is a block diagram showing an example of the hardware configuration of an information processing device 900 according to one embodiment of the present disclosure. The information processing device 900 is an example of a hardware configuration applicable to the imaging device 10, information processing device 20, or display terminal 30 according to this embodiment. Note that the information processing device 900 does not necessarily have all of the hardware configurations shown in Figure 19. 【0136】 As shown in Figure 19, the information processing device 900 includes a processing circuit 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903. The information processing device 900 may also include a host bus 907, a bridge 909, an external bus 911, an interface 913, an input device 915, an output device 917, a storage device 919, a drive 921, a connection port 923, and a communication device 925. 【0137】 The processing circuit 901 functions as an arithmetic processing unit and control unit, and controls the overall operation or a part of the operation within the information processing unit 900 according to various programs recorded in the ROM 902, RAM 903, storage device 919, or removable recording medium 927. The ROM 902 stores programs and calculation parameters used by the processing circuit 901. The RAM 903 temporarily stores programs used in the execution of the processing circuit 901 and parameters that change as appropriate during its execution. The processing circuit 901, ROM 902, and RAM 903 are interconnected by a host bus 907, which is composed of an internal bus. Furthermore, the host bus 907 is connected to an external bus 911, such as a PCI (Peripheral Component Interconnect / Interface) bus, via a bridge 909. 【0138】The input device 915 is a device operated by the user, such as a button. The input device 915 may also include a mouse, keyboard, touch panel, switch, and lever. The input device 915 may also include a microphone that detects the user's voice. The input device 915 may be, for example, a remote control device that uses infrared or other radio waves, or an external connection device 929 such as a mobile phone that is compatible with the operation of the information processing device 900. The input device 915 includes an input control circuit that generates an input signal based on information input by the user and outputs it to the processing circuit 901. By operating this input device 915, the user inputs various data to the information processing device 900 or instructs it to perform processing operations. 【0139】 The input device 915 may also include an imaging device and sensors. The imaging device is a device that captures real space and generates an image using various components such as an image sensor, such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and a lens for controlling the imaging of a subject onto the image sensor. The imaging device may capture still images or motion images. The sensors are various types of sensors, such as distance sensors, acceleration sensors, gyro sensors, geomagnetic sensors, vibration sensors, light sensors, and sound sensors. The sensors acquire information about the state of the information processing device 900 itself, such as the orientation of the housing of the information processing device 900, and information about the surrounding environment of the information processing device 900, such as the brightness and noise around the information processing device 900. The sensors may also include a GPS sensor that receives GPS (Global Positioning System) signals and measures the latitude, longitude, and altitude of the device. 【0140】The output device 917 is comprised of a device capable of visually or audibly notifying the user of the acquired information. The output device 917 may be, for example, a display device such as an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display, or an audio output device such as a speaker or headphones. The output device 917 may also include a PDP (Plasma Display Panel), a projector, a hologram, a printer, etc. The output device 917 outputs the results obtained from the processing of the information processing device 900 as text or images, or as sound such as voice or acoustics. The output device 917 may also include a lighting device that brightens the surroundings. 【0141】 The storage device 919 is a data storage device configured as an example of the storage unit of the information processing device 900. The storage device 919 is composed of, for example, a magnetic storage device such as an HDD (Hard Disk Drive), a semiconductor storage device, an optical storage device, or a magneto-optical storage device. This storage device 919 stores programs and various data executed by the processing circuit 901, as well as various data acquired from external sources. 【0142】 The drive 921 is a reader / writer for removable recording media 927 such as magnetic disks, optical disks, magneto-optical disks, or semiconductor memory, and is either built into or external to the information processing device 900. The drive 921 reads information recorded on the installed removable recording media 927 and outputs it to the RAM 905. The drive 921 also writes data to the installed removable recording media 927. 【0143】The connection port 923 is a port for directly connecting equipment to the information processing device 900. The connection port 923 may be, for example, a USB (Universal Serial Bus) port, an IEEE 1394 port, or a SCSI (Small Computer System Interface) port. Alternatively, the connection port 923 may be an RS-232C port, an optical audio terminal, or an HDMI (High-Definition Multimedia Interface) port. By connecting an external device 929 to the connection port 923, various types of data can be exchanged between the information processing device 900 and the external device 929. 【0144】 The communication device 925 is a communication interface, for example, consisting of a communication device for connecting to an external network 60. The communication device 925 may be, for example, a communication card for wired or wireless LAN (Local Area Network), Bluetooth®, Wi-Fi®, or WUSB (Wireless USB). Alternatively, the communication device 925 may be a router for optical communication, an ADSL (Asymmetric Digital Subscriber Line) router, or a modem for various types of communication. The communication device 925 transmits and receives signals, for example, to the Internet or other communication devices using a predetermined protocol such as TCP / IP. The external network 60 connected to the communication device 925 is a network connected by wire or wireless, for example, the Internet, a home LAN, infrared communication, radio wave communication, or satellite communication. 【0145】 For example, when the information processing device 900 functions as an imaging device 10, an information processing device 20, or a display terminal 30 according to the embodiments of this disclosure, the processing circuit 901 of the information processing device 900 functions as a control unit 130, a control unit 220, or a control unit 330 by executing a program loaded on the RAM 903. The storage device 919 stores the information processing program according to this disclosure and various data stored in the storage unit 150, storage unit 230, or storage unit 360. 【0146】 The processing circuit 901 reads and executes program data from the storage device 919, but as an alternative, these programs may be obtained from other devices via the external network 60. In other words, the storage device 919 is not limited to being inside the information processing device 900, but may be located outside the information processing device 900. 【0147】 The processing circuit 901 is an example of an integrated circuit, and CPU (Central Processing Unit), MPU (Micro Processing Unit), GPU (Graphics Processing Unit), APU (Accelerated Processing Unit), ASIC (Application Specific Integrated Circuit), and FPGA (Field Programmable Gate Array) can all be considered integrated circuits. 【0148】 Furthermore, when the information processing device 900 functions as an imaging device 10, an information processing device 20, or a display terminal 30 according to the embodiments of this disclosure, the communication device 925 corresponds to the communication unit 110, the communication unit 210, or the communication unit 310. Also, the input device 915 and the output device 917 correspond to the operation display unit 160 or the operation display unit 320. In addition, the input device 915 corresponds to the imaging unit 120, the sensor unit 140, the sensor unit 340, or the imaging unit 350. 【0149】 <8. Supplementary Information> Although preferred embodiments of the present disclosure have been described in detail above with reference to the attached drawings, the present technology is not limited to such examples. It is clear that a person with ordinary skill in the art of the present disclosure may conceive of various modifications or alterations within the scope of the technical idea described in the claims, and these will naturally be understood to fall within the technical scope of the present disclosure. 【0150】Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions. 【0151】 The information processing system according to this disclosure may consist of a single device comprising an imaging device 10, an information processing device 20, a display terminal 30, or a server (not shown). Furthermore, the information processing system according to this disclosure may consist of multiple devices. These multiple devices may, for example, be the imaging device 10, the information processing device 20, and the display terminal 30, or the imaging device 10 and the display terminal 30A. 【0152】 Furthermore, the embodiments and modifications of this disclosure described above can be combined as appropriate in areas where the processing content is not contradictory. Also, the order of each step shown in the sequence diagram or flowchart of this embodiment can be changed as appropriate. For example, each step may be processed chronologically, repeatedly, or partially in parallel. 【0153】 Furthermore, one or more computer programs can be created to enable the imaging device 10, information processing device 20, or display terminal 30 to perform their respective functions. A computer-readable storage medium containing these one or more computer programs is also provided. 【0154】 Furthermore, the effects described herein are merely descriptive or illustrative and not limiting. In other words, the technology relating to this disclosure may produce other effects that will be apparent to those skilled in the art from the description herein, in addition to or in lieu of the effects described herein. 【0155】Furthermore, this technology can also be configured as follows: (1) An information processing method comprising: a processor acquiring each captured image obtained by each imaging device imaging a common object, and position and orientation information of the imaging device attached to each captured image; and outputting information for displaying an image showing the result of imaging the object, generated based on the acquired information, in a display format that can be generated according to the progress stage of imaging the object from among a plurality of different display formats. (2) The information processing method according to (1), wherein the plurality of different display formats are a plurality of display formats that differ in processing load level and level of detail of expression. (3) The information processing method according to (2), wherein as the progress stage of imaging advances, it becomes possible to generate images using a display format with more detailed expression, and a display format with more detailed expression requires more processing time and a larger number of captured images to generate the image. (4) The information processing method according to (1), wherein the plurality of different display formats include at least one of a first display format that arranges a predetermined object indicating the imaging position and direction of each captured image, and a second display format that presents an object of the target that is generated based on each captured image. (5) The information processing method according to (4), wherein the second display format requires more processing time and a larger number of captured images than the first display format. (6) The information processing method according to (4) or (5), wherein the second display format includes a display format that performs three-dimensional reconstruction of the target using a mesh object that shows the shape of the target, and a display format that performs three-dimensional reconstruction of the target using a 3D object of the target. (7) The information processing method according to (6), wherein the mesh object contains less information than the 3D object. (8) The information processing method according to any one of (4) to (7), wherein the objects of the target that are generated based on each captured image are generated to the extent that they can be generated at the time of generation. (9) The information processing method according to any one of (1) to (8), wherein the processor generates images showing the results of the imaging in all possible display formats, and controls the processor to preferentially display the image generated in the display format that provides the most detailed representation among the generated images.(10) The information processing method according to (9), wherein the processor controls the display of an image specified by the user from among the generated images. (11) The information processing method according to (8) or (9), wherein the processor further generates an image by combining each image generated in a different display format. (12) The information processing method according to any one of (4) to (11), wherein the processor updates an image showing the result of imaging the object based on an image captured in real time. (13) The information processing method according to any one of (1) to (12), wherein the processor outputs an image that is superimposed on an image captured in real space as an image showing the result of imaging the object. (14) The information processing method according to (13), wherein the image that is superimposed on the image captured in real space is a defined object that indicates the imaging position and direction of each image captured. (15) The information processing method according to (13), wherein the image that is superimposed on the image captured in real space is an object of the object generated based on each image captured. (16) The information processing method according to any one of (1) to (12), wherein the processor outputs a virtual space image as an image showing the result of imaging the object. (17) The information processing method according to (16), wherein the virtual space image is an image in which a defined object indicating the imaging position and direction of each imaging image is placed in a virtual space. (18) The information processing method according to (16), wherein the image superimposed on the imaging image in real space is an image showing the object of the object generated based on each imaging image. (19) An information processing device comprising a control unit that performs: a process of acquiring each imaging image obtained by each imaging device imaging an object common to each imaging device, and position and orientation information of the imaging device attached to each imaging image; and a process of outputting information for displaying an image showing the result of imaging the object, generated based on the acquired information, in a display format that can be generated from a plurality of different display formats according to the progress stage of imaging the object.(20) A program for causing a computer to function as a control unit that performs the following: a process of acquiring each image obtained by each imaging device imaging an object common to that object, and position and orientation information of the imaging device attached to each image; and a process of outputting information for displaying an image showing the result of imaging the object, which is generated based on the acquired information, in a display format that can be generated from among a plurality of different display formats according to the progress stage of imaging the object. 【0156】 1 Information Processing System 10 Imaging Device 110 Communication Unit 120 Imaging Unit 130 Control Unit 131 Parameter Setting Unit 132 Imaging Control Unit 133 Position and Orientation Estimation Unit 134 Image Transmission Control Unit 140 Sensor Unit 141 Position Sensor 142 IMU 143 Geomagnetic Sensor 150 Storage Unit 160 Operation Display Unit 20 Information Processing Device 210 Communication Unit 220 Control Unit 221 Three-Dimensional Processing Unit 230 Storage Unit 30 Display Terminal 310 Communication Unit 320 Operation Display Unit 330 Control Unit 331 Image Generation Unit 332 Display Control Unit 340 Sensor Unit 350 Storage Unit

Claims

1. An information processing method comprising: a processor acquiring each captured image obtained by each imaging device capturing a common object, and position and orientation information of the imaging device attached to each captured image; and outputting information for displaying an image showing the result of imaging the object, generated based on the acquired information, in a display format that can be generated from among a plurality of different display formats according to the progress stage of imaging the object.

2. The information processing method according to claim 1, wherein the plurality of different display formats are plurality of display formats that differ in the level of processing load and the level of detail of representation.

3. The information processing method according to claim 2, wherein as the imaging progresses, it becomes possible to generate images with a more detailed display format, and a more detailed display format requires more processing time and more images to generate the image.

4. The information processing method according to claim 1, wherein the plurality of different display formats include at least one of a first display format that arranges a defined object indicating the imaging position and orientation of each captured image, and a second display format that presents the object of the target generated based on each captured image.

5. The information processing method according to claim 4, wherein the second display format requires more processing time and more captured images than the first display format.

6. The information processing method according to claim 4, wherein the second display format includes a display format for performing a three-dimensional reconstruction of the object using a mesh object that shows the shape of the object, and a display format for performing a three-dimensional reconstruction of the object using a 3D object of the object.

7. The information processing method according to claim 6, wherein the mesh object contains less information than the 3D object.

8. The information processing method according to claim 4, wherein the objects of the target object generated based on each captured image are generated to the extent possible at the time of generation.

9. The information processing method according to claim 1, wherein the processor generates images showing the results of the imaging in all possible display formats, and controls the processor to preferentially display the image generated in the display format that provides the most detailed representation among the generated images.

10. The information processing method according to claim 9, wherein the processor controls the display of an image specified by the user from among the generated images.

11. The information processing method according to claim 8, wherein the processor further generates an image by combining each image generated in a different display format.

12. The information processing method according to claim 4, wherein the processor updates an image showing the result of imaging the object based on an image captured in real time.

13. The information processing method according to claim 1, wherein the processor outputs an image superimposed on the captured image in real space as an image showing the result of imaging the object.

14. The information processing method according to claim 13, wherein the image superimposed on the captured image of real space is a defined object indicating the imaging position and direction of each captured image.

15. The information processing method according to claim 13, wherein the image superimposed on the captured image of real space is an object of the target generated based on each captured image.

16. The information processing method according to claim 1, wherein the processor outputs a virtual space image as an image showing the result of imaging the object.

17. The information processing method according to claim 16, wherein the virtual space image is an image in which a defined object indicating the imaging position and direction of each captured image is placed in the virtual space.

18. The information processing method according to claim 16, wherein the image superimposed on the captured image of real space is an image representing the object of the target, generated based on each captured image.

19. An information processing device comprising: a control unit that performs the following processes: acquiring each captured image obtained by each imaging device imaging a common object, and position and orientation information of the imaging device attached to each captured image; and outputting information for displaying an image showing the result of imaging the object, generated based on the acquired information, in a display format that can be generated from among a plurality of different display formats according to the progress stage of imaging the object.

20. A program for causing a computer to function as a control unit that performs the following: a process of acquiring each captured image obtained by each imaging device capturing a common object, and position and orientation information of the imaging device attached to each captured image; and a process of outputting information for displaying an image showing the result of imaging the object, generated based on the acquired information, in a display format that can be generated according to the progress stage of imaging the object, from among a plurality of different display formats.

Images