Information processing device, information processing method, and program

Abstract

Achieve the objective of utilizing the mobile device. [Solution] The reception unit 121 receives the imaging area specified by the user, which is the area to be imaged by the mobile body 2, the delivery date for providing the three-dimensional structure to be reconstructed from the imaged images, and the specified quality, which includes the quality related to the positional accuracy of the reconstructed three-dimensional structure. The determination unit 122 determines the images to be acquired for the imaging area, provided that the specified quality is met. The prediction unit 123 predicts the encoding time required for image encoding and the transmission time required to transmit the encoded images to the server 3 that performs reconstruction processing, based on the number of images to be acquired, and predicts the reconstruction time required for reconstruction based on the number of images and the computing resources of the server 3 allocated to the user. The generation unit 124 generates an acquisition plan, including the route of the mobile body 2, for acquiring the images to be acquired within the available time calculated by subtracting the encoding time, transmission time, and reconstruction time from the time until the delivery date.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] In recent years, unmanned moving bodies such as drones (hereinafter simply referred to as "moving bodies" in this specification) have been rapidly spreading, and technologies for utilizing moving bodies have also been developing. For example, Patent Document 1 discloses a technique for dynamically considering and comprehensively determining the noise, vibration, and the possibility of falling objects generated during the movement of a moving body, and calculating a route that minimizes the probability of generating noise, vibration, and falling objects.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the utilization of a moving body, not only the processing executed during the movement of the moving body but also post-processing to be performed after the movement may be required. In such a case, even if only the processing executed during the movement of the moving body is considered, there is a possibility that the purpose of utilizing the moving body cannot be sufficiently achieved.

[0005] Therefore, the present invention has been made in view of these points, and an object thereof is to provide a technology capable of achieving the purpose of utilizing a moving body.

Means for Solving the Problems

[0006] A first aspect of the present invention is an information processing device. This information processing device includes: a receiving unit that receives an imaging area specified by a user, which is an area to be imaged by a moving object; a delivery date for providing a three-dimensional structure to be reconstructed from the imaged image; and specified quality, which includes quality related to the positional accuracy of the reconstructed three-dimensional structure; a determination unit that determines images to be acquired for the imaging area, provided that the specified quality is met; a prediction unit that predicts the encoding time required for image encoding and the transmission time required to transmit the encoded images to a server that performs reconstruction processing, based on the number of images to be acquired, and predicts the reconstruction time required for reconstruction based on the number of images and the computing resources of the server allocated to the user; and a generation unit that generates an acquisition plan, including the path of a moving object, for acquiring the images to be acquired within the available time calculated by subtracting the encoding time, the transmission time, and the reconstruction time from the time until the delivery date.

[0007] The information processing device may further include a mobile body control unit that moves the mobile body along the path and causes the mobile body to capture an image, and a modification unit that modifies the acquisition plan when the prediction unit identifies a predetermined delay in the acquisition plan, provided that the specified quality is met.

[0008] The information processing device may further include an encoding unit that encodes the acquired image using an encoding profile from among a plurality of encoding profiles that results in a relatively shorter transmission time for the acquired image captured by the moving object, provided that the specified quality is met, and a transmission unit that transmits the encoded acquired image to the server in the order of the processing units for reconstruction by the server.

[0009] The information processing device may further include a reconstruction startup unit that calculates the computing resources of the server required to reconstruct the encoded acquired image by the deadline and allocates the computing resources, and instructs the server to perform pre-processing for reconstruction on the encoded acquired image for each processing unit; and a server control unit that, when the prediction unit determines from the time remaining until the deadline, the progress of the reconstruction process, and the allocated computing resources of the server that the reconstruction will not be completed by the deadline, calculates the computing resources of the server required to perform the reconstruction by the deadline and allocates the computing resources.

[0010] The determination unit may further determine the image quality, including the ground resolution, for each of the planned images to be captured, provided that it satisfies the specified quality, and the generation unit may further generate the acquisition plan for each of the planned images to be captured, which includes at least one of setting the altitude of the moving object and setting the camera parameters used for capturing the image.

[0011] The generation unit may generate the path of the moving body as a set of line segments connecting one or more waypoints that the moving body should pass through in order to capture the planned image, and may generate the acquisition plan which further includes the ratio of oblique imaging, which is an image captured at an angle greater than or equal to a predetermined angle with respect to the vertical, and the imaging density. The moving body control unit may receive from the moving body performing imaging the relative speed of the moving body with respect to the ground, the time required when the moving body performs a turning motion, and an index indicating the clarity of each acquired image captured by the moving body at predetermined intervals. The modification unit may (1) modify the acquisition plan by adjusting at least one of the imaging density, the ratio of oblique imaging, the order in which the one or more waypoints are reached, the moving speed of the moving body, and the camera parameters used for capturing the image. (2) If the moving body control unit identifies that any of the indicators in the acquired image is outside a predetermined range, it may adjust at least one of the camera parameters and the moving speed of the moving body, on the condition that the indicator falls within the predetermined range.

[0012] The encoding unit may (1) use an encoding profile as a candidate for the encoding profile, which includes at least one setting from the following: setting the encoding method, setting the quantization, setting the bitrate, setting the GOP (Group of Picture) configuration, setting the spatial resolution, and setting the color difference subsampling; and (2) impose a constraint that the acquired image after encoding must satisfy predetermined conditions necessary to satisfy the specified quality, by encoding each of the acquired images selected from the acquired images using the encoding profile, and the acquired image after encoding must satisfy predetermined conditions necessary to satisfy the specified quality.

[0013] The information processing device may further include a storage unit that stores a learning model that has learned to output the encoding time, the transmission time, and the reconstruction time, taking the number of captured images and the available computing resources of the server as inputs, or a regression model that regressively predicts the encoding time, the transmission time, and the reconstruction time, taking the number of captured images and the available computing resources of the server as inputs. The prediction unit may input the number of images to be captured and the computing resources of the server allocated to the user to the learning model or the regression model, and may predict the encoding time, the transmission time, and the reconstruction time based on the results output by the learning model or the regression model.

[0014] The determination unit may divide the imaging area into a plurality of sub-regions, and may determine the image to be acquired for each sub-region; the generation unit may generate the path of the moving object as a series of closed-loop paths that target each of the sub-regions, and which can generate a point cloud of the sub-regions that is sparser than the three-dimensional structure provided to the user, using only the images captured from the paths; the transmission unit may use the sub-regions as the processing unit; and the reconstruction activation unit may independently activate the server a pre-processing step for reconstruction to generate the point cloud of the sub-regions for each sub-region.

[0015] The receiving unit may further receive a priority policy indicating which of the delivery date and the specified quality should be prioritized, and the information processing device may further include a notification unit that, if the prediction unit determines that the total time required to perform any unfinished processes among the execution of the acquisition plan by the mobile device, the encoding process by the encoding unit, the transmission process by the transmission unit, and the reconstruction process by the server is longer than the time until the delivery date, then prioritize either the delivery date or the specified quality indicated by the priority policy.

[0016] The generation unit may generate the acquisition plan which further includes imaging density, the modification unit may modify the acquisition plan which reduces the imaging density included in the acquisition plan which, when the reception unit notifies that the delivery date should be prioritized, the acquired image after encoding satisfies the quality requirement for reconstruction, and the encoding unit may encode the acquired image obtained in accordance with the modified acquisition plan which uses an encoding profile which results in a relatively shorter transmission time, provided that the acquired image after encoding has the quality requirement for reconstruction.

[0017] If the server control unit receives notification from the reception unit to prioritize the specified quality, it may further output at least one of the point clouds for each of the subregions and the model obtained by integrating the point clouds.

[0018] A second aspect of the present invention is an information processing method. This information processing method involves a processor receiving an imaging area, which is an area to be imaged by a moving object, as specified by a user; a delivery date for providing a three-dimensional structure to be reconstructed from the imaged images; and specified quality, which includes quality related to the positional accuracy of the reconstructed three-dimensional structure; determining images to be acquired for the imaging area, provided that the specified quality is met; predicting the encoding time required for image encoding and the transmission time required to transmit the encoded images to a server that performs reconstruction processing, based on the number of images to be acquired; predicting the reconstruction time required for reconstruction based on the number of images to be acquired and the computing resources of the server allocated to the user; and generating an acquisition plan, including the path of the moving object, for acquiring the images to be acquired within the available time calculated by subtracting the encoding time, the transmission time, and the reconstruction time from the time until the delivery date.

[0019] A third aspect of the present invention is a program. This program enables a computer to implement the following functions: an imaging area specified by a user, which is an area to be imaged by a moving object; a delivery date for providing a three-dimensional structure to be reconstructed from the imaged images; a specified quality, which includes quality related to the positional accuracy of the reconstructed three-dimensional structure; a function to determine the images to be captured for the imaging area, provided that the specified quality is met; a function to predict the encoding time required for image encoding and the transmission time required to transmit the encoded images to a server that performs reconstruction processing, based on the number of images to be captured; a function to predict the reconstruction time required for reconstruction based on the number of images to be captured and the computing resources of the server allocated to the user; and a function to generate an acquisition plan, including the path of the moving object, for capturing the images within the available time calculated by subtracting the encoding time, transmission time, and reconstruction time from the time until the delivery date.

[0020] A computer-readable recording medium recording this program may be provided in order to provide this program or update a part of the program, and this program may also be transmitted via a communication line.

[0021] In addition, any combination of the above components, and those obtained by converting the expression of the present invention among a method, an apparatus, a system, a computer program, a data structure, a recording medium, etc. are also effective as aspects of the present invention.

Advantages of the Invention

[0022] According to the present invention, the purpose of utilization of a moving body can be achieved.

Brief Description of the Drawings

[0023] [Figure 1] It is a schematic diagram for explaining an outline of processing executed by an information processing apparatus according to an embodiment. [Figure 2] It is a diagram schematically showing a functional configuration of an information processing apparatus according to an embodiment. [Figure 3] It is a diagram schematically showing a data structure of user order information for managing an imaging area, a delivery date, and a specified quality specified by a user. [Figure 4] It is a diagram schematically showing a data structure of acquisition plan information for managing an acquisition plan. [Figure 5] It is a schematic diagram for explaining a process in which a correction unit corrects an acquisition plan. [Figure 6] It is a diagram schematically showing a data structure of encoding profile information for managing an encoding profile. [Figure 7] It is a schematic diagram for explaining a process in which a correction unit adjusts the arrival order of waypoints. [Figure 8] It is a schematic diagram for explaining a partial area. [Figure 9] It is a flowchart for explaining a flow of information processing executed by an information processing apparatus according to an embodiment. [Modes for carrying out the invention]

[0024] <Overview of the Embodiment> The information processing device according to this embodiment is used to provide a user with a three-dimensional structure reconstructed by a server from an image captured by a moving object. Here, the three-dimensional structure is, for example, a three-dimensional point cloud or a three-dimensional mesh generated by SfM (Structure from Motion). The three-dimensional structure may also be represented in a format such as NeRF (Neural Radiance Fields) or 3D Gaussian Splatting.

[0025] The user specifies, for example, the imaging area to be imaged by a moving object for surveying purposes, the quality of the reconstructed three-dimensional structure, and the delivery date. The information processing device according to the embodiment is used to provide the user with a three-dimensional structure of the imaging area reconstructed by the server to satisfy the specified quality and delivery date conditions. Here, the user may use the information processing device, or a provider who has received a request from the user to provide a three-dimensional structure may use the information processing device.

[0026] Figure 1 is a schematic diagram illustrating the overview of the processing performed by the information processing device 1 according to the embodiment. The information processing device 1 according to the embodiment constitutes a part of the three-dimensional structure reconstruction system S. The three-dimensional structure reconstruction system S includes the information processing device 1, the mobile unit 2, and the server 3. The three-dimensional structure reconstruction system S may also include other terminals, devices, etc.

[0027] Information processing device 1 is a device that controls the mobile body 2 and server 3 to manage the reconstruction of the three-dimensional structure. Information processing device 1 can be, for example, a tablet terminal, a personal computer, a single-board computer, or a drone dock which is a base for drones to charge and communicate data.

[0028] Mobile device 2 is, for example, a drone, equipped with a camera that captures images of imaging area A and a memory device that stores the captured images. Mobile device 2 is an unmanned aerial vehicle such as a UAV (Unmanned Aerial Vehicle), UGV (Unmanned Ground Vehicle), or USV (Unmanned Surface Vehicle).

[0029] Server 3 is a device that reconstructs a three-dimensional structure from images captured by the mobile object 2. Server 3 may be a single computer or multiple computers. Alternatively, Server 3 may be one or more virtual servers operating on a cloud, which is a collection of computer resources.

[0030] Figure 1 shows an example where a user (not shown) specifies a rectangular coastal area measuring 1 km north-south and 1.5 km east-west as the imaging area A, specifies a positional accuracy of 3 cm or less as the quality of the three-dimensional structure (indicating the degree to which the reconstructed three-dimensional structure matches its actual position), and specifies a delivery time of 12 hours as the delivery time D. In the example shown in Figure 1, the mobile object 2 is a drone.

[0031] The following describes the outline of the processes performed by the information processing device 1 according to the embodiment, in order from (1) to (4), with reference to Figure 1. These numbers correspond to (1) to (4) in Figure 1.

[0032] (1) The information processing device 1 receives the imaging area A specified by the user, the specified quality Q which is the quality of the three-dimensional structure after reconstruction, and the delivery date D.

[0033] (2) The information processing device 1 determines the image to be captured TI, which is the image that the moving body 2 needs to capture for the imaging area A, in order to ensure the specified quality Q.

[0034] (3) The information processing device 1 predicts the encoding time Te required for encoding the images captured by the moving object 2 based on the number of images TI to be captured. Similarly, the information processing device 1 predicts the transmission time Tu required for transmitting the encoded images to the server 3 that performs reconstruction processing based on the number of images TI to be captured. Then, the information processing device 1 predicts the reconstruction time Tp required for the server 3 to reconstruct the three-dimensional structure from the encoded images based on the number of images TI to be captured and the computing resources of the server 3 that have been pre-allocated to the user.

[0035] (4) The information processing device 1 generates an acquisition plan P, including the path of the moving object 2 for capturing the planned image TI, within the available time calculated by subtracting the coding time Te, transmission time Tu, and reconstruction time Tp from the time until the delivery date D. As a result, the information processing device 1 can fit the acquisition time Tc required to execute the acquisition plan P within the available time.

[0036] Thus, the information processing device 1 according to this embodiment generates an acquisition plan P considering the encoding process, transmission process, and reconstruction process, and can achieve the purpose of utilizing the mobile device 2, which is to provide a three-dimensional structure while satisfying the specified quality Q and delivery date D specified by the user.

[0037] <Functional configuration of the information processing device 1 according to the embodiment> Figure 2 is a schematic diagram showing the functional configuration of an information processing device 1 according to an embodiment. The information processing device 1 comprises a storage unit 10, a communication unit 11, and a control unit 12. In Figure 2, the arrows indicate the main data flow, and there may be data flows not shown in Figure 2. In Figure 2, each functional block shows a functional unit configuration, not a hardware (device) unit configuration. Therefore, the functional blocks shown in Figure 2 may be implemented in a single device, or they may be implemented separately in multiple devices. Data exchange between functional blocks may be performed via any means, such as a data bus, network, or portable storage medium.

[0038] The memory unit 10 is a large-capacity storage device such as a ROM (Read Only Memory) that stores the BIOS (Basic Input Output System) of the computer that implements the information processing device 1, a RAM (Random Access Memory) that serves as the working area of ​​the information processing device 1, and an HDD (Hard Disk Drive) or SSD (Solid State Drive) that stores the OS (Operating System), application programs, and various information referenced when the application programs are executed.

[0039] The communication unit 11 is a communication interface for the information processing device 1 to communicate with external devices, and is implemented using known communication modules such as a LAN (Local Area Network) module or a Wi-Fi (registered trademark) module. Hereinafter, in this specification, it is assumed that the information processing device 1 communicates with external devices via the communication unit 11, and the description of the communication unit 11 may be omitted.

[0040] The control unit 12 is a processor such as the CPU (Central Processing Unit), GPU (Graphics Processing Unit), or NPU (Neural Network Processing Unit) of the information processing device 1, and functions as a reception unit 121, decision unit 122, prediction unit 123, generation unit 124, mobile unit control unit 125, correction unit 126, encoding unit 127, transmission unit 128, reconstruction startup unit 129, server control unit 130, and notification unit 131 by executing programs stored in the memory unit 10.

[0041] Figure 2 shows an example where the information processing device 1 is composed of a single device. However, the information processing device 1 may be implemented using multiple computing resources such as processors and memory, for example, in a cloud computing system. In this case, each part constituting the control unit 12 is implemented by at least one of the multiple different processors executing a program.

[0042] The reception unit 121 receives the imaging area A, which is the area to be imaged by the mobile body 2, as specified by the user; the delivery date D for the provision of the three-dimensional structure reconstructed from the imaged image; and the specified quality Q, which includes the quality related to the positional accuracy of the reconstructed three-dimensional structure.

[0043] Figure 3 schematically shows the data structure of user order information 100, which manages the imaging area A, delivery date D, and specified quality Q specified by the user. In the example shown in Figure 3, user order information 100 is information that associates a "user identifier" to identify the user, an "order identifier" to identify the order which is the content specified by the user, an "imaging area" which indicates the geographical area occupied by imaging area A, a "delivery date" which indicates the delivery date D, and information that indicates the specified quality Q, including "quality target," "quality index name," and "quality value."

[0044] In the example shown in Figure 3, the "imaging area" is polygon-formatted information where the geographical area occupied by imaging area A is represented by multiple vertex coordinates. The imaging area A received by the reception unit 121 may be a structure such as a transmission tower, a building such as a skyscraper, etc. Alternatively, the imaging area A received by the reception unit 121 may be an area within a specified region where a landslide has occurred, etc. Furthermore, the "imaging area" is not limited to polygon format and may be an address, the name of a natural object, the name of a building, etc.

[0045] In the example shown in Figure 3, the specified quality Q is expressed by "Quality Target," which indicates the object or information to be evaluated; "Quality Indicator Name," which indicates the criteria or scale for evaluating quality; and "Quality Value," which indicates the specific numerical value or grade required as the quality indicator. In the example shown in Figure 3, user order information 100 with order identifier "OID001" has a quality target of a three-dimensional structure, a quality indicator name of positional accuracy, and a quality value of 5 cm or less.

[0046] In the example shown in Figure 3, user order information 100 with order identifier "OID002" has an acquired image as the quality target, and the quality index name is Ground Sample Distance (GSD), which is an index indicating how much of the ground surface one pixel of the image represents, and the quality value is 3 cm or less. Note that the ground resolution of the acquired image is a factor that constrains the positional accuracy of the three-dimensional structure, and the positional accuracy of the three-dimensional structure can be improved by using an image with a higher ground resolution. The specified quality Q may use a quality related to the positional accuracy of the three-dimensional structure, such as the ground resolution of the acquired image, instead of a quality related to the positional accuracy of the three-dimensional structure. In addition, the specified quality Q may include qualities unrelated to the positional accuracy of the three-dimensional structure.

[0047] The determination unit 122 determines the planned images TI to be acquired for imaging area A, provided that the specified quality Q is met. The determination unit 122 determines the planned images TI to be acquired in order to meet the specified quality Q based on the specified quality Q and the area and shape of imaging area A. The determination unit 122 may determine the minimum number of planned images TI to be acquired in order to meet the specified quality Q, or it may determine an increased number of planned images TI to be acquired in order to improve the quality of the three-dimensional structure.

[0048] The prediction unit 123 predicts the encoding time Te required for image encoding and the transmission time Tu required to transmit the encoded images to the server 3 for reconstruction processing, based on the number of images TI to be captured. The prediction unit 123 also predicts the reconstruction time Tp required for reconstruction based on the number of images and the computing resources of the server 3 allocated to the user.

[0049] The prediction unit 123 predicts the coding time Te based on, for example, the number of images TI to be captured, a predetermined coding method used to predict the coding time Te, and the computing resources used for coding. The prediction unit 123 also predicts the transmission time Tu based on, for example, the number of images TI to be captured, the data size of the coded image, and the throughput of the network used for transmission. Finally, the prediction unit 123 predicts the reconstruction time Tp based on, for example, the number of images TI to be captured, the resolution of the coded image, and the computing resources of the server 3 allocated to the user.

[0050] The generation unit 124 generates an acquisition plan P, including the path of the mobile body 2, for capturing the planned image TI within the available time calculated by subtracting the coding time Te, transmission time Tu, and reconstruction time Tp from the time until the delivery date D. Here, the generation unit 124 generates the acquisition plan P such that the acquisition time Tc required to execute the acquisition plan P falls within the available time. The generation unit 124 may generate an acquisition plan P that captures only the planned image TI, or it may generate an acquisition plan P that captures more images than the planned image TI in order to improve the quality of the three-dimensional structure, provided that the acquisition time Tc falls within the available time.

[0051] Figure 4 schematically shows the data structure of acquisition plan information 101 that manages acquisition plan P. In the example shown in Figure 4, acquisition plan information 101 is information that associates an "acquisition plan identifier" for identifying acquisition plan P, an "order identifier" for identifying the order that triggered the generation of acquisition plan P, a "start date and time" indicating the start date and time of acquisition plan P, an "end date and time" indicating the end date and time of acquisition plan P, a "movement path" indicating the movement path of the mobile object 2, an "imaging path" indicating the path within the movement path where imaging is performed, an "oblique imaging ratio," and an "imaging density." The "oblique imaging ratio" and "imaging density" will be described later.

[0052] As described above, the information processing device 1 according to the embodiment generates an acquisition plan P that can be executed within the available time, taking into account the coding time Te, transmission time Tu, and reconstruction time Tp, and can therefore more reliably provide a three-dimensional structure while satisfying the specified quality Q and delivery time D.

[0053] The mobile unit control 125 moves the mobile unit 2 along the path included in the acquisition plan P and causes the mobile unit 2 to capture images. For example, while moving the mobile unit 2 along the movement path included in the acquisition plan P, the mobile unit control 125 causes the mobile unit 2 to capture images corresponding to each of the planned images TI included in the acquisition plan P on the imaging path included in the acquisition plan P.

[0054] The modification unit 126 modifies the acquisition plan P when the prediction unit 123 identifies that a predetermined delay has occurred in relation to the acquisition plan P, provided that the specified quality Q is met.

[0055] Here, the predetermined delay is the degree of delay required for the correction unit 126 to determine whether or not to correct the acquisition plan P. This degree of delay can be determined experimentally, taking into account the number of images to be captured TI, the area of ​​the imaging area A, the predicted acquisition time Tc, and the time available for executing the acquisition plan P. This degree of delay is, for example, when the ratio of the distance already traveled to the elapsed time in the acquisition plan P is less than 10%.

[0056] Figure 5 is a schematic diagram illustrating the process by which the modification unit 126 modifies the acquisition plan P. Figure 5(a) is a schematic diagram showing the acquisition plan P generated by the generation unit 124. Figure 5(b) is a schematic diagram showing the acquisition plan P modified by the modification unit 126.

[0057] Figure 5 shows an example where a rectangular coastal area measuring 1 km north-south and 1.5 km east-west is designated as imaging area A, with the route included in acquisition plan P represented by a dotted line and the route already traveled by mobile object 2 represented by a solid line. In the example shown in Figure 5, a westerly wind of 4 m / s is blowing during the time when mobile object 2 is performing imaging, and it is assumed that mobile object 2 is, for example, a drone, and that the wind direction and wind speed can be measured or estimated using a small anemometer or the like.

[0058] Figure 5(a) shows an example where the path included in the acquisition plan P generated by the generation unit 124 is a meandering path through the imaging area A due to a straight section extending in the north-south direction. In this case, if a westerly wind is blowing, the moving body 2 will encounter crosswinds when turning eastward or westward, which can increase the time required for the moving body 2 to turn and thus increase the time required to execute the acquisition plan P. In the example shown in Figure 5(a), the point where the moving body 2 is shown schematically indicates the starting point of the path.

[0059] In the example shown in Figure 5(b), the prediction unit 123 identifies a predetermined delay at the location where the moving object 2 is shown. In this case, the correction unit 126 corrects the acquisition plan P to satisfy, for example, the conditions that (1) the uncaptured planned image TI is captured, and (2) the remaining acquisition plan P is completed within the available time calculated by subtracting the coding time Te, transmission time Tu, reconstruction time Tp, and elapsed time of acquisition plan P from the time until the deadline D. In the example shown in Figure 5(b), in order to further reduce the effect of westerly winds, the route included in the corrected acquisition plan P is a route that meanders through the uncaptured imaging area A by a straight section extending in the east-west direction.

[0060] As described above, the information processing device 1 can cause the mobile body 2 to perform imaging according to the acquisition plan P via the mobile body control unit 125, and can also correct the acquisition plan P via the correction unit 126 whenever the plan is delayed, thereby enabling the provision of a three-dimensional structure while more reliably meeting the specified quality Q and delivery date D.

[0061] The encoding unit 127 encodes the acquired image using an encoding profile from among multiple encoding profiles that results in a relatively shorter transmission time Tu for the image captured by the mobile body 2, provided that the specified quality Q is met.

[0062] Here, an encoding profile defines a combination of algorithms and parameters used when encoding data such as images and videos. Encoding profiles include standard-compliant profiles such as JPEG Baseline, which conform to international standards, and proprietary profiles defined specifically for the information processing device 1.

[0063] Figure 6 schematically shows the data structure of the encoding profile information 102 that manages the encoding profile. In the example shown in Figure 6, the encoding profile information 102 is information that associates an "encoding profile identifier" for identifying the encoding profile, an "encoding profile type" indicating the type of encoding profile such as a standards-compliant profile or a proprietary profile, an "encoding profile name" indicating the name of the encoding profile, and "quantization" indicating the strength of quantization that is permitted to be used when performing encoding. The encoding profile information 102 may also be associated with known parameters used during encoding. Furthermore, the encoding unit 127 may combine multiple encoding profiles to encode the acquired image.

[0064] By compressing the image data through encoding of the acquired image by the encoding unit 127, the amount of data required for transmission can be reduced, and the transmission time Tu can be shortened. On the other hand, increasing the compression ratio tends to decrease the specified quality Q of the three-dimensional structure reconstructed from the encoded acquired image due to the loss of information associated with the encoding process, and there is a trade-off relationship between the transmission time Tu and the specified quality Q. Furthermore, there are encoding processes that have low information loss and a high compression ratio, but such encoding processes tend to have long processing times. Therefore, the encoding unit 127 selects an encoding profile from the encoding profile information 102 that has a relatively short sum of encoding time Te and transmission time Tu, provided that the specified quality Q is satisfied, and encodes the acquired image using that encoding profile.

[0065] The transmission unit 128 transmits the encoded acquired images to the server 3 in the order of the reconstruction processing units performed by the server 3. Here, the order of the reconstruction processing units refers to, for example, the order based on the imaging time of the processing units when the acquired images are divided into certain units based on the imaging time and each unit is processed in chronological order. For example, the mobile body 2, after the execution of the acquisition plan P has been completed, may be directly connected to the information processing device 1 via a wired cable. By connecting in this way, the information processing device 1 can treat the storage device of the mobile body 2 as its own storage medium, and the encoding unit 127 and the transmission unit 128 can perform the encoding and transmission of the acquired images.

[0066] As a result, the information processing device 1 can shorten the transmission time Tu while satisfying the specified quality Q with the encoding unit 127 and the transmission unit 128, and can more reliably provide a three-dimensional structure while satisfying the specified quality Q and delivery time D.

[0067] The transmission unit 128 preferably chunks the encoded acquired image and transmits it using a resumeable communication method with a protocol such as HTTP / 2 or QUIC / TLS. The transmission unit 128 may also assign a hash value such as SHA-256 for integrity verification of the encoded acquired image. Furthermore, the transmission unit 128 may also add redundant forward error correction such as ProMPEG FEC or RaptorQ. As a result, the information processing device 1 can transmit the encoded acquired image more reliably or more efficiently by the transmission unit 128, and thus can provide a three-dimensional structure while more reliably meeting the specified quality Q and delivery date D.

[0068] The reconstruction activation unit 129 calculates the computing resources required of server 3 to perform reconstruction from the encoded acquired images by the deadline D, allocates those computing resources, and activates pre-processing for reconstruction on server 3 for each processing unit of the encoded acquired images.

[0069] The reconstruction startup unit 129 may allocate the computing resources of server 3 to the user without recalculating them if it can infer that reconstruction can be performed from the encoded acquired images by the deadline D using the computing resources of server 3 allocated to the user. Alternatively, the reconstruction startup unit 129 may calculate the necessary computing resources of server 3 from, for example, the number of encoded images, the resolution of the encoded images, and the deadline D. Here, the preprocessing for reconstruction includes, for example, feature point extraction, which extracts feature points from multiple images within a processing unit, and initial calibration, which initially estimates and adjusts the camera position and orientation of each image.

[0070] If the server control unit 130 determines, based on the time remaining until deadline D, the progress of the reconstruction process, and the allocated computing resources of server 3, that the reconstruction will not be completed by deadline D, it calculates the computing resources of server 3 required to perform the reconstruction by deadline D and allocates those computing resources. For example, the prediction unit 123 predicts whether the reconstruction will be completed or not by deadline D based on the time remaining until deadline D, the number of encoded images, the resolution of the encoded images, the progress of the reconstruction process, and the allocated computing resources of server 3.

[0071] As a result, the information processing device 1 can provide a three-dimensional structure while more reliably meeting the specified quality Q and delivery date D by having the reconstruction startup unit 129 and the server control unit 130 control the processing and computing resources of the server 3 that performs the reconstruction.

[0072] The prediction unit 123 may predict whether or not the reconstruction will be completed by the deadline D, based on the time until the deadline D, the number of encoded images, the resolution of the encoded images, indicators related to the reconstruction time Tp, the progress of the reconstruction process, and the computing resources of the allocated server 3. Indicators related to the reconstruction time Tp include, for example, the feature point density, which is the number of feature points in the encoded images, and the convergence difficulty of the estimation in initial calibration, which is the number of iterations and the difficulty of stability required for the estimated value to converge to the desired accuracy in initial calibration. As a result, the information processing device 1 can improve the specific accuracy of the prediction unit 123 and provide a three-dimensional structure while reliably satisfying the specified quality Q and deadline D.

[0073] The determination unit 122 further determines the image quality, including the ground resolution, for each of the images to be captured TI, provided that the specified quality Q is met.

[0074] For example, let's consider a case where the upper limit of the positional accuracy of the three-dimensional structure and the upper limit of the ground resolution of the acquired image are specified in the specified quality Q. In this case, the determination unit 122 determines a ground resolution value for the planned image TI that satisfies the following conditions: (1) it is less than or equal to the upper limit of ground resolution defined in the specified quality Q, and (2) it is less than or equal to the upper limit of ground resolution necessary to keep the positional accuracy below or equal to the upper limit defined in the specified quality Q, taking into account the imaging conditions and reconstruction conditions.

[0075] The generation unit 124 generates an acquisition plan P for each of the images to be captured, which further includes setting the altitude of the moving object 2 and setting the camera parameters to be used for image acquisition.

[0076] Here, the camera parameter settings refer to, for example, the camera's focal length, aperture value, sensitivity, and shutter speed. The generation unit 124 also generates a path that includes altitude information as the path of the moving object 2 included in the acquisition plan P, based on, for example, the digital elevation model (DEM) of the imaging area A and the camera parameters.

[0077] As a result, the information processing device 1 can provide a three-dimensional structure while more reliably satisfying the specified quality Q by having the determination unit 122 and the generation unit 124 control the ground resolution of the acquired image.

[0078] The generation unit 124 generates the path of the moving object 2 as a set of line segments connecting one or more waypoints that the moving object 2 must pass through in order to capture the planned image TI. In the example shown in Figure 4, the movement path, which is the path of the moving object 2 thus generated, is represented as a sequence of waypoints, which are multiple pieces of positional information in three-dimensional space. Similarly, in the example shown in Figure 4, the imaging path, which is information indicating the path on which imaging is performed within the movement path, is also represented as a sequence of waypoints. Positional information in three-dimensional space can be, for example, a combination of latitude, longitude, and altitude. Another example of positional information in three-dimensional space is relative coordinates in a three-dimensional coordinate system set with the position coordinates of the starting point of the moving object 2 as the origin.

[0079] The generation unit 124 generates an acquisition plan P which further includes the ratio of oblique images, which are images taken at an angle greater than or equal to a predetermined angle with respect to the vertical direction, and the imaging density.

[0080] Here, the predetermined angle is the angle for acquiring the three-dimensional shape of imaging area A. This angle can be determined experimentally by considering the shape of imaging area A, camera parameters, and the characteristics of the reconstruction process, but for example, it is 30 degrees. Imaging density is an index that indicates the degree of overlap between acquired images, and includes the overlap rate in the direction of travel and the overlap rate in the direction of travel. As explained with reference to Figure 4, the acquisition plan information 101 is information that associates not only the movement path and imaging path, but also the "oblique imaging ratio" and "imaging density". For example, in the example shown in Figure 4, in the acquisition plan information 101 with acquisition plan identifier PID001, the oblique imaging ratio is 20%, and the overlap rate in the direction of travel and the overlap rate in the direction of travel included in the imaging density are 60% and 80%, respectively.

[0081] The mobile unit control unit 125 receives from the mobile unit 2 performing imaging, at a predetermined interval, the relative speed of the mobile unit 2 with respect to the ground surface, the time required for the mobile unit 2 to perform a rotational movement, and an index indicating the clarity of each acquired image captured by the mobile unit 2.

[0082] Here, the index indicating the clarity of each acquired image is, for example, the blur index, which is an index that numerically represents the degree of blur in the acquired image. It is known that the Laplacian variance of the image or the MTF (Modulation Transfer Function) estimate is used as the blur index. The predetermined period is the period for periodically communicating information about the moving object 2 and the acquired image. This period can be determined experimentally by taking into account the communication bandwidth, the speed of the moving object 2, the imaging density, etc., but for example, it is once per second.

[0083] The modification unit 126 modifies the acquisition plan P by adjusting at least one of the following: imaging density, ratio of oblique imaging, order of arrival of one or more waypoints, movement speed of the moving object 2, and camera parameters used for image acquisition.

[0084] The modification unit 126 can shorten the acquisition time Tc, although there is a risk of degrading the quality of the three-dimensional structure, by modifying the acquisition plan P, for example, by lowering the imaging density, lowering the ratio of oblique imaging, or increasing the movement speed of the moving body 2. Here, the movement speed is either the absolute speed that does not take into account external conditions such as wind, or the relative speed of the moving body 2 with respect to the ground surface.

[0085] The waypoint arrival order refers to the sequence in which waypoints are traversed, which are factors that determine the route included in the acquisition plan P generated by the generation unit 124. Figure 7 is a schematic diagram illustrating the process by which the modification unit 126 adjusts the waypoint arrival order. Note that Figure 7 shows the waypoints that have not yet arrived at the time the modification unit 126 modifies the acquisition plan P. Figure 7(a) is a schematic diagram showing the waypoint arrival order generated by the generation unit 124. Figure 7(b) is a schematic diagram showing the waypoint arrival order after adjustment by the modification unit 126.

[0086] Figure 7(a) shows an example where the waypoint arrival order is such that it meanders through imaging area A due to straight sections extending in the north-south direction. In this case, if a westerly wind is blowing, the mobile body 2 will experience crosswinds when turning eastward or westward, which can increase the time required for the mobile body 2 to turn and thus increase the time required to execute the acquisition plan P. In the example shown in Figure 7(b), the adjusted waypoints are arranged in an order that meanders through imaging area A due to straight sections extending in the east-west direction to further reduce the effect of the westerly wind.

[0087] Furthermore, the camera parameters that the correction unit 126 adjusts include, for example, imaging parameters including camera exposure parameters. More specifically, the camera parameters that the correction unit 126 adjusts include, for example, aperture value, sensitivity, and shutter speed. For example, if the movement speed of the moving object 2 is increased, the acquired image may become more blurred. To suppress this blurring, it may be necessary to increase the shutter speed, adjust the aperture value, adjust the sensitivity, etc., in accordance with the increase in movement speed.

[0088] Furthermore, if the mobile body control unit 125 determines that any indicator in the acquired image is outside a predetermined range, the correction unit 126 adjusts at least one of the camera parameters and the movement speed of the mobile body 2, provided that the indicator comes within the predetermined range.

[0089] Here, the specified range refers to the range of image clarity required to ensure a predetermined accuracy in the feature point extraction and reconstruction processes. This range can be determined experimentally by considering factors such as the feature point extraction rate and the accuracy of the three-dimensional structure reconstruction, but for example, the blur index (Laplacian variance) value may be between a lower limit of 100 and an upper limit of 400.

[0090] This allows the information processing device 1 to clearly define the control target for which the correction unit 126 will make adjustments in response to delays in the acquisition plan P. By having the correction unit 126 make adjustments when the image quality of the acquired image deteriorates due to corrections to the acquisition plan P, the delivery date D and specified quality Q can be reliably met.

[0091] The encoding unit 127 uses an encoding profile as a candidate encoding profile that includes at least one of the following settings: encoding scheme setting, quantization setting, bitrate setting, GOP (Group Of Picture) configuration setting, spatial resolution setting, and color difference subsampling setting.

[0092] Here, the encoding method refers to the encoding algorithm for images and videos such as JPEG; quantization refers to the strength of quantization permitted for use when performing encoding, as mentioned above; bitrate refers to the amount of data transmitted or recorded per second; GOP refers to the constituent unit of a frame in a video; spatial resolution refers to the pixel density of an image; and chromatic sampling is a compression technique that reduces the amount of data of the chromatic difference component compared to the luminance component. The encoding unit 127 may also use an encoding profile for video by treating the acquired image as a continuous frame.

[0093] The encoding unit 127, as a condition for satisfying the specified quality Q, performs encoding on each of the acquired images selected from the acquired images using an encoding profile, and imposes a constraint that the encoded acquired image must satisfy the predetermined conditions necessary to satisfy the specified quality Q.

[0094] Here, the specified conditions refer to the SfM metrics that the encoded acquired image must satisfy in order to meet the specified quality Q. SfM metrics are indices that evaluate the accuracy and quality of reconstructing a three-dimensional structure. More specifically, SfM metrics include, for example, keypoint retention rate, total number of matches between images, RANSAC in-liar ratio, image gradient retention rate, peak-signal-noise ratio, and blur metric, all of which are known metrics. The upper and lower limits of the SfM metrics can be set based on the specified quality Q.

[0095] For example, the encoding unit 127 performs encoding on each of the selected acquired images using a different encoding profile. For each encoding profile, the encoding unit 127 measures the SfM index and obtains predicted values ​​for the sum of the encoding time Te, transmission time Tu, and reconstruction time Tp from the prediction unit 123. Then, the encoding unit 127 performs encoding on the acquired image using the encoding profile that satisfies the conditions for the specified quality Q and the SfM index, and minimizes the predicted values ​​for the sum of the encoding time Te, transmission time Tu, and reconstruction time Tp.

[0096] As a result, the information processing device 1 can clarify the contents and selection criteria of the encoding profile, improve the reproducibility of the processing of the encoding unit 127, and thus ensure that the delivery date D and specified quality Q are met.

[0097] The encoding unit 127 may select an acquired image from the images captured by the mobile body 2 to be used for reconstruction by the server 3, provided that it satisfies the specified quality Q, and encode the selected acquired image using an encoding profile. This allows the information processing device 1 to reduce the number of images subject to encoding by the encoding unit 127, transmission by the transmission unit 128, and reconstruction by the server 3, thereby shortening the encoding time Te, transmission time Tu, and reconstruction time Tp, and further ensuring that the delivery date D is met.

[0098] The storage unit 10 may store a learning model that has been trained to output encoding time Te, transmission time Tu, and reconstruction time Tp, taking the number of captured images and the available computing resources of the server 3 as input, or a regression model that predicts encoding time Te, transmission time Tu, and reconstruction time Tp, taking the number of captured images and the available computing resources of the server 3 as input. The prediction unit 123 may input the number of images TI to be captured and the computing resources of the server 3 allocated to the user to the learning model or regression model stored in the storage unit 10, and predict the results output by the learning model or regression model as encoding time Te, transmission time Tu, and reconstruction time Tp. This increases the likelihood that the information processing device 1 can make highly accurate predictions based on past performance using the learning model or regression model by the prediction unit 123, and further increases the likelihood of meeting the deadline D and specified quality Q.

[0099] The determination unit 122 divides the imaging area A into multiple sub-regions PA and determines the image to be acquired for each sub-region PA. The generation unit 124 generates the path of the moving body 2 as a series of paths that are closed-loop paths targeting each sub-region PA, and that can generate a point cloud of the sub-region PA that is sparser than the three-dimensional structure provided to the user, using only the images acquired from the paths. Here, the point cloud of the sub-region PA that is sparser than the three-dimensional structure (hereinafter referred to as the "sparse point cloud" in this specification) is, for example, a point cloud composed only of the feature points of the sub-region PA.

[0100] Figure 8 is a schematic diagram illustrating a partial region PA. Figure 8(a) is a schematic diagram showing the imaging area A before it is divided into partial region PAs. Figure 8(b) is a schematic diagram showing a partial region PA and a path that targets the partial region PA for imaging. Figure 8(a) shows an example in which a rectangular coastal area measuring 1 km north-south and 1.5 km east-west is designated as the imaging area A. Figure 8(b) shows an example in which the imaging area A is divided into six partial region PAs, from partial region PA1 to partial region PA6. Figure 8(b) shows an example in which the path that targets partial region PA6 for imaging has a concave closed-loop shape. Here, a closed-loop path in this specification is not limited to a path in which the start point and end point of the movement of the moving body 2 coincide, but also includes paths in which the start point and end point are relatively close to each other.

[0101] The transmission unit 128 uses a sub-region PA as the processing unit. The reconstruction startup unit 129 independently starts the pre-processing for reconstruction on the server 3 for each sub-region PA, generating a sparse point group for the sub-region PA. For example, the reconstruction startup unit 129 independently starts the following pre-processing steps on the server 3 for each sub-region PA: (1) the feature point extraction described above, (2) the initial calibration described above, and (3) the process of generating a sparse point group using the extracted feature points.

[0102] This allows the information processing device 1 to have the server 3 perform sparse point group generation for each sub-region PA in parallel, thereby shortening the reconstruction time Tp and ensuring that the deadline D is met. Alternatively, the reconstruction activation unit 129 may independently activate the pre-reconstruction processing for generating the sparse point group for each sub-region PA on the server 3 each time the transmission of each sub-region PA is completed. This allows the information processing device 1 to activate the processing on the server 3 for each sub-region PA in advance of the completion of the transmission process, further ensuring that the deadline D is met.

[0103] The reception unit 121 further receives a priority policy indicating which of the delivery date D and specified quality Q the user will prioritize. If the prediction unit 123 determines that the total time required to perform any unfinished processes among the execution of the acquisition plan P by the mobile device 2, the encoding process by the encoding unit 127, the transmission process by the transmission unit 128, and the reconstruction process by the server 3 is longer than the time until delivery date D, the notification unit 131 notifies that either delivery date D or specified quality Q indicated by the priority policy will be prioritized. Here, the notification unit 131 may, for example, display a notification regarding which to prioritize on the display being operated by the user of the information processing device 1, or it may send the notification to the user who requested the provision of the three-dimensional structure by email.

[0104] As a result, the information processing device 1 can detect early on, using the prediction unit 123, the possibility that it may not be able to meet both the delivery date D and the specified quality Q, and can notify the user of matters that should be given higher priority based on a predetermined priority policy, using the notification unit 131. Therefore, the information processing device 1 can reliably meet the higher priority requirements among the delivery date D and the specified quality Q, and prevent a decrease in user satisfaction regarding the purpose of using the mobile device 2.

[0105] The generation unit 124 generates an acquisition plan P that further includes the imaging density. The modification unit 126 modifies the acquisition plan P to reduce the imaging density included in the acquisition plan P, provided that the acquired images after encoding meet the quality requirements for reconstruction, when the reception unit 121 notifies that the delivery date D should be prioritized. For example, the modification unit 126 pre-sets a lower limit for the imaging density required to reconstruct the three-dimensional structure of the imaging area A. Then, when the modification unit 126 is notified that the delivery date D should be prioritized, it modifies the acquisition plan P to reduce the imaging density within a range that does not fall below this lower limit. This lower limit is determined independently of the lower limit for imaging density required to meet the specified quality Q. By reducing the imaging density, the information processing device 1 can shorten the acquisition time Tc and the number of acquired images, thereby shortening the encoding time Te, transmission time Tu, and reconstruction time Tp.

[0106] The encoding unit 127 encodes the acquired image according to the modified acquisition plan P using an encoding profile that relatively shortens the transmission time Tu, provided that the encoded acquired image has a quality that allows for reconstruction. For example, the encoding unit 127 pre-sets a lower limit for the keypoint retention rate of the encoded acquired image necessary to reconstruct the three-dimensional structure of imaging area A. Then, for example, provided that the keypoint retention rate of the encoded acquired image is equal to or greater than the lower limit, the encoding unit 127 selects an encoding profile from the encoding profile information 102 that relatively shortens the sum of the encoding time Te and transmission time Tu, and encodes the acquired image using that encoding profile. Note that this lower limit for keypoint retention rate is determined independently of the lower limit for keypoint retention rate necessary to satisfy the specified quality Q. Since a decrease in keypoint retention rate generally reduces the amount of data, the information processing device 1 may be able to shorten the reconstruction time Tp in addition to shortening the encoding time Te and transmission time Tu.

[0107] As a result, when the information processing device 1 cannot meet both the delivery date D and the specified quality Q, it can prioritize the delivery date D in the correction unit 126 and the encoding unit 127, thereby increasing the probability that the higher priority delivery date D can be met.

[0108] If the prediction unit 123 determines that the total time required to execute any unfinished processes among the execution of the acquisition plan P by the mobile device 2, the encoding process by the encoding unit 127, the transmission process by the transmission unit 128, and the reconstruction process by the server 3 is shorter than the time until the deadline D, the modification unit 126 may modify the acquisition plan P to improve the quality of the three-dimensional structure, provided that the deadline D is met. Then, the encoding unit 127 encodes the acquired image acquired according to the modified acquisition plan P using an encoding profile that improves the quality of the three-dimensional structure, provided that the deadline D is met. As a result, the information processing device 1 can improve the quality of the three-dimensional structure when there is sufficient time until the deadline D, even if both the deadline D and the specified quality Q cannot be met, thereby preventing a decrease in user satisfaction regarding the purpose of using the mobile device 2. Furthermore, even when both the deadline D and the specified quality Q can be met, the information processing device 1 can improve the quality of the three-dimensional structure when there is sufficient time until the deadline D, thereby improving user satisfaction regarding the purpose of using the mobile device 2.

[0109] When the server control unit 130 receives notification from the reception unit 121 to prioritize the specified quality Q, it further outputs at least one of the sparse point groups for each sub-region PA, and the model formed by integrating the sparse point groups. Here, the sparse point groups for each sub-region PA, and the model formed by integrating the sparse point groups, are intermediate products generated in the process of the server 3 reconstructing the three-dimensional structure.

[0110] If the reception unit 121 notifies that the specified quality Q should be prioritized, the information processing device 1 allows the three-dimensional structure to be provided to the user after the deadline D has passed. In this case, for example, the server control unit 130 causes the server 3 to output the intermediate deliverables sequentially each time the server 3 generates them. The output by the server 3 may be done, for example, by making the intermediate deliverables viewable by the user on a web browser, or by providing the intermediate deliverables to the user in a downloadable format.

[0111] As a result, when the information processing device 1 cannot meet both the delivery date D and the specified quality Q, it is possible to have the server control unit 130 sequentially provide intermediate deliverables to the user while adhering to the higher priority specified quality Q, thereby preventing a decrease in user satisfaction regarding the purpose of using the mobile device 2.

[0112] <Processing flow of the information processing method executed by the information processing device 1> Figure 9 is a flowchart illustrating the flow of information processing performed by the information processing device 1 according to this embodiment. The processing in this flowchart starts, for example, when the information processing device 1 is started up.

[0113] The reception unit 121 receives the imaging area A, delivery date D, and specified quality Q specified by the user (S1). The decision unit 122 determines the planned image TI to be acquired for imaging area A, provided that the specified quality Q is met (S2).

[0114] The prediction unit 123 predicts the coding time Te and transmission time Tu from the number of images TI to be captured (S3). The prediction unit 123 predicts the reconstruction time Tp from the number of images TI to be captured and the computing resources of the server 3 allocated to the user (S4).

[0115] The generation unit 124 generates an acquisition plan P, including the path of the mobile body 2, for acquiring the planned image TI within the available time calculated by subtracting the coding time Te, transmission time Tu, and reconstruction time Tp from the time until the delivery date D (S5). Once the generation unit 124 generates the acquisition plan P, the processing in this flowchart is completed.

[0116] <Effects of the information processing device 1 according to the embodiment> As described above, the information processing device 1 according to the embodiment can achieve the purpose of utilizing the mobile body 2.

[0117] Furthermore, this invention will make it possible to contribute to Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), "Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation."

[0118] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of its gist. For example, all or part of the apparatus can be configured by functionally or physically distributing and integrating in any unit. Furthermore, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effects of the new embodiments resulting from the combinations are combined with the effects of the original embodiments.

[0119] <First variation> The above primarily describes a case where the information processing device 1 controls the mobile body 2 and the server 3 to manage the reconstruction of the three-dimensional structure as a three-dimensional structure reconstruction system S. However, the processing performed by the information processing device 1 may also be performed by the mobile body 2 or the server 3. The processing performed by the information processing device 1 may be distributed and integrated into the mobile body 2 or the server 3 in any unit.

[0120] For example, after removing the information processing device 1 from the three-dimensional structure reconstruction system S, the mobile body 2 may comprise the receiving unit 121, determination unit 122, prediction unit 123, generation unit 124, mobile body control unit 125, correction unit 126, encoding unit 127, and transmission unit 128 of the information processing device 1. The server 3 may comprise the reconstruction activation unit 129, server control unit 130, and notification unit 131 of the information processing device 1.

[0121] In this case, the mobile unit 2 may encode the acquired image for each partial region PA using the encoding unit 127 each time imaging for each partial region PA is completed, and the transmission unit 128 may transmit the encoded acquired image. This allows the three-dimensional structure reconstruction system S to perform the execution of the acquisition plan P by the mobile unit 2, the encoding process of the acquired image, the transmission process of the encoded image, and the reconstruction process of the encoded image by the server 3 in parallel, thereby more reliably providing the three-dimensional structure while meeting the deadline D.

[0122] Furthermore, in this case, if the imaging area A is in an environment with unstable communication, such as a mountainous area or a disaster site, the mobile unit 2 may encode the acquired image for each partial region PA using the encoding unit 127 each time imaging for each partial region PA is completed. Then, after the execution of the acquisition plan P is completed, the mobile unit 2 may transmit the encoded acquired image using the transmission unit 128. This allows the three-dimensional structure reconstruction system S to execute the acquisition plan P even when the mobile unit 2 is offline, and to perform the execution of the acquisition plan P and the encoding of acquired images in parallel. Therefore, even when the imaging area A is in an environment with unstable communication, such as a mountainous area or a disaster site, the three-dimensional structure reconstruction system S can more reliably provide a three-dimensional structure while meeting the deadline D.

[0123] <Second variation> For example, after removing the information processing device 1 from the three-dimensional structure reconstruction system S, the mobile body 2 may comprise some functional blocks of the information processing device 1, including the receiving unit 121, the determination unit 122, the prediction unit 123, the generation unit 124, the mobile body control unit 125, the modification unit 126, and the encoding unit 127, as well as the transmission unit 128. The server 3 may comprise some functional blocks of the encoding unit 127, the reconstruction activation unit 129, the server control unit 130, and the notification unit 131.

[0124] In this case, the encoding unit 127 of the mobile device 2 may transmit selected acquired images from the acquired images of the partial region PA to the server 3 each time imaging of the partial region PA is completed. The encoding unit 127 of the server 3 may then encode each transmitted acquired image using an encoding profile, and, provided that the encoded acquired image satisfies predetermined conditions necessary to satisfy the specified quality Q, set the encoding profile used by the encoding unit 127 of the mobile device 2 from among multiple encoding profiles that relatively shortens the transmission time Tu of the acquired image captured by the mobile device 2. The encoding unit 127 of the mobile device 2 encodes the acquired image using the set encoding profile. As a result, the three-dimensional structure reconstruction system S can perform the execution of the acquisition plan P by the mobile device 2, the encoding processing of the acquired image, the transmission processing of the encoded image, and the reconstruction processing of the encoded image by the server 3 in parallel, and can provide the three-dimensional structure while more reliably meeting the delivery date D.

[0125] <Third variation> For example, after removing the information processing device 1 from the three-dimensional structure reconstruction system S, the mobile body 2 may comprise some functional blocks of the encoding unit 127 and some functional blocks of the transmission unit 128 in the information processing device 1. The server 3 may comprise the receiving unit 121, the determination unit 122, the prediction unit 123, the generation unit 124, the mobile body control unit 125, the correction unit 126, some functional blocks of the encoding unit 127, some functional blocks of the transmission unit 128, the reconstruction activation unit 129, the server control unit 130, and the notification unit 131 in the information processing device 1.

[0126] In this case, an example of processing corresponding to the processing performed by the encoding unit 127 and the transmission unit 128 of the information processing device 1 will be described. For example, the mobile device 2 transmits selected acquired images from the acquired images to the server 3. The server 3 then encodes each transmitted acquired image using an encoding profile, and, on the condition that the encoded acquired image satisfies predetermined conditions necessary to satisfy the specified quality Q, sets the encoding profile that the mobile device 2 will use from among multiple encoding profiles, which results in a relatively shorter transmission time Tu of the acquired image captured by the mobile device 2. The mobile device 2 encodes the acquired image using the set encoding profile. Furthermore, the server 3 controls the mobile device 2 to transmit the encoded acquired images to the server 3 in the order of the reconstruction processing units.

[0127] As another example, for instance, the mobile device 2 may transmit the acquired image to the server 3, and the server 3 may encode the acquired image to shorten the reconstruction time Tp. Furthermore, as yet another example, the mobile device 2 may reversibly compress the acquired image and then transmit it to the server 3, and the server 3 may restore the acquired image and then encode it to shorten the reconstruction time Tp. This allows the three-dimensional structure reconstruction system S to concentrate its functions in the server 3, simplify its operation, and provide a stable three-dimensional structure.

[0128] <Fourth variation> The information processing device 1 according to the fourth modification may further include, in addition to the configuration shown in Figure 2, a fee estimation unit 132, a billing unit 133, a payment linkage unit 134, a distribution unit 135, an audit log unit 136, and a contract / SLA management unit 137, which are not shown. The control unit 12 functions as each of these units by executing a program. Here, SLA stands for Service Level Agreement, and refers to a contract that defines the quality and level of service exchanged between the administrator of the information processing device 1, which is the service provider, and the user.

[0129] The fee estimation unit 132 calculates an estimated amount before execution based on the imaging area A, delivery date D, specified quality Q and priority policy received by the reception unit 121, as well as the coding time Te, transmission time Tu, reconstruction time Tp, acquisition time Tc, and the unit price and availability of the server 3's computing resources output by the prediction unit 123. The billing unit 133 collects meter information related to the usage of each process and calculates the final billed amount according to the fee policy. The payment linkage unit 134 links with payment methods such as invoice issuance, direct debit, credit card, bank transfer, prepaid credit consumption, and connection to intercompany payment networks. The distribution unit 135 distributes revenue among multiple businesses, such as the operator of the mobile device 2, the cloud provider of the server 3, and the platform provider of the system, according to the distribution rules. The audit log unit 136 stores and manages time-stamped meter information, model input and output values, fee calculation formula version, and policy identifier that serve as the basis for estimates, finalizations, refunds, surcharges, and distributions, in a manner that enables detection of tampering. The Contracts & SLA Management Department (137) manages plan types, included resource allocations, refund conditions in case of breach of contract, priority ranks, tiered delivery levels of deliverables, and retention periods.

[0130] The storage unit 10 may further store pricing policy information, SLA information, usage history information, billing information, and distribution rule information. Pricing policy information includes acquisition time unit price Uacq, encoded data volume unit price Uenc, transmitted data volume unit price Utx, GPU second unit price Ugpu, storage capacity unit price Ust, surcharge / discount coefficients, tiered pricing Kiter, regional coefficient Kregion, and time zone coefficient Ktime. SLA information includes delivery class, quality class, refund coefficient in case of breach of contract, and conditions for providing tiered deliverables. Usage history information records, for each job identifier, encoding time Te, transmission time Tu, reconstruction time Tp, acquisition time Tc, encoded data volume Ssenc, transmission volume Stx, GPU consumption Ggpu, storage occupancy Sst, progress for each sub-area PA, and deliverable output time. Billing information includes estimated amount, pre-approved credit limit, final amount, payment status, and history of refunds and credit grants. Distribution rule information includes distribution ratios and fixed fees for each party involved, referral fees, etc.

[0131] The billing unit 133 may calculate charges by using the acquisition process, encoding process, transmission process, reconstruction process, and storage process as billing units, and by applying coefficients according to priority and congestion. The charge R is calculated by adding the sum of the product of the unit price corresponding to each process and the actual or predicted value, and the total of the option charges to the base charge R0, and further multiplying by the dynamic coefficients of priority, quality, and congestion: the delivery urgency coefficient Kd, the quality strictness coefficient Kq, and the congestion coefficient Kl. The billing unit 133 may also multiply the unit price corresponding to each process by the tiered charge Kiter, the regional coefficient Kregion, and the time zone coefficient Ktime as multiplier coefficients for the unit price. Specifically, it is defined as R = R0 + Kprio·(Uacq´·Tc_est + Uenc´×Senc_est + Utx´×Stx_est + Ugpu´×Ggpu_est + Ust´·Sst_est) + Σoi. Here, Tc is the acquisition time, Senc is the amount of data after encoding, Stx is the amount of data transmitted, Ggpu is the GPU consumption, Sst is the product of storage capacity and dwell time, and Σoi is the total of options. The dash symbol ' indicates that a multiplier coefficient has been applied. For example, Uacq' is given in the form Uacq'=Uacq·Ktier·Kregion·Ktime. Also, _est is a subscript indicating that it is an estimate. For example, Tc_est indicates the predicted acquisition time. Kprio is defined as the product of at least the delivery urgency coefficient Kd, the quality strictness coefficient Kq, and the congestion coefficient Kl, for example, Kd=1+kd·max(0,1-Trem / Tbase), Kq=1+kq·f(required positional accuracy, specified GSD, oblique imaging ratio, imaging density), and Kl=1+kl·ρ. Here, kd is a non-negative real number that is a surcharge or discount coefficient for delivery time, Trem is the remaining time until delivery, Tbase is the base time defined in the pricing policy information, kq is a non-negative real number that is a surcharge or discount coefficient for quality, f is a monotonically increasing function for all four arguments and is a default monotonically increasing function representing the strictness of the quality requirements, kl is a non-negative real number that is a surcharge or discount coefficient for congestion, and ρ is the computing resource utilization rate of server 3.

[0132] The billing unit 133 may adjust the final amount by applying the quality performance coefficient Kqa and the delivery date achievement coefficient Kdly based on the verification results output by the server 3. Specifically, the billing unit 133 may adjust the final amount by multiplying the variable part of the billing amount by the quality performance coefficient Kqa and the delivery date achievement coefficient Kdly, for example. If quality is not met, the billing unit 133 applies a refund coefficient such as Kqa = 1 - rq, but if the user explicitly agrees to lower the imaging density or coding profile at their own discretion, the billing unit 133 may not apply a refund. Here, rq is the refund rate for quality (0 ≤ rq ≤ ​​1). If a delivery delay occurs, the billing unit 133 applies Kdly = 1 - rd, which is proportional to the delay, but if the notification unit 131 notifies the user of the expected delay in advance and the user switches the priority policy to "delivery date priority" and agrees to quality relaxation, the billing unit 133 does not perform a delay refund. Here, rd is the refund rate for delay (0 ≤ rd ≤ 1). Conversely, if a higher quality or earlier delivery than required is achieved, the billing unit 133 applies a small additional coefficient.

[0133] The reception unit 121 may further accept a job-specific budget limit Bmax from the user. Based on the required computing resources output by the prediction unit 123 and the estimated cost output by the cost estimation unit 132, the server control unit 130 dynamically determines and allocates the number of GPU nodes, priority queue settings, and degree of parallelism for reconstruction preprocessing necessary to meet the delivery date D and specified quality Q within the budget limit Bmax. If the prediction does not exceed the budget limit Bmax, the server control unit 130 immediately starts the required number of GPU instances. If it is expected to exceed the budget limit, the notification unit 131 presents the user with several alternatives, along with their cost impact, such as reducing imaging density, reducing the oblique imaging ratio, changing the coding profile, or switching to phased delivery in partial region PA units. The generation unit 124 and the modification unit 126 update the acquisition plan P according to the option selected by the user, and the server control unit 130 adjusts the allocation. Through this series of controls, the information processing device 1 according to the fourth modified example can provide a technical effect that increases the accuracy of meeting delivery deadlines by coordinating the allocation of costs and computing resources.

[0134] The transmission unit 128 and the reconstruction startup unit 129 each initiate advance transmission and preprocessing using the partial area PA unit as the processing unit, and the server 3 generates intermediate deliverables such as sparse point groups for each partial area PA and outputs them sequentially based on the phased delivery conditions set by the contract / SLA management unit 137. When the milestone defined for each partial area PA is reached, the billing unit 133 proportionally allocates the variable cost portion excluding the base fee R0 and fixed costs and executes an intermediate bill or consumes prepaid credits, and applies the necessary refund coefficient based on the quality verification results for the partial area PA. As a result, the information processing device 1 according to the fourth modified example can synchronize the payment for partial deliverables and the securing of computing resources even in environments with unstable communication or in large-scale projects.

[0135] The audit log unit 136 records the start and end times of each process, equipment identifier, software version, encoding profile, network throughput sample, GPU allocation history, model estimates, and intermediate and final values ​​of the fee calculation as time-stamped records linked to the job identifier and PA identifier. The audit log unit 136 hashes each record and records them in a chain, making it possible to detect tampering with records. The audit log unit 136 also records the fee calculation formula and policy version, so the difference between the estimated amount and the final amount can be verified by recalculating under the same input conditions, and it is possible to fulfill accountability based on technical evidence in response to external audits and objections from users.

[0136] The Contract / SLA Management Unit 137 may manage subscription-type plans that include a predetermined amount of computing resources, storage space, imaging area space, etc., in a monthly base fee. In this case, the Billing Unit 133 applies a usage-based rate to any portion exceeding the allocated space and handles any unused portion according to the provisions for carrying over or expiring within a certain period. The Fee Estimation Unit 132 calculates an estimated amount using the effective rate that takes into account the remaining space, and the Server Control Unit 130 adjusts the parallelism and priority settings based on the relationship between the space consumption rate and the delivery time D. For example, if the remaining space is small, the Server Control Unit 130 proposes to the user that the reconfiguration process be shifted to late-night or low-congestion periods in order to suppress the impact of the congestion coefficient Kl on Kprio, and reschedules based on the user's consent.

[0137] The payment integration unit 134 may also support a prepaid credit-type advance payment method. In this case, the distribution unit 135 reserves the estimated amount or a certain percentage of it as transaction reserve at the start of the job, and sequentially distributes it from the transaction reserve to the relevant businesses each time a milestone (sparse point group generation, partial mesh generation, quality verification completion, etc.) for each partial area PA is reached. If an SLA is not met, the payment integration unit 134 can automatically process a refund for the unused portion or grant credits to the next job based on the verification results and policies recorded by the audit log unit 136.

[0138] The pricing estimation unit 132 updates the congestion coefficient Kl in real time using operational indicators such as the computing resource utilization rate ρ of server 3, the startup queue length, and the market price of spot instances, and generates price curves during the estimation phase and execution. The notification unit 131 presents the user with proposals to change the time zone or switch to phased delivery along with the cost impact when surge pricing occurs, and if the user accepts, the server control unit 130 changes the allocation to match the proposal. If the user chooses cost priority, the server control unit 130 attempts to reduce the total cost by switching to preemptible computing resources or applying a pause / resume strategy, while the notification unit 131 presents revised pricing and schedules that incorporate the cost of re-execution and the impact on delivery time when preemption occurs.

[0139] The information processing device 1 according to the fourth modification may collaborate with external mobile operators, surveying companies, cloud service providers, etc., via API (Application Programming Interface), and the distribution unit 135 may automatically calculate and distribute usage fees and referral fees according to the contribution of each service provider. The contract / SLA management unit 137 manages the conditions for resale licenses for third parties, terminal-based licenses, and API call-based licenses. The audit log unit 136 records partner identifiers, case identifiers, and license clause versions along with timestamps to ensure transparency and reproducibility of compensation distribution.

[0140] According to the information processing device 1 of the fourth modification, the billing process is not merely a post-payment settlement, but is linked bidirectionally with the allocation of computing resources based on predictions. As a result, the degree of parallelism and priority required to meet technical requirements such as delivery time and quality are dynamically optimized under the cost constraints. This allows the information processing device 1 of the fourth modification to reduce the risk of SLA violations while suppressing the over-allocation of unnecessary resources and unexpected cost overruns. Furthermore, it enables the provision of deliverables in stages and tiered billing at the sub-area PA level, allowing for both cash flow management and securing computing resources even for long-running jobs. In addition, audit logs ensure the reproducibility and explainability of charges, reducing litigation costs.

[0141] In the event of a transmission interruption or computing node failure, the server control unit 130 performs failover to an alternative node and resets priorities. For partial area PAs deemed unachievable, the settlement linkage unit 134 suspends phased billing and provides immediate refunds or credits based on the provisions of the contract / SLA management unit 137. The notification unit 131 immediately presents the expected delay, alternatives, and price impact, and the billing unit 133 recalculates the delivery date achievement coefficient Kdly and the quality performance coefficient Kqa based on user acceptance and quality mitigation or delivery date changes. The audit log unit 136 records the timeline of failure events, impact assessments, applied compensation coefficients, and their basis, enabling verification of technical and contractual consistency in the event of future disputes.

[0142] In a hybrid configuration using both on-premises reconfiguration servers and cloud servers 3, the server control unit 130 divides processing according to security policies and data residency, and the cost estimation unit 132 incorporates the depreciation and maintenance costs of the on-premises side into the virtual unit price to provide a consistent estimate of the total cost. If the spot price or congestion level on the cloud side increases, the server control unit 130 prioritizes available slots on the on-premises side to achieve both cost minimization and on-time delivery. The billing unit 133 issues a unified invoice to the user, and the distribution unit 135 automatically performs cost apportionment and revenue distribution between the on-premises owner and the cloud provider. [Explanation of symbols]

[0143] 1. Information Processing Device 10...Storage section 11. Communications Department 12. Control Unit 121...Reception Department 122...Decision Section 123... Prediction Department 124...Generation section 125... Mobile Unit Control Unit 126...Revision section 127...encoding section 128...Transmission section 129...Reconfiguration startup part 130...Server Control Unit 131... Notification Department 2. Mobile 3. Server

Claims

1. A reception unit that receives the imaging area specified by the user, which is the area to be imaged by a moving object; the delivery date for providing the three-dimensional structure reconstructed from the imaged image; and the specified quality, including the quality related to the positional accuracy of the reconstructed three-dimensional structure. A determination unit that determines the number of images to be acquired for the imaging area, provided that the specified quality is met, A prediction unit predicts the encoding time required for image encoding and the transmission time required to transmit the encoded images to the server that performs the reconstruction processing, based on the number of images to be captured, and predicts the reconstruction time required for reconstruction based on the number of images and the computing resources of the server allocated to the user. A generation unit generates an acquisition plan, including the path of a moving object, for capturing the planned image within the available time calculated by subtracting the encoding time, the transmission time, and the reconstruction time from the time until the delivery date, An information processing device equipped with the following features.

2. The aforementioned information processing device is A mobile body control unit that moves the mobile body along the aforementioned path and causes the mobile body to capture images, The system further includes a modification unit that modifies the acquisition plan on the condition that the specified quality is met when the prediction unit identifies that a predetermined delay has occurred in relation to the acquisition plan. The information processing apparatus according to claim 1.

3. The aforementioned information processing device is A coding unit that, provided that the specified quality is met, codes the acquired image using a coding profile from among multiple coding profiles that results in a relatively shorter transmission time for the acquired image captured by the moving object, The system further includes a transmission unit that transmits the acquired images, after encoding, to the server in the order of the processing units used for reconstruction by the server. The information processing apparatus according to claim 2.

4. The aforementioned information processing device is A reconstruction startup unit calculates the computing resources of the server required to perform reconstruction from the encoded acquired image by the aforementioned deadline, allocates said computing resources, and starts pre-processing for reconstruction on the server for each processing unit of the encoded acquired image. The server control unit further comprises, if the prediction unit determines that the reconstruction will not be completed by the deadline based on the time remaining until the deadline, the progress of the reconstruction process, and the allocated computing resources of the server, it calculates the computing resources of the server required to perform the reconstruction by the deadline and allocates those computing resources. The information processing apparatus according to claim 3.

5. The determination unit further determines the image quality, including the ground resolution, for each of the images to be captured, provided that the specified quality is met. The generation unit generates the acquisition plan, which further includes at least one of the following for each of the images to be captured: setting the altitude of the moving object and setting the camera parameters to be used for capturing the image. The information processing apparatus according to claim 1.

6. The generation unit generates the path of the moving body as a set of line segments connecting one or more waypoints that the moving body should pass through in order to capture the planned image, and generates the acquisition plan which further includes the ratio of oblique images, which are images captured at an angle greater than or equal to a predetermined angle with respect to the vertical, and the imaging density. The mobile unit control unit receives from the mobile body performing the imaging, at a predetermined interval, at least one of the relative speed of the mobile body with respect to the ground surface and the time required for the mobile body to perform a rotational movement, and an index indicating the clarity of each acquired image captured by the mobile body. The modification unit (1) modifies the acquisition plan by adjusting at least one of the following: the imaging density, the ratio of oblique imaging, the order in which one or more waypoints are reached, the movement speed of the moving body, and the camera parameters used for image acquisition; and (2) when the moving body control unit identifies that any of the indicators in the acquired image is outside a predetermined range, it adjusts at least one of the camera parameters and the movement speed of the moving body, provided that the indicator comes within the predetermined range. The information processing apparatus according to claim 2.

7. The encoding unit (1) uses an encoding profile as a candidate for the encoding profile, which includes at least one setting from the following: setting the encoding method, setting the quantization, setting the bitrate, setting the GOP (Group Of Picture) configuration, setting the spatial resolution, and setting the color difference subsampling; and (2) as a condition for satisfying the specified quality, it imposes a constraint that it encodes each of the acquired images selected from the acquired images using the encoding profile, and that the encoded acquired image satisfies predetermined conditions necessary to satisfy the specified quality. The information processing apparatus according to claim 3.

8. The information processing device further includes a storage unit that stores a learning model that has learned to output the encoding time, the transmission time, and the reconstruction time, taking the number of captured images and the available computing resources of the server as input, or a regression model that regressively predicts the encoding time, the transmission time, and the reconstruction time, taking the number of captured images and the available computing resources of the server as input. The prediction unit inputs the number of images to be captured and the computing resources of the server allocated to the user into the learning model or the regression model, and predicts the encoding time, transmission time, and reconstruction time from the output of the learning model or the regression model. The information processing apparatus according to claim 1.

9. The determination unit divides the imaging area into a plurality of sub-regions and determines the number of images to be acquired for each sub-region. The generation unit generates the path of the moving object as a series of paths that are closed paths with each of the sub-regions as the target of imaging, and which can generate a point cloud of the sub-regions that is sparser than the three-dimensional structure provided to the user, using only the images captured from the path. The transmission unit uses the sub-region as the processing unit, The reconstruction startup unit initiates the reconstruction preprocessing for generating the point cloud of the sub-regions on the server independently for each sub-region. The information processing apparatus according to claim 4.

10. The reception department further receives a priority policy indicating which of the above-mentioned delivery date and the above-mentioned specified quality should be prioritized. The information processing device further includes a notification unit that, when the prediction unit determines that the total time required to perform any unfinished processes among the execution of the acquisition plan by the mobile device, the encoding process by the encoding unit, the transmission process by the transmission unit, and the reconstruction process by the server is longer than the time until the delivery date, the notification unit will prioritize either the delivery date indicated by the priority policy or the specified quality. The information processing apparatus according to claim 9.

11. The generation unit generates the acquisition plan which further includes the imaging density, The modification unit, upon notification from the reception unit that the delivery date should be prioritized, modifies the acquisition plan to reduce the imaging density included in the acquisition plan, provided that the acquired image after encoding meets the quality requirements for reconstruction. The encoding unit encodes the acquired image obtained in accordance with the modified acquisition plan using an encoding profile that results in a relatively shorter transmission time, provided that the encoded acquired image has a quality that allows for reconstruction. The information processing apparatus according to claim 10.

12. When the server control unit receives notification from the reception unit to prioritize the specified quality, it further outputs at least one of the point clouds for each of the subregions and the model obtained by integrating the point clouds. The information processing apparatus according to claim 10.

13. The processor, A step of receiving the imaging area, which is the area to be imaged by the moving object as specified by the user, the delivery date for providing the three-dimensional structure reconstructed from the imaged image, and the specified quality, which includes the quality related to the positional accuracy of the reconstructed three-dimensional structure. A step of determining the number of images to be acquired for the imaging area, provided that the specified quality is met, The steps include predicting the encoding time required for image encoding and the transmission time required to transmit the encoded images to the server that performs reconstruction processing, based on the number of images to be captured, A step of predicting the reconstruction time required for reconstruction based on the number of images to be captured and the computing resources of the server allocated to the user, A step of generating an acquisition plan, including the path of a moving object, for capturing the planned image within the available time calculated by subtracting the encoding time, the transmission time, and the reconstruction time from the time until the delivery date, An information processing method that performs the following.

14. On the computer, A function that accepts the imaging area, which is the area to be imaged by a moving object as specified by the user, the delivery date for providing the three-dimensional structure reconstructed from the imaged image, and the specified quality, which includes the quality related to the positional accuracy of the reconstructed three-dimensional structure. A function to determine the number of images to be acquired for the imaging area, provided that the specified quality is met, A function that predicts the encoding time required for image encoding and the transmission time required to transmit the encoded image to the server that performs reconstruction processing, based on the number of images to be captured. A function to predict the reconstruction time required for reconstruction based on the number of images to be captured and the computing resources of the server allocated to the user, A function to generate an acquisition plan, including the path of a moving object, for capturing the planned image within the available time calculated by subtracting the encoding time, the transmission time, and the reconstruction time from the time until the delivery date, A program that makes this possible.

Images