Program, information processing apparatus, information processing method, and learning model

Abstract

To generate data at high frequency which is obtained by observing an observation area by using a plurality of flying objects.SOLUTION: A program causes a computer to execute, on the basis of first observation data that is information on an observation area at first timing and that is observed by first observation means, and second observation data that is information on the observation area at second timing different from the first timing and that is observed by second observation means different from the first observation means, generating integrated data including first information based on the first observation data and second information based on the second observation data.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a program, an information processing apparatus, an information processing method, and a learning model.

Background Art

[0002] Observation of the state of the Earth's surface including the ground and the sea using flying objects such as artificial satellites, airplanes, or drone devices is widely carried out. Observation methods using flying objects include an observation method performed by acquiring an optical image, and an observation method performed by acquiring a so-called SAR image, which is a radar image obtained using synthetic aperture radar (SAR) technology.

[0003] In Patent Document 1, in order to improve the frequency of grasping the situation of the Earth's surface, a new image obtained by photographing a predetermined observation point on the Earth's surface by a first flying object flying on a first orbit and a first orbit are used. A method for grasping the situation of the Earth's surface is described in which a change area is detected by aligning and comparing one most recent past image obtained by photographing the same point as the predetermined observation point by a second flying object flying on a different second orbit.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the method described in Patent Document 1, a new image observed from the first orbit is treated as a different image from the past image observed from the second orbit. In Patent Document 1, the new image is a new image observed by the first flying object from the first orbit, and is not data considering the observation of the observation point by the second flying object. Therefore, in Patent Document 1, the data of the observation point itself is not generated frequently, and only the frequency of change detection is improved. When performing observation using a flying object, it is preferable that the generation of observation data is performed frequently in order to grasp the state of the observed area in more detail.

[0006] Therefore, an object of the present invention is to generate data obtained by observing an observation area using a plurality of flying objects frequently.

Means for Solving the Problems

[0007] A program according to an aspect of the present invention causes a computer to generate integrated data including first information based on first observation data, which is information on an observation area at a first timing and observed by a first observation means, and second information based on second observation data, which is information on the observation area at a second timing different from the first timing and observed by a second observation means different from the first observation means, based on the first observation data and the second observation data.

[0008] According to this aspect, since the integrated data includes first information based on the first observation data and second information based on the second observation data, when at least one of the first observation data or the second observation data is updated or acquired, the integrated data will also be updated. Therefore, the computer can generate the integrated data frequently by generating the integrated data based on the information on the observation area at either the first timing or the second timing.

[0009] In the above aspect, the first observation data is data observed under the first observation conditions, and the computer is provided with information on the observation area at a third timing different from the first timing. The data is observed under the third observation conditions by a third observation means different from the first observation means, and the computer is caused to further execute acquiring third observation data observed based on an observation method common to the first observation data, and inputting learning observation data that is information on the learning observation area, learning observation conditions corresponding to the learning observation data, and a learning observation timing at which the learning observation data is observed, and outputting conversion observation data obtained by converting the learning observation data, the conversion observation data indicating data obtained by observing the learning observation area under the first observation conditions at the learning observation timing, to a conversion model, and inputting the third observation data, the third observation conditions, and the third timing to the conversion model to obtain conversion observation data obtained by converting the third observation data. Generating integrated data may include updating the first information based on the conversion observation data to generate the integrated data.

[0010] According to this aspect, when there is third observation data that has the same observation method as the first observation data, the third observation data can be converted into data observed under the first observation conditions. Therefore, for example, when performing analysis using the integrated data, it becomes possible to handle data observed under different observation conditions as the first information as data observed under the first observation conditions, and the first information in the integrated data is updated frequently, and as a result, it becomes possible to generate the integrated data frequently.

[0011] In the above aspect, the first observation conditions may include first airframe information of a first flying object that is the first observation means and first environment information indicating an observation environment by the first flying object, and the third observation conditions may include third airframe information of a third flying object that is the third observation means and third environment information indicating an observation environment by the third flying object.

[0012] According to this aspect, it becomes possible to convert data in consideration of the information of the airframe and the observation environment. Therefore, it becomes possible to accurately convert the third observation data into observation data under the first observation conditions.

[0013] In the above aspect, the first observation data is data observed under first observation conditions. The computer is caused to acquire the first observation data and the first observation conditions, and information on an observation area at a third timing different from the first timing, which is data observed under third observation conditions by a third observation means different from the first observation means, and to acquire third observation data observed based on an observation method common to the first observation data. Learning observation data that is information on a learning observation area, learning observation conditions corresponding to the learning observation data, and a learning observation timing at which the learning observation data was observed are input, and a conversion model that outputs conversion observation data obtained by converting the learning observation data, which indicates data obtained by observing the observation area under reference observation conditions at the learning observation timing, is caused to input the first observation data, the first observation conditions, and the first timing, and to acquire first conversion observation data as the conversion observation data obtained by converting the first observation data. The conversion model is caused to input the third observation data, the third observation conditions, and the third timing, and to acquire second conversion observation data as the conversion observation data obtained by converting the third observation data at the third timing. Further executing generating integrated data may include updating first information based on the first conversion observation data with first information based on the second conversion observation data to generate integrated data.

[0014] According to this aspect, each of the first observation data and the third observation data observed under a common observation method can be converted into conversion observation data observed under reference observation conditions that are common observation conditions. By generating integrated data based on the conversion observation data, the differences between the respective data in the integrated data can be reduced, and the information processing burden during the analysis of the integrated data can be alleviated. Also, the first information in the integrated data is updated frequently, and as a result, it becomes possible to generate the integrated data frequently.

[0015] In the above aspect, the first observation condition may include the first airframe information of the first flying object which is the first observation means and the first environmental information indicating the observation environment by the first flying object, and the third observation condition may include the second airframe information of the third flying object which is the third observation means and the second environmental information indicating the observation environment by the third flying object.

[0016] According to this aspect, it becomes possible to convert data in consideration of the information of the airframe and the observation environment. Therefore, it becomes possible to accurately convert the second observation data into the observation data under the first observation condition.

[0017] In the above aspect, the first observation data may be data observed based on the first observation method, and the second observation data may be data observed based on a second observation method different from the first observation method.

[0018] According to this aspect, data observed based on different observation methods is generated as integrated data. For example, the first observation method may be an observation method by SAR, and the second observation method can be an observation method by an optical image. Thereby, data observed by a plurality of observation methods can be integrated into one data, and it becomes possible to obtain more information by analysis or perform highly accurate analysis.

[0019] In the above aspect, further causing a computer to acquire third observation data which is information on an observation area at a third timing different from the first timing, data observed under the third observation condition by a third observation means different from the first observation means, and third observation data observed based on an observation method different from the first observation method, inputting the learning observation data and the learning observation timing at which the learning observation data was observed, inputting the third observation data and the third timing to a prediction model that outputs prediction observation data indicating data obtained by observing the observation area based on the first observation method at the learning observation timing, and acquiring prediction observation data at the third timing, and generating integrated data may include updating the first information based on the prediction observation data to generate integrated data.

[0020] According to this aspect, the observation data at the third timing based on the first observation method can be inferred from the third observation data at the third timing based on the second observation method. Thereby, for example, even when the first observation data cannot be obtained due to factors such as bad weather, the data can be supplemented with the third observation data, and integrated data including the data observed based on the first observation method can be generated frequently.

[0021] In the above aspect, causing a computer to further execute obtaining first observation data, learning observation data, learning observation timing at which the learning observation data was observed, and a timing different from the learning observation timing, inputting the first observation data, the first timing, and the second timing to a prediction model that outputs prediction observation data indicating data obtained by observing an observation area based on the second observation method at different timings, and obtaining prediction observation data at the second timing, and generating integrated data may include generating integrated data including second information based on the prediction observation data at the second timing.

[0022] According to this aspect, the observation data at the second timing based on the second observation method can be inferred from the first observation data at the first timing based on the first observation method. Thereby, for example, even when the second observation data cannot be obtained due to factors such as bad weather, the data can be supplemented with the first observation data, and integrated data can be generated.

[0023] In the above aspect, cause the computer to obtain third observation data when the observation area is observed at a third timing before the first timing, and input the learning observation data, the learning observation timing when the learning observation data is observed, and a prediction timing after the learning observation timing into a prediction model that outputs speculative observation data indicating data obtained by observing the observation area based on the first observation method at the prediction timing, input the third observation data, the third timing, and the first timing, and further cause the computer to obtain speculative observation data at the first timing, and generating integrated data may include generating integrated data including first information based on the speculative observation data at the first timing.

[0024] According to this aspect, it is possible to infer the observation data at the first timing based on the first observation method from the third observation data at the third timing based on the first observation method. Thereby, for example, even when the first observation data cannot be obtained due to factors such as bad weather, data can be complemented with the third observation data observed at the third timing before the first timing to generate integrated data.

[0025] In the above aspect, the computer may be further caused to input integrated learning data based on learning observation data that is information on the learning observation area, and input the integrated data into a meta-information output model that outputs meta-information on the learning observation area, and obtain meta-information on the observation area.

[0026] According to this aspect, it is possible to obtain meta-information on the observation area by using integrated data generated frequently based on the observation of the observation area. Thereby, the accuracy of the meta-information on the observation area can be increased.

[0027] An information processing apparatus according to another aspect of the present invention includes an integrated data generation unit that generates integrated data including first information based on first observation data, which is information on an observation area at a first timing and is observed by first observation means, and second information based on second observation data, which is information on the observation area at a second timing different from the first timing and is observed by second observation means different from the first observation means, based on the first observation data and the second observation data.

[0028] An information processing method according to another aspect of the present invention includes a computer generating integrated data including first information based on first observation data, which is information on an observation area at a first timing and is observed by first observation means, and second information based on second observation data, which is information on the observation area at a second timing different from the first timing and is observed by second observation means different from the first observation means, based on the first observation data and the second observation data.

[0029] A learning model according to another aspect of the present invention is trained using teacher data that takes, as input, learning observation data, which is information on a learning observation area, and a learning observation condition corresponding to the learning observation data, and outputs first converted observation data obtained by converting the learning observation data as data obtained by observing the learning observation area according to a reference observation condition at the timing when the learning observation data is acquired. For an input of observation data, which is information on a target observation area, and an observation condition corresponding to the observation data, the computer is caused to function so as to output second converted observation data indicating data obtained by observing the target observation area under the reference observation condition at the timing when the observation data is acquired.

[0030] A learning model according to another aspect of the present invention uses, as inputs, learning observation data when a learning observation area is observed based on a first observation method, the observation timing when the learning observation data is observed, and a prediction timing different from the observation timing, and is learned using teacher data that outputs, as an output, first estimated observation data indicating data obtained by observing the learning observation area based on a second observation method different from the first observation method at the prediction timing. The computer is caused to function so as to output second estimated observation data indicating data obtained by observing the target observation area based on the second observation method at the timing when the observation data is acquired, for an input of the observation data that is information on the target observation area and the observation conditions corresponding to the observation data.

Advantages of the Invention

[0031] According to the present invention, it is possible to generate data obtained by observing an observation area using a plurality of flying objects with high frequency.

Brief Description of the Drawings

[0032]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Embodiments for Carrying Out the Invention

[0033] Preferred embodiments of the present invention will be described with reference to the accompanying drawings. In each figure, those denoted by the same reference numerals have the same or similar configurations.

[0034] (First Embodiment) FIG. 1 shows an observation system 10 according to the first embodiment. The observation system 10 is the same also in the second and third embodiments.

[0035] Observation system 10 includes an observation data processing device 101, artificial satellites 201, 202, 203, and an observation data receiving device R.

[0036] In observation system 10, observation data obtained by observing the Earth's surface O by a plurality of satellites including artificial satellites 201, 202, 203 is transmitted to the observation data receiving device R. At this time, the observation conditions indicating the environment at the time of observation of the observation data may be transmitted to the observation data receiving device R so as to be associated with or included in the observation data. The observation data processing device 101 acquires the observation data from the observation data receiving device R and performs information processing on the observation data. Note that the device for observing the observation data in the observation system 10 is a flying object that is arranged in space as an artificial satellite capable of acquiring observation data and orbits around the Earth, as exemplified by artificial satellites 201, 202, 203. The flying object for observing the observation data may be a geostationary satellite. Further, the flying object may be any device that can be located in the airspace above the Earth, such as an aircraft, a helicopter, or a drone device.

[0037] In observation system 10, the observation data observed by artificial satellites 201, 202, 203 may be data observed based on different observation methods from each other, or may be data observed based on a common observation method.

[0038] The observation method is, for example, a method using SAR signals (SAR method). The observation data is a signal corresponding to the electromagnetic wave reflected by the observation target object, where the microwave (electromagnetic wave) irradiated from the artificial satellite equipped with the radar device to the observation target object. The observation data (SAR data) based on the SAR method is, for example, subjected to visualization processing by the observation data processing device 101 to become a SAR image. As other SAR data, there is also second-level SAR data obtained by performing range compression and multi-look azimuth compression on the observed SAR data. With the second-level SAR data, visualization by the SAR image can be performed in a geometrically corrected state. As other SAR data, there is also third-level SAR data obtained by performing range compression, single-look azimuth compression, and ortho correction on the observed SAR data. By adding ortho correction, a SAR image that can be overlaid with an optical image described later can be obtained.

[0039] As another observation method, there is a method using optical data obtained using an optical sensor (optical method). The optical data is information obtained by a sensor that observes light having wavelengths in the visible light range or near-infrared range, for example. The optical data is data based on the light generated by the reflection of sunlight on the earth's surface.

[0040] Also, as another observation method, there is a method using meteorological data obtained using an infrared sensor. The meteorological data is a signal obtained by observing infrared rays radiated from clouds, the earth's surface, and the atmosphere, and is used for generating infrared images and water vapor images. In the first embodiment and subsequent embodiments, the optical data and the meteorological data are described separately according to the difference in their respective used wavelength ranges.

[0041] Referring to FIG. 2, the generation of integrated data by the observation data processing device 101 will be described. In FIG. 2, the case where each of the artificial satellites 201, 202, and 203 performs observations based on different observation methods, namely method 1, method 2, or method 3, will be described as an example. Here, each method is, for example, method 1 is a method using SAR data, method 2 is a method using optical data, and method 3 is a method using meteorological data.

[0042] At a certain time T1 on the time axis T, the artificial satellite 201 observes the observation area and acquires observation data D11. The observation data D11 is transmitted to the observation data processing device 101, and the observation data processing device 101 generates integrated data ID1 based on the observation data D11.

[0043] After that, the artificial satellite 202 observes observation data D22 at a certain time T2 on the time axis T. The observation data D22 is transmitted to the observation data processing device 101, and the observation data processing device 101 generates integrated data ID2 based on the previously observed observation data D11 and observation data D22.

[0044] The integrated data ID2 is data based on the observation data based on method 1 at time T1 and the observation data based on method 2 at time T2, and is data including the latest information on the observation area at time T2.

[0045] The artificial satellite 203 observes observation data D33 at a certain time T3 on the time axis T. The observation data D33 is transmitted to the observation data processing device 101, and the observation data processing device 101 generates integrated data ID3 based on the previously observed observation data D11, D22, and observation data D33. The integrated data ID3 is data based on the observation data based on method 1 at time T1, the observation data based on method 2 at time T2, and the observation data based on method 3 at time T3, and is data including the latest information on the observation area at time T3.

[0046] Thereafter, at time T4, the artificial satellite 201 observes the same observation area as the area observed at time T1 and acquires observation data D14. Here, the time from the observation of the observation area by the artificial satellite 201 to the next observation is, for example, an interval determined according to the orbiting path of the artificial satellite 201. The observation data D14 is transmitted to the observation data processing device 101, and the observation data processing device 101 generates integrated data ID4 based on the observation data D14. The observation data processing device 101 replaces the observation data D11 obtained by the observation of the artificial satellite 201 at time T1 with the newly obtained observation data D14 and generates integrated data ID14. The integrated data ID4 is data including the latest information on the observation area at time T4.

[0047] Similarly, at time T5, the artificial satellite 202 observes the same observation area as the area observed at time T2 and acquires observation data D25. The observation data D25 is transmitted to the observation data processing device 101, and the observation data processing device 101 generates integrated data ID5 based on the observation data D25. The observation data processing device 101 replaces the observation data D22 obtained by the observation of the artificial satellite 202 at time T2 with the newly obtained observation data D25 and generates integrated data ID5. The integrated data ID5 is data including the latest information on the observation area at time T5. By integrating the data at each time, the observation data processing device 101 can generate the data obtained by observing the observation area using a plurality of flying objects more frequently than the observation by a single flying object.

[0048] The information included in the integrated data is associated with the information of the time when the observation data that is the basis of each piece of information was observed, and the information in the integrated data can be sorted based on time. For example, in integrated data ID5, it is possible to order the observation data D33 observed at time T3, the observation data D14 observed at time T4, and the observation data D25 observed at time T5 in chronological order. Note that the observation data does not include time information and may be sorted in the order of acquisition. That is, the integrated data is generated in a state where it can be sorted along a time series. By using the integrated data generated by enabling the observation data to be sorted along a time series, it becomes possible to more accurately detect changes in the observation area. Note that the integrated data does not necessarily have to be generated in a manner that enables the observation data to be sorted along a time series. The format of the integrated data can be appropriately set according to the use of the integrated data.

[0049] Here, the data to be integrated may be the signals themselves from each satellite or the image data generated based on each signal. When generating integrated data based on the image data, each image data is integrated as each channel input to the meta-information output model described later. Also, the data to be integrated may be the meta-information obtained from the signals or image data from each satellite. Further, the signals, image data, or meta-information from each satellite may be, for example, information generated by simulation processing performed by an information processing device based on observation data. That is, the data included in the integrated data may be information based on the observation data.

[0050] Observation data processing device 101 generates integrated data each time it acquires observation data from each satellite. As a result, the integrated data is updated to include the latest observation results of the observation area. For example, if a change occurs in the observation area between time T2 and time T5, when only artificial satellite 202 is used, the change will not appear in the observation data until time T5. However, based on the observation data D33 from artificial satellite 203 at time T3 and the observation data D14 from artificial satellite 201 at time T4, the integrated data is generated as integrated data ID3, ID4. Since the integrated data ID3, ID4 includes the changes that occurred in the observation area between time T2 and time T5, the changes that occurred in the observation area are also reflected in the integrated data ID3, ID4. By analyzing the integrated data based on the observation data observed by multiple satellites, it becomes possible to timely grasp changes and the like that occurred within the period of the observation interval of a single satellite. Also, even when an event occurs where artificial satellite 202 cannot observe the observation area at time T5, for example, an event such as bad weather, it is possible to grasp the changes in the observation area by analyzing the integrated data generated including data by the SAR method that is not affected by the weather.

[0051] The observation data processing device 101 enables the data regarding the observation area to be updated at shorter time intervals, and enables application to tasks that require real-time performance by analysis. As an example of a task, there is the measurement of NDVI (Normalized Difference Vegetation Index), which is one of the vegetation indices, using satellites and the like. While the measurement of NDVI can be performed more accurately using an optical satellite, optical data is limited to observations during the day, and there are also restrictions on time and weather, such as being unable to observe due to clouds and the like. Therefore, by measuring NDVI using integrated data including SAR data from an SAR satellite, the number of measurements can be increased and the real-time performance can be enhanced.

[0052] In addition, since the integrated data is data obtained by integrating information from each satellite, for example, when acquiring meta-information using a learning model and analyzing the observation area, it is possible to perform high-frequency and multi-faceted analysis rather than inputting information from each satellite into the learning model individually.

[0053] It is also possible to utilize an observation system that has been operating using single satellite data as a system that generates integrated data using other satellite data to enable analysis.

[0054] Each part of the observation data processing apparatus 101 will be described. FIG. 3 shows a block diagram of the observation data processing apparatus 101 according to the first embodiment. The observation data processing apparatus 101 includes a communication unit 1011, a storage unit 1012, an observation data acquisition unit 1013, an integrated data generation unit 1014, and a meta-information acquisition unit 1015. Each part of the observation data processing apparatus 101 can be realized by a program stored in a storage device being executed by a processor in an information processing apparatus such as a personal computer.

[0055] The communication unit 1011 controls communication between the observation data processing apparatus 101 and an external information processing apparatus including the observation data receiving apparatus R. Further, the observation data processing apparatus 101 may acquire observation data from an information processing apparatus such as a server that records observation data, and the communication unit 1011 controls communication with other information processing terminals.

[0056] The storage unit 1012 stores various types of information used in the processing by the observation data processing apparatus 101. The storage unit 1012 stores a meta-information output model 10121.

[0057] The meta-information output model 10121 will be described with reference to FIG. 4. The meta-information output model 10121 is a learning model that outputs meta-information of the observation area in response to the input of integrated data obtained from the observation of the observation area.

[0058] The meta - information output model 10121 is trained using the training data LD1 as the teacher data. The training data LD1 includes a set of integrated training data based on a plurality of training observation data obtained by observing the observation area and meta - information MD0 corresponding to the integrated training data. The training model 10212 is trained with the integrated training data as the input and the meta - information MD0 as the output.

[0059] The meta - information MD0 has various items depending on the observation target. For the meta - information regarding land - based moving objects such as automobiles and animals, information on the moving trajectory of the moving object is included. This information is obtained from changes in SAR data due to changes in the interference of reflected electromagnetic waves generated by the moving object and changes in the position of the moving object in the optical image.

[0060] When the flooded area during a disaster is the observation target, the meta - information may be information on the range of the flooded area. Also, as meta - information used for crop management, the NDVI indicating the growth degree of the crop may be used as meta - information. The growth degree of the crop is estimated based on the correlation calculated based on the backscattering coefficient of the SAR data and the actually observed growth degree of the crop, as well as color changes and spectral reflection characteristics in the optical data. In this case, the meta - information may include information on the actual height of the crop measured.

[0061] In the detection regarding buildings, information on new buildings may be used as meta - information. The information on new buildings is estimated based on the correlation calculated based on the backscattering coefficient of the SAR data and the actually observed information on new buildings, as well as color changes in the optical data. Also, information on the type of building obtained based on a map or the like may be used as meta - information.

[0062] When the integrated data generated by the observation data processing device 101 is input to the trained meta - information output model 10121, the meta - information output model 10121 outputs the meta - information MD1 corresponding to the integrated data. Note that the meta - information output model 10121 does not necessarily have to be stored in the observation data processing device 101 and may be stored in other information processing devices.

[0063] Returning to FIG. 2, the observation data acquisition unit 1013 acquires the observation data (first observation data) observed by the artificial satellite 201 and the observation conditions (first observation conditions) in the observation by the artificial satellite 201. The observation data acquisition unit 1013 may acquire the observation data and the observation conditions from, for example, the observation data receiving device R, or may acquire the observation data and the observation conditions from a server in which the actual observation data or the observation data generated by simulation is recorded. Note that when generating integrated data, the observation data processing device 101 may generate integrated data based on the observation data without acquiring the observation conditions. The same applies to the observation data (second observation data) from the artificial satellite 202, the observation conditions (second observation conditions) in the observation by the artificial satellite 202, and the observation data and the observation conditions from the artificial satellite 203.

[0064] The observation conditions are information including the aircraft information, which is information based on the characteristics of the flying object, and the environmental information indicating the observation environment by the flying object. The aircraft information includes the frequency of light used by the flying object for observation, the polarization of light, pulse information, and the beam pattern. Further, when the flying objects form a constellation, for example, the identifier of the constellation and the aircraft number or identifier of each flying object within the constellation may be included in the aircraft information.

[0065] The environmental information includes the orbital direction (ascending northbound orbit or descending southbound orbit) at the time of observation, the orbital angle, the observation direction (left or right) at the time of observation, the aircraft speed, the attitude information of the aircraft, and the parameters of the receiving system. When the observation data is image data, the environmental information may include information on the resolution of the image. When the observation data is SAR image data, the environmental information may include information on the incident angle of the irradiation signal for each pixel of the image. When noise removal is performed on the observation data, the environmental information may include information on the noise, the type of noise filter, and the parameters for noise removal.

[0066] The integrated data generation unit 1014 performs a process of generating integrated data. The integrated data generation unit 1014 generates integrated data ID1 based on first information based on observation data D11 and second information based on observation data D22, for example, observation data D11 which is information on the observation area at time T1 and is observed by the artificial satellite 201, and observation data D22 which is information on the observation area at time T2 and is observed by the artificial satellite 202.

[0067] The meta information acquisition unit 1015 inputs the integrated data into the meta information output model 10121 and acquires the meta information of the observation area. The acquired meta information may be presented to the user of the observation data processing device 101 by a display unit included in the observation data processing device 101, or may be transmitted from the observation data processing device 101 to another information processing device and presented to the user.

[0068] Referring to FIG. 5, the integrated data generation process will be described. In step S501, the observation data acquisition unit 1013 acquires first observation data at a first timing. For example, the observation data acquisition unit 1013 acquires observation data D11 at time T1.

[0069] In step S502, the observation data acquisition unit 1013 acquires second observation data at a second timing. For example, the observation data acquisition unit 1013 acquires observation data D22 at time T2.

[0070] In step S503, the integrated data generation unit 1014 acquires first information based on the first observation data. Here, the first information is, for example, image data generated based on the first observation data, or meta information obtained from a signal or image data from the artificial satellite 201. That is, the data included in the integrated data may be information based on the observation data. Also, the first information may be, for example, the observation data itself from the artificial satellite 201.

[0071] In step S504, the integrated data generation unit 1014 acquires second information based on the second observation data. The second information is the same as the first information.

[0072] In step S505, the integrated data generation unit 1014 generates integrated data including the first information and the second information. At this time, the integrated data generation unit 1014 may store the generated integrated data in the storage unit 1012. Further, the integrated data generation unit 1014 may transmit the generated integrated data to an external information processing device.

[0073] The integrated data generation unit 1014 may generate integrated data each time observation data is acquired. For example, the integrated data generation unit 1014 may generate integrated data such that the observation data D14 obtained when the artificial satellite 201 observes the observation area at time T4 replaces the observation data D11 obtained at time T1 before time T4.

[0074] Referring to FIG. 6, the meta information acquisition process based on the integrated data will be described. In step S601, the meta information acquisition unit 1015 acquires the integrated data generated by the integrated data generation unit 1014. The meta information acquisition unit 1015 acquires the integrated data by referring to, for example, the integrated data stored in the storage unit 1012. In step S602, the meta information acquisition unit 1015 inputs the integrated data into the meta information output model 10121. In step S603, the meta information acquisition unit 1015 acquires the meta information of the observation area from the meta information output model 10121.

[0075] Through the processing so far, the observation data processing device 101 can generate data obtained by observing the observation area using a plurality of flying objects at high frequency and acquire the meta information of the observation area.

[0076] (Second Embodiment) The second embodiment will be described. In the embodiments after the second embodiment, descriptions of matters common to the first embodiment will be omitted, and only the differences will be described. The second embodiment is different from the first embodiment in that the observation data is converted using a conversion model, and integrated data is generated based on the converted data.

[0077] FIG. 7 shows a block diagram of the observation data processing apparatus 101A according to the second embodiment. The observation data processing apparatus 101A includes a storage unit 1012A having a first conversion model 10122 and a second conversion model 10123, and a converted observation data acquisition unit 1016, in addition to each unit of the observation data processing apparatus 101.

[0078] The first conversion model 10122 will be described with reference to FIG. 8. The first conversion model 10122 is a learning model that outputs converted observation data in response to inputs of third observation data, third observation conditions, and third observation timings obtained by observing an observation area.

[0079] The first conversion model 10122 is learned using the learning data LD2 as teacher data. The learning data LD2 includes a set of learning observation data, learning observation conditions, and learning observation timings obtained by observing a learning observation area, and converted observation data corresponding thereto. The converted observation data is data observed under the first observation conditions, and the observation timing thereof is the learning observation timing. The first conversion model 10122 is learned by inputting the learning observation data, the learning observation conditions, and the learning observation timings and outputting the converted observation data.

[0080] The first conversion model 10122 is a model that generates observation data obtained by changing the observation conditions for the observation data at a certain timing. By the first conversion model 10122, observation data with different observation conditions is converted into observation data under a certain observation condition (for example, the first observation condition), and integrated data is generated.

[0081] The conversion observation data acquisition unit 1016 acquires converted observation data obtained by converting the observation data using the first conversion model 10122 and the second conversion model 10123 described later.

[0082] Referring to FIGS. 9 and 10, the generation process of integrated data using the first conversion model 10122 in the observation data processing apparatus 101A will be described. Here, it is assumed that the artificial satellites 201, 202, and 203 acquire observation data based on a common observation method (for example, the SAR method). Further, the observation system 10 further includes an artificial satellite 204 that acquires observation data based on an observation method (for example, an optical method) different from that of the artificial satellites 201, 202, and 203.

[0083] In step S901, the observation data acquisition unit 1013 acquires first observation data at a first timing. For example, the observation data acquisition unit 1013 acquires observation data D11 at time T1.

[0084] In step S902, the observation data acquisition unit 1013 acquires second observation data at a second timing. For example, the observation data acquisition unit 1013 acquires observation data D42 at time T2.

[0085] In step S903, the observation data acquisition unit 1013 acquires third observation data at a third timing. For example, the observation data acquisition unit 1013 acquires observation data D23 at time T3.

[0086] In step S904, the conversion observation data acquisition unit 1016 acquires third observation conditions for the third observation data. For example, the conversion observation data acquisition unit 1016 acquires observation conditions for the observation data D23. More specifically, the aircraft information of the artificial satellite 202 and the environmental information of the observation environment of the observation data D23 are acquired as the observation conditions.

[0087] In step S905, the conversion observation data acquisition unit 1016 inputs the third observation data, the third observation condition, and the third timing into the first conversion model 10122. For example, the conversion observation data acquisition unit 1016 inputs the observation data D23, the observation condition for the observation data D23, and the time T3 into the first conversion model 10122.

[0088] In step S906, the conversion observation data acquisition unit 1016 acquires the converted observation data obtained by converting the third observation data. For example, the conversion observation data acquisition unit 1016 acquires the converted observation data D13 obtained by converting the observation data D23 from the first conversion model 10122.

[0089] In step S907, the integrated data generation unit 1014 updates the first information with the first information based on the conversion observation data, and generates integrated data including the first information and the second information. For example, the integrated data generation unit 1014 generates integrated data ID2 in which the observation data D11 and the observation data D42 are integrated. Next, the integrated data generation unit 1014 generates integrated data ID3 in which the observation data D11 is updated by the conversion observation data D13.

[0090] Similarly, the integrated data generation unit 1014 generates integrated data ID4 based on the conversion observation data D14 obtained by converting the observation data D34. Further, the integrated data generation unit 1014 generates integrated data ID5 in which the conversion observation data D14 is updated by the observation data D15 in the integrated data.

[0091] As a result, for example, even when there are differences in aircraft characteristics and observation environments between the artificial satellite 201 and the artificial satellites 202 and 203, it is possible to align the observation data used for generating integrated data as data observed under the observation conditions of the artificial satellite 201. Thereby, integrated data with reduced differences between observation means is generated. Also, it becomes possible to generate data observed under the observation conditions of the artificial satellite 201 frequently, and as a result, it becomes possible to update the integrated data frequently. By using such integrated data, meta-information of the observation area can be obtained with higher accuracy.

[0092] With reference to FIG. 11, the second conversion model 10123 will be described. The second conversion model 10123 is a learning model that outputs converted observation data in response to the input of first observation data, first observation conditions, and first observation timing obtained by observing an observation area.

[0093] The second conversion model 10123 is learned using the learning data LD3 as teacher data. The learning data LD3 includes a set of learning observation data, learning observation conditions, and learning observation timing obtained by observing a learning observation area, and converted observation data corresponding thereto. The converted observation data is data observed under reference observation conditions, and the observation timing thereof is the learning observation timing. The learning model 10212 is learned with the learning observation data, learning observation conditions, and learning observation timing as input and the converted observation data as output. Here, the reference observation conditions are predetermined observation conditions different from the respective observation conditions of the artificial satellites 201, 202, and 203. The reference observation conditions are conditions determined in advance by the user.

[0094] The second conversion model 10123 is a model that generates observation data obtained by changing the observation conditions for the observation data at a certain timing. By the second conversion model 10123, observation data with different observation conditions is converted into observation data based on the reference observation conditions, and integrated data is generated.

[0095] Referring to FIGS. 12 and 13, the generation process of integrated data using the second conversion model 10123 in the observation data processing apparatus 101A will be described. Here, it is assumed that the artificial satellites 201, 202, and 203 acquire observation data based on a common observation method (for example, the SAR method). Further, the observation system 10 further includes an artificial satellite 204 that acquires observation data based on an observation method (for example, the optical method) different from that of the artificial satellites 201, 202, and 203.

[0096] In step S1201, the observation data acquisition unit 1013 acquires the first observation data at the first timing. For example, the observation data acquisition unit 1013 acquires the observation data D11 at time T1.

[0097] In step S1202, the converted observation data acquisition unit 1016 acquires the first observation condition for the first observation data. For example, the converted observation data acquisition unit 1016 acquires the observation condition for the observation data D11. More specifically, the aircraft information of the artificial satellite 201 and the environmental information of the observation environment are acquired as the observation conditions.

[0098] In step S1203, the converted observation data acquisition unit 1016 inputs the first observation data, the first observation condition, and the first timing into the second conversion model 10123. For example, the converted observation data acquisition unit 1016 inputs the observation data D11, the observation condition for the observation data D11, and the time T1 into the second conversion model 10123.

[0099] In step S1204, the converted observation data acquisition unit 1016 acquires the first converted observation data obtained by converting the first observation data. For example, the converted observation data acquisition unit 1016 acquires the converted observation data D51 obtained by converting the observation data D11 from the first conversion model 10122.

[0100] In step S1205, the observation data acquisition unit 1013 acquires the second observation data at the second timing. For example, the observation data acquisition unit 1013 acquires the observation data D42 at time T2.

[0101] In step S1206, the integrated data generation unit 1014 generates integrated data based on the first information based on the first converted observation data and the second information based on the second observation data. For example, the integrated data generation unit 1014 generates integrated data ID2 based on the converted observation data D51 and the observation data D42.

[0102] In step S1207, the observation data acquisition unit 1013 acquires the third observation data at the third timing. For example, the observation data acquisition unit 1013 acquires the observation data D23 at time T3.

[0103] In step S1208, the converted observation data acquisition unit 1016 acquires the third observation condition for the third observation data. For example, the converted observation data acquisition unit 1016 acquires the observation condition for the observation data D23. More specifically, the aircraft information of the artificial satellite 201 and the environmental information of the observation environment are acquired as the observation conditions.

[0104] In step S1209, the converted observation data acquisition unit 1016 inputs the third observation data, the third observation condition, and the third timing into the second conversion model 10123. For example, the converted observation data acquisition unit 1016 inputs the observation data D23, the observation condition for the observation data D23, and the time T3 into the second conversion model 10123.

[0105] In step S1210, the converted observation data acquisition unit 1016 acquires the third converted observation data obtained by converting the third observation data. For example, the converted observation data acquisition unit 1016 acquires the converted observation data D53 obtained by converting the observation data D23 from the first conversion model 10122.

[0106] In step S1211, the integrated data generation unit 1014 updates the first information based on the first converted observation data with the first information based on the second converted observation data, and generates integrated data including the first information and the second information. For example, the integrated data generation unit 1014 generates integrated data ID3 in which the observation data D51 is updated by the converted observation data D53.

[0107] Similarly, the integrated data generation unit 1014 generates integrated data ID4 based on the converted observation data D54 obtained by converting the observation data D34. Further, the integrated data generation unit 1014 generates integrated data ID5 based on the converted observation data D55 obtained by converting the observation data D15.

[0108] Thereby, for example, even when there are differences in aircraft characteristics and observation environments between the artificial satellite 201 and the artificial satellites 202 and 203, it is possible to align the observation data used for generating the integrated data as data observed under predetermined observation conditions. Thereby, integrated data with reduced differences between observation means is generated. By using such integrated data, meta-information in the observation area can be obtained with higher accuracy.

[0109] (Third Embodiment) The third embodiment will be described. The third embodiment is different from the first and second embodiments in that the observation data is inferred using an inference model, and the integrated data is generated based on the inferred data. Here, the inference of the observation data means inferring the observation data observed at a certain timing from other observation data.

[0110] FIG. 14 shows a block diagram of the observation data processing apparatus 101B according to the third embodiment. The observation data processing apparatus 101B includes a storage unit 1012B having a first estimation model 10124, a second estimation model 10125, and a third estimation model 10126, and an estimated observation data acquisition unit 1017, in addition to the respective units of the observation data processing apparatus 101. The estimated observation data acquisition unit 1017 acquires estimated observation data obtained by estimating the observation data using the first estimation model 10124, the second estimation model 10125, or the third estimation model 10126, which will be described later.

[0111] The first estimation model 10124 will be described with reference to FIG. 15. The first estimation model 10124 is a learning model that outputs estimated observation data in response to the input of first observation data obtained by observing an observation area and second observation timing. The input first observation data is data observed based on a first observation method (for example, the SAR method), and the output estimated observation data is data observed based on a second observation method (for example, the optical method) different from the first observation method.

[0112] The first estimation model 10124 is learned using the learning data LD4 as teacher data. The learning data LD4 includes a set of learning observation data obtained by observing a learning observation area and learning observation timing A and the estimated observation data corresponding thereto. The estimated observation data is data observed based on the first observation method, and the observation timing thereof is timing A. The first estimation model 10124 is learned by inputting the learning observation data and the learning observation timing and outputting the estimated observation data.

[0113] The first estimation model 10124 is a model that generates observation data obtained by changing the observation method of the observation data at a certain timing to another observation method. Based on a certain observation data, the first estimation model 10124 estimates observation data with different observation methods and generates integrated data.

[0114] Referring to FIGS. 16 and 17, the generation process of integrated data using the first inference model 10124 in the observation data processing apparatus 101B will be described. Here, it is assumed that the artificial satellite 201 acquires observation data based on the first observation method (for example, the SAR method), the artificial satellites 202 and 204 acquire observation data based on the second observation method (for example, the optical method), and the artificial satellite 203 acquires observation data based on the third observation method (for example, the observation method using meteorological data).

[0115] In step S1601, the observation data acquisition unit 1013 acquires the first observation data at the first timing. For example, the observation data acquisition unit 1013 acquires the observation data D11 at time T1.

[0116] In step S1602, the observation data acquisition unit 1013 acquires the second observation data at the second timing. For example, the observation data acquisition unit 1013 acquires the observation data D42 at time T2.

[0117] In step S1603, the observation data acquisition unit 1013 acquires the third observation data at the third timing. For example, the observation data acquisition unit 1013 acquires the observation data D23 at time T3.

[0118] In step S1604, the converted observation data acquisition unit 1016 acquires the third observation condition for the third observation data. For example, the converted observation data acquisition unit 1016 acquires the observation condition for the observation data D23. More specifically, the aircraft information of the artificial satellite 202 and the environmental information of the observation environment of the observation data D23 are acquired as the observation conditions.

[0119] In step S1605, the converted observation data acquisition unit 1016 inputs the third observation data, the third observation condition, and the third timing to the first inference model 10124. For example, the converted observation data acquisition unit 1016 inputs the observation data D23, the observation condition for the observation data D23, and the time T3 to the first conversion model 10122.

[0120] In step S1606, the conversion observation data acquisition unit 1016 acquires the converted observation data obtained by converting the third observation data. For example, the conversion observation data acquisition unit 1016 acquires the estimated observation data D13 obtained by converting the observation data D23 from the first estimation model 10124.

[0121] In step S1607, the integrated data generation unit 1014 updates the first information based on the estimated observation data, and generates integrated data including the first information and the second information. For example, the integrated data generation unit 1014 generates integrated data ID2 obtained by integrating the observation data D11 and the observation data D42. Next, the integrated data generation unit 1014 generates integrated data ID3 in which the observation data D11 is updated by the estimated observation data D13.

[0122] Similarly, the integrated data generation unit 1014 generates integrated data ID4 based on the converted observation data D14 obtained by converting the observation data D34. Further, the integrated data generation unit 1014 generates integrated data ID5 in which the converted observation data D14 is updated by the observation data D15 in the integrated data.

[0123] Thereby, for example, even when there are differences in the observation methods among the artificial satellites 201, 202, and 203, it is possible to align the observation data used for generating the integrated data as data observed under a certain observation method, for example, the observation method of the artificial satellite 201. In addition, it is possible to generate the data observed under the observation method of the artificial satellite 201 with high frequency. As a result, the differences between the observation methods are aligned, and the integrated data is generated with high frequency. By using such integrated data, the meta information of the observation area can be obtained more accurately.

[0124] Referring to FIG. 18, the second inference model 10125 will be described. The second inference model 10125 is a learning model that outputs inferred observation data for inputs of first observation data, a first observation timing, and a second observation timing obtained by observing an observation area. The input first observation data is data observed based on a first observation method (for example, the SAR method), and the output inferred observation data is data observed based on a second observation method (for example, the optical method) different from the first observation method.

[0125] The second inference model 10125 is learned using the learning data LD5 as teacher data. The learning data LD5 includes a set of learning observation data obtained by observing a learning observation area, a learning observation timing A, a timing B different from the learning observation timing A, and the inferred observation data corresponding thereto. The inferred observation data is data observed based on the first observation method, and the observation timing thereof is the timing B. The second inference model 10125 is learned with the learning observation data, the learning observation timing, and the timing B as inputs and the inferred observation data as the output.

[0126] The second inference model 10125 is a model that changes the observation method of the observation data at a certain timing to another observation method and generates observation data obtained at another timing. The second inference model 10125 infers observation data with different observation methods and observation timings based on a certain observation data, and generates integrated data.

[0127] Referring to FIGS. 19 and 22, the generation process of integrated data using the second inference model 10125 in the observation data processing apparatus 101B will be described. Here, it is assumed that the artificial satellites 201 and 203 acquire observation data based on a first observation method (for example, the SAR method), and the artificial satellite 202 acquires observation data based on a second observation method (for example, the optical method).

[0128] In step S1901, the observation data acquisition unit 1013 acquires first observation data at a first timing. For example, the observation data acquisition unit 1013 acquires observation data D22 at time T2.

[0129] In step S1902, the estimated observation data acquisition unit 1017 inputs the first observation data, the first timing, and the second timing into the first estimation model. For example, as shown in FIG. 22, the estimated observation data acquisition unit 1017 inputs the observation data D22, the time T2, and the time T3 into the second estimation model 10125.

[0130] In step S1903, the estimated observation data acquisition unit 1017 acquires estimated observation data at a second timing from the second estimation model 10125. For example, as shown in FIG. 22, the estimated observation data acquisition unit 1017 acquires the estimated observation data D63 at time T3 from the second estimation model 10125.

[0131] In step S1904, the integrated data generation unit 1014 generates integrated data based on first information based on the first observation data and second information based on the estimated observation data. For example, as shown in FIG. 22, the integrated data generation unit 1014 generates integrated data ID3 based on the observation data D11 and the estimated observation data D63.

[0132] Thereby, for example, even when there are differences in the observation methods between the artificial satellite 202 and the artificial satellites 201 and 203, it is possible to align the observation data used for generating the integrated data as data observed under the observation methods of the artificial satellites 201 and 203. Thereby, integrated data with reduced differences between the observation means is generated. By using such integrated data, the meta information of the observation area can be acquired more accurately.

[0133] Referring to FIG. 20, the third inference model 10126 will be described. The third inference model 10126 is a learning model that outputs inferred observation data for the first observation data obtained by observing an observation area, the third observation timing before the first observation timing, and the input at the first observation timing. The first observation data to be input and the inferred observation data to be output are data observed based on a common observation method (for example, the SAR method). The third inference model 10126 is a model that performs time-series regression prediction and generates inferred observation data.

[0134] The third inference model 10126 is learned using the learning data LD6 as teacher data. The learning data LD6 includes a set of learning observation data obtained by observing a learning observation area, a learning observation timing A, a timing C before the learning observation timing A, and the inferred observation data corresponding thereto. The learning observation data and the inferred observation data are data observed based on the first observation method, and the observation timing thereof is the timing C. The third inference model 10126 is learned with the learning observation data, the learning observation timing, and the timing C as inputs and the inferred observation data as outputs.

[0135] The third inference model 10126 is a model that generates observation data obtained at timings after a certain timing based on the observation data at a certain timing without changing the observation method. The third inference model 10126 infers observation data with different observation timings based on a certain observation data and generates integrated data.

[0136] Referring to FIGS. 20 and 21, the generation process of integrated data using the third inference model 10126 in the observation data processing apparatus 101B will be described. Here, it is assumed that the artificial satellites 201 and 203 acquire observation data based on the first observation method (for example, the SAR method), and the artificial satellite 202 acquires observation data based on the second observation method (for example, the optical method).

[0137] In step S2101, the observation data acquisition unit 1013 acquires third observation data when the observation area is observed at a third timing before the first timing. For example, when the observation area is observed at a time T2 before the time T5, the observation data acquisition unit 1013 acquires the observation data D22.

[0138] In step S2102, the estimated observation data acquisition unit 1017 inputs the third observation data, the third timing, and the first timing into the estimation model. For example, as shown in FIG. 22, the estimated observation data acquisition unit 1017 inputs the observation data D22, the time T2, and the time T5 into the third estimation model 10126.

[0139] In step S2103, the estimated observation data acquisition unit 1017 acquires the estimated observation data at the first timing from the third estimation model 10126. For example, as shown in FIG. 22, the estimated observation data acquisition unit 1017 acquires the estimated observation data D75 at the time T5 from the third estimation model 10126.

[0140] In step S2104, the integrated data generation unit 1014 generates integrated data based on first information based on the estimated observation data at the first timing and second information based on the second observation data. For example, as shown in FIG. 22, the integrated data generation unit 1014 generates integrated data ID5 based on the estimated observation data D53 and the observation data D14.

[0141] Thereby, for example, even when the artificial satellite 202 cannot perform observation at a certain point in time, it is possible to estimate the observation data used for generating the integrated data and use it as the integrated data. Thereby, even when a part of the observation data cannot be obtained, it is possible to generate integrated data with a short time interval. By using such integrated data, it becomes possible to acquire the meta information of the observation area with higher accuracy.

[0142] The embodiments described above are for facilitating the understanding of the present invention and are not for limiting and interpreting the present invention. Each element included in the embodiments, as well as its arrangement, conditions, shape, size, etc. are not limited to those exemplified and can be changed as appropriate. Also, it is possible to partially substitute or combine the configurations shown in different embodiments with each other.

Explanation of Reference Numerals

[0143] 10…Observation system, 101, 101A, 101B…Observation data processing device, 201, 202, 203…Artificial satellite, 1013…Observation data acquisition unit, 1014…Integrated data generation unit, 1015…Meta information acquisition unit, 1016…Converted observation data acquisition unit, 1017…Estimated observation data acquisition unit

Claims

1. On the computer, generating integrated data including first information based on the first observation data and second information based on the second observation data, the first information being information of an observation area at a first timing and observed by a first observation means, and second information being information of the observation area at a second timing different from the first timing and observed by a second observation means different from the first observation means; A program that executes the following.

2. The program according to claim 1, the first observation data is data observed under a first observation condition, The computer includes: acquiring third observation data, which is information on the observation area at a third timing different from the first timing, observed under third observation conditions by a third observation means different from the first observation means, and observed based on a common observation method to the first observation data; inputting the third observation data, the third observation conditions, and the third timing into a conversion model which receives learning observation data, which is information about a learning observation area, learning observation conditions corresponding to the learning observation data, and a learning observation timing at which the learning observation data was observed, and outputs converted observation data into which the learning observation data is converted, the converted observation data indicating data observed in the learning observation area under the first observation conditions at the learning observation timing, and acquiring the converted observation data into which the third observation data is converted; Then, generating the integrated data, updating the first information based on the converted observation data to generate the integrated data.

3. The program according to claim 2, the first observation condition includes first aircraft information of a first flying object which is the first observation means, and first environmental information indicating an observation environment by the first flying object; A program, wherein the third observation conditions include third aircraft information of a third flying object which is the third observation means, and second environmental information which indicates the observation environment by the third flying object.

4. The program according to claim 1, the first observation data is data observed under a first observation condition, The computer includes: acquiring the first observation data and the first observation condition; acquiring third observation data, which is information on the observation area at a third timing different from the first timing, observed under third observation conditions by a third observation means different from the first observation means, and observed based on a common observation method to the first observation data; a conversion model that receives input of learning observation data, which is information on a learning observation area, learning observation conditions corresponding to the learning observation data, and a learning observation timing at which the learning observation data was observed, and outputs converted observation data obtained by converting the learning observation data, the converted observation data indicating data observed in the observation area under reference observation conditions at the learning observation timing, inputting the first observation data, the first observation conditions, and the first timing, and acquiring first converted observation data as the converted observation data obtained by converting the first observation data; inputting the third observation data, the third observation condition, and the third timing into the conversion model, and acquiring second converted observation data as the converted observation data obtained by converting the third observation data at the third timing; Then, generating the integrated data, updating the first information based on the first converted observation data with the first information based on the second converted observation data to generate the integrated data.

5. The program according to claim 4, the first observation condition includes first aircraft information of a first flying object which is the first observation means, and first environmental information indicating an observation environment by the first flying object; A program, wherein the third observation conditions include third aircraft information of a third flying object which is the third observation means, and second environmental information which indicates the observation environment by the third flying object.

6. The program according to claim 1, the first observation data is data observed based on a first observation method, The second observation data is data observed based on a second observation method different from the first observation method.

7. The program according to claim 6, The computer includes: acquiring third observation data, which is information on the observation area at a third timing different from the first timing, which is data observed under third observation conditions by a third observation means different from the first observation means, and which is observed based on an observation method different from the first observation method; inputting the third observation data and the third timing into an estimation model which receives the learning observation data and the learning observation timing at which the learning observation data was observed, and outputs estimated observation data indicating data at which the observation domain was observed based on the first observation method at the learning observation timing, and acquiring the estimated observation data at the third timing; Then, generating the integrated data, updating the first information based on the estimated observation data to generate the integrated data.

8. The program according to claim 6, The computer includes: acquiring the first observation data; inputting the first observation data, the first timing, and the second timing into an estimation model which receives as input learning observation data, a learning observation timing at which the learning observation data was observed, and a timing different from the learning observation timing, and outputs estimated observation data indicating data obtained by observing the observation area based on the second observation method at the different timing, and acquiring the estimated observation data at the second timing; Then, generating the integrated data, generating the integrated data including the second information based on the estimated observation data at the second timing.

9. The program according to claim 6, acquiring third observation data when the observation area is observed at a third timing that is earlier than the first timing; inputting the third observation data, the third timing, and the first timing into an estimation model which receives as input learning observation data, a learning observation timing at which the learning observation data was observed, and a prediction timing after the learning observation timing, and outputs estimated observation data indicating data at which the observation area was observed based on the first observation method at the prediction timing, and acquiring the estimated observation data at the first timing; Then, generating the integrated data, generating the integrated data including the first information based on the estimated observation data at the first timing.

10. The program according to claim 1, The computer includes: inputting learning integrated data based on learning observation data, which is information of a learning observation area, into a meta-information output model which outputs meta-information of the learning observation area, and acquiring meta-information of the observation area; Further execute the program.

11. an integrated data generating unit that generates integrated data including first information based on the first observation data and second information based on the second observation data, the first information being information of an observation area at a first timing and observed by a first observation means, and second observation data being information of the observation area at a second timing different from the first timing and observed by a second observation means different from the first observation means; An information processing device comprising:

12. The computer generating integrated data including first information based on the first observation data and second information based on the second observation data, the first information being information of an observation area at a first timing and observed by a first observation means, and second information being information of the observation area at a second timing different from the first timing and observed by a second observation means different from the first observation means; An information processing method comprising:

13. learning is performed using teacher data in which learning observation data, which is information on a learning observation area, and learning observation conditions corresponding to the learning observation data are input, and first converted observation data in which the learning observation data is converted as data observed in the learning observation area according to a reference observation condition at the timing when the learning observation data is acquired is output; A learning model that causes a computer to function in response to input of observation data, which is information about a target observation area, and observation conditions corresponding to the observation data, so as to output second converted observation data that indicates data observed in the target observation area under the standard observation conditions at the time the observation data was acquired, and in which the observation data is converted.

14. learning is performed using teacher data in which learning observation data when a learning observation area is observed based on a first observation method, an observation timing when the learning observation data is observed, and a prediction timing different from the observation timing are input, and first estimated observation data indicating data observed in the learning observation area based on a second observation method different from the first observation method at the prediction timing is output; A learning model that causes a computer to function in response to input of observation data, which is information about a target observation area, and observation conditions corresponding to the observation data, such that the computer outputs second estimated observation data indicating data observed of the target observation area based on the second observation method at the time the observation data was acquired.

Images