Information processing apparatus, information processing method, program, and sensing system

Abstract

Provided is an information processing apparatus including a macro measurement analysis calculation section configured to calculate detection data from a macro measurement section adapted to sense a first measurement range of a measurement object at a first spatial resolution; a micro measurement analysis calculation section configured to calculate detection data from a micro measurement section adapted to sense a second measurement range at a second spatial resolution higher than the first spatial resolution, the second measurement range being included in the first measurement range of the measurement object; and an inverse model calculation section configured to acquire a model parameter for inverse model calculation using a result of calculation from the macro measurement analysis calculation section, based on the detection data from the micro measurement section determined by the micro measurement analysis calculation section.

Description

[0001] Cross-reference to related applications

[0002] This application claims the benefit of Japanese priority patent application JP2019-124764, filed on July 3, 2019, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This technology relates to information processing devices, information processing methods, programs, and sensing systems, and in particular, to techniques suitable for generating measurement results such as vegetation status. Background Technology

[0004] For example, much effort has been made to remotely sense vegetation conditions by using imaging devices mounted on small flying objects (such as drones) to image the vegetation conditions of plants as the flying objects fly over farmland.

[0005] PTL 1 discloses a technique for performing remote sensing by imaging farmland.

[0006] Citation List

[0007] Patent documents

[0008] PTL 1

[0009] Japanese Patent No. 5162890 Summary of the Invention

[0010] Technical issues

[0011] In addition to shape measurements based on visible light (R (red), G (green), and B (blue)), this remote sensing also allows for the measurement of an object's physical properties, physiological state, and other characteristics using various light wavelengths and methods. However, sensing devices that can be installed in small flying objects are typically limited in terms of size and weight.

[0012] For example, hyperspectral cameras capable of acquiring a large number of wavelengths and performing component analysis often require a scanning mechanism configured to acquire two-dimensional images and are relatively large. Therefore, it is difficult to mount hyperspectral cameras on small drones or similar devices.

[0013] On the other hand, satellites equipped with advanced devices capable of performing sophisticated sensing are in operation to enable sensing using artificial satellites. However, this sensing is insufficient in terms of spatial resolution.

[0014] Because various objects are mixed within a single spatial resolution unit, low spatial resolution not only precludes shape determination but also hinders measurement of only the object to be inspected.

[0015] To cover low spatial resolution, measurements of a specific object are determined using an inverse model calculation (inverse computation) based on a model that contains information about the object's form (e.g., a model that associates "shape" with "characteristic / environmental response," specifically, a radiative transfer characteristic model). However, in reality, differences in shape and other properties between the object and the model make accurate measurements difficult.

[0016] Therefore, it is desirable to provide a system and an information processing apparatus for the system, which can obtain more accurate measurement results based on remote sensing using high-performance sensors, such as those from satellites.

[0017] Solution to the problem

[0018] An information processing apparatus according to an embodiment of the present technology includes a macroscopic measurement analysis and calculation unit configured to calculate detection data from a macroscopic measurement unit, the macroscopic measurement unit being adapted to sense a first measurement range of a measurement object at a first spatial resolution; a microscopic measurement analysis and calculation unit configured to calculate detection data from a microscopic measurement unit, the microscopic measurement unit being adapted to sense a second measurement range at a second spatial resolution higher than the first spatial resolution, the second measurement range being included in the first measurement range of the measurement object; and an inverse model calculation unit configured to obtain model parameters for inverse model calculation using the calculation results from the macroscopic measurement analysis and calculation unit, based on the detection data from the microscopic measurement unit determined by the microscopic measurement analysis and calculation unit.

[0019] The model parameters used for inverse computation are generated based on the measurement results of the second space decomposition.

[0020] Furthermore, in the information processing apparatus according to the embodiments of the present technology described above, the inverse model calculation unit can use model parameters based on detection data from the micro-measurement unit determined by the micro-measurement analysis calculation unit as parameters of the inverse model in the inverse model calculation using the calculation results from the macro-measurement analysis calculation unit.

[0021] Advanced measurements can be achieved based on inverse model computation by using macroscopic measurements capable of high-performance sensing. In this case, parameters obtained from actual measurements and through high spatial resolution sensing are used as parameters for the inverse model.

[0022] In the information processing apparatus according to the embodiments of the present technology described above, the inverse model calculation unit can use model parameters based on detection data of a second measurement range to determine the calculation result in units of a first spatial resolution.

[0023] The inverse model computation component determines the computational results in the form of, for example, features or environmental responses. In this case, model parameters of a second measurement range associated with the microscopic measurement component are used to determine the computational results in units of a first spatial resolution associated with the macroscopic measurement component.

[0024] In the information processing apparatus according to the embodiments of the present technology described above, the inverse model calculation unit can determine the characteristics or environmental response of the measurement object as the calculation result in units of a first spatial resolution.

[0025] A characteristic refers to the static shape or properties of a measured object. Environmental response refers to the dynamic shape or properties of a measured object.

[0026] In the information processing apparatus according to the embodiments of the present technology described above, the macroscopic measurement component can sense at a distance longer than that of the measurement object than the microscopic measurement component.

[0027] Compared to micro-measuring components, macro-measuring components perform measurements over a wider range at a greater distance from the object being measured. Conversely, compared to macro-measuring components, micro-measuring components perform measurements over a relatively narrower range at a shorter distance from the object being measured.

[0028] In the information processing apparatus according to the embodiments of the present technology described above, the inverse model calculation unit can obtain model parameters of representative individuals in each measurement region for clustering, switch model parameters for each clustered measurement region, and apply the obtained model parameters to the inverse model.

[0029] The model parameters for each measurement region generated by clustering are determined by calculating the detection data from the micro-measuring component. The inverse model calculation component can use different model parameters for each measurement region generated by clustering.

[0030] In the information processing apparatus according to the embodiments of the present technology described above, the clustering can be performed based on user input from a specified region.

[0031] For example, farmland is divided into areas for planting different crops. Users, such as farmers, can input this information.

[0032] In the information processing apparatus according to the embodiments of the present technology described above, the clustering can be performed based on detection data from macroscopic measurement components or detection data from microscopic measurement components.

[0033] For example, using detection data from macroscopic or microscopic measurement components allows regions with significantly different shapes or states to be distinguished from each other, thereby automatically performing clustering calculations.

[0034] In the information processing apparatus according to the embodiments of the present technology described above, the clustering can be performed based on user input from a designated area and detection data from macroscopic measurement components or detection data from microscopic measurement components.

[0035] That is, perform clustering calculations that reflect both manual input and automatic differentiation.

[0036] In the information processing apparatus according to the embodiments of the present technology described above, the model parameters may include any one of the following: the three-dimensional structure of the plant, plant height, average leaf angle (average leaf tilt angle), plant coverage, LAI, chlorophyll concentration, soil spectral characteristics, or sun-leaf ratio.

[0037] Please note that LAI (Leaf Area Index) is a leaf area index that represents the number of leaves in a plant; therefore, a higher value indicates more leaves.

[0038] In the information processing apparatus according to the above-described embodiments of the present technology, the micro-measurement component may include any one of the following as a micro-measurement sensor: a visible light image sensor, a stereo camera, a sensor for laser image detection and ranging, a polarization sensor, or a ToF (Time-of-Flight) sensor.

[0039] Please note that the sensor used for laser image detection and ranging is called LiDAR (Light Detection and Ranging).

[0040] In the information processing apparatus according to the embodiments of the present technology described above, the macroscopic measurement component may include any one of a multispectral camera, a hyperspectral camera, an FTIR (Fourier transform infrared spectrometer), or an infrared sensor as a macroscopic measurement sensor.

[0041] The information processing apparatus according to the embodiments of the present technology described above may further include an output component configured to generate and output image data based on the calculation results from the inverse model calculation component.

[0042] In other words, the information processing device allows information about the calculation results from the inverse model calculation unit to be imaged and presented to the user.

[0043] In the information processing apparatus according to the embodiments of the present technology described above, the output unit can generate output image data obtained from the color mapping of the calculation results from the inverse model calculation unit.

[0044] Given the computation results from the inverse model computation component for each of the multiple regions, an image is generated to be presented to the user, such that a different color is assigned to each region.

[0045] In the information processing apparatus according to the embodiments of the present technology described above, the output unit can generate output image data obtained by combining an image obtained from a color mapping derived from a calculation result from an inverse model calculation unit with another image.

[0046] Composite images, each with different colors assigned to its region, are created by means of overlay or overwriting.

[0047] In the information processing apparatus according to the embodiments of the present technology described above, the macroscopic measurement component can be installed in an artificial satellite.

[0048] Macroscopic measurement components are installed on satellites to measure objects, such as farmland, from a distance in the sky.

[0049] In the information processing apparatus according to the embodiments of the present technology described above, the micro-measuring component can be installed in a flight body that can be radio-controlled or automatically controlled.

[0050] Examples of flying bodies capable of radio or automatic control include so-called drones, small radio-controlled fixed-wing aircraft, small radio-controlled helicopters, etc.

[0051] Another embodiment of the information processing method according to the present technology includes: performing macroscopic measurement analysis processing by an information processing device to calculate detection data from a macroscopic measurement component, the macroscopic measurement component being configured to sense a first measurement range of a measurement object at a first spatial resolution; performing microscopic measurement analysis processing by the information processing device to calculate detection data from a microscopic measurement component, the microscopic measurement component being configured to sense a second measurement range at a second spatial resolution higher than the first spatial resolution, the second measurement range being included in the first measurement range of the measurement object; and performing inverse model calculation processing by the information processing device to obtain model parameters for inverse model calculation using the calculation results in the macroscopic measurement analysis processing, based on the detection data from the microscopic measurement component determined in the microscopic measurement analysis processing.

[0052] Therefore, the information processing device is able to generate advanced and accurate measurement result information for the object being measured, which is a combination of macroscopic and microscopic measurements.

[0053] According to another embodiment of the present technology, the program is a program that causes an information processing device to execute the above-described method. This facilitates the realization of a computer device that generates advanced measurement results.

[0054] A sensing system according to another embodiment of the present technology includes: a macroscopic measurement component configured to sense a first measurement range of a measurement object with a first spatial resolution; a microscopic measurement component configured to sense a second measurement range with a second spatial resolution higher than the first spatial resolution, the second measurement range being included in the first measurement range of the measurement object; and the aforementioned information processing device.

[0055] Therefore, it is possible to construct a system that performs macroscopic and microscopic measurements and further uses the results of these measurements to generate measurement results. Attached Figure Description

[0056] Figure 1 This is an explanatory diagram of the macroscopic measurement component and the microscopic measurement component in the sensing system according to an embodiment of the present technology.

[0057] Figure 2 This is an explanatory diagram of a remote sensing example of farmland according to an embodiment.

[0058] Figure 3 This is an explanatory diagram illustrating the measurement of the macroscopic and microscopic measurement components according to an embodiment.

[0059] Figure 4 A diagram showing the measurement range and resolution of the macroscopic and microscopic measurement components according to this embodiment is provided.

[0060] Figure 5 A diagram illustrating a situation where inverse computation leads to incorrect results.

[0061] Figure 6 This is a descriptive diagram of clustering based on an embodiment.

[0062] Figure 7 This is a block diagram of the hardware configuration of an information processing apparatus according to an embodiment.

[0063] Figure 8 This is a block diagram illustrating the functional configuration of the information processing apparatus according to an embodiment.

[0064] Figure 9 This is a flowchart illustrating a processing example according to an embodiment.

[0065] Figure 10 This is a flowchart of the microscopic measurement analysis and calculation process according to the embodiment.

[0066] Figure 11 This is an illustrative diagram of an image used for microscopic measurement analysis calculations according to an embodiment.

[0067] Figure 12 This is a flowchart of clustering based on an embodiment.

[0068] Figure 13A diagram illustrating clustering according to an embodiment is shown.

[0069] Figure 14 This is a flowchart of the inverse model calculation according to the embodiment.

[0070] Figure 15 A diagram is shown showing the macroscopic and microscopic measurement component model parameters used for inverse model calculation according to an embodiment.

[0071] Figure 16 A diagram showing the output image obtained using color mapping according to an embodiment is illustrated.

[0072] Figure 17 A diagram showing a composite of a color-mapped image and another image according to an embodiment is shown.

[0073] Figure 18 A diagram showing a composite of a color-mapped image and another image according to an embodiment is shown. Detailed Implementation

[0074] The embodiments will be described in the following order.

[0075] <1. Configuration of the Sensing System>

[0076] <2. Configuration of Information Processing Devices>

[0077] <3. Processing Example>

[0078] <4. Various Examples>

[0079] <5. Conclusion and Modification Examples>

[0080] <1. Configuration of the Sensing System>

[0081] First, the sensing system according to an embodiment will be described.

[0082] Figure 1 The macroscopic measurement component 2 and the microscopic measurement component 3 included in the sensing system are shown.

[0083] The micro-measuring component 3 performs sensing at a position relatively close to the object being measured 4. The measurement range for which one sensing unit is performed is a relatively narrow range denoted as the micro-measuring range RZ3. Note that a unit used here, for example in the case of a camera, refers to the range of capturing one frame of an image, but this unit can vary depending on the sensor type.

[0084] In contrast, the macroscopic measuring component 2 performs sensing at a location farther from the object 4 than the microscopic measuring component 3. The measurement range for which one sensing unit is performed is represented as the macroscopic measurement range RZ2, which is wider than the microscopic measurement range RZ3. However, the measurement range for which one sensing unit is performed by the macroscopic measuring component 2 can be the same as the microscopic measurement range RZ3.

[0085] In this embodiment, the microscopic measurement range RZ3 is the same as or narrower than the macroscopic measurement range RZ2. That is, the area of ​​the microscopic measurement range RZ3 in the measurement object 4 is also covered by the macroscopic measurement range RZ2. In other words, the microscopic measurement range RZ3 is the range within which both the microscopic measurement by the microscopic measurement component 3 and the macroscopic measurement by the macroscopic measurement component 2 are performed.

[0086] An example of a sensing system using the macroscopic measurement component 2 and the microscopic measurement component 3 as described above is a system for sensing, for example, the vegetation status of farmland 300. Figure 2 As shown.

[0087] Figure 2 The condition of farmland 300 is shown. Recently, much effort has been made to utilize equipment installed on small flying bodies 200 (e.g., such as...). Figure 2 The imaging device 250 in the drone shown remotely senses the vegetation status.

[0088] The flying vehicle 200 can fly over the farmland 300, for example, by operator radio control or automatic control.

[0089] Imaging device 250 is disposed in flying body 200 to image, for example, an area below flying body 200. When flying body 200 flies over farmland 300 along a predetermined route, imaging device 250 captures still images, for example, periodically.

[0090] As described above, the imaging device 250 installed in the flight body 200 serves as Figure 1 The micro-measurement component 3 is located within the imaging device 250. The image captured by the imaging device 250 is used as detection data for the micro-measurement. The imaging range of the imaging device 250 corresponds to the micro-measurement range RZ3.

[0091] also, Figure 2 An artificial satellite 210 is shown in the sky. An imaging device 220 is installed in the artificial satellite 210 and is capable of sensing the surface side of the Earth.

[0092] Imaging device 220 can sense (image) farmland 300. That is, imaging device 220 serves as a macroscopic measurement component 2. The image captured by imaging device 220 is used as detection data for macroscopic measurement. The imaging range of imaging device 220 corresponds to the macroscopic measurement range RZ2.

[0093] The imaging device 250 (in other words, a specific micro-measurement sensor) used as the micro-measurement component 3 installed in the flight body 200 is assumed to be a visible light image sensor (an image sensor that images visible light in R (red), G (green) and B (blue)), a stereo camera, a photoradar (a sensor for laser image detection and ranging), a polarization sensor, a ToF sensor, a NIR (near-infrared) imaging camera, etc.

[0094] Furthermore, the microscopic measurement sensor can be a multispectral camera that performs imaging in multiple bands, such as capturing NIR and R (red) images, and is capable of calculating NDVI (Normalized Differential Vegetation Index) based on the acquired images, provided that the sensor has the device size to be operatively mounted in the aircraft 200. NDVI is an indicator of vegetation distribution and activity.

[0095] The aforementioned sensors are intended to be suitable for analyzing, for example, the characteristics of the measured object, environmental response, and environmental conditions (range, distribution, etc.). Note that characteristics refer to the static shape and properties of the measured object. Environmental response refers to the dynamic shape and properties of the measured object. Environmental conditions refer to the state of the environment in which the measured object exists, including the range of the measured object, its distribution, or environmental characteristics.

[0096] In addition, the sensor is expected to be relatively small in size and lightweight, and easy to install in the aircraft 200.

[0097] On the other hand, the imaging device 220 (in other words, a specific macroscopic measurement sensor) used as the macroscopic measurement component 2 installed in the satellite 210 can be a multispectral camera, hyperspectral camera, FTIR (Fourier transform infrared spectroscopy), infrared sensor, etc., that captures images (e.g., NIR images and R images) in multiple bands. In this case, a relatively large sensing device is accepted, and it is assumed that accurate sensing can be performed.

[0098] These macroscopic measurement sensors are suitable for analyzing various physical property values ​​(e.g., information about photosynthesis, etc.).

[0099] Furthermore, due to factors such as device size and weight, these sensors are difficult to install in the small flying object 200. However, in the sensing system of this example, such sensors are installed in the artificial satellite 210.

[0100] In addition, tag information is attached to the images captured and acquired by imaging devices 220 and 250. The tag information includes imaging date and time information, location information (latitude / longitude information) in the form of GPS (Global Positioning System) data, imaging device information (individual identification information and model information about the camera, etc.), and information about each image data (such as image size, wavelength, and imaging parameters, etc.).

[0101] Note that location information, as well as imaging date and time information, are also used as information to associate the image (detection data) from imaging device 220 with the image (detection data) from imaging device 250.

[0102] As described above, image data and tag information acquired by the imaging device 250 installed in the aircraft 200 and the imaging device 220 installed in the satellite 210 are sent to the information processing device 1. The information processing device 1 uses the image data and tag information to generate analysis information using farmland 300 as the measurement object. In addition, the information processing device 1 performs the process of presenting the analysis results as images to the user.

[0103] The information processing device 1 is implemented as, for example, a PC (personal computer), an FPGA (field programmable gate array), a terminal device such as a smartphone or tablet computer, etc.

[0104] Notice, Figure 1 An information processing device 1 separate from the imaging device 250 is shown; however, for example, a computing device (microcomputer, etc.) that serves as the information processing device 1 may be provided in the unit that includes the imaging device 250.

[0105] refer to Figure 3 The functions of macroscopic measurement component 2 and microscopic measurement component 3 will be described.

[0106] The macroscopic measurement component 2 performs measurements within the measurement range RZ2 and uses a model to perform inverse calculations to obtain outputs or measurement results such as vegetation. For example, by using a prepared model to interpret the macroscopic measurements, the correct output can be obtained from a mixture of measurement values.

[0107] However, accurate output cannot be obtained if there are differences between the model used for inverse model calculations and the actual measured object. In particular, plants and animals change shape due to growth or differ significantly in shape due to variety, so the model is often unsuitable.

[0108] Therefore, measurements from representative individuals were used to update the model. For this purpose, detection data from microscopic measurement components were used.

[0109] The micro-measurement component 3 can measure each individual to be measured. For example, individuals OBa1, OBa2, OBa3...OBan are illustrated, and the micro-measurement component 3 can measure or determine the characteristics, environmental response and environmental state of each individual, and the area identification based on the characteristics, environmental response and environmental state, etc.

[0110] The detection data from the micro-measurement component 3 is used to measure the characteristics or environmental responses of representative individuals, and these characteristics or environmental responses are used to modify the model parameters. Then, inverse model calculations can be performed using a model based on the actual measurements.

[0111] The primary purpose of measurements using a sensing system is to determine macroscopic trends (averages, totals, distributions, etc.), which can be obtained through inverse model calculations based on information from the detection data of the macroscopic measurement component 2. In this case, measurement accuracy is ensured by updating the model parameters based on the detection data from the microscopic measurement component 3.

[0112] Figure 4 A two-dimensional diagram shows the macroscopic measurement range RZ2 and the microscopic measurement range RZ3, and also shows the resolution of the macroscopic measurement component 2 and the microscopic measurement component 3.

[0113] Larger cells represent macroscopic measurement resolution, while smaller cells represent microscopic measurement resolution. The information obtained using resolution corresponds to, for example, the information of a single pixel in a captured image.

[0114] That is, the macroscopic measurement sensor installed in the macroscopic measurement component 2 is a sensor with a resolution corresponding to the large cell, while the microscopic measurement sensor installed in the microscopic measurement component 3 is a sensor with a resolution corresponding to the small cell.

[0115] For example, given a measurement object as shown by the dashed line, the resolution corresponding to the small cell shown by the thin line can be used to determine the characteristics, environmental response, area, etc. of the measurement object, while the resolution corresponding to the large cell shown by the thick line can be used to measure physical property values, etc.

[0116] The background for the need for such a sensing system will be described.

[0117] As mentioned above, sensing using flying objects 200, such as drones, has been frequently performed recently. This allows for the measurement of features not only based on visible light (RGB) measurements but also the use of various light wavelengths and techniques to measure the physical properties, physiological states, etc., of objects. However, sensing devices that can be installed in small flying objects 200 are often limited in terms of size and weight.

[0118] Hyperspectral cameras that acquire a wide range of wavelengths and perform component analysis typically require scanning mechanisms to obtain two-dimensional images and are relatively large. Therefore, unless the flying object is very large, hyperspectral cameras are difficult to install.

[0119] Furthermore, scanning may take time and may require hovering, thus extending the measurement time. Scanning also often hinders the use of the battery capacity in the aircraft 200 to measure the continental surface.

[0120] Furthermore, the potential vibration of the aircraft 200 during scanning may reduce measurement accuracy.

[0121] Furthermore, using a higher spectral resolution FTIR scheme in principle involves equipment that is longer and difficult to install in the aircraft 200.

[0122] To achieve accurate sensing, the signal-to-noise ratio (S / N) can be improved by installing a large imager or providing multiple exposures. However, large imagers involve large optical systems and are not suitable for installation within the flying body 200. Due to the hovering of the flying body 200, multiple exposures result in longer measurement times, and potential vibrations of the flying body 200 reduce accuracy.

[0123] In addition, the shell of the aircraft 200 is typically exposed to sunlight and has a temperature higher than normal.

[0124] In precise sensing, sensors are kept at low temperatures to reduce thermal noise. Some sensors used indoors (such as spectrophotometers) maintain accuracy by using Peltier elements to keep them at low temperatures. However, Peltier elements consume a lot of power and are therefore unsuitable for installation in power-constrained aircraft bodies 200.

[0125] Heat pump-type temperature control devices, such as those using compressors in air conditioners, have high power efficiency, but are not suitable for installation in the aircraft 200 in terms of size or weight.

[0126] On the other hand, in satellite sensing, artificial satellites equipped with devices that enable advanced sensing are in operation. However, satellite sensing is insufficient in terms of spatial resolution.

[0127] For Satellite 210, installing a hyperspectral camera, FTIR or a large imager, as well as cryogenic control as described above, is not so difficult.

[0128] However, because various objects are mixed within a single spatial resolution unit, low spatial resolution not only precludes shape determination but also hinders measurement of only the objects to be examined. In particular, in the example of vegetation measurement, soil, shadows, etc., are unfavorably mixed in.

[0129] More specifically, in remote sensing using Satellite 210, measurements are performed from a distant location. Therefore, in addition to the object to be measured, different objects are mixed within a single pixel corresponding to the spatial resolution of the measuring instrument. The measurements are weighted according to the ratio of each object in the pixel and are called a mixel (mixed pixel). For example, when measuring vegetation, measuring the mixture of plants and soil hinders the determination of the physical property values ​​of the plants themselves (chlorophyll concentration, etc.).

[0130] Various methods have been proposed to solve this problem.

[0131] "SAVI (Soil Adjusted Vegetation Index)" is a vegetation index used to correct for fluctuations caused by soil reflectance. When LAI is represented as "L", L=0 (equal to NDVI) is used for high LAI, and L=1 is used for low LAI.

[0132] SAVI=〔(NIR-RED) / (NIR+RED+L)〕×(1+L)

[0133] However, it may be necessary to use assumed values ​​as LAI values, which cannot be accurately corrected in situations where LAI may vary with location (such as in farmland).

[0134] Furthermore, adverse effects on remote sensing are not limited to the two-dimensional ratio of objects. Reflections from plant leaves are not Lambertian reflections and are affected by the incident angle of the light source, the leaf angle, and the angle of the measuring instrument (this effect is often referred to as BRDF).

[0135] The three-dimensional shape of plants causes shadows, multiple reflections of light beams in the community, and other factors, which in turn cause fluctuations in the measured values.

[0136] To handle such complex mechanisms, optical behavior is simulated using a leaf radiative transfer model that describes the angle dependence of reflectivity (example: PROSAIL model) or an optical model of a community that includes shade.

[0137] In other words, in order to cover the low spatial resolution of satellite remote sensing and to deal with the complex factors affecting the measurement, inverse calculations using a "model (such as a radiative transfer characteristic model)" that includes information about the form of the object being measured are used to determine the measurement values ​​for a specific object.

[0138] However, this is appropriate when there is no difference between the extent of the object being measured and the shape of the model (such as in the case of vegetation in a tropical rainforest), but for example, in the measurement (survey) of farmland 300, the shape itself must be measured and cannot be identified or accurately measured. For example, characteristics such as these are unknown because, for example, the shape of the crop changes during its growth, or for some reason, the crop cannot grow properly or is about to die.

[0139] For example, Figure 5 A shows plant individuals with essentially the same shape arranged extensively in a line, and, for example, the LAI is substantially consistent within the macroscopic measurement range RZ2.

[0140] on the other hand, Figure 5B illustrates region H1, where plants with large leaves (growing very large) are planted, and region H2, where plants with small leaves are planted. The macroscopic measurement range RZ2 shown in the diagram spans regions H1 and H2. In this case, LAI, or plant height, varies with location. Therefore, there may be a significant difference between the macroscopic measurement range RZ2 and the model used for inverse model calculations.

[0141] Therefore, in the sensing system according to this embodiment, the high-resolution measurements performed by the micro-measurement component 3 include measurements of the plant status (LAI, plant cover and height, average leaf angle, chlorophyll concentration, etc.) and ridge installation status in the actual field, and inverse model calculations of satellite sensing are performed using parameters of the actual object. This allows measurements to be performed even when the object being measured has a changing shape or state or a shape different from the standard shape indicated by the model.

[0142] In a specific example, inverse model calculations are performed using parameters of the real object, based on detection data from the microscopic measurement component 3. Therefore, even when the shape and state of the measured object are unknown, the correct SIF (solar-induced chlorophyll fluorescence) is calculated to obtain information about the rate of photosynthesis.

[0143] The model parameters are assumed to be the three-dimensional structure of the plant, plant height, average leaf angle, plant cover, LAI, chlorophyll concentration, soil spectral characteristics, and sun leaf ratio, etc.

[0144] Furthermore, the sensing system according to this embodiment also includes the concept of clustering. In other words, the sensing system includes the clustering concept of dividing the measurement object area into clusters, measuring representative individuals of each cluster, and switching models sequentially from cluster to cluster.

[0145] Figure 6 The illustration schematically shows the scenario of performing sensing in each of the specific regions H10 and H20.

[0146] Suppose that individuals OBa1, OBa2, OBa3…OBan exist in region H10, and individuals OBb1, OBb2, OBb3…OBbn with characteristics different from those of individuals OBa1, OBa2, OBa3…OBan exist in region H20.

[0147] Clustering is used to identify regions H10 and H20 as different clusters.

[0148] In addition, for each cluster, representative individuals are measured to determine model parameters.

[0149] Different model parameters are used for their respective clustering.

[0150] That is, when measuring region H10 using macroscopic measurement component 2 and microscopic measurement component 3, inverse model calculation is performed using model parameters of representative individuals based on correlation clustering.

[0151] When measuring region H20 using macroscopic measurement component 2 and microscopic measurement component 3, inverse model calculation is performed using model parameters of representative individuals based on correlation clustering.

[0152] In this way, different models are used for each region generated from the clustering to obtain accurate measurement results.

[0153] In addition, Figure 4 A and Figure 4 In diagram B, the individual object being measured is shown with dashed lines, but in this figure, the microscopic measurement component 3 covers both regions H1 and H2. Assume that regions H1 and H2 include, for example, Figure 5 Groups shown in B have different leaf sizes (growth patterns), representing different plants, etc.

[0154] In the same case, the following configuration is possible: perform inverse model calculations such that, for the macroscopic measurement range RZ2, the model based on the microobservations of region H1 is used for the part related to region H1, and the model based on the microobservations of region H2 is used for the part related to region H2. Figure 4 C schematically illustrates the switching of the model used for inverse model computation.

[0155] <2. Configuration of Information Processing Devices>

[0156] The information processing device 1 in the above-described sensing system acquires detection information from the macroscopic measurement component 2 and the microscopic measurement component 3 and performs processing such as analysis.

[0157] Figure 7 The hardware configuration of the information processing device 1 is shown. The information processing device 1 includes a CPU (Central Processing Unit) 51, a ROM (Read-Only Memory) 52, and a RAM (Random Access Memory) 53.

[0158] The CPU 51 performs various types of processing according to the program stored in the ROM 52 or the program loaded into the RAM 53 from the storage unit 59. The RAM 53 also appropriately stores data, etc., for the CPU 51 to perform various types of processing.

[0159] CPU 51, ROM 52 and RAM 53 are connected together via bus 54. Input / output interface 55 is also connected to bus 54.

[0160] The input / output interface 55 can be connected to display components 56, including LCD panels, organic EL (electroluminescent) panels, etc.; input components 57, including keyboards, mice, etc.; speakers 58; storage components 59; communication components 60, etc.

[0161] The display component 56 can be integrated with the information processing device 1, or it can be a separate device from the information processing device 1.

[0162] Display unit 56 displays various analysis results on the screen based on instructions from CPU 51. Additionally, based on instructions from CPU 51, display unit 56 displays various operation menus, icons, messages, etc., i.e., it provides display as a GUI (Graphical User Interface) interface.

[0163] Input component 57 represents the input device used by the user of information processing device 1.

[0164] For example, input component 57 is assumed to be any of a variety of operators and operating devices such as a keyboard, mouse, keys, dial, touch panel, touchpad, or remote control.

[0165] Input unit 57 detects user operations, and CPU 51 interprets the signals corresponding to the input operations.

[0166] Storage component 59 includes a storage medium, such as an HDD (hard disk drive), solid-state memory, etc. Storage component 59 stores, for example, detection data and analysis results received from macroscopic measurement component 2 and microscopic measurement component 3, as well as various other types of information. Additionally, storage component 59 is used to store program data for analysis and processing, etc.

[0167] The communication component 60 performs communication processing via a network including the Internet and communicates with equipment in the peripheral part.

[0168] For example, the communication component 60 may be a communication device that communicates with the micro-measurement component 3 (imaging device 250) and the macro-measurement component 2 (imaging device 220).

[0169] The driver 61 is also connected to the input / output interface 55 as needed, and a storage device 6, such as a memory card, is installed in the input / output interface 55 as needed to allow data to be written to and read from the storage device 6.

[0170] For example, computer programs read from storage device 6 are installed in storage component 59, and data processed by CPU 51 as needed is stored in storage component. Needless to say, drive 61 can be a recording and playback drive for removable storage media such as disks, optical discs, or magneto-optical discs. Disks, optical discs, magneto-optical discs, etc., are also types of storage device 6.

[0171] Note that the information processing device 1 according to this embodiment is not limited to having, for example, Figure 7 The hardware configuration shown is a single information processing device (computer device) 1, but multiple computer devices can be configured into a system. The multiple computer devices can be configured as a system using a LAN, or deployed in a remote location using a VPN (Virtual Private Network) or the Internet. These multiple computer devices may include computer devices available through cloud computing services.

[0172] in addition, Figure 7 The information processing device 1 can be implemented as a personal computer such as a desktop or laptop computer, or a portable terminal such as a tablet or smartphone. Furthermore, electronic equipment such as measuring devices, television devices, monitor devices, imaging devices, or facility management devices used as information processing device 1 can be equipped with the information processing device 1 according to this embodiment.

[0173] For example, an information processing device 1 with this hardware configuration includes the computing functions of a CPU 51, the storage functions of a ROM 52, RAM 53 and storage unit 59, the data acquisition functions of a communication unit 60 and a driver 61, and the output functions of a display unit 56. Installed software functions provide, for example... Figure 8 The functional configuration shown.

[0174] That is, as a main component, the information processing device 1 is equipped with Figure 8 The data input unit 10, the analysis execution unit 20, and the data storage and output unit 30 are shown.

[0175] These processing functions are implemented by software started by the CPU 51.

[0176] The program included in the software is downloaded from the network or read from storage device 6 (e.g., removable storage media) and installed. Figure 7 The program is stored in the information processing device 1. Alternatively, the program can be pre-stored in the storage unit 59, etc. Then, the CPU 51 starts the program to activate the function of each component as described above.

[0177] In addition, storage functions such as various buffers are implemented using storage areas, such as those in RAM 53 or storage components 59.

[0178] in addition, Figure 8 The diagram shows a macroscopic measurement component 2, a microscopic measurement component 3, a network 5, a storage device 6, and an operation input component 7, which are external devices of the information processing device 1.

[0179] As described above, the macroscopic measurement component 2 is, for example, installed in the satellite 210. The macroscopic measurement sensor 2S is a large sensor, such as a hyperspectral camera or FTIR, and can be installed in the satellite 210, but is difficult to install in the flying body 200. These sensors typically operate outside the visible spectrum and are primarily used to measure physical properties.

[0180] The micro-measuring component 3 is installed in the aircraft body 200. The micro-measuring sensor 3S is a small sensor, such as an RGB camera or a stereo camera, and is easy to install in the aircraft body 200. Generally, the micro-measuring sensor 3S operates in the visible spectrum and is mainly used to measure the characteristics of the measured object and its environmental response.

[0181] Network 5 is assumed to be, for example, the Internet, a home network, a LAN (local area network), a satellite communication network, or any other network.

[0182] As described above, storage device 6 is primarily assumed to be a removable storage medium, such as a memory card or a disk-shaped recording medium.

[0183] Operation input component 7 is a device through which the user can provide manual input, and can be considered as Figure 7 Input component 57.

[0184] The operation input component 7 can be integrated with the information processing device 1, or it can be a separate device from the information processing device 1.

[0185] The data input unit 10 in the information processing device 1 corresponds to the function of receiving data input from the aforementioned external device, and includes sensor input units 11 and 12 and program and model input unit 13.

[0186] The sensor input unit 11 receives detection information from the macroscopic measurement sensor 2S of the macroscopic measurement unit 2. For example, the macroscopic measurement unit 2 and Figure 7 The communication between the communication components 60 can directly receive detection data from the macroscopic measurement sensor 2S.

[0187] Alternatively, detection data from the macroscopic measurement sensor 2S can be received via the network 5 through the communication component 60.

[0188] In addition, detection data from the macroscopic measurement sensor 2S can be obtained via the storage device 6.

[0189] The sensor input unit 12 inputs detection information from the micro-measurement sensor 3S of the micro-measurement unit 3. For example, detection data from the micro-measurement sensor 3S can be received directly through communication between the micro-measurement unit 3 and the communication unit 60, or the detection data from the micro-measurement sensor 3S can be received via the network 5 or obtained via the storage device 6 through the communication unit 60.

[0190] Note that processing such as spectral correction of the light source can be performed in sensor input components 11 and 12.

[0191] The program and model input component 13 obtains a suitable program or model by downloading the program or model from the server via network 5 or reading the program or model from storage device 6. The model refers to the model parameters used for inverse calculation.

[0192] The analysis execution unit 20 includes a macroscopic measurement analysis calculation unit 21, a macroscopic measurement analysis value buffer 22, a microscopic measurement analysis calculation unit 23, a microscopic measurement analysis value buffer 24, a position mapping unit 25, an inverse model calculation program and a model holding unit 26 (hereinafter referred to as "holding unit 26"), an inverse model calculation unit 27, and a clustering calculation unit 28.

[0193] The macroscopic measurement analysis and calculation unit 21 performs calculations to determine the amount of substance components, etc., based on the detection data from the macroscopic measurement sensor 2S acquired by the sensor input unit 11.

[0194] For example, the macroscopic measurement and analysis calculation component 21 calculates vegetation index and SIF (chlorophyll fluorescence) based on multi-wavelength data from hyperspectral cameras or FTIR, using methods such as NIRS (near-infrared spectroscopy) and FLD (fraunhofer line discrimination) methods (solar dark line: Fraunhofer line).

[0195] Note that the wavelength of solar dark rays O2A is approximately 1 nm and relatively thin, therefore, this sensing is typically suitable using sensors such as hyperspectral cameras or FTIR. In the sensing system of this example, since such equipment is easily installed in satellite 210, the macroscopic measurement analysis calculation unit 21 performs calculations based on the detection data from the macroscopic measurement unit 2.

[0196] The macroscopic measurement analysis value buffer 22 temporarily stores the data processed by the macroscopic measurement analysis calculation unit 21.

[0197] For example, the macroscopic measurement analysis value buffer 22 stores the SIF calculated by the macroscopic measurement analysis calculation unit 21, the location information notified from the macroscopic measurement unit 2, etc.

[0198] The micro-measurement analysis and calculation unit 23 performs calculations to obtain appropriate information from the detection data from the micro-measurement sensor 3S acquired by the sensor input unit 12.

[0199] For example, the microscopic measurement and analysis calculation component 23 calculates LAI, average leaf angle, sun leaf ratio, etc. A sun leaf refers to a leaf or part of a leaf exposed to sunlight.

[0200] The microscopic measurement analysis value buffer 24 temporarily stores the data processed by the microscopic measurement analysis calculation unit 23.

[0201] For example, the micro-measurement analysis value buffer 24 stores information about LAI, average leaf angle, sun leaf ratio, etc., determined by the micro-measurement analysis calculation unit 23, as well as position information notified from the micro-measurement unit 3, and further stores RGB images, NDVI images, etc.

[0202] The position mapping unit 25 performs calculations to extract identical points from a set of images with different resolutions or imaging units (measurement ranges RZ2 and RZ3). For example, GPS information or orthogonal mosaic processing is used to align the information processed by the macroscopic measurement analysis calculation unit 21 with the information processed by the microscopic measurement analysis calculation unit 23.

[0203] The inverse model calculation component 27 is a function that performs inverse model calculations using the calculation results from the macroscopic measurement analysis calculation component 21 and the microscopic measurement analysis calculation component 23.

[0204] That is, the inverse model calculation unit 27 generates model parameters based on the detection data from the micro-measurement unit 3 determined by the micro-measurement analysis calculation unit 23.

[0205] Then, the inverse model calculation unit 27 uses the model parameters as parameters of the inverse model to perform inverse model calculations using the calculation results from the macroscopic measurement analysis calculation unit 21.

[0206] Note that after generating model parameters based on the detection data from the micro-measurement component 3, the inverse model calculation component 27 can associate the model parameters with information about the date, time, and location, and store the results in a predetermined storage component. The date, time, and location refer to the date, time, and location (e.g., GPS information) of the original micro-observations used to determine the model parameters.

[0207] The holding component 26 retains the default parameters of the inverse model calculation program and model obtained by the program and model input component 13. The inverse model calculation component 27 performs inverse model calculations based on these programs and models.

[0208] The clustering calculation unit 28 performs clustering calculations. For example, based on user input via the operation input unit 7, the clustering calculation unit 28 performs clustering corresponding to the segmentation of the area to be measured, such as farmland 300. For example, the user specifies field boundaries, and different crops or the same crop at different developmental stages are planted across the boundaries. This allows the user to perform optional clustering segmentation.

[0209] In addition, the clustering calculation component 28 can obtain information about instructions for clustering via the network 5 or via the storage device 6.

[0210] Furthermore, the clustering calculation unit 28 can perform automatic clustering based on information obtained from the microscopic measurement analysis calculation unit 23 or from the macroscopic measurement analysis calculation unit 21.

[0211] The information obtained from the microscopic measurement and analysis calculation unit 23 and used for clustering can be detection data from the microscopic measurement unit 3, or LAI, average leaf angle, sun leaf ratio, etc. calculated by the microscopic measurement and analysis calculation unit 23 based on the detection data.

[0212] The information obtained from the macroscopic measurement analysis and calculation unit 21 and used for clustering can be detection data (multispectral information, etc.) from the macroscopic measurement unit 2, or SIF, etc., calculated by the macroscopic measurement analysis and calculation unit 21 based on the detection data.

[0213] Clustering and segmentation can be performed based on the differences between the different types of information mentioned above. Furthermore, it is assumed that image texture analysis, machine learning, etc., will be performed.

[0214] Furthermore, any difference in physiological characteristics or environmental responses, as well as in form or features, can be used for clustering.

[0215] In addition, in the combination of manual input and automatic clustering determination, if a user-specified region includes multiple clusters automatically identified by the macroscopic measurement component, automatic sub-segmentation clustering can be set.

[0216] Clustering information from clustering calculation unit 28 is provided to inverse model calculation unit 27 and referenced during inverse model calculation.

[0217] The data storage and output component 30 includes an analysis data buffer 31, a color mapping component 32, an image compositing component 33, a graphics generation component 34, an image output component 35, and a data output component 36.

[0218] Information about the calculation results from the inverse model calculation unit 27 is temporarily stored in the analysis data buffer 31.

[0219] When the inverse model calculation unit 27 determines, for example, the SIF, the analysis data buffer 31 retains this information. Alternatively, the analysis data buffer 31 can retain either an RGB image or an NDVI image.

[0220] In order to visualize and display the physical values ​​obtained as the result of calculation from the inverse model calculation unit 27, the color mapping unit 32 performs calculation processing, for example, using each level of RGB primary color to convert a specific range of physical values ​​into color gradations from blue to red.

[0221] The image compositing component 33 performs calculations to paste color-mapped physical value data into the original spatial region or to overlay color-mapped physical value data onto an RGB image.

[0222] To visualize and display data, the graphics generation component 34 performs computational processing to create graphics, such as representing physical values ​​with dashed lines or converting two-dimensional physical values ​​into scatter plots.

[0223] The image output unit 35 outputs the image data generated by the processing of the color mapping unit 32, the image compositing unit 33, and the graphics generation unit 34 to the external display unit 56 for display. Alternatively, the image output unit 35 performs output to send the generated image data to an external device using the network 5, or archives the image data and saves the result file in the storage device 6.

[0224] The data output unit 36 ​​outputs information about the calculation results from the inverse model calculation unit 27 stored in the analysis data buffer 31. For example, the data output unit 36 ​​performs output to send the information about the inverse model calculation results to an external device using the network 5, or to archive the information about the inverse model calculation results and save the result file in the storage device 6.

[0225] <3. Processing Example>

[0226] A processing example of the information processing device 1, which includes the above-described functions, will be described.

[0227] Figure 9 An example of processing by the information processing device 1 is shown.

[0228] In step S101, the information processing device 1 uses the function of the sensor input component 11 to input the measurement value from the macroscopic measurement component 2.

[0229] In step S102, the information processing device 1 performs macroscopic measurement analysis calculations using the functions of the macroscopic measurement analysis calculation unit 21. For example, for information about photosynthesis, SIF calculations are performed. For SIF calculations, the FLD method based on dark lines in the solar spectrum is known.

[0230] In step S103, the information processing device 1 uses the function of the sensor input component 12 to input the measurement value from the micro-measurement component 3.

[0231] In step S104, the information processing device 1 performs micro-measurement analysis calculations using the function of the micro-measurement analysis calculation unit 23.

[0232] Figure 10 An example of the processing in the microscopic measurement analysis calculation in step S104 is shown.

[0233] Note that it is assumed that the microscopic measurement and analysis calculation component 23 has already acquired... Figure 11 The image shown includes RGB, NIR, R images, and polarization sensor angle information.

[0234] exist Figure 10 In step S201, the microscopic measurement and analysis calculation unit 23 performs image structure analysis calculation. Specifically, the image is segmented. In a simple case, such as Figure 11 As shown, an image can be divided into cells.

[0235] However, images can be segmented by using image recognition or similar methods to identify objects. For example, individuals can be identified as measurement objects, and the image can be segmented based on the identified individuals.

[0236] In step S202, the micro-measurement analysis calculation component 23 calculates the LAI, average leaf angle, and solar leaf ratio based on the segmentation unit.

[0237] LAI can be determined based on vegetation cover. Vegetation cover can be determined by dividing the number of pixels corresponding to a specific or larger NDVI by the number of measurement points (pixels) in the relevant segmentation unit.

[0238] Note that NDVI can be determined based on both the R and NIR images. That is, the value of NDVI is determined by the following formula:

[0239] NDVI = (NIR - R) / (NIR + R)

[0240] Here, "R" represents the reflectance in the visible red region, and "NIR" represents the reflectance in the near-infrared region. NDVI has values ​​normalized to the range of "-1" to "1". The larger the positive value of NDVI, the denser the vegetation.

[0241] The leaf angle can be determined by obtaining angle information from a polarization sensor.

[0242] The sun-leaf ratio is the ratio of leaves exposed to sunlight. Sun-leaf ratios can be extracted as pixels with a specific or greater NDVI and a specific or greater NIR value. Therefore, the sun-leaf ratio can be determined by dividing the number of such extracted pixels by the number of measurement points (pixels) in the relevant segmentation unit.

[0243] In step S203, the microscopic measurement and analysis calculation unit 23 determines whether the calculation of LAI, average leaf angle, and solar leaf ratio has been completed for all segmented units. If the calculation is not completed, then in step S202, the LAI, average leaf angle, and solar leaf ratio are calculated for the next segmented unit.

[0244] Once all segmented units have been processed, the microscopic measurement and analysis calculations are complete.

[0245] exist Figure 10 In the above processing, Figure 9 The execution of step S104 causes the information processing device 1 to perform clustering in step S105 using the function of the clustering calculation unit 28.

[0246] Figure 12 An example of clustering calculation is shown.

[0247] In step S301, the clustering calculation unit 28 specifies the segmentation of the measurement object that is manually input by the user.

[0248] Figure 13 A schematic diagram illustrates farmland 300 corresponding to the measured object. Note that the definitions of areas AR1 to AR6 are for ease of description and do not necessarily correspond to areas with different types of vegetation. However, it is assumed that the shaded area AR3 is an area where crops different from those in other areas are grown.

[0249] In this case, the user specifies the boundary, indicated by a thick line, by manually entering the boundary information. For example, suppose the entire area of ​​a field where crops are grown (field boundary) is indicated, and the boundary of the area AR3 that distinguishes between different types of crops is specified.

[0250] The clustering calculation unit 28 performs clustering segmentation indicated by thick lines, reflecting input based on such information initially known to the user.

[0251] In step S302, the clustering calculation unit 28 performs automatic clustering using information obtained from the macroscopic measurement analysis calculation unit 21 and information obtained from the microscopic measurement analysis calculation unit 23. Clustering is performed using, for example, SIF quantity, LAI, average leaf angle, and sun-leaf ratio.

[0252] Figure 13B illustrates measurement ranges a, b, c, and d as microscopic measurement ranges RZ3 associated with multiple measurements. In this case, measurement range a corresponds to the microscopic measurement range RZ3 for measurement region AR3, measurement range b corresponds to the microscopic measurement range RZ3 for measurement region AR4, measurement range c corresponds to the microscopic measurement range RZ3 for measurement region AR5, and measurement range d corresponds to the microscopic measurement range RZ3 for measurement region AR6.

[0253] Suppose that automatic clustering involves performing processes such as dividing regions into clusters with different LAIs. Suppose that the LAI values ​​vary between measurement ranges a, b, and c, but are substantially the same between measurement ranges c and d.

[0254] The crops and LAIs in region AR3 and region AR4 are different. Assuming that regions AR4, AR5, and AR6 have the same crops, but only region AR4 involves different growth conditions, then setting region AR4 as a separate cluster is appropriate.

[0255] In step S303, the clustering calculation unit 28 performs sub-segmentation on the clustering settings based on user input, so that the sub-segmentation reflects the automatic clustering determination in step S304.

[0256] In the example above, such as Figure 13 As shown by the thick line in C, region AR4 has been redefined as a cluster.

[0257] For example, regions AR3 and AR4 are assumed to be clusters CL3 and CL4, respectively. Note that regions AR1 and AR2 are not mentioned, but if regions AR1 and AR2 have the same LAI as regions AR5 and AR6, regions AR1, AR2, AR5, and AR6 are collectively designated as a single cluster CL1.

[0258] For example, as described above, clustering computation is performed by performing clustering based on user input and using values ​​related to macroscopic or microscopic measurements to perform automatic clustering determination for subsegmentation.

[0259] Needless to say, clustering can be set exclusively based on user input or by performing automatic clustering determination for subsegmentation using values ​​related to macroscopic or microscopic measurements.

[0260] After performing the above clustering calculations, Figure 9 In step S106, the information processing device 1 performs position mapping using the function of the position mapping component 25.

[0261] That is, the information processing device 1 aligns the macroscopic measurement and analysis calculation results with the microscopic measurement and analysis calculation results.

[0262] In step S107, the information processing device 1 performs inverse model calculation using the function of the inverse model calculation unit 27. Figure 14 An example of the inverse model computation process is shown.

[0263] Notice, Figure 15 A schematic diagram illustrates the area of ​​SIF calculation based on macroscopic measurements. SIF is determined in cells, with each cell illustrated as the macroscopic measurement resolution (macroscopic resolution units W1 to Wn).

[0264] In step S401, the inverse model calculation unit 27 reads the SIF calculated by the macroscopic measurement analysis calculation unit 21 for a macroscopic resolution unit. For example, the inverse model calculation unit 27 first reads the SIF for a macroscopic resolution unit W1.

[0265] In step S402, the inverse model calculation unit 27 obtains the parameters determined by the microscopic measurement analysis calculation unit 23, namely LAI, average leaf angle, and solar leaf ratio, for clustering corresponding to macroscopic resolution units.

[0266] Figure 15 B shows the LAI, average leaf angle, and sun-leaf ratio for the above measurement ranges a, b, and c (=d). In other words, as Figure 13 As shown in C, LAI, average leaf angle, and sun leaf ratio are the model parameters for cluster CL3 in region AR3, cluster CL4 in region AR4, and cluster CL1 in region AR1+AR2+AR5+A6.

[0267] For example, the macroscopic resolution unit W1 corresponds to cluster CL1, thus obtaining Figure 15 Model parameters for cluster CL1 in B.

[0268] In step S403, the inverse model calculation unit 27 performs inverse model calculation. That is, it determines the desired physical property value (e.g., characteristics of the measured object) based on the SIF obtained from macroscopic measurements.

[0269] In this case, the parameters based on the micro-observations obtained in step S402 are used as model parameters.

[0270] Therefore, even if the shape or state of the object being measured is unknown solely through sensing performed by the macroscopic measurement component 2, the correct SIF can be determined based on the actual shape and state of the object being measured.

[0271] The above process is repeated by returning from step S404 to step S401 until processing is performed on all macroscopic resolution units W1 to Wn. In other words, from macroscopic resolution units W1 to Wn, inverse model calculations are performed using model parameters based on the microobservations of the corresponding clusters.

[0272] Note that an example of SIF has been described, but for example, in the case where NDVI is obtained based on macroscopic measurements, the required physical property value (e.g., chlorophyll concentration as an environmental response) can be determined by inverse calculation based on NDVI based on macroscopic measurements.

[0273] Once processing has been completed for all macroscopic resolution units, the inverse model calculation unit 27 proceeds to step S405 to write the calculation results into the analysis data buffer 31. In this case, the calculation results are written for each macroscopic resolution unit W1 to Wn.

[0274] Note that in the above processing, the model parameters corresponding to the clusters for each macroscopic resolution unit are applied to the macroscopic resolution unit, but clustering may not be specifically performed. For example, if the same plant is grown under similar growing conditions throughout the entire farmland 300, the clustering in step S105 may not result in cluster segmentation. Additionally, consider a processing example where the processing in step S105 is not performed.

[0275] In these cases, it is sufficient to perform inverse model calculations by applying the model parameters obtained by measuring a single microscopic measurement range RZ3 to all macroscopic resolution units.

[0276] In the above process, during execution Figure 9 After step S107, the information processing device 1 uses the function of the data storage and output unit 30 to perform color mapping in step S108, image synthesis in step S109, and image output in step S110.

[0277] Therefore, users can use display components 56, etc., to check the calculation results.

[0278] An example of an output image that undergoes color mapping, etc., in this case will be described.

[0279] Figure 16 This is an example of generating an image by performing color assignment (color mapping) on ​​the inverse model calculation results obtained as described above for each macroscopic resolution unit. "Color assignment" as used here refers to pre-setting colors corresponding to each numerical range, selecting colors based on the object's values, and assigning those colors to the relevant pixels.

[0280] Figure 16 A shows the inverse model calculation results for each macroscopic resolution unit. For example... Figure 16As shown in B, colors are assigned to these values ​​to generate a color-mapped image. In the image, the color corresponding to the calculated value is assigned to each region.

[0281] Note that the attached diagram uses shading lines, dots, and other types to represent color differences. Furthermore, for macroscopic resolution units where no valid calculation results exist (e.g., areas without sun leaves), "No Data" is indicated. For example, a background color (white) is assigned to areas indicating "No Data".

[0282] When presenting such a color-mapped image to a user, the inverse model calculation results for each region in each macroscopic resolution unit are represented by color, and the image allows the user to easily identify macroscopic measurement results.

[0283] Next, Figure 17 This is an example of a synthesized image with colors assigned to regions where vegetation is in a specific state. Figure 17 A shows the value of the inverse model calculation result for each macroscopic resolution unit. Figure 17 B is an image depicting the extracted sun leaf NDVIp-pr (NDVI plant filter Par filter). The image depicting the extracted sun leaf NDVIp-pr refers to the image depicting the extraction range, which has a specific or greater NDVI value and a specific or greater NIR value.

[0284] Then, colors are assigned to the solar leaf portion within each macroscopic resolution unit to generate, as shown below. Figure 17 The color-mapped image shown in C. Only the sun leaf portion has colors corresponding to the inverse model calculation results. Therefore, this image allows users to easily identify macroscopic measurements and the distribution of sun leaves in each region.

[0285] Next, Figure 18 This is an example of an overlay display on a visible light image (RGB image).

[0286] Figure 18 A shows the value of the inverse model calculation result for each macroscopic resolution unit. Figure 18 B is an RGB image.

[0287] like Figure 18 As shown in Figure C, in an RGB image, colors are superimposed on values ​​assigned to each macroscopic resolution unit based on the results calculated from the inverse model. This figure illustrates the application of color to the relevant pixel portion.

[0288] In other words, in the image, the color representing the calculation result is displayed on the RGB image. Therefore, macroscopic measurement results are easily identifiable on images that users typically view.

[0289] Note that you can perform an overwrite using the color assigned to the relevant pixel instead of an overlay.

[0290] For example, generated as described above and in Figure 16 , Figure 17 and Figure 18 The output image is shown on display unit 56 and transmitted to an external device via network 5, or archived together with the result file stored in storage device 6. The user can then utilize the analysis results.

[0291] <4. Various Examples>

[0292] The above description assumes vegetation sensing. However, the technology according to this disclosure is applicable to various fields.

[0293] For example, in buildings such as office buildings where a central heat source is used, the energy usage of the entire building can be identified through macroscopic measurements.

[0294] In addition, specific measurements can be taken of a part of a building (e.g., a commercial office occupying a floor), as micro-measurements.

[0295] When performing inverse model calculations using information based on macroscopic measurements, model parameters can be set based on microscopic measurements.

[0296] Furthermore, for example, in fields such as labor statistics, the change in the unemployment rate over a certain period can be measured as a macro-level measurement, while the characteristics of unemployed persons in each season can be measured as a micro-level measurement.

[0297] When performing inverse model calculations using information based on macroscopic measurements, model parameters can be set based on microscopic measurements.

[0298] <5. Conclusion and Modification Examples>

[0299] The above embodiments produce the following effects.

[0300] The information processing apparatus 1 according to this embodiment includes a macroscopic measurement analysis and calculation unit 21, which calculates detection data from a macroscopic measurement unit 2. The macroscopic measurement unit 2 performs sensing of a macroscopic measurement range RZ2 (first measurement range) for the measured object at a macroscopic measurement resolution (first spatial resolution). The information processing apparatus 1 also includes a microscopic measurement analysis and calculation unit 23, which calculates detection data from a microscopic measurement unit 3. The microscopic measurement unit 3 performs sensing of a microscopic measurement range RZ3 (second measurement range) at a microscopic measurement resolution (second spatial resolution), where the microscopic measurement resolution is higher than the macroscopic measurement resolution, and the microscopic measurement range RZ3 is included in the macroscopic measurement range RZ2. Furthermore, the information processing apparatus 1 includes an inverse model calculation unit 27, which obtains model parameters for inverse model calculation using the calculation results from the macroscopic measurement analysis and calculation unit 21 based on the detection data from the microscopic measurement unit 3 determined by the microscopic measurement analysis and calculation unit 23.

[0301] By using the parameters of the microscopic measurement calculation model as described above, the parameters of the inverse model applicable to calculations using the calculation results from the macroscopic measurement analysis calculation component can be obtained.

[0302] In addition, in this embodiment, the inverse model calculation unit 27 uses model parameters based on the detection data from the micro-measurement unit 3 determined by the micro-measurement analysis calculation unit 23 as parameters for the inverse model in the inverse model calculation using the calculation results from the macro-measurement analysis calculation unit 21.

[0303] When performing inverse model calculations, and in cases of shape or state changes, the shape or state of the measured object may differ from the standard shape or state indicated by the model, leading to reduced measurement accuracy. In this embodiment, inverse model parameters are generated based on actual measurements using microscopic measurements. Therefore, accurate measurements can be achieved even when the shape or state changes or the measured object does not have the standard shape indicated by the model.

[0304] In particular, in this case, by using detection data from the microscopic measurement component 3 (which can perform sensing with high spatial resolution), the inverse model parameters can be made more suitable. This enables measurements to be performed based on inverse model calculations corresponding to the characteristics of the measured object and the environmental response, which cannot be achieved solely by the macroscopic measurement component 2.

[0305] More specifically, in high-resolution measurements using the flying object 200, the plant status (plant cover and height, average leaf angle, chlorophyll concentration, etc.) or the set state of the ridges in the actual field is measured, and the parameters of the real object are used for inverse model calculations based on the sensing data from the satellite 210. Then, accurate measurements can be achieved even when the shape or state changes or the measured object does not have the standard shape indicated by the model.

[0306] Furthermore, for example, the use of such a measurement system allows the spacecraft 200 to obtain photosynthetic information without measuring SIF. Advantageously, the satellite 210 can also acquire high-resolution information, which was not anticipated until now.

[0307] In this embodiment, the inverse model calculation unit 27 uses model parameters based on detection data of the microscopic measurement range RZ3 (second measurement range) to determine the calculation result of the unit of macroscopic measurement resolution (first spatial resolution) (see [link]). Figure 14 and Figure 15 ).

[0308] Therefore, by using inverse model calculations, measurements reflecting microscopic observations can be determined in units of macroscopic measurement resolution within the macroscopic measurement range RZ2.

[0309] In particular, images from satellite 210 are included in the output range, thus obtaining the inverse model calculation results as images covering a range larger than that covered by the flying body 200.

[0310] In this embodiment, the inverse model calculation unit 27 determines the characteristics of the measured object or the environmental response as the calculation result in units of macroscopic measurement resolution.

[0311] This enables the sensing of determining the static shape or characteristics of a measured object or the dynamic shape or characteristics of a measured object.

[0312] For example, remote sensing suitable for agriculture can be achieved by obtaining information such as plant shape, vegetation index, and photosynthesis.

[0313] For example, as information about photosynthesis, even when the shape or state of the object being measured is unknown, the correct SIF (solar-induced chlorophyll fluorescence) and various types of information calculated from the correct SIF can be obtained.

[0314] In this embodiment, the macroscopic measurement component 2 performs sensing at a greater distance from the measurement object 4 (e.g., farmland 300) than the microscopic measurement component 3.

[0315] When the macroscopic measuring component 2 is located relatively far from the object being measured 4, a relatively large device or equipment can easily be implemented as the macroscopic measuring component 2 or a device equipped with the macroscopic measuring component 2.

[0316] Note that the micro-measuring component 3 is installed in the flying body 200, while the macro-measuring component 2 is installed in the artificial satellite 210. However, the macro-measuring component 2 can also be installed in the flying body 200, such as a drone. For example, the macro-measuring component 2 can be installed in the flying body 200 flying at a higher position in the sky to sense the macro-measuring range RZ2.

[0317] In the example mentioned in the embodiment, the inverse model calculation unit 27 obtains the model parameters of representative individuals of each measurement region generated from the clustering, switches the model parameters of each measurement region generated from the clustering, and applies the obtained model parameters to the inverse model (see...). Figure 14 and Figure 15 ).

[0318] By calculating the detection data from the micro-measuring component to determine the model parameters for each measurement region generated from the cluster, the inverse model calculation component 27 can use different model parameters for each measurement region generated from the cluster.

[0319] Therefore, even within the macroscopic measurement range RZ2, appropriate and different measurement results can be obtained for each cluster region (e.g., for each region where different crops are grown).

[0320] Please note that by specifying a crop name for each cluster, not only can model parameters (such as height) be switched automatically, but the model itself (a model that reflects shape differences between varieties (such as tomatoes and corn)) can also be switched automatically.

[0321] In the example mentioned in the embodiments, the clustering is based on user input for a specified region.

[0322] For example, in farmland 300, different crops are planted in different areas. For example, users such as farmers can input such information.

[0323] This allows for clear access to information about regions where different crops are grown, regions where crops are planted at different times, and so on. Therefore, it is possible to appropriately determine crop characteristics or environmental responses for each region.

[0324] Furthermore, clustering that reflects user input allows for measurement results to be obtained for each region as desired by the user.

[0325] Notice, Figure 12 Clustering based on user input, macroscopic measurements, and microscopic measurements is illustrated; however, for example, clustering computation based solely on user input can be performed, where only... Figure 12 Step S301 in the process.

[0326] In addition, in the example mentioned in this embodiment, clustering is performed based on detection data from macroscopic measurement component 2 or detection data from microscopic measurement component 3.

[0327] For example, by using detection data (or information calculated from detection data) from macroscopic measurement component 2 or microscopic measurement component 3, regions with distinctly different shapes or states can be distinguished from each other, thereby allowing for automatic clustering calculations.

[0328] This allows for automatic clustering of areas where different crops are grown, areas where crops are planted at different times, and so on. Therefore, the characteristics of crops or environmental responses in each area can be accurately determined without significant effort from the user.

[0329] Figure 12 Clustering based on user input, macroscopic measurements, and microscopic measurements is illustrated. However, for example, clustering computation based solely on macroscopic and microscopic measurements can be performed, where only... Figure 12 Step S302 in the process. Furthermore, clustering calculations based solely on macroscopic measurements and clustering calculations based solely on microscopic measurements are possible.

[0330] In the example mentioned in the embodiment, the clustering is based on user input for a specified region and detection data from either a macroscopic measurement component or a microscopic measurement component.

[0331] In other words, clustering calculations are performed that reflect both manual input and automatic identification.

[0332] exist Figure 12 In the example shown, clustering calculation unit 28 performs clustering using detection data from macroscopic measurement unit 2, detection data from microscopic measurement unit 3, and input data from operation input unit 7. In this case, in addition to the accuracy of user input and responsiveness to requests, automatic discrimination coefficients for areas with different vegetation conditions are provided, allowing measurement results for each more suitable area to be obtained.

[0333] exist Figure 12 In the example, subsegmentation of user-input-based clusters is performed using automatic discrimination. However, in contrast, user input can be used to subsegment automatically discriminate clusters.

[0334] The model parameters described in this embodiment can be any one or more of the following: plant three-dimensional structure, plant height, average leaf angle (average leaf tilt angle), plant coverage, LAI, chlorophyll concentration, soil spectrum characteristics, or sun-leaf ratio.

[0335] Figure 15 The examples mentioned in B are LAI, average leaf angle, and sun-leaf ratio. Other model parameters can be applied depending on the purpose of the measurement.

[0336] Therefore, the model parameters can be appropriately applied to measurements suitable for vegetation remote sensing.

[0337] In the example mentioned in this embodiment, the micro-measurement component 3 includes any one of a visible light image sensor, a stereo camera, a sensor for laser image detection and ranging, a polarization sensor, or a ToF sensor, as the micro-measurement sensor 3S.

[0338] These sensors are suitable for analyzing the characteristics, environmental response, range, distribution, etc. of the measured object, such as shape analysis.

[0339] In addition, the sensors are relatively easy to install in the flying body 200 and are suitable for operation as a small unmanned flying body (such as a drone).

[0340] In the example mentioned in this embodiment, the macroscopic measurement component 2 includes any one of a multispectral camera, a hyperspectral camera, a Fourier transform infrared spectrometer, or an infrared sensor, as the macroscopic measurement sensor 2S.

[0341] These sensors are suitable for analyzing various physical property values, such as information about photosynthesis.

[0342] Furthermore, sensors are relatively difficult to install in the flying body 200. Therefore, for example, installing sensors in a satellite 210 allows for the operation of the flying body 200 as a small unmanned flying body (e.g., a drone).

[0343] The information processing device 1 in this embodiment includes a data storage and output component 30, which generates and outputs image data based on the calculation results from the inverse model calculation component 27.

[0344] Without altering the calculation results, the results from the inverse model calculation unit 27 may not be suitable for human viewing (the evaluation results from the image are difficult to understand). Therefore, the data storage and output unit 30 converts the calculation results into an image suitable for human presentation and outputs the resulting image to the display unit 56, network 5, or storage device 6. Thus, an image that is easy for the user to understand can be provided with the calculation results.

[0345] In the example mentioned in this embodiment, the data storage and output unit 30 generates an output image produced by the color mapping of the calculation results from the inverse model calculation unit 27 (see [link to example]). Figure 16 ).

[0346] That is, given the calculation results obtained from the inverse model calculation unit 27 for each region corresponding to the macroscopic resolution unit, the image presented to the user is generated as an image in which colors are assigned to each region.

[0347] Therefore, users can be provided with images that allow for color-by-color identification and analysis results.

[0348] In the example mentioned in this embodiment, the data storage and output unit 30 generates an output image produced by combining an image derived from a color map of the calculation result from the inverse model calculation unit 27 and another image (see [link]). Figure 17 and Figure 18 ).

[0349] By overlaying or overwriting, by synthesizing another image and an image generated by color mapping, the data storage and output component 30 can provide the user with an image that allows the evaluation results of each region to be identified color by color, while also allowing the identification of each region with the aid of another image.

[0350] In the example mentioned in this embodiment, the macroscopic measurement component 2 is installed in the artificial satellite 210.

[0351] Satellite 210 includes relatively high functionality, and large sensors can be easily mounted on satellite 210. Therefore, satellite 210 is suitable for mounting macroscopic measurement components 2 that perform advanced sensing.

[0352] For example, by allowing a large number of farmers, sensing organizations, etc. to share the macroscopic measurement component 2 of satellite 210, operating costs can be reduced and macroscopic measurement sensors 2S can be used effectively.

[0353] Note that instead of satellite 210, flying body 200 or a relatively large flying body may be equipped with macroscopic measurement component 2 and perform sensing at a position higher than that of microscopic measurement component 3.

[0354] In the example mentioned in this embodiment, the micro-measuring component 3 is installed in the flight body 200, which can be radio-controlled or automatically controlled.

[0355] Examples of flying bodies 200 that can be radio-controlled or automatically controlled include so-called drones, small radio-controlled fixed-wing aircraft, small radio-controlled helicopters, etc.

[0356] The small flying object 200 performs sensing at a relatively low altitude from the object being measured (e.g., farmland 300). In this case, the small flying object 200 is suitable for sensing with high spatial resolution.

[0357] In addition, avoiding the installation of macroscopic measurement components 2 in the flying body 200 facilitates the operation of the small flying body 200 and enables the reduction of sensing costs.

[0358] In the above example, the information processing device 1 according to this embodiment includes a holding component 26, which holds the inverse model calculation program and the inverse model input from an external device.

[0359] That is, the information processing device 1 allows the acquisition of a program from an external device that defines the calculation algorithm of the interpolation analysis calculation component.

[0360] For example, a program for interpolation analysis calculations is obtained from an external device such as network 5 or storage device 6 and stored in holding unit 26. The inverse model calculation unit operates based on the program. This enables information processing unit 1 to perform various inverse model calculations.

[0361] The program according to this embodiment causes the information processing device 1 to perform macroscopic measurement analysis and calculation processing, calculating detection data from macroscopic measurement component 2, which senses the macroscopic measurement range RZ2 of the measured object at macroscopic measurement resolution. The program also causes the information processing device 1 to perform microscopic measurement analysis and calculation processing, calculating detection data from microscopic measurement component 3, which senses the microscopic measurement range RZ3 at microscopic measurement resolution higher than the macroscopic measurement resolution, and the microscopic measurement range RZ3 is included within the macroscopic measurement range RZ2. The program further causes the information processing device 1 to perform inverse model calculation processing, using model parameters based on the detection data from the microscopic measurement component 3 determined by the microscopic measurement analysis and calculation component 23 as parameters for an inverse model calculated using the calculation results from the macroscopic measurement analysis and calculation component 21.

[0362] That is, the program causes the information processing device to execute... Figure 9 , Figure 10 and Figure 14 The processing in the middle.

[0363] Such a procedure helps to realize the image processing apparatus 1 according to this embodiment.

[0364] Such programs can be pre-stored in, for example, a recording medium built into a device such as a computer, or in the ROM of a microcomputer including a CPU. Alternatively, the program can be temporarily or permanently stored on a removable recording medium, such as semiconductor memory, memory cards, optical discs, magneto-optical discs, or magnetic disks. Furthermore, such removable recording media can be provided as a so-called software package.

[0365] In addition to being installed on a personal computer from a removable recording medium, such programs can also be downloaded from download sites via networks such as LANs or the Internet.

[0366] Note that the effects described in this article are illustrative only and are not intended to be limiting; other effects may also occur.

[0367] Those skilled in the art will understand that various modifications, combinations, sub-combinations and alterations can be made depending on design requirements and other factors, as long as they are within the scope of the appended claims or their equivalents.

[0368] Note that this technology can also be configured as described below.

[0369] (1) An information processing device, comprising:

[0370] A macroscopic measurement analysis and calculation unit is configured to calculate detection data from a macroscopic measurement unit, which is adapted to sense a first measurement range of the object to be measured with a first spatial resolution.

[0371] A microscopic measurement and analysis computing component is configured to calculate detection data from a microscopic measurement component adapted to sense a second measurement range at a second spatial resolution higher than a first spatial resolution, the second measurement range being included within a first measurement range of the object being measured; and

[0372] The inverse model calculation unit is configured to acquire model parameters for inverse model calculation using calculation results from the macroscopic measurement analysis calculation unit, based on detection data from the microscopic measurement unit determined by the microscopic measurement analysis calculation unit.

[0373] (2) The information processing device according to (1) above, wherein

[0374] The inverse model calculation unit uses model parameters based on detection data from the micro-measurement unit, determined by the micro-measurement analysis calculation unit, as parameters of the inverse model in the inverse model calculation using calculation results from the macro-measurement analysis calculation unit.

[0375] (3) The information processing device according to (2) above, wherein

[0376] The inverse model calculation unit uses model parameters based on the detection data from the second measurement range to determine the calculation results in units of the first spatial resolution.

[0377] (4) The information processing device according to (2) or (3) above, wherein

[0378] The inverse model calculation component determines the characteristics or environmental response of the measured object as the calculation result in units of the first spatial resolution.

[0379] (5) An information processing apparatus according to any one of (1) to (4) above, wherein

[0380] Macroscopic measuring components sense objects at a greater distance than microscopic measuring components.

[0381] (6) An information processing apparatus according to any one of (1) to (5) above, wherein

[0382] The inverse model computation component obtains the model parameters of representative individuals in each measurement region for clustering, switches the model parameters for each clustered measurement region, and applies the obtained model parameters to the inverse model.

[0383] (7) The information processing device according to (6) above, wherein

[0384] The clustering is performed based on user input from a specified region.

[0385] (8) The information processing apparatus according to (6) or (7) above, wherein

[0386] The clustering is performed based on detection data from macroscopic measurement components or detection data from microscopic measurement components.

[0387] (9) An information processing apparatus according to any one of (6) to (8) above, wherein

[0388] The clustering is performed based on user input from a specified region and detection data from either a macroscopic or microscopic measurement component.

[0389] (10) An information processing apparatus according to any one of (1) to (9) above, wherein

[0390] Model parameters include any one of the following: plant three-dimensional structure, plant height, average leaf angle, plant cover, LAI, chlorophyll concentration, soil spectral characteristics, or sun-leaf ratio.

[0391] (11) An information processing apparatus according to any one of (1) to (10) above, wherein

[0392] The micro-measurement component includes any one of the following as a micro-measurement sensor: a visible light image sensor, a stereo camera, a sensor for laser image detection and ranging, a polarization sensor, or a ToF sensor.

[0393] (12) An information processing apparatus according to any one of (1) to (11) above, wherein

[0394] The macroscopic measurement component includes any one of a multispectral camera, a hyperspectral camera, a Fourier transform infrared spectrometer, or an infrared sensor, which serves as a macroscopic measurement sensor.

[0395] (13) The information processing apparatus according to any one of (1) to (12) above further includes:

[0396] The output component is configured to generate and output image data based on the computation results from the inverse model computation component.

[0397] (14) The information processing device according to (13) above, wherein

[0398] The output component generates output image data obtained from the color mapping of the computation results from the inverse model computation component.

[0399] (15) The information processing apparatus according to (13) above, wherein

[0400] The output component generates output image data by combining an image obtained from a color map based on the calculation results from the inverse model calculation component with another image.

[0401] (16) An information processing apparatus according to any one of (1) to (15) above, wherein

[0402] Macroscopic measurement components are installed in the artificial satellite.

[0403] (17) An information processing apparatus according to any one of (1) to (16) above, wherein

[0404] Microscopic measurement components are installed in the aircraft that can be controlled by radio or automatically.

[0405] (18) An information processing method, comprising:

[0406] The information processing device performs macroscopic measurement analysis processing to calculate the detection data from the macroscopic measurement component, which is configured to sense a first measurement range of the object to be measured with a first spatial resolution.

[0407] A micro-measurement analysis process is performed by an information processing device to calculate detection data from a micro-measuring component, the micro-measuring component being configured to sense a second measurement range at a second spatial resolution higher than the first spatial resolution, the second measurement range being included within the first measurement range of the object being measured; and

[0408] The information processing device performs inverse model calculation processing, which uses model parameters based on detection data from the micro-measuring component determined in the micro-measuring analysis processing as parameters for an inverse model calculated using the calculation results in the macro-measuring analysis processing.

[0409] (19) A procedure for causing an information processing device to perform the following processes:

[0410] Macroscopic measurement analysis processing is performed to calculate detection data from a macroscopic measurement component, which is configured to sense a first measurement range of the object to be measured with a first spatial resolution.

[0411] Microscopic measurement analysis processing is performed to calculate detection data from a microscopic measurement component, which is configured to sense a second measurement range at a second spatial resolution higher than the first spatial resolution, the second measurement range being included within the first measurement range of the object being measured; and

[0412] Inverse model calculation processing uses model parameters based on detection data from the micro-measuring component determined in the micro-measuring analysis process as parameters of the inverse model calculated using the inverse model of the calculation results in the macro-measuring analysis process.

[0413] (20) A sensing system, comprising:

[0414] A macroscopic measurement component is configured to sense a first measurement range of the object being measured with a first spatial resolution;

[0415] A micro-measuring component is configured to sense a second measurement range with a second spatial resolution higher than the first spatial resolution, the second measurement range being included in the first measurement range of the object being measured;

[0416] The macroscopic measurement and analysis calculation unit is configured to calculate the detection data from the macroscopic measurement unit;

[0417] A microscopic measurement and analysis computing component is configured to calculate detection data from the microscopic measurement component; and

[0418] The inverse model calculation unit is configured to use model parameters based on detection data from the micro-measurement unit determined by the micro-measurement analysis calculation unit as parameters for an inverse model calculated using the calculation results from the macro-measurement analysis calculation unit.

[0419] Reference Mark List

[0420] 1. Information processing device; 2. Macroscopic measurement component; 2S macroscopic measurement sensor; 3. Microscopic measurement component; 3S microscopic measurement sensor; 4. Measurement object; 5. Network; 6. Storage device; 7. Operation input component; 10. Data input component; 11. Sensor input component; 12. Sensor input component; 13. Program and model input component; 20. Analysis execution component; 21. Macroscopic measurement analysis and calculation component; 22. Macroscopic measurement analysis value buffer; 23. Microscopic measurement analysis and calculation component; 24. Microscopic measurement analysis value buffer; 25. Position mapping component; 26. Reverse calculation program and model holding component. 27 Reverse model calculation unit, 28 Clustering calculation unit, 30 Data storage and output unit, 31 Analysis data buffer, 32 Color mapping unit, 33 Image compositing unit, 34 Graphics generation unit, 35 Image output unit, 36 Data output unit, 51 CPU, 52 ROM, 53 RAM, 54 Bus, 55 Input / output interface, 56 Display unit, 57 Input unit, 58 Speaker, 59 Storage unit, 60 Communication unit, 61 Driver, 200 Flying vehicle, 210 Artificial satellite, 220 Imaging device, 250 Imaging device, 300 Farmland

Claims

1. An information processing apparatus, comprising: A macroscopic measurement analysis and calculation unit is configured to calculate detection data from a macroscopic measurement unit, which is adapted to sense a first measurement range of the object to be measured with a first spatial resolution. A microscopic measurement analysis and computing component is configured to calculate detection data from a microscopic measurement component, the microscopic measurement component being adapted to sense a second measurement range at a second spatial resolution higher than the first spatial resolution, the second measurement range being included in the first measurement range of the object being measured; The inverse model calculation unit is configured to acquire model parameters for inverse model calculation using calculation results from the macroscopic measurement analysis calculation unit, based on detection data from the microscopic measurement unit determined by the microscopic measurement analysis calculation unit. The model parameters include any one of the following: three-dimensional structure of the plant, plant height, average leaf angle, plant coverage, LAI, chlorophyll concentration, soil spectral characteristics, or solar leaf ratio. as well as The clustering calculation component is configured to perform clustering calculations to cluster the regions of the measurement objects into at least one clustered measurement region. Specifically, the inverse model calculation component acquires model parameters for representative individuals in each cluster measurement region, switches the model parameters for each cluster measurement region, and applies the obtained model parameters to the inverse model calculation. The inverse model calculation component uses the model parameters in the inverse model calculation to calculate the characteristics of the measured object or the environmental response in units of a first spatial resolution.

2. The information processing apparatus according to claim 1, wherein... The inverse model calculation unit uses model parameters based on the detection data from the second measurement range to determine the calculation results in units of the first spatial resolution.

3. The information processing apparatus according to claim 1, wherein... Macroscopic measuring components sense objects at a greater distance than microscopic measuring components.

4. The information processing apparatus according to claim 1, wherein... The clustering is performed based on user input from a specified region.

5. The information processing apparatus according to claim 1, wherein... The clustering is performed based on detection data from macroscopic measurement components or detection data from microscopic measurement components.

6. The information processing apparatus according to claim 1, wherein The clustering is performed based on user input from a specified region and detection data from either a macroscopic or microscopic measurement component.

7. The information processing apparatus according to claim 1, wherein... The micro-measurement component includes any one of the following as a micro-measurement sensor: a visible light image sensor, a stereo camera, a sensor for laser image detection and ranging, a polarization sensor, or a ToF sensor.

8. The information processing apparatus according to claim 1, wherein The macroscopic measurement component includes any one of a multispectral camera, a hyperspectral camera, a Fourier transform infrared spectrometer, or an infrared sensor, which serves as a macroscopic measurement sensor.

9. The information processing apparatus according to claim 1, further comprising: The output component is configured to generate and output image data based on the computation results from the inverse model computation component.

10. The information processing apparatus according to claim 9, wherein The output component generates output image data obtained from the color mapping of the computation results from the inverse model computation component.

11. The information processing apparatus according to claim 9, wherein The output component generates output image data by combining an image obtained from a color map based on the calculation results from the inverse model calculation component with another image.

12. The information processing apparatus according to claim 1, wherein Macroscopic measurement components are installed in the artificial satellite.

13. The information processing apparatus according to claim 1, wherein Microscopic measurement components are installed in the aircraft that can be controlled by radio or automatically.

14. An information processing method, comprising: The information processing device performs macroscopic measurement analysis processing to calculate the detection data from the macroscopic measurement component, which is configured to sense a first measurement range of the object to be measured with a first spatial resolution. The information processing device performs micro-measurement analysis processing to calculate the detection data from the micro-measuring component, the micro-measuring component is configured to sense a second measurement range with a second spatial resolution higher than the first spatial resolution, and the second measurement range is included in the first measurement range of the object being measured; The information processing device performs inverse model calculation processing based on detection data from microscopic measurement components determined in microscopic measurement analysis processing, to acquire model parameters for inverse model calculation using the calculation results in macroscopic measurement analysis processing. These model parameters include any one of the following: plant three-dimensional structure, plant height, average leaf angle, plant cover, LAI (leaf area index), chlorophyll concentration, soil spectral characteristics, or sun-leaf ratio; and The information processing device performs clustering calculations to cluster the regions of the measurement objects into at least one clustered measurement region. Specifically, model parameters for representative individuals in each cluster measurement region are obtained, the model parameters are switched for each cluster measurement region, and the obtained model parameters are applied to the inverse model calculation. In the inverse model calculation, the model parameters are used to calculate the characteristics of the measured object or the environmental response in units of a first spatial resolution.

15. A computer program product comprising instructions, which, when executed by a processor, cause an information processing apparatus to perform the following processes: Macroscopic measurement analysis processing is performed to calculate detection data from a macroscopic measurement component, which is configured to sense a first measurement range of the object to be measured with a first spatial resolution. Microscopic measurement analysis processing is performed to calculate detection data from a microscopic measurement component, which is configured to sense a second measurement range with a second spatial resolution higher than the first spatial resolution, and the second measurement range is included in the first measurement range of the object being measured. Based on detection data from microscopic measurement components determined in microscopic measurement analysis processing, an inverse model calculation process is performed to obtain model parameters for inverse model calculation using the calculation results in macroscopic measurement analysis processing. These model parameters include any one of the following: plant three-dimensional structure, plant height, average leaf angle, plant cover, LAI (leaf area index), chlorophyll concentration, soil spectral characteristics, or sun-leaf ratio; and Perform clustering computations to cluster the regions of the measured objects into at least one cluster of the measured regions. Specifically, model parameters for representative individuals in each cluster measurement region are obtained, the model parameters are switched for each cluster measurement region, and the obtained model parameters are applied to the inverse model calculation. In the inverse model calculation, the model parameters are used to calculate the characteristics of the measured object or the environmental response in units of a first spatial resolution.

16. A sensing system, comprising: A macroscopic measurement component is configured to sense a first measurement range of the object being measured with a first spatial resolution; A micro-measuring component is configured to sense a second measurement range with a second spatial resolution higher than the first spatial resolution, the second measurement range being included in the first measurement range of the object being measured; The macroscopic measurement and analysis calculation unit is configured to calculate the detection data from the macroscopic measurement unit; The microscopic measurement and analysis computing component is configured to calculate the detection data from the microscopic measurement component; The inverse model calculation unit is configured to acquire model parameters for inverse model calculation using calculation results from the macroscopic measurement analysis calculation unit, based on detection data from the microscopic measurement unit determined by the microscopic measurement analysis calculation unit. The model parameters include any one of the following: three-dimensional structure of the plant, plant height, average leaf angle, plant coverage, LAI, chlorophyll concentration, soil spectral characteristics, or solar leaf ratio. as well as The clustering calculation component is configured to perform clustering calculations to cluster the regions of the measurement objects into at least one clustered measurement region. Specifically, the inverse model calculation component acquires model parameters for representative individuals in each cluster measurement region, switches the model parameters for each cluster measurement region, and applies the obtained model parameters to the inverse model calculation. The inverse model calculation component uses the model parameters in the inverse model calculation to calculate the characteristics of the measured object or the environmental response in units of a first spatial resolution.

Images