Information processing device, information processing method, and program

Abstract

To provide an information processing apparatus configured to measure a height of a plant body more accurately, an information processing method, and a program.SOLUTION: An information processing apparatus 100 includes: a digital surface model acquisition unit 150 which acquires a digital surface model (DSM) representing a distribution of altitude values of spots in an image, the model being generated from three-dimensional data based on the image including plant bodies captured from the above; an analysis section setting unit 170 which sets an analysis section, using a measurement target region in the image as a reference; a digital terrain model calculation unit 180 which generates a digital terrain model DTM representing a distribution of altitude values excluding the plant bodies in a region including the measurement target region, on the basis of the altitude values in the analysis section included in the digital surface model; and a crop surface model generation unit 190 which generates a crop surface model (CSM) representing a distribution of altitude values in the measurement target regions, as a distribution of a height of the plant bodies, on the basis of the digital surface model and the digital terrain model.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Measuring the height (plant height) of plants (crops) in experimental fields or production sites where they are cultivated is effective for, for example, evaluating the growth rate of the plants and determining the harvest time. For this reason, various remote sensing methods have been proposed that use aerial images to evaluate cultivated plants. These remote sensing methods include applying a technique called Structure from Motion (SfM) to aerial images to generate three-dimensional data, and then measuring plant height based on the generated three-dimensional data. SfM is a technique for constructing a three-dimensional model from images taken from multiple viewpoints, and its use in the biomass field is being considered.

[0003] For example, Non-Patent Document 1 describes a method of generating three-dimensional data from aerial images taken by a small unmanned aerial vehicle (UAV) using SfM and using the generated three-dimensional data to measure the height (tree height) of trees living in forests, etc. In the measurement method described in Non-Patent Document 1, a digital surface model (DSM) representing the distribution of elevation values ​​including trees and a digital terrain model (DTM) representing the distribution of elevation values ​​excluding trees are generated from the generated three-dimensional data, and tree height is measured by calculating the difference between the DSM and the DTM (subtracting the DTM from the DSM). Furthermore, Non-Patent Document 1 also describes generating a DTM of an area covered by trees by interpolating the elevation values ​​(DSM) of the surrounding area. In this case, the DTM of the area covered by trees is expected to represent flat terrain. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Tamura, T. et al., "Tree height measurement using aerial photography with a small UAV and SfM," Journal of the Japanese Society of Green Technology, 41(1), p163-168, 2015, https: / / www.jstage.jst.go.jp / article / jjsrt / 41 / 1 / 41_163 / _pdf [Non-patent document 2] Girod, LMR, Filhol, SVP (2020), “ABSOLUTE COREGISTRATION AND DOMING CORRECTION IN ADVERSE CONDITIONS, OR HOW TO RETRIEVE SNOW DEPTH FROM DRONE FLIGHTS”, ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, VOL.V-3-2020, pp.375-379. Summary of the Invention [Problem to be solved by the invention]

[0005] However, the target area for remote sensing, i.e., the area covered by trees, is not necessarily flat. Furthermore, when constructing a 3D model using SfM, a phenomenon called doming occurs, in which an area that is originally flat is constructed as a sphere. For this reason, in the measurement method described in Non-Patent Document 1, tree heights measured by subtracting DTM from DSM contain errors due to topography and doming.

[0006] In this regard, Non-Patent Document 2 describes a measurement method for measuring snowfall amount by performing correction to reduce errors. In the measurement method described in Non-Patent Document 2, three-dimensional data is generated by SfM from aerial images taken when there is no snow, and a digital elevation model (DEM) is generated based on the generated three-dimensional data. In this measurement method, the generated DEM is fitted to a quadratic polynomial to correct errors caused by doming, and then the snowfall amount is measured.

[0007] However, when correcting errors due to doming using the measurement method described in Non-Patent Document 2, it is necessary to take aerial images of the plant in a state where it is being cultivated and aerial images of the plant in a state where it is not present (such as after harvesting). In other words, it is necessary to take two images at different times. Generally, when constructing a three-dimensional model in SfM, vertical (height) misalignment is likely to occur between the three-dimensional data generated based on aerial images taken at different times. As a result, a new error, namely a vertical misalignment between the two three-dimensional data, may be introduced.

[0008] The present invention has been made based on the recognition of the above-mentioned problems, and aims to provide an information processing device, an information processing method, and a program that can measure the height of a plant body more accurately. [Means for solving the problem]

[0009] The information processing device, the information processing method, and the program according to the present invention employ the following configuration. An information processing device according to one embodiment of the present invention is an information processing device that includes: a digital ground surface model acquisition unit that acquires a digital ground surface model that represents the distribution of elevation values ​​at each point in an image, generated from three-dimensional data based on an image including plant bodies photographed from above; an analysis area setting unit that sets an analysis area based on a measurement target area in the image; a digital ground surface model generation unit that generates a digital terrain model that represents the distribution of elevation values ​​excluding the plant bodies in an area including the measurement target area, based on the elevation values ​​within the analysis area included in the digital ground surface model; and a crop surface model generation unit that generates a crop surface model that represents the distribution of elevation values ​​within the measurement target area as a distribution of heights of the plant bodies, based on the digital ground surface model and the digital ground surface model.

[0010] The analysis area setting unit receives the input of the measurement target area and sets the analysis area around the measurement target area.

[0011] The digital terrain model generating unit generates the digital terrain model by setting the elevation value of a predetermined percentile value of the elevation values ​​included in the analysis area as the elevation value within the analysis area.

[0012] The digital terrain model generating unit generates, as the digital terrain model, a plane or a curved surface that approximates the entire area of ​​the image by the least squares method from the elevation values ​​within the analysis area.

[0013] The digital terrain model generating unit generates the digital terrain model that approximates a plane or curved surface expressed by an n-th degree polynomial (n is a natural number of 1 to 3, including at least 0).

[0014] The crop surface model generating unit displays an image based on the crop surface model on a display device.

[0015] Another aspect of the present invention is an information processing method in which a computer acquires a digital ground surface model generated from three-dimensional data based on an image including plant bodies photographed from above, the digital ground surface model representing the distribution of elevation values ​​at each point in the image, sets an analysis area based on a measurement target area in the image, generates a digital terrain model representing the distribution of elevation values ​​excluding the plant bodies in an area including the measurement target area based on the elevation values ​​in the analysis area included in the digital ground surface model, and generates a crop surface model representing the distribution of elevation values ​​in the measurement target area as a distribution of heights of the plant bodies based on the digital ground surface model and the digital terrain model.

[0016] Another aspect of the present invention is a program that causes a computer to acquire a digital ground surface model that is generated from three-dimensional data based on an image including plant bodies photographed from above, and that represents the distribution of elevation values ​​at each point in the image; set an analysis area based on a measurement target area in the image; generate a digital terrain model that represents the distribution of elevation values ​​excluding the plant bodies in an area including the measurement target area based on the elevation values ​​in the analysis area included in the digital ground surface model; and generate a crop surface model that represents the distribution of elevation values ​​in the measurement target area as a distribution of heights of the plant bodies, based on the digital ground surface model and the digital terrain model. [Effects of the Invention]

[0017] According to the present invention, the height of a plant can be measured more accurately. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a diagram illustrating an example of a configuration of an information processing apparatus according to an embodiment and an example of a usage environment of the information processing apparatus; [Figure 2] 10 is an example of three-dimensional data representing an area of ​​an aerial image of a farm field acquired by a three-dimensional data acquisition unit. [Figure 3] This is an example of a digital surface model acquired by the DSM acquisition unit. [Figure 4]FIG. 10 is a diagram showing an example of a screen for setting a measurement target region and an analysis area in the information processing device. [Figure 5] FIG. 2 is a diagram showing an example of a digital terrain model generated by a DTM calculation unit. [Figure 6] FIG. 10 is a diagram illustrating an example of processing in a CSM generation unit. [Figure 7] FIG. 2 is a diagram showing an example of a crop surface model generated by a CSM generation unit. [Figure 8] 10 is a flowchart showing an example of a processing flow for measuring the height of a plant using an information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0019] Hereinafter, an information processing device, an information processing method, and a program according to an embodiment of the present invention will be described with reference to the drawings.

[0020] [Configuration of information processing device] 1 is a diagram showing an example of the configuration of an information processing device according to an embodiment and an example of a usage environment of the information processing device. The information processing device 100 generates a three-dimensional model representing the height (grass height) of a plant (crop) based on an aerial image including the plant taken from above (the sky). The aerial image is taken from the sky by a flying object such as a drone or a small unmanned aerial vehicle (UAV), including a target area for generating a three-dimensional model (hereinafter referred to as a "measurement target area").

[0021] Figure 1 shows an example in which the flying object FO is a drone. In Figure 1, the measurement target area is an area in which the height of plants grown in a field F is measured, and the flying object FO is shown photographing the measurement target area including the plants from above the field F.

[0022] The flight of the flying object FO is controlled by, for example, the terminal device T1. Alternatively, the flying object FO may fly autonomously according to a program stored in an internal memory. The flying object FO is equipped with, for example, an imaging device C such as a digital camera. The imaging device C captures aerial images of the field F including the plants from the sky, for example, at regular intervals or whenever the flying object FO travels a certain distance. The imaging device C may capture aerial images of the field F including the plants from the sky in response to an operation from the terminal device T1. The flying object FO transmits the captured aerial images to the terminal device T1. Furthermore, the flying object FO is also equipped with a positioning device (not shown) that determines the position of the flying object FO itself based on signals received from satellites constituting a Global Navigation Satellite System (GNSS), such as a Global Positioning System (GPS). The imaging device C associates the captured aerial image of the field F with information indicating the position of the flying object FO (hereinafter referred to as "GPS information") and transmits the associated image to the terminal device T1. The GPS information includes, for example, information such as the latitude, longitude, and altitude of the flying object FO. The flying object FO may be equipped with, for example, an altimeter that measures altitude based on air pressure, and may transmit information on the altitude of the flying object FO measured by the altimeter in association with the aerial image in place of or in addition to the altitude information included in the GPS information.

[0023] The terminal device T1 controls, for example, the flight of the flying object FO and the capture of aerial images by the imaging device C. The terminal device T1 is a computing device such as a personal computer, smartphone, or tablet terminal. An application for controlling the flight of the flying object FO and the capture of images by the imaging device C is executed on the terminal device T1. The application transmits information for controlling the flight of the flying object FO and the capture of images by the imaging device C to the flying object FO and the imaging device C in response to operations by a user P1 using the information processing device 100. The application displays, on a display device provided in the terminal device T1, images including operation buttons for controlling the flight of the flying object FO and the capture of images by the imaging device C, as well as an image showing the current capture range transmitted by the imaging device C. When the user P1 operates the terminal device T1 to capture an image of the field F using the imaging device C, the user P1 may capture images multiple times while moving the flying object FO so that an area of ​​the field F including plants is captured. The imaging device C associates GPS information with each captured aerial image and transmits the associated information to the terminal device T1. The terminal device T1 transmits to the information processing device 100 the aerial image transmitted by the imaging device C provided in the flying object FO.

[0024] The terminal device T1 transmits, for example, aerial images to the information processing device 100 via a network NW. The network NW includes, for example, the Internet, a wide area network (WAN), a local area network (LAN), a provider device, a wireless base station, etc. The terminal device T1 may, for example, temporarily store the aerial images transmitted by the imaging device C in a portable memory and then transfer the aerial images to the information processing device 100 by attaching the portable memory to the information processing device 100.

[0025] The information processing device 100 includes, for example, a communication unit 110, an image acquisition unit 120, a storage unit 130, a three-dimensional data acquisition unit 140, a DSM acquisition unit 150, a measurement target region setting unit 160, an analysis region setting unit 170, a DTM calculation unit 180, and a CSM generation unit 190. The information processing device 100 may further communicate with a terminal device T2 separate from the terminal device T1 via a network NW.

[0026] The terminal device T2 is operated by, for example, a user P2 who uses the information processing device 100 to check measurement results of the information processing device 100 connected via the network NW. Like the terminal device T1, the terminal device T2 is also a computer device such as a personal computer, a smartphone, or a tablet terminal. The terminal device T2 runs applications for controlling measurements in the information processing device 100 and checking measurement results. The applications running on the terminal device T2 control the setting of measurement conditions when the information processing device 100 performs measurements and the display of measurement results of the information processing device 100 in response to operations by the user P2. The applications running on the terminal device T2 display images including operation buttons for controlling the setting of measurement conditions and images showing measurement results of the information processing device 100 on a display device provided in the terminal device T2. The display devices provided in one or both of the terminal devices T1 and T2, or one or both of the terminal devices T1 and T2, are examples of the "display device" in the claims.

[0027] The components of the information processing device 100, excluding the communication unit 110 and the storage unit 130, are implemented by a hardware processor, such as a CPU (Central Processing Unit), executing a program (software). Some or all of the functions of these components may be implemented by hardware (including circuitry), such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be implemented by a combination of software and hardware. Some or all of the functions of these components may be implemented by a dedicated LSI. The program may be stored in advance in a storage device (a storage device having a non-transitory storage medium), such as an HDD (Hard Disk Drive) or flash memory, included in the information processing device 100. Alternatively, the program may be stored in a removable storage medium (a non-transitory storage medium), such as a DVD or CD-ROM, and installed in the storage device included in the information processing device 100 by inserting the storage medium into a drive device included in the information processing device 100. The information processing device 100 may be realized, for example, as a computer device such as a personal computer or a storage device. The information processing device 100 may also be realized as a server device or a storage device incorporated in a cloud computing system. In this case, the functions of the information processing device 100 may be realized by multiple server devices and storage devices in the cloud computing system.

[0028] The communication unit 110 is a communication interface such as a network card for connecting to the network NW, and communicates with the terminal device T1 via the network NW.

[0029] The image acquisition unit 120 acquires aerial images captured by an imaging device C mounted on the flying object FO from the terminal device T1 that communicates with the communication unit 110. The image acquisition unit 120 acquires multiple aerial images (or all aerial images) captured by the imaging device C via the communication unit 110 and the terminal device T1. The image acquisition unit 120 stores the acquired aerial images in the memory unit 130. The aerial images acquired by the image acquisition unit 120 may be stored in the memory unit 130 by DMA (Direct Memory Access).

[0030] The storage unit 130 stores data and information used when the components of the information processing device 100 perform processing. The storage unit 130 is, for example, a storage device such as an HDD or a flash memory.

[0031] The three-dimensional data acquisition unit 140 acquires three-dimensional point cloud data (hereinafter referred to as "three-dimensional data") of the area depicted in the aerial image, which is constructed by applying the SfM technique to the same aerial image as the aerial image acquired by the image acquisition unit 120. The three-dimensional data acquisition unit 140 acquires three-dimensional data constructed by another processing device (not shown) connected to the network NW, for example, via the communication unit 110. The three-dimensional data acquisition unit 140 may acquire the three-dimensional data constructed by the other processing device (not shown), for example, via a portable memory or directly. FIG. 2 shows an example of three-dimensional data representing an area of ​​the aerial image of the field F acquired by the three-dimensional data acquisition unit 140. FIG. 2 shows an example of three-dimensional data represented as an image. The image shown in FIG. 2 is, for example, called an orthoimage. The three-dimensional data acquisition unit 140 stores the acquired three-dimensional data in the storage unit 130. The three-dimensional data acquired by the three-dimensional data acquisition unit 140 may be stored in the storage unit 130 by DMA.

[0032] Instead of or in addition to acquiring three-dimensional data, the three-dimensional data acquisition unit 140 may be configured to apply an SfM technique to the aerial images acquired by the image acquisition unit 120 and stored in the storage unit 130 to construct (generate) three-dimensional data itself. In this case, the three-dimensional data acquisition unit 140 aligns each aerial image based on latitude and longitude information extracted from GPS information included in each of the multiple aerial images captured by the imaging device C and acquired by the image acquisition unit 120, and the subjects depicted in each aerial image. The three-dimensional data acquisition unit 140 then constructs one piece of three-dimensional data corresponding to the aligned aerial image. The three-dimensional data is data in which latitude, longitude, and altitude information extracted from GPS information included in the aerial image are associated with data on each point. The altitude information extracted from the GPS information is associated with the three-dimensional data as the elevation value of each point. The three-dimensional data acquisition unit 140 stores the constructed three-dimensional data in the storage unit 130.

[0033] The DSM acquisition unit 150 acquires a digital surface model (DSM), which is a three-dimensional model representing the distribution of elevation values ​​at each point, including plants, generated based on the same three-dimensional data as the three-dimensional data acquired by the three-dimensional data acquisition unit 140. The DSM acquisition unit 150 acquires a DSM generated by another processing device (not shown) connected to the network NW (which may be the same as or different from the processing device that constructs the three-dimensional data), for example, via the communication unit 110. The DSM acquisition unit 150 may acquire the DSM generated by the other processing device (not shown), for example, via a portable memory or directly. The DSM is, for example, image data in which the elevation value of each point is used as a pixel value. The DSM acquisition unit 150 displays an image representing the pixel values ​​of the acquired DSM using colors or color gradations (e.g., gradations) on, for example, a display device provided in a terminal device T2 operated by a user P2 using the information processing device 100. This allows the user P2 to visually confirm the elevation height of the farm field F containing the plants. The DSM acquisition unit 150 may display an image of the acquired DSM, for example, on a display device provided on a terminal device T1 operated by the user P1, so that the user P1 can visually confirm the elevation height of the field F containing the plant body.

[0034] FIG. 3 shows an example of a digital surface model (DSM) acquired by the DSM acquisition unit 150. The example shown in FIG. 3 is an example of a DSM displayed as an image on a display device provided on a terminal device T2 operated by a user P2 or a terminal device T1 operated by a user P1. FIG. 3 shows an example in which the elevation value of each point in the DSM is represented by a color gradation (shade of black and white), and elevation information EI is added to associate the color in the image with the elevation value. Here, the DSM acquired by the DSM acquisition unit 150 is a three-dimensional model in which a predetermined reference elevation value is uniformly subtracted from all three-dimensional data. Therefore, the DSM acquired by the DSM acquisition unit 150 contains an error due to doming. For example, doming is a barrel distortion that increases from the optical center of the imaging lens of the imaging device C due to the curved surface of the imaging lens. This is a cause of error in which points closer to the center of the DSM range are represented as higher elevation values ​​and points further out from the DSM range are represented as lower elevation values. In Figure 3, the plant height at the center point is high, while the surrounding ground surface and plant height are low.

[0035] The DSM acquisition unit 150 stores the acquired DSM data in the storage unit 130. The DSM acquisition unit 150 may store the acquired DSM data in the storage unit 130 by DMA.

[0036] DSM acquisition part 150 Alternatively, or in addition to acquiring a DSM, the DSM acquisition unit 150 may generate a DSM itself based on three-dimensional data acquired or constructed by the three-dimensional data acquisition unit 140. The DSM acquisition unit 150 stores the generated DSM data in the storage unit 130. The storage of the generated DSM data in the DSM acquisition unit 150 in the storage unit 130 may be performed by DMA.

[0037] The combined configuration of the three-dimensional data acquisition unit 140 and the DSM acquisition unit 150 (which may also include the communication unit 110) is an example of the "digital earth surface model acquisition unit" in the claims.

[0038] The measurement target area setting unit 160 sets the range of the measurement target area for generating a three-dimensional model. For example, the measurement target area setting unit 160 sets, as the measurement target area, an area specified (input) by a user P2 using the information processing device 100 connected via the network NW, by operating an input operation device such as a mouse or keyboard provided on the terminal device T2. Here, the users P1 and P2 may be different people or the same person. Furthermore, the terminal devices T1 and T2 may be different terminal devices or the same terminal device. The measurement target area is specified by the user P2, for example, by displaying an image representing the DSM acquired or generated by the DSM acquisition unit 150 on a display device provided on or connected to the terminal device T2, and the user P2 operating the input operation device to specify the range of the measurement target area on the displayed image. The measurement target area setting unit 160 outputs information representing the range of the specified measurement target area to the analysis area setting unit 170 and the DTM calculation unit 180. The measurement target region setting section 160 may store information indicating the range of the specified measurement target region in the storage section 130.

[0039] The analytical region setting unit 170 sets analytical regions to approximate the topography within the measurement region, which are used when generating a three-dimensional model. The analytical region setting unit 170 applies the range of the measurement region output by the measurement region setting unit 160 to the DSM acquired or generated by the DSM acquisition unit 150, and then sets multiple analytical regions within a predetermined range around the measurement region, using the applied measurement region as a reference. The analytical region setting unit 170 outputs information indicating the range of each of the set analytical regions to the DTM calculation unit 180. The analytical region setting unit 170 may store information indicating the range of each of the set analytical regions in the storage unit 130.

[0040] Here, an example of a method for setting the measurement target area in the measurement target area setting unit 160 and the analysis area in the analysis area setting unit 170 will be described. FIG. 4 is a diagram showing an example of a screen for setting the measurement target area and the analysis area in the information processing device 100. FIG. 4 shows an example of a state in which an image representing the DSM acquired or generated by the DSM acquisition unit 150 is displayed on a display device provided in the terminal device T2, the range of the measurement target area TA specified by the user P2 is superimposed on the image of the DSM, and the analysis area setting unit 170 has set multiple analysis areas AA around the measurement target area TA. For example, the user P2 sets the conditions for the measurement target area TA on the measurement target area setting panel TS shown in FIG. 4 and specifies within the DSM using an input operation device (e.g., a mouse), thereby setting the measurement target area TA. As a result, the analysis area setting unit 170 sets a predetermined number of analysis areas AA around the measurement target area TA. The conditions when the analysis area setting unit 170 sets the analysis areas AA may be specified, for example, by the user P2. FIG. 4 also shows an analysis area setting panel AS, which allows user P2 to specify conditions for setting the analysis area AA. The analysis area setting panel AS shown in FIG. 4 is an example of a case where user P2 specifies conditions such as the length, width, distance (margin) from the measurement target area TA, and number of analysis areas AA. After the measurement target area TA is set, the analysis area setting unit 170 sets multiple analysis areas AA according to the conditions specified by user P2 using the analysis area setting panel AS. The analysis area setting unit 170 displays the range of the set analysis area AA by superimposing it on an image of the DSM. This allows user P2 to visually confirm the measurement target area TA and the analysis area AA set on the DSM. Although FIG. 4 displays an image representing the DSM, an image of three-dimensional data (an aerial image such as an orthoimage) acquired or generated by the three-dimensional data acquisition unit 140 may be displayed instead.

[0041] Returning to FIG. 1, the DTM calculation unit 180 performs calculations based on the information representing the range of the measurement target area output by the measurement target area setting unit 160 and the information representing the range of the analysis area output by the analysis area setting unit 170 to generate a digital terrain model (DTM), which is a three-dimensional model representing the distribution of elevation values ​​that approximates the terrain within the measurement target area and is used when generating a three-dimensional model. In other words, the DTM calculation unit 180 generates a DTM that represents the distribution of elevation values ​​excluding (excluding) plant bodies. The DTM generated by the DTM calculation unit 180 enables the terrain within the measurement target area to be considered (corrected) to have the same elevation value.

[0042] More specifically, the DTM calculation unit 180 first extracts the three-dimensional data for each analysis area output by the analysis area setting unit 170 from the three-dimensional data for each location represented by the DSM acquired or generated by the DSM acquisition unit 150. The DTM calculation unit 180 then selects a representative value from the extracted three-dimensional data for each analysis area. The representative value for one analysis area is, for example, three-dimensional data of elevation values ​​located at a predetermined percentage (percentile value) when the elevation values ​​contained in multiple three-dimensional data belonging to the same analysis area are sorted in ascending order. The DTM calculation unit 180 selects an elevation value with a low percentile value, such as the 0.1 percentile value, as the representative value for each analysis area. The percentile value used by the DTM calculation unit 180 when selecting the representative value is not limited to the 0.1 percentile value and may be any low percentile value. For example, the DTM calculation unit 180 may use any of the percentile values, such as the 0.5 percentile, 1 percentile, or 5 percentile, as the percentile value used when selecting a representative value for each analysis plot. However, the analysis plot for which the representative value is selected, i.e., the area surrounding the measurement target area (field F), may contain depressions, such as tire marks left by agricultural machinery such as tractors. In this case, selecting an elevation value with a low percentile value as the representative value for each analysis plot may result in the DTM generated by the DTM calculation unit 180 containing many errors. In other words, selecting an elevation value with a relatively high percentile value (e.g., the median = 50th percentile value) as the representative value for each analysis plot may reduce the errors contained in the DTM generated by the DTM calculation unit 180. For this reason, for example, an item (not shown) for setting percentile values ​​may be provided on the analysis region setting panel AS on the screen shown in Fig. 4, and the user P2 may operate the input operation device to specify the percentile values ​​that the DTM calculation unit 180 will use when selecting representative values ​​for each analysis region. In other words, a wide range of percentile values ​​may be applied as the percentile values ​​that the DTM calculation unit 180 will use.

[0043] The DTM calculation unit 180 then sets the latitude and longitude associated with the three-dimensional data of the representative values ​​of each analysis area as the coordinates (x, y) of the earth's surface, and applies the altitude (elevation value) as the height coordinate (z) onto the three-dimensional coordinates (x, y, z), and generates a DTM that approximates the height of the terrain to be fitted to the measurement target area by using the least squares method for a two-variable nth-order polynomial as shown in the following equation (1).

[0044]

number

[0045] The DTM calculation unit 180 generates a DTM that represents the height of the terrain within the measurement target area using, for example, a zero-order plane, a linear plane, a quadratic surface, or a cubic surface, using at least the above formula (1). That is, the DTM calculation unit 180 generates the DTM using the least squares method, where n in the above formula (1) is a natural number between 1 and 3, including 0. More specifically, the DTM calculation unit 180 generates a DTM that represents the zero-order plane f0(x,y) using the following formula (2), generates a DTM that represents the linear plane f1(x,y) using the following formula (3), generates a DTM that represents the quadratic surface f2(x,y) using the following formula (4), and generates a DTM that represents the cubic surface f3(x,y) using the following formula (5).

[0046]

number

[0047]

number

[0048]

number

[0049]

number

[0050] In the above formulas (1) to (5), the coefficient a may be, for example, a value assumed in the measurement area, or an optimal value may be found by trial and error from a similar process previously performed, or from actual measurements of the height when there are no plants in the measurement area, etc. In the above formulas (1) to (5), the coefficient a00 is, for example, the average value of the altitude (elevation value) represented by the representative value of each analysis plot.

[0051] Here, an example of a DTM generated by the DTM calculation unit 180 will be described. FIG. 5 is a diagram showing an example of a digital terrain model (DTM) generated by the DTM calculation unit 180. In order to easily understand the shape of the DTM generated by the DTM calculation unit 180, FIG. 5 shows an example of a DTM obtained by plotting representative values ​​(latitude (x), longitude (y), and altitude (z)) of each analysis area against three-dimensional coordinates (x, y, z) and using the least squares method according to the above equations (2) to (5). FIG. 5(a) shows an example of a DTM with a zero-order plane, FIG. 5(b) shows an example of a DTM with a linear plane, FIG. 5(c) shows an example of a DTM with a quadratic surface, and FIG. 5(d) shows an example of a DTM with a cubic surface. Here, the zero-order plane shown in FIG. 5(a) approximates the topography of the measurement area as flat, similar to the conventional method of measuring grass height. The DTM calculation unit 180 stores the generated DTM data in the storage unit 130. The DTM data generated by the DTM calculation unit 180 may be stored in the storage unit 130 by DMA. The DTM calculation unit 180 is an example of a "digital terrain model generation unit" in the claims.

[0052] Returning to FIG. 1 , the CSM generation unit 190 generates a three-dimensional model that represents the distribution of plant heights of only plants based on the DSM acquired or generated by the DSM acquisition unit 150 and the DTM generated by the DTM calculation unit 180. In other words, the CSM generation unit 190 generates a three-dimensional model that represents only the plant heights of plants by correcting the topography of the field, which is the measurement target area, to the same elevation value. More specifically, the CSM generation unit 190 calculates the difference between the elevation values ​​of the same point in the DSM and the DTM, and generates a three-dimensional model in which the difference value represents the plant height of the plant. The three-dimensional model generated by the CSM generation unit 190 is, for example, called a crop surface model (CSM). In the following description, the three-dimensional model generated by the CSM generation unit 190 is referred to as a "CSM." Similar to the DSM, the CSM is image data in which the pixel values ​​are, for example, the elevation values ​​of each point. Similar to the DSM acquisition unit 150, the CSM generation unit 190 displays an image representing the pixel values ​​of the generated CSM using colors and color gradations (for example, gradations) on, for example, a display device provided in a terminal device T2 operated by a user P2 who uses the information processing device 100. This allows the user P2 to visually check the plant height of only the plants in the field F. The CSM generation unit 190 may also display an image of the generated CSM on, for example, a display device provided in a terminal device T1 operated by a user P1, allowing the user P1 to visually check the plant height of only the plants in the field F.

[0053] Fig. 6 is a diagram schematically illustrating an example of processing (processing for generating a CSM) in the CSM generation unit 190. Fig. 6 illustrates a case in which a CSM is generated by subtracting a DTM generated by the DTM calculation unit 180 (in Fig. 6, the DTM shown in Fig. 5(c)) from a DSM acquired or generated by the DSM acquisition unit 150. In this way, the CSM generation unit 190 generates a CSM that has been corrected so that the altitude values ​​represented by the DSMs acquired or generated by the DSM acquisition unit 150 can be regarded as being based on the same altitude value.

[0054] The CSM generation unit 190 stores the generated CSM data in the storage unit 130. The CSM generation unit 190 may store the generated CSM data in the storage unit 130 by DMA. The CSM generation unit 190 is an example of a "crop surface model generation unit" in the claims.

[0055] Here, an example of a CSM generated by the CSM generation unit 190 will be described. Fig. 7 is a diagram showing an example of a crop surface model (CSM) generated by the CSM generation unit 190. Fig. 7 shows an example of a CSM generated by the CSM generation unit 190 by subtracting each DSM generated by the DTM calculation unit 180 from the DSM acquired or generated by the DSM acquisition unit 150. Fig. 7 shows the range of the measurement target area TA specified by the user P2 superimposed on the image of the CSM.

[0056] Figure 7(a) is an example of a CSM generated by subtracting a DTM of a zero-order plane (flat surface) such as that shown in Figure 5(a) from a DSM. The CSM shown in Figure 7(a) corresponds to one generated using a conventional method for measuring plant height. In the CSM shown in Figure 7(a), for example, area A2 is lower than area A1. In this way, simply subtracting a flat surface from the DSM does not sufficiently correct for the topography (elevation values) around field F. For this reason, it is expected that the CSM shown in Figure 7(a) contains a large amount of error in the plant height of the plants in field F.

[0057] Figure 7(b) is an example of a CSM generated by subtracting a linear-plane DTM such as that shown in Figure 5(b) from a DSM. When a linear plane is subtracted from a DSM, as in the CSM shown in Figure 7(b), the slope of the topography (elevation values) of field F is corrected to some extent compared to the CSM shown in Figure 7(a), but it is still considered to be insufficient. More specifically, in the CSM shown in Figure 7(b), area A3 in the center of field F is distorted, being higher than areas A4 and A5 in the peripheral areas of field F. This is thought to be due to doming. For this reason, it is expected that the CSM shown in Figure 7(b) also contains errors in the plant height of the plants in field F, although the errors are smaller than those in the CSM shown in Figure 7(a).

[0058] Figure 7(c) is an example of a CSM generated by subtracting a quadratic DTM, such as that shown in Figure 5(c), from a DSM. Figure 7(d) is an example of a CSM generated by subtracting a cubic DTM, such as that shown in Figure 5(d), from a DSM. By subtracting a quadratic or cubic surface from a DSM, as in the CSMs shown in Figures 7(c) and 7(d), it is believed that the topography (elevation values) of field F is sufficiently corrected. More specifically, in the CSMs shown in Figures 7(c) and 7(d), in addition to correcting the slope of the topography (elevation values) of field F, the topographic distortion of field F, which is thought to be due to doming and which appeared in the CSM shown in Figure 7(b), is also corrected. Therefore, it is expected that the CSMs shown in Figures 7(c) and 7(d) contain almost no errors in the plant height of the plants in field F. In the CSM shown in Figure 7(d), area A6, which is outside the measurement target area TA, has a slightly lower topography than area A7, a similar area in the CSM shown in Figure 7(c), but this is due to a commonly occurring phenomenon in which the difference becomes greater the further away from the measurement target area TA that approximates the topography, depending on the conditions for approximating a cubic curved surface. However, because area A6, which is the lower topography in the CSM shown in Figure 7(d), is outside the measurement target area TA that is the target of approximating the topography, it has no effect on the plant height of the plants in the field F represented by the CSM.

[0059] [Measurement and processing of plant height using an information processing device] Next, an example of a process for measuring the plant height of a plant body using the information processing device 100 will be described. Fig. 8 is a flowchart showing an example of the flow of a process for measuring the height (plant height) of a plant body using the information processing device 100. In the following description, it is assumed that the flight of the flying object FO and the capture of aerial images by the imaging device C are controlled by the terminal device T1, and that the measurement in the information processing device 100 is controlled and the measurement results are confirmed by the terminal device T2. Furthermore, in the following description, it is assumed that the three-dimensional data acquisition unit 140 is configured to construct three-dimensional data by itself, and that the DSM acquisition unit 150 is configured to generate a DSM by itself.

[0060] When measuring the plant height of a plant body in the information processing device 100, first, the image acquisition unit 120 acquires a plurality of aerial images (or all aerial images) taken by the imaging device C via the communication unit 110 and the terminal device T1 (step S100). The image acquisition unit 120 stores the acquired aerial images in the memory unit 130. The image acquisition unit 120 may notify the three-dimensional data acquisition unit 140 that the aerial images have been stored in the memory unit 130.

[0061] The three-dimensional data acquisition unit 140 reads out the multiple aerial images stored in the storage unit 130, and applies the SfM technique to the multiple aerial images that have been read out to construct three-dimensional data (step S102). The three-dimensional data acquisition unit 140 stores the constructed three-dimensional data in the storage unit 130. The three-dimensional data acquisition unit 140 may notify the DSM acquisition unit 150 that the three-dimensional data has been stored in the storage unit 130.

[0062] The DSM acquisition unit 150 reads out the three-dimensional data constructed by the three-dimensional data acquisition unit 140 and stored in the storage unit 130, and generates a DSM based on the read three-dimensional data (step S104). The DSM acquisition unit 150 stores the generated DSM in the storage unit 130. The DSM acquisition unit 150 may notify the measurement target region setting unit 160 that the DSM has been stored in the storage unit 130.

[0063] The measurement target area setting unit 160 sets the range of the measurement target area (step S106). At this time, the measurement target area setting unit 160 reads out the DSM stored in the storage unit 130 and displays an image representing the read DSM (or an aerial image such as an orthoimage based on the three-dimensional data generated by the three-dimensional data acquisition unit 140) on the display device provided in the terminal device T2. Then, the measurement target area setting unit 160 sets the area specified (input) by the user P2 operating the input operation device as the measurement target area. The measurement target area setting unit 160 displays the range of the set measurement target area by superimposing it on the image of the DSM (see FIG. 4). Thereafter, the measurement target area setting unit 160 may prompt the user P2 to input whether or not to confirm the setting (designation) of the measurement target area. The measurement target area setting unit 160 outputs information representing the range of the specified measurement target area to the analysis area setting unit 170 and the DTM calculation unit 180. When the measurement target area setting unit 160 stores information representing the range of the specified measurement target area in the memory unit 130, it may notify the analysis area setting unit 170 and the DTM calculation unit 180 that the information on the measurement target area has been stored in the memory unit 130.

[0064] The analytical region setting unit 170 sets analytical regions (step S108). At this time, the analytical region setting unit 170 sets a predetermined number of analytical regions around the measurement region set by the measurement region setting unit 160. The analytical region setting unit 170 displays the range of the set analytical regions by superimposing it on the image of the DSM (see FIG. 4). Thereafter, the analytical region setting unit 170 may prompt the user P2 to input whether or not to confirm the setting of the analytical regions. The analytical region setting unit 170 outputs information indicating the range of each set analytical region to the DTM calculation unit 180. When storing information indicating the range of each set analytical region in the storage unit 130, the analytical region setting unit 170 may notify the DTM calculation unit 180 that the information of the analytical regions has been stored in the storage unit 130.

[0065] The DTM calculation unit 180 reads the DSM stored in the storage unit 130 and generates a DTM by performing calculations based on the read DSM, the measurement target region set by the measurement target region setting unit 160, and the analysis region set by the analysis region setting unit 170 (step S110). At this time, as described above, the DTM calculation unit 180 may generate a DTM by using, as a representative value for each analysis region, the elevation value of a percentile value specified by a percentile value setting item (not shown) provided on the analysis region setting panel AS on the screen shown in FIG. 4. The DTM calculation unit 180 may generate DTMs for, for example, a zero-order plane, a linear plane, a quadratic surface, and a cubic surface, as described above. Alternatively, the DTM calculation unit 180 may display, on a display device provided with the terminal device T2, a DTM specification panel (not shown) below the analysis region setting panel AS on the screen shown in FIG. 4, allowing the user P2 to specify the DTM to be generated, and generate the specified DTM by operating an input operation device. The DTM calculation unit 180 stores the generated DTM in the storage unit 130. The DTM calculation unit 180 may notify the CSM generation unit 190 that the DTM has been stored in the storage unit 130.

[0066] The CSM generation unit 190 reads out the DSM and DTM stored in the storage unit 130, and generates a CSM from the read DSM and DTM (step S112). The CSM generation unit 190 stores the generated CSM in the storage unit 130. Then, the CSM generation unit 190 displays an image of the generated CSM on a display device provided in the terminal device T2 (step S114). This allows the user P2 to check the plant height of only the plant represented by the CSM displayed on the display device.

[0067] With this configuration and processing, the information processing device 100 acquires an aerial image captured by the imaging device C mounted on the flying object FO. The information processing device 100 then acquires or generates three-dimensional data corresponding to the same aerial image as the acquired aerial image, and acquires or generates a DSM corresponding to the same aerial image as the acquired or generated aerial image. Furthermore, the information processing device 100 generates a DTM that approximates the topography of the measurement target area specified by the user P2. The information processing device 100 then generates a CSM that represents the distribution of plant heights of only the plants within the measurement target area, based on the acquired or generated DSM and the generated DTM. This allows the information processing device 100 to reduce errors due to the slope of the topography of the measurement target area (field F) as well as errors due to doming, which is thought to be caused by the curved surface of the imaging lens of the imaging device C, thereby enabling the plant height of the plants to be measured with higher accuracy.

[0068] Moreover, the information processing device 100 can measure the plant height of a plant in its current state as it is being cultivated, without having to take two images at different times, as is the case with conventional methods for measuring plant height. In other words, the information processing device 100 can measure the plant height of a plant by approximating the topography of the measurement target area from a series of aerial images taken at the same time, even when the measurement target area is covered with plants. Moreover, the information processing device 100 does not require the installation or maintenance of ground control points (GCPs), etc., which are considered to be used to correct (measure) the topography of the measurement target area in conventional methods for measuring plant height.

[0069] As a result, the information processing device 100 can be effectively used to measure the plant height of plants (crops) in test fields or production sites. For example, in the field of agricultural research where plant (crop) cultivation tests are conducted in test fields, it is possible to reduce the labor required to measure the traits of plants, which has traditionally been measured manually. For example, at plant (crop) production sites, it can be used to observe (monitor) the growth state and predict harvest yields.

[0070] As described above, the information processing device 100 of the embodiment acquires an aerial image captured by the imaging device C mounted on the flying object FO, and acquires or generates three-dimensional data and a DSM corresponding to the same aerial image. Furthermore, the information processing device 100 of the embodiment generates a DTM that approximates the topography of the measurement target area specified by the user P2. Then, the information processing device 100 of the embodiment generates a CSM that represents the distribution of plant heights of only the plants in the measurement target area, based on the acquired or generated DSM and the generated DTM. This reduces errors caused by the slope of the topography of the measurement target area (field F) as well as errors caused by doming, which is thought to be due to the curved surface of the imaging lens of the imaging device C, and enables more accurate measurement of the heights of the plants (plant heights).

[0071] The information processing device 100 of the embodiment has been described as having a configuration in which the image acquisition unit 120 acquires aerial images, the three-dimensional data acquisition unit 140 acquires three-dimensional data, and the DSM acquisition unit 150 acquires a DSM. However, the information processing device 100 may also have a configuration in which, for example, the image acquisition unit 120 acquires aerial images, three-dimensional data, and a DSM. In this case, the three-dimensional data acquisition unit 140 and the DSM acquisition unit 150 may be omitted from the information processing device 100. In this case, the operation and processing of the information processing device may be equivalent to the operation and processing of the information processing device 100 of the above-described embodiment.

[0072] In the information processing device 100 according to the embodiment, the measurement target region setting unit 160, the analytical region setting unit 170, and the CSM generation unit 190 are each configured to display information (images) for the user P2 to set or check on a display device included in the terminal device T2. However, the information processing device 100 may also be configured to include, for example, a display control unit (not shown) that controls the display of images on the display device. In this case, the measurement target region setting unit 160, the analytical region setting unit 170, and the CSM generation unit 190 may each request (instruct) the display control unit (not shown) to display an image on the display device. For example, the measurement target region setting unit 160 may be configured to cause the display control unit (not shown) to read a DSM stored in the storage unit 130 and to instruct the display device included in the terminal device T2 to display an image of the read DSM. Similarly, the analytical region setting unit 170 and the CSM generation unit 190 may each be configured to read information from the storage unit 130 and instruct the display device included in the terminal device T2 to display the information. In this case, the operation and processing of the information processing device may be equivalent to the operation and processing of the information processing device 100 of the above-described embodiment.

[0073] In the above-described embodiment, the terminal device T1 controls the flight of the flying object FO and the capture of aerial images by the imaging device C, and the image acquisition unit 120 included in the information processing device 100 acquires the aerial images captured by the imaging device C via the communication unit 110 and the terminal device T1. However, the information processing device 100 may be configured to include, for example, a control unit that directly controls the flight of the flying object FO and the capture of aerial images by the imaging device C. In other words, the information processing device 100 may be configured to operate in cooperation with the flying object FO and the terminal device T1. In this case, the operation and processing of the information processing device may be equivalent to the operation and processing of the information processing device 100 and the terminal device T1 in the above-described embodiment.

[0074] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0075] 100 Information processing device 110 Communications Department 120 Image acquisition unit 130...Storage section 140...3D data acquisition unit 150...DSM acquisition department 160 Measurement target area setting unit 170...Analysis area setting department 180...DTM calculation section 190...CSM generation section TA···Measurement target area AA...Analysis area F···field FO...Flying object C. Imaging device T1, T2... Terminal equipment NW...Network

Claims

1. a digital ground surface model acquisition unit that acquires a digital ground surface model that is generated from three-dimensional data based on an image including a plant body photographed from above, the digital ground surface model representing the distribution of elevation values ​​at each point in the image; an analysis region setting unit that sets a plurality of analysis regions based on a measurement target region in the image; a digital terrain model generation unit that generates a digital terrain model representing a distribution of elevation values ​​excluding the plant bodies within an area including the measurement target area based on the elevation values ​​for each of the plurality of analysis areas included in the digital earth surface model; a crop surface model generation unit that generates a crop surface model that represents a distribution of elevation values ​​within the measurement target area as a distribution of heights of the plant bodies based on the digital earth surface model and the digital terrain model; An information processing device comprising:

2. the analysis region setting unit receives the input of the measurement target region and sets the plurality of analysis regions around the measurement target region. The information processing device according to claim 1 .

3. the digital terrain model generation unit generates the digital terrain model for each of the analysis areas by using an elevation value of a predetermined percentile value among the elevation values ​​included in the analysis area as a representative elevation value of the analysis area.

3. The information processing device according to claim 1.

4. the digital terrain model generation unit generates, as the digital terrain model, a plane or a curved surface that approximates the entire area of ​​the image by a least squares method from the elevation values ​​of each of the plurality of analysis areas; The information processing device according to any one of claims 1 to 3.

5. the digital terrain model generation unit generates the digital terrain model approximating a plane or a curved surface expressed by an n-th degree polynomial (n is a natural number of 1 to 3, including at least 0); The information processing device according to any one of claims 1 to 4.

6. the crop surface model generation unit causes a display device to display an image based on the crop surface model; The information processing device according to any one of claims 1 to 5.

7. The computer obtaining a digital earth surface model that is generated from three-dimensional data based on an image including a plant body photographed from above, and that represents the distribution of elevation values ​​at each point in the image; A plurality of analysis areas are set based on a measurement target area in the image; generating a digital terrain model that represents a distribution of elevation values ​​excluding the plant bodies within an area that includes the measurement target area based on the elevation values ​​for each of the plurality of analysis areas included in the digital earth surface model; generating a crop surface model that represents the distribution of elevation values ​​within the measurement target area as a distribution of heights of the plant bodies based on the digital earth surface model and the digital terrain model; Information processing methods.

8. On the computer, obtaining a digital earth surface model that represents the distribution of elevation values ​​at each point in an image that includes a plant body and that is generated from three-dimensional data based on the image; A plurality of analysis areas are set based on a measurement target area in the image; generating a digital terrain model representing a distribution of elevation values ​​excluding the plant bodies within an area including the measurement target area based on the elevation values ​​for each of the plurality of analysis areas included in the digital earth surface model; generating a crop surface model that represents a distribution of elevation values ​​within the measurement target area as a distribution of heights of the plant bodies based on the digital earth surface model and the digital terrain model; program.

Images