Information processing device, information processing method, program, and information processing system

Abstract

The object of the present invention is to enable high-precision measurement of a measurement object such as a farmland in a short time. To this end, a multispectral camera that captures images of a plurality of different wavelength bands is used to capture a portion of the range of the measurement object to obtain a processed sample image. The sample image as the processing object is calculated to generate a calculation result image that provides information for evaluating the entire measurement object.

Description

Technical Field

[0001] The present technology relates to an information processing device, an information processing method, a program, and an information processing system, and in particular, to a technology suitable for generating an evaluation result of a measurement object such as a vegetation state. Background Art

[0002] As a technology for generating required information by using an image captured by a camera, the following technology is known.

[0003] The following Patent Document 1 discloses a technology for collecting basic data by shooting a video while moving and performing image processing thereon to obtain altitude data.

[0004] The following patent document 2 describes a method for classifying vegetation in a forest area based on satellite data, as described below. According to the tree species, an upper limit and a lower limit or a threshold are set for the measured value of each wavelength. Then, a tree species object extraction map is created, the number of which corresponds to the number of tree species. In addition, a normalized difference vegetation index is generated for the area to be surveyed. Then, the tree species object extraction map is masked by using the vegetation index. Thus, the mountain forest part or the tree species distribution is extracted.

[0005] The following Patent Document 3 describes a method of detecting defects in a semiconductor wafer by scanning the semiconductor wafer in a direction different from the longitudinal direction of wires to detect defective elements and then scanning the semiconductor wafer in the longitudinal direction.

[0006] Citation List

[0007] Patent Literature

[0008] Patent Document 1: Japanese Patent Application Laid-Open No. 8-159762

[0009] Patent Document 2: Japanese Patent Application Laid-Open No. 2006-85517

[0010] Patent Document 3: Japanese Patent No. 4903469 Summary of the invention

[0011] Problems to be solved by the present invention

[0012] For example, there is a method of remotely sensing the state of vegetation by mounting an imaging device on a small aircraft and imaging the state of vegetation of plants while moving over a farmland.

[0013] As a result of generating evaluation results of the vegetation state or the like by using images captured by the camera in this manner, observation over a wide range can be performed efficiently.

[0014] Here, in the case where high-precision observation is desired, it is conceivable to perform imaging with higher resolution. However, performing high-resolution imaging over a wide range such as a farmland leads to degradation in measurement efficiency.

[0015] An object of the present technology is to enable highly accurate measurement to be performed efficiently while imaging and observing a measurement object.

[0016] Technical solutions to problems

[0017] An information processing apparatus according to the present technology includes a calculation unit that performs arithmetic processing on a sampling image as a processing object acquired by imaging a part of a measurement object using a multispectral camera, and generates a calculation result image serving as evaluation information of the measurement object.

[0018] A multispectral camera that captures and records multispectral images of electromagnetic waves in different wavelength bands is used to sample and image at least a portion of the entire range of the measurement object. For example, the entire measurement object area is divided into a plurality of regions, and an image of a portion of the range of each region is captured by the multispectral camera. The information processing device performs necessary calculation processing on the sampled image as a processing object obtained by such sampling imaging, and generates a calculation result image serving as one of the measurement results.

[0019] Note that the multispectral camera mentioned in this specification is a general term for cameras capable of capturing images of multiple wavelength bands. In addition to cameras generally referred to as multispectral cameras, examples of multispectral cameras include cameras referred to as hyperspectral cameras.

[0020] In the above-mentioned information processing device according to the present technology, it is conceivable that the calculation unit includes a filter generation unit that generates a filter image that can be used for filtering processing by using the sampling image.

[0021] The filter image is an image showing a portion having a specific composition or a specific state, and is an image corresponding to information that acts as a filter when filtering is performed. In other words, it can also be said that the filter image corresponds to the filter condition. That is, the calculation unit uses one or more of the sampled images as processing objects to generate a filter image for generating a calculation result image representing the measurement result.

[0022] In the above-mentioned information processing device according to the present technology, it is conceivable that the calculation unit includes an evaluation index calculation unit that generates an evaluation index image by using the sampling image.

[0023] The evaluation index is acquired as one of the evaluations of the measurement object. Examples of the evaluation index include a vegetation index, etc. The evaluation index image is an image representing a vegetation index, etc., for each pixel.

[0024] In the above-mentioned information processing device according to the present technology, it is conceivable that the evaluation index image is a vegetation index image indicating a vegetation index.

[0025] An image showing a vegetation index such as NDVI is generated.

[0026] In the above-mentioned information processing device according to the present technology, it can be imagined that the calculation unit includes: an evaluation index calculation unit, which generates an evaluation index image by using a sampling image; and a filter generation unit, which generates a filter image that can be used for filtering processing by using the evaluation index image.

[0027] The filter image (image corresponding to the filter condition) is generated by using an image (evaluation index image) representing an evaluation index (for example, a vegetation index).

[0028] In the above-mentioned information processing device according to the present technology, it can be imagined that the calculation unit includes: an evaluation index calculation unit, which generates an evaluation index image by using a sampling image; and a filter generation unit, which generates a filtered image that can be used for filtering processing by using a filtered image generated using the sampling image and a filtered image generated using the evaluation index image.

[0029] That is, the filter image is generated by using a plurality of filter images.

[0030] In the above-mentioned information processing device according to the present technology, it is conceivable that the calculation unit includes: a filter generation unit that generates a filter image that can be used for filtering processing; and a filter execution unit that generates a filter result image by using the filter image.

[0031] That is, filtering is performed according to the conditions represented by the filter image (the image corresponding to the filter conditions) to generate a filter result image.

[0032] In the above-mentioned information processing device according to the present technology, it can be imagined that the calculation unit includes: an evaluation index calculation unit, which generates an evaluation index image by using a sampling image; a filter generation unit, which generates a filtered image that can be used for filtering processing; and a filter execution unit, which generates a filtered result image of the evaluation index image by using the filtered image.

[0033] That is, the evaluation index image (for example, the vegetation index image) is filtered according to the condition represented by the filter image (the image corresponding to the filter condition) to generate a filtering result image.

[0034] In the above-mentioned information processing device according to the present technology, it can be imagined that the calculation unit includes: an evaluation index calculation unit, which generates an evaluation index image by using a sampling image; a filter generation unit, which generates a filter image that can be used for filtering processing; a filter execution unit, which generates a filter result image of the evaluation index image by using the filter image; and a presentation image generation unit, which generates a presentation image by using the filter result image.

[0035] The filtering result image is obtained by extracting pixels from the image before filtering according to the conditions represented by the filtering image, and may not be suitable for human visual recognition in some cases (the evaluation result may be difficult to understand). Therefore, an image in a state suitable for presentation to humans is generated.

[0036] In the above-mentioned information processing device according to the present technology, it is conceivable that the rendered image generation unit divides the filtering result image into areas and generates a rendered image in which each area is color-mapped.

[0037] The filtered result image is divided into a plurality of regions, and a color is assigned to each region, thereby generating a rendered image.

[0038] In the above-mentioned information processing device according to the present technology, it is conceivable that the presentation image generation unit divides the filtering result image into areas and generates a color-mapped presentation image by using a representative value of each area.

[0039] The representative value refers to the average value, maximum value, minimum value, centroid value, mode value, etc. of the field.

[0040] In the above-mentioned information processing device according to the present technology, it is conceivable that the presentation image generation unit divides the filtering result image into areas, and generates the presentation image by synthesizing the image in which each area is color-mapped and other images.

[0041] For example, an image to which a color is assigned for each area and other images are synthesized in a form such as superimposition or overwriting.

[0042] In the above-mentioned information processing device according to the present technology, it is conceivable that the other image is any one of a sampling image, an evaluation index image, a filtered image, or a filtering result image.

[0043] The image and the sampled image, etc., to which the color is assigned for each area, are synthesized in a superimposed or overwritten manner, for example.

[0044] In the above-mentioned information processing device according to the present technology, it can be imagined that the calculation unit also includes: an image synthesis unit, which synthesizes a low-resolution image and a presentation image, wherein the low-resolution image is obtained by imaging the measured object as an image with a resolution lower than that of the sampling image.

[0045] The presentation image is generated based on the filtering result image, and the filtering result image is based on the sampling image. That is, the presentation image is not based on the image of the entire measurement object, but is based on the image of a part of the measurement object. Therefore, the presentation image is synthesized with a low-resolution image obtained by imaging a wider range than the sampling image.

[0046] In the above-mentioned information processing device according to the present technology, it is conceivable that the image synthesis unit performs image synthesis by using correspondence information that associates the presentation image with a low-resolution image acquired by imaging a range including a sampling image as a source of the presentation image.

[0047] In the synthesis process, the rendered image of the imaging range of the sampled image is synthesized with the low-resolution image obtained by imaging a wider range including the imaging range of the sampled image. Therefore, the rendered image and the low-resolution image are synthesized by using corresponding information indicating a corresponding relationship (such as information about an imaging position, information about an imaging time, etc.).

[0048] In the above-mentioned information processing device according to the present technology, it is conceivable that the image synthesis unit performs synthesis to arrange the presentation image at a position corresponding to the imaging range of the sampling image as the source of the presentation image in the low resolution image.

[0049] For example, in the synthesis process, the sampled image is superimposed or overwritten with the rendered image in the low-resolution image. In this case, the rendered image is arranged at a position in the low-resolution image corresponding to the imaging range of the sampled image. In other words, the rendered image and the low-resolution image are aligned and attached together.

[0050] In the above-mentioned information processing device according to the present technology, it is conceivable that the image synthesis unit performs synthesis reflecting the representative value of the presentation image in the low-resolution image.

[0051] The representative value of the presentation image refers to the average value, maximum value, minimum value, centroid value, mode value, etc. of the presentation image. The representative value of the presentation image is synthesized on the low-resolution image.

[0052] The information processing method according to the present technology is an information processing method to be performed by an information processing device, the method including: performing arithmetic processing on a sample image as a processing object acquired by imaging a part of a measurement object using a multispectral camera; and performing a process of generating a calculation result image serving as evaluation information of the entire measurement object. As a result, evaluation information on the measurement object can be efficiently generated in the information processing device.

[0053] The program according to the present technology is a program that causes an information processing device to perform the processing of the above-mentioned method. This facilitates the realization of a computer device that efficiently generates highly accurate evaluation information.

[0054] An information processing system according to the present technology includes: a multispectral camera that performs imaging to capture images of a plurality of different wavelength bands; and the above-mentioned information processing device. This makes it possible to construct a system suitable for generating evaluation information of the entire measurement object.

[0055] Effects of the Invention

[0056] According to the present technology, high-precision and in-depth analysis can be achieved by using high-resolution information without increasing measurement time or data volume.

[0057] Note that the effects described here are not necessarily restrictive, and any of the effects described in the present disclosure can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is an explanatory diagram of remote sensing to be performed in an embodiment of the present technology.

[0059] Figure 2 is a block diagram of the system configuration according to the first embodiment.

[0060] Figure 3 is a block diagram of a computer device according to an embodiment.

[0061] Figure 4 is an explanatory diagram of a high-resolution image and a low-resolution image according to an embodiment.

[0062] Figure 5 It is an explanatory diagram of the correspondence information according to the embodiment.

[0063] Figure 6 is a flowchart of a process of the information processing apparatus according to the embodiment.

[0064] Figure 7 is a flowchart of a process of the information processing apparatus according to the embodiment.

[0065] Figure 8 It is an explanatory diagram of the calculation of the vegetation index according to the embodiment.

[0066] Fig. 9 It is an explanatory diagram of filtering generation according to the embodiment.

[0067] Fig.10 It is an explanatory diagram of filtering execution according to the embodiment.

[0068] Fig.11 : is a flowchart of imaging processing example 1 according to the embodiment.

[0069] Fig.12 1 is an explanatory diagram of imaging processing example 1 according to the embodiment.

[0070] Fig.13 is a flowchart of imaging processing example II according to the embodiment.

[0071] Fig.14 is an explanatory diagram of imaging processing example II according to the embodiment.

[0072] Fig.15 is a flowchart of imaging processing example III according to the embodiment.

[0073] Fig.16 is an explanatory diagram of imaging processing example III according to the embodiment.

[0074] Fig.17 : is a flowchart of synthesis processing example I according to the embodiment.

[0075] Fig.18 1 is an explanatory diagram of synthesis processing example I according to the embodiment.

[0076] Fig.19 is a flowchart of synthesis processing example II according to the embodiment.

[0077] Fig. 20 is an explanatory diagram of synthesis processing example II according to the embodiment.

[0078] Fig.21 is an explanatory diagram of synthesis processing example II according to the embodiment.

[0079] Fig. 22 is a flowchart of synthesis processing example III according to the embodiment.

[0080] Fig.23 is an explanatory diagram of synthesis processing example III according to the embodiment.

[0081] Fig.24 is a flowchart of synthesis processing example IV according to the embodiment.

[0082] Fig.25is an explanatory diagram of synthesis processing example IV according to the embodiment.

[0083] Fig.26 is a block diagram of a system configuration according to the second embodiment.

[0084] Fig. 27 is a block diagram of a control configuration of an aircraft according to an embodiment.

[0085] Fig.28 is an explanatory diagram of an imaging interval between high-resolution images according to an embodiment.

[0086] Fig.29 is a flowchart of setting a scanning interval and a coverage rate according to an embodiment.

[0087] Fig.30 It is an explanatory diagram of a setting example according to the embodiment.

[0088] Fig.31 is a flow chart of flight control according to an embodiment.

[0089] Fig.32 is an explanatory diagram of flight control according to the embodiment.

[0090] Fig.33 is a flow chart of flight control according to an embodiment.

[0091] Fig.34 is an explanatory diagram of flight control according to the embodiment.

[0092] Fig.35 is an explanatory diagram of flight control according to the embodiment.

[0093] Fig.36 is an explanatory diagram of flight control according to the embodiment.

[0094] Fig.37 is an explanatory diagram of flight control according to the embodiment.

[0095] Fig.38 is an explanatory diagram of flight control according to the embodiment.

[0096] Fig.39 is a flow chart of flight control according to an embodiment.

[0097] Fig.40 is an explanatory diagram of the configuration of a flight plan processing unit according to the embodiment.

[0098] Fig.41 is an explanatory diagram of the configuration of a flight plan processing unit according to the embodiment. DETAILED DESCRIPTION

[0099] Hereinafter, each embodiment is described in the following order.

[0100] <1. System Configuration of First Embodiment>

[0101] <2. Generation of evaluation information based on sample images>

[0102] <3. Generation of Presentation Image>

[0103] <4. Synthesis with low-resolution images>

[0104] <5. System Configuration of Second Embodiment>

[0105] <6. Flight plan control>

[0106] <7. Summary and Modifications>

[0107] <1. System Configuration of First Embodiment>

[0108] In the embodiment, a case where sensing of the vegetation state of a farmland is performed will be described as an example.

[0109] For example, Figure 1 As shown, it is assumed that the vegetation of the farmland 300 as the measurement object is remotely sensed by using the imaging device 250 mounted on the aircraft 200 .

[0110] The small aircraft 200 can move over the farmland 300 through, for example, wireless control by an operator, automatic control, etc.

[0111] The imaging device 250 is provided on the aircraft 200 so as to capture downward view images, for example. When the aircraft 200 moves over the farmland 300 along a path DR set, for example, as indicated by an arrow, the imaging device 250 captures still images, for example, periodically.

[0112] Here, the imaging device 250 is a multispectral camera that captures images of multiple wavelength bands. For example, a camera that captures near infrared (NIR) images and red (R) images may be used, and the normalized difference vegetation index (NDVI) may be calculated based on these images. NDVI is an index that indicates the distribution state and activity of vegetation.

[0113] Of course, this is just an example. It is sufficient as long as the imaging device 250 captures images of two or more wavelength bands. A camera that captures a green (G) image, a blue (B) image, an ultraviolet region image, etc. may also be used.

[0114] Note that the imaging device 250 only needs to capture images of two or more wavelength bands. Therefore, multiple cameras can also be installed. For example, two cameras, a NIR image capture camera and an R image capture camera, can be installed to act as a multi-spectral camera as a whole. In any case, in the case of the present embodiment, it is sufficient for the aircraft 200 to be equipped with a necessary number of cameras of the necessary type for capturing images of multiple wavelength bands. Regardless of the form of the camera, these cameras are collectively referred to as "multi-spectral cameras".

[0115] Furthermore, the imaging device 250 is capable of capturing a high-resolution sampling image (hereinafter also referred to as a “high-resolution image”) HR and a low-resolution image LR.

[0116] The high-resolution image HR is, for example, a captured image of a range A-HR indicated by a dotted line. The low-resolution image LR is a captured image of a range A-LR indicated by a dotted line.

[0117] The high-resolution image HR is an image obtained by imaging a part of the region imaged as the low-resolution image LR.

[0118] The high-resolution image HR and the low-resolution image LR as described above may be captured by, for example, installing a plurality of cameras for high-resolution imaging and low-resolution imaging as the imaging device 250 on the aircraft 200 .

[0119] Note that the high-resolution image HR and the low-resolution image LR may be captured simultaneously (including at approximately the same time) or may be captured at different timings. For example, the high-resolution image HR may be captured in the first flight on the path DR, the low-resolution image LR may be captured in the second flight on the path DR, and so on. Alternatively, the low-resolution camera and the high-resolution camera may be interchanged to capture images in two flights.

[0120] Furthermore, tag information is attached to an image obtained as a result of imaging using the imaging device 250. The tag information contains, for example, imaging date / time information, location information (latitude / longitude information) as global positioning system (GPS) data, imaging device information (such as individual identification information and model information of a camera), and information about each image data (information about image size, wavelength, imaging parameters, etc.).

[0121] Note that the position information and the imaging date / time information also serve as correspondence information that associates the low-resolution image LR and the high-resolution image HR.

[0122] The image data (high resolution image HR and low resolution image LR) obtained by the imaging device 250 mounted on the aircraft 200 as described above and the label information are sent to the information processing device 1. The information processing device 1 uses the image data and the label information to generate evaluation information about the farmland 300 as the measurement object. In addition, the information processing device 1 performs a process of presenting the evaluation result as an image to the user.

[0123] The information processing device 1 is realized as, for example, a personal computer (PC), a field programmable gate array (FPGA), or the like.

[0124] Note that Figure 1 The information processing device 1 is provided separately from the imaging device 250. However, an arithmetic unit (microcomputer or the like) serving as the information processing device 1 may be provided in a unit including the imaging device 250.

[0125] Will refer to Figure 2 The configuration of the information processing apparatus 1 is described. Note that Figure 2 The configuration of the information processing device 1 is shown, wherein, for example, a calculation function by software and a buffer function using a storage area are shown as modules.

[0126] The information processing device 1 includes a data input unit 21, an input buffer 22, a vegetation index calculation unit 23, a filter generation unit 24, a calculation result buffer 25, a filter execution unit 26, a presentation image generation unit 27, an output buffer 28, an image synthesis unit 29, an image output unit 30 and a user input unit 31.

[0127] Note that as the calculation unit 2, a vegetation index calculation unit 23, a filter generation unit 24, a filter execution unit 26, a rendering image generation unit 27, and an image synthesis unit 29 are shown. The calculation unit 2 is a specific example of the calculation unit described in the claims. However, the calculation unit of the present disclosure does not necessarily have all the functions shown as the calculation unit 2. It is sufficient to have at least one of these functions.

[0128] Each unit included in the calculation unit 2 may be configured by hardware. However, in the present embodiment, each unit included in the calculation unit 2 is described as a function to be realized by software in the computer device 100 described later.

[0129] The data input unit 21 obtains image data related to images (high resolution image HR and low resolution image LR) captured by the imaging device 250 mounted on the above-mentioned aircraft 200, and label information serving as correspondence information P, for example.

[0130] Then, the data input unit 21 causes the input buffer 22 to store the high resolution image HR, the low resolution image LR, and the correspondence information P.

[0131] Note that the image data obtained by the data input unit 21 refers to an image data file sent from the imaging device 250 or the relay device by wired transmission or wireless transmission and received by the information processing device 1, or an image data file recorded on a recording medium on the imaging device 250 side or recorded on a recording medium by another recording device and obtained by the information processing device 1 when the information processing device 1 runs the recording medium.

[0132] In addition, the data input unit 21 can obtain image data related to the captured image (high resolution image HR and low resolution image LR) in real time (at the time of imaging) or at a later time. It is sufficient to store the image data in the input buffer 22 at least when the measurement result is calculated by the information processing device 1.

[0133] Furthermore, in the present embodiment, the data input unit 21 obtains an image data file of an image acquired by the imaging device 250 as a multispectral camera. For example, the image data file contains measurement images of two or more wavelengths as a high-resolution image HR acquired by imaging of a single range.

[0134] The vegetation index calculation unit 23 generates an evaluation index image (for example, a vegetation index image) by using the high-resolution image HR which is a sampling image of the farmland 300. Specifically, it is conceivable to generate an NDVI image.

[0135] The filter generation unit 24 generates a filter image that can be used for filtering processing using the high-resolution image HR that is a sampling image of the farmland 300 .

[0136] The filter image is an image corresponding to information (filtering conditions) that acts as a filter when filtering is performed. In other words, the filter image is an image showing a portion having a specific composition or a specific state, and is an image showing a portion to be extracted in the filtering process.

[0137] That is, the filter generation unit 24 generates filter conditions for generating a calculation result image showing the measurement result using one or more high-resolution images HR.

[0138] Note that, as will be described later, the filter generation unit 24 generates a filter image from an image of a specific wavelength range of the high resolution image HR or from a vegetation index image such as an NDVI image. Alternatively, another filter image is generated using the generated filter image in some cases.

[0139] The filter execution unit 26 generates a filter result image using the filter image generated by the filter generation unit 24 .

[0140] For example, the filtering execution unit 26 filters the vegetation index image by using the filtering image to generate a filtering result image.

[0141] Furthermore, the filtering execution unit 26 filters the image of the specific wavelength range of the input high resolution image HR by using the filter image to generate a filtering result image.

[0142] In addition, the filtering execution unit 26 may also filter the filtering image by using another filtering image to generate a filtering result image.

[0143] The calculation result buffer 25 is used to store the vegetation index image generated by the vegetation index calculation unit 23 .

[0144] Furthermore, the calculation result buffer 25 is also used to store the filter image generated by the filter generation unit 24 .

[0145] In addition, the calculation result buffer 25 is also used to store the filtering result image generated by the filtering execution unit 26 .

[0146] The presentation image generation unit 27 generates a presentation image by using the filtering result image.

[0147] As a result of extracting pixels from the image before filtering according to the conditions represented by the filter image, a filter result image is acquired. It may be difficult for a person to visually recognize the filter result image. Therefore, the presentation image generation unit 27 performs processing to generate an image in a state suitable for presentation to a person. As will be described later, as a process for generating a presentation image, screen division and color mapping are performed.

[0148] The image synthesizing unit 29 performs a synthesizing process for synthesizing the low resolution image LR and the presentation image generated by the presentation image generating unit 27 based on the high resolution image HR. The image synthesizing unit 29 refers to the correspondence information P for the synthesizing process.

[0149] The output buffer 28 is used to store the presentation image generated by the presentation image generation unit 27 .

[0150] In addition, the output buffer 28 is also used to store the synthesized image generated by the image synthesis unit 29.

[0151] The image output unit 30 outputs the image data (output image Pout) read from the output buffer 28. For example, the presentation image generated by the presentation image generation unit 27 and the synthesized image generated by the image synthesizing unit 29 are read from the output buffer 28 and output by the image output unit 30 as the output image Pout.

[0152] For example, the output image is displayed on a monitoring display, sent to another information processing device, or stored in a storage medium by a storage device.

[0153] The output image is an image showing the evaluation result based on the measurement (remote sensing) of the farmland 300. The user can understand, for example, the state of the farmland 300 and the state of plant growth by viewing the presented image and the synthesized image.

[0154] Note that although not shown in the figure, the vegetation index image, the filtered image, and the filtered result image stored in the calculation result buffer 25 can be output as the output image Pout to be displayed, sent, or stored, for example, as a processing result of the image output unit 30.

[0155] The user input unit 31 receives various user operation inputs, setting inputs, and the like.

[0156] For example, on / off of each processing function in the calculation unit 2 , processing parameters, selection of measurement details, selection of an output image, and the like are designated by user operations.

[0157] By having, for example, Figure 3 The computer device 100 composed of the hardware shown in the figure implements the following Figure 2 The information processing device 1 having the above-mentioned functional configuration is shown.

[0158] like Figure 3 As shown, the computer device 100 includes a central processing unit (CPU) 51 , a read only memory (ROM) 52 , and a random access memory (RAM) 53 .

[0159] The CPU 51 performs various processes according to the program stored in the ROM 52 or the program loaded from the storage unit 59 into the RAM 53. The RAM 53 also stores data and the like necessary for the CPU 51 to perform various processes as appropriate.

[0160] The CPU 51, the ROM 52, and the RAM 53 are connected to each other via a bus 54. To the bus 54, an input / output interface 55 is also connected.

[0161] The input / output interface 55 can be connected to a display 56, an input unit 57, a speaker 58, a storage unit 59, a communication unit 60, etc. The display 56 includes a liquid crystal panel, an organic electroluminescent (EL) panel, etc. The input unit 57 includes a keyboard, a mouse, etc. The storage unit 59 includes an HDD, etc.

[0162] The display 56 may be formed integrally with the computer device 100, or may be formed as a separate device. For example, the display 56 displays the output image Pout.

[0163] The input unit 57 refers to an input device to be used by a user who uses the computer device 100 .

[0164] The communication unit 60 performs communication processing via a network including the Internet, or communication with peripheral devices of each unit. For example, the communication unit 60 can also perform communication with the imaging device 250 and the like.

[0165] The input / output interface 55 is also connected to a drive 61 in which a memory card 62 is mounted, as necessary, so that a computer program read from the memory card 62 is installed in the storage unit 59 as necessary, or data processed in the CPU 51 is stored in the memory card 62. Of course, the drive 61 may be a recording / reproducing drive for a removable storage medium such as a magnetic disk, an optical disk, or a magneto-optical disk.

[0166] This hardware configuration enables processing by the calculation unit 2 of the information processing apparatus 1 according to the embodiment, that is, by Figure 2 The vegetation index calculation unit 23, the filter generation unit 24, the filter execution unit 26, the presentation image generation unit 27, and the image synthesis unit 29 shown in the figure are processed. In addition, this hardware configuration also enables processing by the data input unit 21 and the image output unit 30. That is, the software to be started by the CPU 51 realizes the processing of these units. The program included in the software is downloaded from the network or read from the removable storage medium and installed in Figure 3 Alternatively, the program may be stored in advance in a HDD or the like as the storage unit 59. Then, when the program is started in the CPU 51, the function of each unit described above is realized.

[0167] Furthermore, the input buffer 22 , the calculation result buffer 25 , and the output buffer 28 are realized by using, for example, a storage area of ​​the RAM 53 or a storage area of ​​the storage unit 59 .

[0168] The user input unit 31 performs a function of the CPU 51 to detect an operation input by the input unit 57 .

[0169] As the high-resolution image HR and the low-resolution image LR, the image data file, the corresponding information P, etc. are received by the communication unit 60 or read from the storage medium by the driver 61, and stored in, for example, the storage unit 59. The CPU 51 having the function as the data input unit 21 obtains an image data file required for generating an evaluation result from among the image data files and the like thus input, and treats the obtained image data file as a processing object.

[0170] The image output by the image output unit 30 is displayed on the display 56, stored in the storage unit 59, sent to an external device by the communication unit 60, or stored in a storage medium such as a memory card 62 through the drive 61. Thus, the output of the image is performed in various forms.

[0171] Note that the information processing apparatus 1 according to the embodiment is not limited to being configured as Figure 3 The information processing device (computer device) 100 of a single device composed of the hardware shown. On the contrary, the system may be constructed so that a plurality of computer devices are included in the system. The plurality of computer devices may form a system through a LAN or the like, or may be located at a remote location and connected through a virtual private network (VPN) or the like using the Internet or the like. The plurality of computer devices may include computer devices that can be used in cloud computing services.

[0172] In addition, you can Figure 3 The computer device 100 shown is implemented as a personal computer such as a stationary computer or a notebook computer, or a portable terminal such as a tablet terminal or a smart phone. In addition, electronic devices having functions as the computer device 100 such as a measuring device, a television device, a monitor device, an imaging device, and a facility management device may also be equipped with the information processing device 1 according to the present embodiment.

[0173] <2. Generation of evaluation information based on sample images>

[0174] An example of evaluation information generation processing to be performed by the information processing device 1 based on the sample images is described below.

[0175] First, refer to Figure 4 Describe the imaging range of the high-resolution image HR and the low-resolution image LR.

[0176] In the case of imaging the farmland 300 by the aircraft 200 , a plurality of images are captured in each of longitudinal and lateral directions in a form having a certain margin to form a single large image by stitching.

[0177] exist Figure 4 Regions #1 to #5, which are imaged as low-resolution images LR#1 to LR#5, are illustrated in A. Each of regions #1 to #5 is a portion of the farmland 300 as a measurement object, and represents a range to be imaged as a single low-resolution image LR. Regions #1 to #5 partially overlap with each other. The overlap between regions #1 to #5 corresponds to a range to be imaged as an edge in the low-resolution images LR#1 to LR#5.

[0178] Here, only regions #1 to #5 are shown. Figure 1 During the flight on the path DR, the image capture continues, thereby also capturing low-resolution image LR#5 and subsequent images. As a result, the required number of low-resolution images LR covering the entire farmland 300 are acquired. Then, the process of stitching the low-resolution images LR is performed. Thus, an image of the entire farmland 300 can be generated.

[0179] Note that the areas #1, #2, . . . in the farmland 300 are determined by the viewing angle of the imaging device 250 and the flight altitude of the aircraft 200, rather than being areas divided in a fixed manner.

[0180] At the same time, the high-resolution image HR is captured in a discontinuous manner in one or both of the longitudinal direction and the transverse direction. Figure 4 B shows high-resolution images HR#1 to HR#5, which are images acquired as a result of imaging discrete sampling areas #1C to #5C, respectively.

[0181] Here, sampling area #1C corresponds to the area approximately located Figure 4 The sampling area #1C is an area in the center of the area #1 of A. However, the sampling area #1C is not necessarily located in the center thereof, and is an area located at least within the area #1.

[0182] Similarly, sampling areas #2C to #5C are areas included in areas #2 to #5, respectively (for example, areas located approximately in the center).

[0183] As described above, a single high resolution image HR is acquired by imaging the sampling region included in the imaging region of the corresponding low resolution image LR.

[0184] Note that high-resolution images HR are also captured sequentially during the flight on the path DR above the farmland 300. However, even if the high-resolution images HR are acquired at a certain point in time after the entire farmland is imaged, it is not possible to generate a high-resolution image of the entire farmland 300 by stitching. This is because the high-resolution images HR are images of individual discrete sampling areas.

[0185] In this embodiment, a high-resolution image HR is acquired by imaging such discrete sampling areas, so that the large farmland 300 can be efficiently imaged, and the sampling areas #1C, #2C, ... corresponding to areas #1, #2, ... respectively can be finely analyzed and the evaluation results calculated by using the high-resolution image HR.

[0186] Next, in Figure 5 An example of the corresponding information P is shown in FIG.

[0187] The correspondence information P is information that associates the imaging region of the high resolution image HR with the imaging region of the low resolution image LR as described above.

[0188] For example, consider a case where a camera for a low-resolution image LR and a camera for a high-resolution image HR are installed as the imaging device 250 and simultaneously capture images. Figure 5 As shown, the correspondence information P in this case is a file including a table that records file names and coordinates indicating the range of the viewing angle of the high resolution image HR in the low resolution image LR to associate imaging results with each other.

[0189] For example, the table shown as the corresponding information P in the figure shows a set of image data files File1 and File2. Image data file File1 is an image data file of the high-resolution image HR#1. Image data file File2 is an image data file of the low-resolution image LR#1. In addition, the start coordinates Start#1 (x1, y1) are associated with the end coordinates End#1 (x2, y2). The start coordinates Start#1 (x1, y1) and the end coordinates End#1 (x2, y2) represent the range of the viewing angle of the high-resolution image HR in the low-resolution image LR.

[0190] Similarly, the table associates the image data file File3 with the image data file File4, and also associates the start coordinates Start#2 (x1, y1) with the end coordinates End#2 (x2, y2). The image data file File3 is the image data file of the high-resolution image HR#2. The image data file File4 is the image data file of the low-resolution image LR#2. The start coordinates Start#2 (x1, y1) and the end coordinates End#2 (x2, y2) represent the range of the viewing angle of the high-resolution image HR in the low-resolution image LR.

[0191] Note that there is a technology for recording information about the location where the image was captured in the image file by using GPS. In this case, its location information can also be used as the corresponding information between the imaging range of the high-resolution image HR (sampling area #1C, etc.) and the imaging range of the low-resolution image LR (area #1, etc.).

[0192] Furthermore, in the case where the low-resolution image LR and the high-resolution image HR are captured simultaneously in a single flight, imaging date / time information is attached to each image data so that the imaging date / time information can be used as the corresponding information P. That is, based on the imaging date / time information, the imaging range (sampling area #1C, etc.) of the high-resolution image HR is associated with the imaging range (area #1, etc.) of the low-resolution image LR regarded as an image captured at the same time as the high-resolution image HR.

[0193] A processing example of the information processing device 1 using the high resolution image HR, the low resolution image LR, and the correspondence information P as described above is described below.

[0194] Figure 6 and 7 The information processing device 1 (CPU 51) is shown. Figure 2 An example of the processing performed by each processing function is shown.

[0195] exist Figure 6 and 7 In the process of, the information processing device 1 determines whether to perform each process in steps S100, S110, S120, S130, S140, S150, and S160. In the case of starting the process of the captured image of the specific measurement object, each process is sequentially performed in a predetermined order, in an order freely set for the process, in response to an external trigger or reception of information, or individually in response to a user operation. In each of the above steps, it is determined whether it is time to perform each process according to the program, sequence setting, user operation, etc.

[0196] When one or both of the image data files of the high-resolution image HR and the low-resolution image LR are input to the data input unit 21 or the corresponding information P is input to the data input unit 21, the information processing device 1 proceeds from step S100 to S101 and causes the input data to be stored in the input buffer 22.

[0197] At the timing when the function of the vegetation index calculation unit 23 performs vegetation index calculation, the information processing device 1 proceeds from step S110 to S111 .

[0198] In step S111, the information processing device 1 (vegetation index calculation unit 23) reads the required high-resolution image HR to be processed from the input buffer 22, performs vegetation index calculation in step S112, and stores the vegetation index image as the calculation result in the calculation result buffer 25 in step S113.

[0199] Figure 8 A specific example of the processing in steps S111 to S113 is shown in FIG.

[0200] For example, the vegetation index calculation unit 23 reads an image (R image) with a wavelength of 660 nm and an image (NIR image) with a wavelength of 800 nm from the input buffer 22. Then, the vegetation index calculation unit 23 generates an NDVI image based on the R image and the NIR image. For example, NDVI is calculated as NDVI=(1-(R / NIR)) / (1+(R / NIR)). The NDVI image is an image having pixel values ​​corresponding to the NDVI values ​​calculated for each corresponding pixel of the R image and the NIR image. Figure 8 The NDVI image is schematically shown as a shadow image. The NDVI image acquired in this manner is stored in the calculation result buffer 25 .

[0201] Note that although the following describes an example of acquiring NDVI as a vegetation index, this is merely an example, and it is also conceivable to acquire any other vegetation index as described later. Of course, in the case of acquiring another vegetation index, the image (wavelength range, etc.) to be read from the input buffer 22 may be different.

[0202] When the filter generation unit 24 generates a filter image, the information processing device 1 Figure 6 Step S120 in the process proceeds to S121.

[0203] In step S121, the information processing device 1 (filter generation unit 24) reads the desired high resolution image HR to be processed from the input buffer 22, and generates a filter image (an image showing a portion having a specific composition or a specific state) for filtering in step S122. Then, the information processing device 1 stores the filter image in the calculation result buffer 25 in step S123.

[0204] exist Fig. 9 A specific example of the processing in steps S121 to S123 is shown in . Here, an example of generating three filter images FP1, FP2, and FP3 by using an image (NIR image) with a wavelength of 800 nm and an NDVI image is shown.

[0205] The filtered image FP1 is an image showing the sunny part of the high-resolution image HR. For example, the filtered image FP1 is an image generated as a result of extracting pixels whose NIR values ​​(pixel values) are equal to or greater than a specific value from the NIR image. For example, only the sunny part in the image is represented as a high-brightness pixel in the figure. The filtered image FP1 generated in this way is stored in the calculation result buffer 25.

[0206] The filtered image FP2 is an image obtained by detecting a portion of the high-resolution image HR corresponding to the vegetation portion. The filtered image FP2 is an image in which the plant portion in the high-resolution image HR is represented as a high-brightness pixel (e.g., a white pixel) and other portions of the high-resolution image HR (e.g., a soil portion) are represented as low-brightness pixels (e.g., a black pixel).

[0207] First, an NDVI image is generated or obtained. For example, the filter generation unit 24 reads the NIR image and the R image from the input buffer 22. Then, the vegetation index calculation unit 23 generates an NDVI image based on the R image and the NIR image. Note that in the case where the NDVI image is generated by the vegetation index calculation unit 23 and stored in the calculation result buffer 25, it is only necessary to read the NDVI image from the calculation result buffer 25.

[0208] Then, pixels whose NDVI values ​​are equal to or greater than a certain value are extracted from the NDVI image. As a result, the filtered image FP2 is an image in which only the vegetation portion is represented as high-brightness pixels, as shown in the figure. The filtered image FP2 generated in this way is stored in the calculation result buffer 25.

[0209] The filtered image FP3 is an image obtained by detecting the parts corresponding to both the vegetation part and the sunny part in the high-resolution image HR. The filtered image FP3 is generated by using the filtered images FP1 and FP2. The filtered image FP3 is an image representing the specific parts corresponding to both the vegetation part and the sunny part as high-brightness pixels (e.g., white pixels) and the other parts as low-brightness pixels (e.g., black pixels).

[0210] In this case, it is sufficient to determine the high-brightness pixel using the "AND" condition for the corresponding pixels (pixels at the same position) of the filter pixels FP1 and FP2. That is, when the high-brightness pixel is set to "1" and the low-brightness pixel is set to "0" in the filter pixels FP1 and FP2, it is determined whether the pixel is "1" or "0" according to the "AND" condition for each pixel position in the filter images FP1 and FP2 to determine the corresponding pixel of the filter image FP3. As a result, as an image in which the parts corresponding to both the vegetation part and the sunny part are represented as high-brightness pixels, the filter image FP3 is obtained, as shown in the figure. The filter image FP3 generated in this way is stored in the calculation result buffer 25.

[0211] Here, three filter images FP1, FP2, and FP3 are described as an example. However, it is conceivable to generate more different filter images according to various filtering purposes, as will be described later.

[0212] That is, the filter image is an image showing a portion having a specific composition or a specific state, and is an image showing a portion to be extracted in the filtering process. Thus, it is conceivable that various filter images are generated according to the filtering conditions.

[0213] At the timing of filtering by the function of the filter execution unit 26, the information processing device 1 Figure 6 Step S130 in proceeds to S131.

[0214] In step S131, the information processing device 1 (filter execution unit 26) reads the vegetation index image and the filter image to be processed from the calculation result buffer 25. Then, in step S132, the information processing device 1 performs filtering processing. The filtering processing is a process of extracting a corresponding portion (pixel) from the vegetation index image by using a filter image showing a portion having a specific composition or a specific state.

[0215] Then, in step S133 , the information processing device 1 stores the image as the filtering result (filtering result image) in the calculation result buffer 25 .

[0216] Fig.10 A specific example of the processing in steps S131 to S133 is shown in . Here, an example is shown in which the NDVI image and the filter image FP3 as the vegetation index image are read from the calculation result buffer 25 and filtered.

[0217] As described above, the filter image FP3 is an image obtained by detecting portions corresponding to both the vegetation portion and the sunny portion in the high-resolution image HR as an image showing a portion having a specific composition or a specific state.

[0218] The filtering process is a process of extracting pixels corresponding to the portion shown in the filter image FP3 from the NDVI image. As a result, as shown in the figure, an image of the filtering result is acquired. The filtering result image is stored in the calculation result buffer 25.

[0219] Finally, the filtering result image is an image obtained by extracting a portion to be measured or a portion suitable for measurement from the high resolution image HR.

[0220] For example, there is a technique that measures the spectrum of the light source (daylight) while measuring with a camera, and corrects the result of the measurement with the camera to reduce the fluctuation of the spectrum of the light source when measuring a vegetation index (e.g., NDVI). If the shaded portion (the portion shaded mainly by plant leaves, i.e., the shadow with a complex light source spectrum affected by leaf-specific light absorption and reflection) can be removed, then measurement can be performed with higher accuracy. The filtered result image shown in the figure is an image obtained by extracting pixels that satisfy the conditions for such high-precision measurement.

[0221] Of course, it is conceivable to perform various filtering processes according to the purpose of measurement. Examples of the various filtering processes include, for example, filtering of the vegetation index image by using the above-mentioned filter images FP1 and FP2.

[0222] It is also conceivable to filter by using other filtered images. For example, imagine a filtered image showing shaded areas. Shaded areas (especially those located below plant communities) may remain in the shade all day long. Thus, by measuring only the sunlit areas and not the shaded areas in the measurement of the photosynthetic potential of the plant community, a more accurate measurement can be achieved. With this in mind, as an example, it is also conceivable to filter a vegetation index image such as an NDVI image by using a filtered image showing the location of the shaded areas.

[0223] The filtering result image acquired in the filtering process as described above is an image showing a portion of a specific composition or a specific state of the vegetation index, and shows one measurement result.

[0224] Such a filtering result image may be stored in an appropriate recording medium as a measurement result, or may be read from the calculation result buffer 25 and output from the image output unit 30 .

[0225] Of course, the filtering result image may be stored together with the original high-resolution image HR, etc. Alternatively, only the filtering result image may be stored. Furthermore, a configuration may be adopted in which the user can select to store both the high-resolution image HR, etc. and the filtering result image or to store only the filtering result image.

[0226] However, if Fig.10 As shown, there are cases where the filtering result image is not very suitable for the user as an image showing the result of measuring the measurement object. That is, it may be difficult for the human eye to understand what the image represents.

[0227] Therefore, in the present embodiment, the processing of the presentation image generation unit 27 enables generation and output of a presentation image so that a person can more easily understand the measurement result when viewing the presentation image.

[0228] <3. Generation of Presentation Image>

[0229] exist Figure 6 and 7 In the process of Figure 7 Step S140 in proceeds to S141.

[0230] In step S141, the information processing device 1 (presentation image generation unit 27) reads the filter result image to be processed and the like from the calculation result buffer 25. Then, the information processing device 1 performs presentation image generation processing in step S142, and stores the generated presentation image in the output buffer 28 in step S143.

[0231] Hereinafter, as examples of the presentation image generation process of step S142 , presentation image generation process examples I, II, and III will be described.

[0232] First, refer to Fig.11 and 12 Describe the presentation image generation processing example I.

[0233] Presentation image generation processing example 1 is an example of generating an image in which a filter result image is segmented into specific units and a color is assigned to a representative value of a pixel included in each segmented region (color mapping is applied).

[0234] In this case, Fig.11 In step S201 of the above, the information processing device 1 divides the filtered image into regions. For example, Fig.12 An example of segmentation into 16 areas is shown.

[0235] Then, the information processing device 1 Fig.11 In step S202, the variable n is set so that n=1, and in step S208, the processing in steps S203 to S206 is performed while the variable n is incremented, until in step S207, the variable n reaches a maximum value so that n=nmax.

[0236] Note that the maximum value nmax of the variable n corresponds to the number of regions set in step S201. For example, in the case where the filter result image is divided into 16 regions, the maximum value nmax is set so that nmax=16.

[0237] In step S203 , the information processing device 1 checks whether or not there is a pixel extracted by the filtering process in the n-th region among the segmented regions, thereby branching the process.

[0238] When there are extracted pixels, the information processing device 1 calculates representative values ​​of pixels in the n-th region in step S204 .

[0239] The representative value refers to the average value, maximum value, minimum value, centroid value, mode value, etc. of the vegetation index of pixels existing in the area. Here, for example, the average value is used as the representative value.

[0240] In step S205, the information processing device 1 performs "color allocation" by using the calculated representative value (average value). Here, "color allocation" means allocating a color selected from the corresponding colors preset for each numerical range according to the calculated average value to the relevant pixel.

[0241] For example, colors are set for NDVI values ​​as follows.

[0242] - Less than 0.700: Dark green

[0243] -0.700 to 0.710: Green

[0244] -0.710 to 0.720: yellow-green

[0245] -0.720 to 0.745: Yellow

[0246] -0.746 to 0.760: Orange

[0247] -0.761 to 0.770: Dark Orange

[0248] -0.771 to 0.799: Red

[0249] -0.800 or greater: Dark red.

[0250] Then, the color is determined based on the average value.

[0251] Furthermore, a color corresponding to the average value is arranged in the nth area. That is, this color is set as the color of the pixel in the nth area.

[0252] Meanwhile, if it is confirmed in step S203 that there are no pixels extracted by the filtering process in the nth area, the information processing device 1 configures the background color in the nth area in step S206. For example, if white is used as the background color, the pixels in the nth area are set as white pixels.

[0253] As a result of performing the processing in steps S203 to S206 for each area, a color map presentation image is generated.

[0254] For example, Fig.12 As shown in FIG. 1 , the result buffer 25 is read from the calculation result buffer 25. Fig.10 The filtering result image described in . The filtering result image is an image containing pixels corresponding to the NDVI values ​​of pixels in the portion corresponding to both the vegetation portion and the sunny portion.

[0255] The filtered result image is divided into 16 areas. Then, the average value (representative value) of NDVI is obtained for each area. Note that areas where there are no pixels (valid pixels as NDVI values) are represented as "no data".

[0256] The area where the average value is obtained is assigned a color, and the area indicated as "no data" is assigned a background color (white).

[0257] As a result, a color map image is generated as shown in the figure. Note that the difference in color is represented by the type of diagonal lines, stippling, etc. in the figure.

[0258] This color map image is generated as a presentation image. In this case, the vegetation index (average value) of each of the 16 areas is represented by color. Thus, the image enables the user to easily grasp the vegetation status in each area.

[0259] Note that, of course, the segmentation into 16 domains is an example, and the image may be segmented into, for example, 2 domains, 3 domains, 4 domains, or 8 domains. Alternatively, the image may be segmented more finely into, for example, 24 domains or 30 domains. Each domain does not necessarily have to have the same square size or shape.

[0260] Next, refer to Fig.13 and 14 Describe the presentation image generation processing example II.

[0261] This is an example in which an image acquired as a result of assigning colors to the filter result image is synthesized with a partially synthesized image having a specific vegetation composition or a specific vegetation state.

[0262] exist Fig.13 The same step numbers are used to represent Fig.11 The steps in the same process are described to avoid repeated description.

[0263] In this case, after performing domain division and setting of variable n (S201 and S202), the information processing device 1 performs the processing in steps S203, S206 and S210 to S213 while incrementing the variable n in step S208 until the variable n reaches the maximum value so that n=nmax in step S207.

[0264] Note that in the case of performing this processing example, in addition to filtering the result image, the information processing device 1 also Figure 7 In step S141, an image showing a portion having a specific vegetation composition or a specific vegetation state (e.g., filtered image FP2) is read as an image to which the color-assigned image is to be attached. Then, in step S201, the filtered result image and, for example, the filtered image FP are similarly segmented into a plurality of areas.

[0265] Note that the filter image FP2 is an image (vegetation detection image) obtained by extracting pixels whose NDVI values ​​are equal to or greater than a certain value.

[0266] In step S203 , the information processing device 1 checks whether or not there is a pixel extracted by the filtering process in the n-th region among the segmented regions in the filtering result image, thereby branching the process.

[0267] If there is an extracted pixel, the information processing device 1 confirms in step S210 whether there is a vegetation detection pixel in the n-th area of ​​the vegetation detection image (filter image FP2).

[0268] In the case where there are filtered extracted pixels in the nth field and there is also a vegetation detection image in the nth field (for example, pixels having NDVI values ​​equal to or greater than a specific value), in step S211, the information processing device 1 calculates a representative value (for example, an average value) of the pixels in the field.

[0269] Then, in step S212 , the information processing device 1 sets a color corresponding to the calculated representative value (average value).

[0270] Furthermore, in step S213, the information processing device 1 assigns colors to pixels existing in the n-th area of ​​the vegetation detection pixels (filter image FP2).

[0271] At the same time, when it is confirmed in step S203 that there are no pixels extracted by filtering processing in the nth field, or when it is confirmed in step S210 that there are no pixels extracted as a vegetation detection image in the nth field, the information processing device 1 configures the background color (for example, white) in step S206.

[0272] As a result of the above processing for each area, a color map rendering image is generated.

[0273] For example Fig.14 The above-mentioned filtering result image and the filtered image FP2 are used as shown in FIG.

[0274] The filtering result image is an image containing pixels corresponding to the NDVI values ​​of pixels in the portion corresponding to both the vegetation portion and the sunny portion. The filtering result image is divided into 16 areas to obtain the average value (representative value) of NDVI for each area.

[0275] Then, a color corresponding to the average value is selected and assigned to pixels in the same area of ​​the filtered image FP2.

[0276] A background color (white) is assigned to all pixels in an area indicated as “no data” where the average value cannot be calculated or an area of ​​the filter image FP2 where there are no pixels with an NDVI value equal to a specific value.

[0277] As a result, as shown in the figure, a color map image is generated. This is an image obtained by assigning colors to the extracted pixels in the filter image FP2 according to the average value.

[0278] Therefore, the average value is expressed by color at the position where the NDVI in the image is equal to or greater than a certain value. Thus, the image enables the user to easily grasp the vegetation status in each field.

[0279] Next, refer to Fig.15 and 16 Describe the presentation image generation processing example III.

[0280] This is an example of an overlay display on a visible light image.

[0281] exist Fig.15 The same step numbers are used to represent Fig.11 The processing steps are the same as those in .

[0282] In this case, after domain division and setting of variable n (S201 and S202), the information processing device 1 performs the processing in steps S203, S220 and S221 while incrementing the variable n in step S223 until the variable n reaches the maximum value so that n=nmax in step S222.

[0283] Note that in the case of this processing example, in addition to filtering the result image, Figure 7 In step S141 in FIG. 2 , an RGB image (visible light image) is read. Note that it is conceivable that, for example, an RGB image supplied from the imaging device 250 and stored in the input buffer 22 is read.

[0284] In step S201 , the filtered result image and the RGB image are similarly segmented into a plurality of areas.

[0285] In step S203 , the information processing device 1 checks whether or not there is a pixel extracted by the filtering process in the n-th region among the segmented regions in the filtering result image, thereby branching the process.

[0286] When there are extracted pixels, the information processing device 1 calculates a representative value (for example, an average value) of the pixels in the area in step S220 .

[0287] Then, in step S221, the information processing device 1 sets a color corresponding to the calculated representative value (average value), and sets this color as the color of each pixel existing in the area of ​​the filtering result image.

[0288] Meanwhile, in the case where it is confirmed in step S203 that there are no pixels extracted by the filtering process in the nth area, the processes of steps S220 and S221 are not performed on the nth area.

[0289] As a result of performing the above-described processing for each area, a color is set for each area in which there are pixels extracted by the filtering processing.

[0290] When the variable n reaches the maximum value such that n=nmax in step S222, the information processing device 1 proceeds to step S224 and superimposes the pixels of the filter result image to which the colors are set on the RGB image.

[0291] That is, Fig.16 As shown in , the color assigned to each area is superimposed on the pixels of the RGB image present in the filtered result image. This figure shows the state in which the corresponding pixels are colored.

[0292] That is, the color indicating the filtering result is represented on the RGB image. Therefore, the representative value of the NDVI of the sunny part where the NDVI is equal to or greater than a specific value is represented by color on the image to be visually recognized by the user under normal circumstances. Thus, the image enables the user to easily grasp the vegetation status.

[0293] Note that instead of overlaying, the corresponding pixels can be overwritten with the assigned color.

[0294] The presentation image generation processing examples I, II, and III are described above as examples.

[0295] Presentation image generation processing example 1 is an example of generating an image by using a filtering result image in which an imaging range is segmented into specific units and colors are assigned to representative values ​​of the segmented units.

[0296] In the presentation image generation processing example II, the filtering result image and the image of the portion having a specific vegetation composition or a specific vegetation state (filtered image FP2) are used, and the filtering result image is assigned a color. Then, the filtering result image that has been color-assigned is synthesized with the image of the portion having a specific vegetation composition or a specific vegetation state (filtered image FP2).

[0297] The presentation image generation processing example III is an example of assigning colors to the filtering result image and superimposing and displaying it on the visible light image.

[0298] In either example, colors are assigned to the filtering result images so that the evaluation results can be expressed by colors.

[0299] In particular, as a result of assigning colors in units of regions, an easily understandable image representing the state in the range imaged as the high-resolution image HR is generated.

[0300] exist Figure 7 The rendered images generated in these examples are stored in the output buffer 28 in step S143.

[0301] Each of these presentation images may be output as an output image Pout by the image output unit 30 and presented to the user.

[0302] That is, at the timing of outputting an image, the information processing device 1 Figure 7 Step S160 in proceeds to S161.

[0303] In step S161, the information processing device 1 (image output unit 30) reads the image to be output from the output buffer 28 (for example, the rendered image), and outputs the image to the outside as the output image Pout. As a result, the rendered image is displayed or recorded, which clearly indicates the measurement result of the sampling area based on, for example, color mapping.

[0304] Note that the above-mentioned presented image is an image in the imaging range of the high-resolution image HR.

[0305] In the present embodiment, it is also possible to perform image synthesis by also using the low-resolution image LR, and output an image representing the entire measurement object of the farmland 300 as the output image Pout.

[0306] <4. Synthesis with low-resolution images>

[0307] exist Figure 6 and 7 In the processing of Figure 7 Step S150 in proceeds to S151.

[0308] In step S151 , the information processing device 1 (image synthesis unit 29 ) reads a desired image, performs synthesis processing, and stores the generated synthesized image in the output buffer 28 in step S152 .

[0309] Hereinafter, as examples of the synthesis process of step S151 , synthesis process examples I, II, III, and IV will be described.

[0310] First, refer to Fig.17 and 18 Synthesis processing example I is described.

[0311] This is an example of aligning and attaching the rendered image and the low-resolution image LR together.

[0312] exist Fig.17In step S301 in , the information processing device 1 sets a variable x so that x=1. Note that the variable x is a variable for representing each region of the measurement object.

[0313] For example, suppose Fig.18 The image 300P is obtained by placing it as Figure 4 An image of the entire measured object is generated by stitching together low-resolution images LR#1, LR#2, LR#3, ..., LR#LN (although area #LN is not shown, "LN" is defined as the last area number) captured respectively from images of areas #1, #2, #3, ..., #LN of the farmland 300 shown in A.

[0314] The variable x is a variable representing a certain region #x as one of the regions #1 to #LN in the measurement object to be imaged as a low-resolution image LR to form the image 300P as described above. Note that here, the region refers to a range to be imaged as a single low-resolution image LR, and the overlap between adjacent regions serves as an edge for stitching.

[0315] The information processing device 1 repeats the process of incrementing the variable x in step S307. Fig.17 The processing of steps S302 to S305 in is repeated until the variable x reaches the maximum value xmax in step S306. The maximum value xmax is a value corresponding to the above-mentioned "LN". That is, the processing of steps S302 to S305 is performed on each single low resolution image LR.

[0316] In step S302 , the information processing device 1 reads the low resolution image LR and related information of the region #x from the input buffer 22 .

[0317] In step S303, the information processing device 1 refers to the relevant information and reads the rendered image of the area #x from the output buffer 28 (for example, Fig.12 ). The rendered image of region #x is a rendered image generated based on the high resolution image HR of region #x determined based on the relevant information.

[0318] In step S304, the information processing device 1 refers to relevant information to determine the position of the high-resolution image HR in the low-resolution image LR, and superimposes (synthesis of transparent presentation images) or overwrites (synthesis of opaque presentation images) the presentation image at the position of the high-resolution image HR.

[0319] Then, in step S305 , the information processing device 1 sets (temporarily stores) an image obtained by superimposing the presentation image on the low resolution image LR or the like as the image G#x of the region #x.

[0320] This process is repeated for each region. For example, Fig.18 Only images G#10 to G#18 of areas #10 to #18 are illustrated.

[0321] Each image G#x is an image acquired as a result of arranging the presentation image CP in the low-resolution image LR.

[0322] When it is confirmed in step S306 that the above processing has been completed for all areas, the information processing device 1 proceeds to step S308 and generates the entire image 300P of all measurement areas by performing stitching processing. That is, the information processing device 1 generates the entire image 300P by stitching each of the images G#1 to G#(xmax) temporarily stored in step S305. For ease of description, as an example of stitching a low-resolution image LR synthesized with the presentation image CP, Fig.18 Only the portion corresponding to the stitched images G#10 to G#18 is shown. This stitching process is performed on the entire area to generate the entire image 300P. Although not shown for simplicity, for areas other than areas #10 to #18, the low-resolution image LR is also synthesized similarly with the presentation image CP, and the resulting synthesized images of the respective areas are stitched to generate the image 300P.

[0323] In this way, the presentation image CP and the low-resolution image LR are aligned and attached together. As a result, for each area imaged as the low-resolution image LR, the measurement result is represented by the presentation image CP. That is, the user can confirm the evaluation result obtained based on the high-resolution image HR while grasping the position of each area in the farmland 300.

[0324] Note that although Fig.17 An example of performing stitching processing in step S308 is shown, but instead of stitching images, the entire image 300P may be generated by a process of attaching images G#1 to G#(xmax) at a stage such as step S308. That is, the image 300P may be generated by a process of mapping based on position information corresponding to each low-resolution image LR, for example. For example, GPS information or the like when capturing the low-resolution image LR or the high-resolution image HR may be used as the position information.

[0325] In addition, in some cases Fig.17The stitching process shown may result in some disadvantages in image quality. Depending on the software used for the stitching process, the processing of the images G#1 to G#(xmax) obtained as a result of synthesizing the low-resolution image LR with the color rendering image CP may affect the detection of feature points, thereby possibly affecting the stitching process performance. In the event of such a concern, it is conceivable that the stitching process is first performed on the low-resolution image LR, and then the rendering image CP generated from the high-resolution image HR corresponding to the corresponding low-resolution image LR is attached to the low-resolution image LR. That is, in this case, as in Fig.17 The processing before step S301 is to read the low-resolution images LR#1, ..., LR#LN obtained by imaging the respective regions #1, ..., #LN, obtain the relevant information, and perform stitching processing so as to generate the entire image 300P. Then, steps S301, S303, S304, S306, and S307 are performed. In this case, it is sufficient to attach the presentation image CP to the corresponding low-resolution image LR by using the relevant information (position information) in step S304. Therefore, steps S302, S305, and S308 are unnecessary.

[0326] Next, refer to Fig.19 , 20 and 21 describe synthesis processing example II.

[0327] This is an example in which the range of the low-resolution image LR is superimposed to present a representative value (average value or maximum value) of the image, also including a portion that cannot be imaged as the high-resolution image HR in each area.

[0328] exist Fig.19 The same step numbers are used to represent Fig.17 The processing steps are the same as those in .

[0329] exist Fig.19 In the process of Fig.17 The processing of steps S301, S306 and S307 is similar to the steps in , and the processing of steps S302 to S312 is performed for each region (for each low-resolution image LR).

[0330] In step S302 , the information processing device 1 reads the low resolution image LR and related information of the region #x from the input buffer 22 .

[0331] In step S303, the information processing device 1 reads the rendering image corresponding to the area #x from the output buffer 28 with reference to the relevant information (for example, Fig.12Note that if information on the average value or maximum value of each field calculated when generating the above-mentioned rendering image is stored in the output buffer 28, the information can be read therefrom.

[0332] In step S310 , the information processing device 1 obtains the average color or the maximum color of the presented image.

[0333] In the case of color mapping the rendered image as described above, an average value or a maximum value is obtained for each area. In contrast, for example, an average value or a maximum value is obtained for the entire range of the high resolution image HR (rendered image) in step S310.

[0334] For example, the average color in the rendered image as a color map image is obtained. It is conceivable to obtain the color as follows. The color of each field in the color map image is converted into a value. Then, the average value or the maximum value of the converted values ​​is obtained. Thus, the color is selected again based on the value.

[0335] Alternatively, if information about the average or maximum value of each field calculated when generating the rendered image is stored as described above, recalculation can be performed by using those values ​​and obtaining the average or maximum value of the entire rendered image. The color of the entire rendered image can be selected based on this value.

[0336] In either case, a color corresponding to the average value or the maximum value of the entire presented image (average value color or maximum value color) is acquired in step S310.

[0337] In step S311 , the information processing apparatus 1 performs synthesis by superimposing the average value color or the maximum value color on the range of the low resolution image LR of the region #x.

[0338] Then, in step S312, the information processing device 1 sets (temporarily stores) an image obtained by superimposing the average color or maximum color of the rendering image on the entire low resolution image LR, as the image G#x of the region #x.

[0339] This process is performed for each region.

[0340] Then, when it is confirmed in step S306 that the above processing has been completed for all areas, the information processing device 1 proceeds to step S308 and generates the entire image 300P of all measurement areas by performing stitching processing. That is, the information processing device 1 generates the entire image 300P by stitching each image G#1 to G#(xmax) temporarily stored in step S305.

[0341] Fig. 20shows a state where an image obtained by superimposing the average color of the rendered image on the low-resolution image has been stitched. Fig.18 Only images G#10 to G#18 are shown, but each of images G#1 to G#(xmax) (not shown) is spliced ​​in this way to generate the entire image 300P.

[0342] also, Fig.21 The state in which images G#10 to G#18 acquired by superimposing the maximum value color of the rendering image on the low-resolution image has been spliced ​​is shown. Each of the images G#1 to G#(xmax) is spliced ​​in this way to generate the entire image 300P.

[0343] The image 300P synthesized in this way represents the measurement result (evaluation result) acquired from the high resolution image HR for each area imaged as the low resolution image LR. That is, the user can confirm the average value or maximum value of the measurement result for each area in the farmland 300.

[0344] In particular, in Fig. 20 When the average color is superimposed as shown, it is easy to understand the status of each area. According to image 300P, when the coverage of sampling measurement is low, the visibility of the overall trend can be improved.

[0345] Furthermore, in Fig.21 In the case where the maximum value color is superimposed as shown, it is easier to understand, for example, areas with good growth, areas that require attention, etc., although it depends on the vegetation index to be used. For example, visibility of finding small abnormal parts can be improved.

[0346] Note that although the average value color and the maximum value color are used in the above example, it is also conceivable to superimpose a color corresponding to the minimum value, centroid value, mode value, etc. of each region on the low resolution image LR in this case.

[0347] Furthermore, it is also conceivable that, as in the synthesis processing example I, the stitching processing of the low-resolution images LR is also first performed in the synthesis processing example II and the synthesis processing examples III and IV to be described later. However, in the case of the synthesis processing example II, if the low-resolution images LR are stitched first, each boundary between the original low-resolution images LR becomes unrecognizable, making the superimposed range of the presented image unrecognizable. In this case, it is conceivable that by using the coverage ratio to make the low-resolution images LR, which will be referred to later, be stitched together. Fig.28 The width a and width d of the high-resolution image HR described are enlarged to the size with interval c and interval f in the same figure, and the average color or maximum value color of the presented image is superimposed on the range. The same applies to the synthesis processing examples III and IV to be described next.

[0348] Note that if the boundary information of each original low-resolution image LR used during the stitching process can be obtained, there is no such problem. In this case, it is sufficient to superimpose the average value color or the maximum value color on the image 300P generated as a result of the stitching process of the low-resolution image LR previously performed by using the boundary information.

[0349] Will refer to Fig. 22 and 23 Combination processing example III is described.

[0350] This is an example in which different colors are assigned to respective ranges of the low resolution image LR, which also includes portions that cannot be imaged as the high resolution image HR in respective areas, depending on whether the sampling measurement results are within a specific value range.

[0351] exist Fig. 22 The same step numbers are used to represent Fig.17 The processing steps are the same as those in .

[0352] exist Fig. 22 In the process of Fig.17 The processing of steps S301, S306 and S307 is similar to the steps in , and the processing of steps S302, S303 and S320 to S325 is performed for each region (for each low-resolution image LR).

[0353] In step S302 , the information processing device 1 reads the low resolution image LR and related information of the region #x from the input buffer 22 .

[0354] In step S303, the information processing device 1 reads the rendering image corresponding to the area #x from the output buffer 28 with reference to the relevant information (for example, Fig.12 ). If information about the average value or maximum value for each field calculated when generating the rendered image is stored in the output buffer 28, this information can also be read therefrom in this case.

[0355] In step S320 , the information processing device 1 calculates the average value or the maximum value of the presented image.

[0356] In this case, the information processing device 1 obtains, for example, the average color or the color as the maximum value in the presentation image as a color map image, and converts the color into a numerical value. Alternatively, if information about the average value or the maximum value of each field calculated when generating the presentation image is stored as described above, the average value or the maximum value of the entire presentation image can be obtained by using those values.

[0357] In step S321 , the information processing apparatus 1 determines whether the calculated average value or maximum value is within a predetermined range as upper and lower thresholds.

[0358] For example, an appropriate range of the average value or an appropriate range of the maximum value is set in advance.

[0359] Then, if the calculated value is within the appropriate range, the information processing device 1 assigns the first color to the region #x in step S322.

[0360] Furthermore, if the calculated value is not within the appropriate range, the information processing device 1 assigns the second color to the region #x in step S323.

[0361] In step S324, the information processing apparatus 1 performs synthesis by superimposing the color (first color or second color) assigned in step S322 or S323 on the range of the low resolution image LR of the region #x.

[0362] Then, in step S325, the information processing device 1 sets (temporarily stores) the image obtained by superimposing the first color or the second color on the entire low-resolution image LR, as the image G#x of the region #x.

[0363] This process is performed for each region.

[0364] Then, when it is confirmed in step S306 that the above processing has been completed for all areas, the information processing device 1 proceeds to step S308 and generates an entire image 300P of all measurement areas by splicing each image G#1 to G#(xmax) temporarily stored in step S305.

[0365] Fig.23 A state in which images G#10 to G#18 acquired by superimposing the first color or the second color on the low-resolution image have been spliced ​​is shown. Each of the images G#1 to G#(xmax) (not shown) is thus spliced ​​to generate the entire image 300P.

[0366] Each region is colored in one of the two colors. Thus, it is possible to clearly identify whether the state of each region is appropriate or inappropriate (for example, whether some growth abnormality occurs, etc.) based on the image.

[0367] Note that in the above example, the average value or the maximum value is obtained, and it is determined whether the obtained value is within an appropriate range to select the first color or the second color. However, it is also conceivable to determine the suitability of the minimum value, the centroid value, the mode value, etc. of each area based on comparison with the threshold range, and select a color and superimpose it on the low-resolution image LR.

[0368] Will refer to Fig.24 and 25 Synthesis processing example IV is described.

[0369] This is an example in which the presented image and the low-resolution image LR are aligned and attached together as in the above-mentioned synthesis processing example, and in each area, a color selected by any of the techniques of the above-mentioned synthesis processing examples II and III is superimposed on the portion that cannot be imaged as the high-resolution image HR.

[0370] exist Fig.24 The same step numbers are used to represent Fig.17 The processing steps are the same as those in .

[0371] exist Fig.24 In the process of Fig.17 The processing of steps S301, S306 and S307 is similar to the steps in , and the processing of steps S302, S303 and S330 to S333 is performed for each region (for each low-resolution image LR).

[0372] In step S302 , the information processing device 1 reads the low resolution image LR and related information of the region #x from the input buffer 22 .

[0373] In step S303, the information processing device 1 refers to the relevant information and reads the rendering image corresponding to the area #x from the output buffer 28. If information about the average value or the maximum value of each field calculated when generating the rendering image is stored in the output buffer 28, the information can also be read therefrom in this case.

[0374] In step S330, the information processing device 1 obtains the average color or the maximum color (such as Fig.19 as in step S310).

[0375] In step S331, the information processing device 1 superimposes the average value color or the maximum value color on the range of the low resolution image LR of the region #x.

[0376] In step S332, the information processing device 1 refers to relevant information to determine the position of the high-resolution image HR in the low-resolution image LR, and superimposes (synthesis of transparent presentation image) or overwrites (synthesis of opaque presentation image) the presentation image (color mapping image) at the position of the high-resolution image HR.

[0377] Then, in step S333, the information processing device 1 sets (temporarily stores) the synthesized image generated as described above as the image G#x of the region #x.

[0378] This process is performed for each region.

[0379] Then, when it is confirmed in step S306 that the above processing has been completed for all areas, the information processing device 1 proceeds to step S308 and generates an entire image 300P of all measurement areas by splicing each image G#1 to G#(xmax) temporarily stored in step S305.

[0380] Fig.25 The stitched images G#10 to G#18 are shown, in which the low-resolution image LR is superimposed or overwritten to present the image CP, and further, the range of the low-resolution image LR other than the present image CP is superimposed with the average color or the maximum color. Each of the images G#1 to G#(xmax) (not shown) is stitched in this way to generate the entire image 300P.

[0381] As a result, the image 300P that enables the user to confirm the presentation image CP acquired based on the high-resolution image HR and recognize the overall state of each region #x can be generated.

[0382] Note that the above example of superimposing the average value color or the maximum value color in step S331 corresponds to the case of combining the synthesis processing examples I and II. In this case, it is also conceivable to select a color based on the minimum value, centroid value, or mode value of each area and then superimpose.

[0383] In addition, in the case of combining the synthesis processing examples I and III, it is determined for each region #x whether the representative value of the entire region is within the appropriate range, and the first color or the second color is selected and superimposed. In this way, it is possible to generate an image 300P that enables the user to confirm the presentation image CP acquired based on the high-resolution image HR and clearly recognize whether the state of each region #x is appropriate.

[0384] The synthesis processing examples I, II, III, and IV are given above as examples.

[0385] exist Figure 7 The synthesized image of the low resolution image LR and the rendered image generated in these examples is stored in the output buffer 28 in step S152.

[0386] Each of these synthesized images may be output by the image output unit 30 as an output image Pout and presented to the user.

[0387] That is, at the timing of outputting an image, the information processing device 1 Figure 7 Step S160 in proceeds to S161.

[0388] In step S161, the information processing device 1 (image output unit 30) reads the image to be output from the output buffer 28 (for example, a composite image of the low-resolution image LR and the presentation image), and outputs the image to the outside as an output image Pout. As a result, for example, a composite image covering the entire farmland 300 is displayed or recorded.

[0389] <5. System Configuration of Second Embodiment>

[0390] Fig.26 The system configuration according to the second embodiment is shown. Figure 2 In addition to the configuration of the first embodiment shown in FIG. 1 , the information processing device 1 according to the second embodiment further includes a flight control unit 3 and a communication unit 44. Figure 2 , and are denoted by the same reference numerals. Therefore, their description is omitted.

[0391] The flight control unit 3 in the information processing device 1 includes a flight plan creation unit 40, a condition judgment unit 41, a flight plan modification unit 42, and a second and subsequent flight plan creation unit 43. These represent functions to be realized by software in the information processing device 1.

[0392] Furthermore, the communication unit 44 communicates with the aircraft 200 through wireless communication.

[0393] With these constituent elements, in the second embodiment, the information processing device 1 controls the flight path and imaging operation (imaging timing, etc.) of the aircraft 200 so that flight and imaging can be performed according to the situation.

[0394] Each function of the flight control unit 3 will be described.

[0395] The flight plan creation unit 40 functions to set a flight path of the aircraft 200 over the measurement object (the farmland 300 ).

[0396] The condition judgment unit 41 obtains information on a vegetation index such as NDVI from the calculation result buffer 25 and determines whether a value (eg, a representative value) satisfies a specific condition. The flight plan is modified using the determination result.

[0397] The flight plan modification unit 42 functions to automatically modify the flight plan during the flight of the aircraft 200, for example, to automatically relax the coverage of the imaging range when the value obtained as a result of the synthetic calculation or the state of the measurement object falls within a specific condition so as to shorten the time required to complete the measurement, or to automatically increase the coverage when the value obtained as a result of the synthetic calculation or the state of the measurement object deviates from the specific condition.

[0398] The second and subsequent flight plan creation unit 43 functions to automatically create a flight plan for measuring an area around a portion not measured in the sampling measurement when a synthesized calculated value or the state of the measurement object as a result of the sampling inspection during the first flight deviates from a predetermined condition.

[0399] In addition, the second and subsequent flight plan creation unit 43 also performs processing for automatically creating a flight plan in which the aircraft 200 flies only the remaining portion of the route specified in the initial flight plan when the aircraft 200 cannot complete the flight corresponding to the initial flight plan during the first flight.

[0400] Fig. 27 An example of the internal configuration of the aircraft 200 that communicates with the information processing device 1 including the flight control unit 3 and the communication unit 44 as described above is shown.

[0401] The aircraft 200 includes, for example, a control unit 201 , a GPS module 202 , an internal measurement unit (IMU) module 203 , an altimeter 204 , an orientation system 205 , a communication unit 206 , a battery 207 , a motor 208 , and the above-mentioned imaging device 250 .

[0402] The GPS module 202 provides position information to the control unit 201 .

[0403] The IMU module 203 detects the angle, angular velocity information or acceleration information between three axes, and provides the angle or information to the control unit 201 .

[0404] The altimeter 204 detects the altitude and provides the altitude information to the control unit 201 .

[0405] The compass meter 205 detects a direction and provides direction information to the control unit 201 .

[0406] The control unit 201 controls the flight operation of the aircraft 200 by using these information.

[0407] The motor 208 is a motor for a propeller to be used for the flight of the aircraft 200. For example, in the case of the aircraft 200 such as a multicopter including a plurality of propellers, the motor 208 is provided for each corresponding propeller. The control unit 201 controls, for example, the driving / stopping of each motor 208 and the number of revolutions of each motor 208. As a result, the aircraft 200 can ascend / descend and fly forward, backward, leftward, and rightward.

[0408] The battery 207 refers to a power supply unit, and supplies an operation power supply voltage Vcc to each unit.

[0409] The communication unit 206 is a wireless communication unit, and is capable of sending and receiving various data to and from, for example, the communication unit 44 of the information processing device 1. For example, the communication unit 206 enables the aircraft 200 to send a captured high-resolution image HR, a captured low-resolution image LR, tag information including correspondence information P, flight operation information, and the like to the information processing device 1.

[0410] The control unit 201 includes, for example, a microcomputer including a CPU, a RAM, a ROM, an interface unit, and the like, and controls the flight operation of the aircraft 200 according to the flight plan data.

[0411] <6. Flight plan control>

[0412] A specific example of the flight plan control of the aircraft 200 will be described.

[0413] First, the function of the flight plan creating unit 40 in the information processing device 1 will be described.

[0414] The flight plan creation unit 40 automatically creates a flight plan for the aircraft 200 in response to a user setting of a coverage rate of sampling measurement (imaging) based on the high-resolution image HR.

[0415] The flight plan of the aircraft 200, which is a multirotor helicopter or the like, is basically created under the condition that allows the entire farmland 300 to be imaged as a low-resolution image LR.

[0416] In the system according to the present embodiment, a flight plan is created with priority given to the coverage of the sampling measurement in consideration of sampling imaging of the high-resolution image HR. That is, a flight plan suitable for the situation can be created, such as flying more finely to increase the coverage, or flying roughly when the coverage can be low.

[0417] Scan intervals are specified by the user or computer to create a flight plan.

[0418] Fig.28 and Figure 4 As in B, discrete sampling areas #1C to #5C serving as imaging positions of high-resolution images HR (HR#1 to HR#5) in the farmland 300 as a measurement object are shown.

[0419] Here, Fig.28 The width a and width d in are the widths of the imaging range of the high-resolution image HR (the range in the longitudinal direction and the range in the lateral direction, respectively). In addition, the width b and width e refer to the longitudinal and lateral intervals between adjacent high-resolution images HR, respectively.

[0420] Here, the above-mentioned scanning intervals correspond to the interval c (=a+b) and the interval f (=d+e), and can be specified by, for example, the user.

[0421] At this time, width a and width d define the imaging range of a single high-resolution image HR to be captured in the sampling imaging. Meanwhile, width b (=ca) or width e (=fd) corresponds to the width of a portion that cannot be imaged (=cannot be measured).

[0422] Thus, by specifying the scanning interval, an optimal value can be set according to, for example, the distribution of events to be observed (eg, flooding due to a malfunction of an irrigation system or an outbreak of pests).

[0423] Note that the coverage rate refers to the ratio of the range that can be imaged as the high-resolution image HR to the entire measurement object. The coverage rate M in the longitudinal direction (the direction DR in which the aircraft 200 travels over the farmland 300) is written as M=a / c. The coverage rate N in the transverse direction (the direction of the next sampling imaging row) is written as N=d / f. Instead of the scanning interval, the user can also input the coverage rate.

[0424] Fig.29 A processing example of the function of the information processing device 1 (flight plan creating unit 40 ) is shown.

[0425] In step S401, the information processing device 1 determines the type specified by the user using the user input unit 31. When creating a flight plan, the user can select whether to specify a scan interval or a coverage rate. The user inputs the scan interval or the coverage rate by using the user interface of the information processing device 1 (e.g., the display screen and the user input unit 31).

[0426] Note that it is also assumed that the value of the scanning interval or coverage ratio is automatically set and input in step S401 by some kind of computer device instead of being input by the user.

[0427] For example, in a case where the user has made an input for designating a scanning interval, the information processing device 1 proceeds to step S402 and detects an input scanning interval c in the traveling direction of the aircraft and an input scanning interval f in the lateral direction.

[0428] Then, in step S405, a flight plan is created that satisfies the specified scanning intervals c and f. For example, the position coordinates and flight speed of various points are set. Examples of the various points include the starting point of the flight path DR, the turning point and the direct flight restart point, the point (or imaging timing) for capturing the high-resolution image HR, etc.

[0429] Then, in step S406, the communication unit 44 is caused to transmit information related to the flight plan created in this manner to the aircraft 200. In the aircraft 200, the communication unit 206 receives the information related to the flight plan, and the control unit 201 inputs and stores the information. The control unit 201 controls each unit based on the information related to the flight plan. Thus, the flight specified in the flight plan is executed.

[0430] When the user has inputted the coverage, the information processing device 1 proceeds from step S401 to step S403 and detects the inputted coverage M in the traveling direction of the aircraft and the coverage N in the lateral direction. Then, the scanning intervals c and f are calculated by using the inputted coverages M and N in step S404.

[0431] Then, in a manner similar to the above, in step S405 , a flight plan that satisfies the scanning intervals c and f is created. Then, in step S406 , the communication unit 44 is caused to transmit information related to the flight plan to the aircraft 200 .

[0432] Next, modification of the flight plan will be described.

[0433] The flight plan modification unit 42 can shorten the time required to complete the measurement by automatically relaxing the coverage rate during the flight of the aircraft 200 when the value obtained as a result of the synthetic calculation or the state of the measurement object falls within a specific condition. In addition, the flight plan modification unit 42 can also automatically modify the flight plan to automatically increase the coverage rate when the value obtained as a result of the synthetic calculation or the state of the measurement object deviates from the specific condition.

[0434] For these processes, the condition determination unit 41 obtains measurement results and performs condition determination.

[0435] Incidentally, the present embodiment exemplifies an example in which the information processing device 1 obtains the high-resolution image HR substantially in real time during the flight of the aircraft 200 to modify the flight plan based on condition determination.

[0436] Therefore, for example, each time the imaging device 250 on the aircraft 200 side captures a high-resolution image HR, the high-resolution image HR is transmitted from the communication unit 44 to the information processing device 1 .

[0437] The information processing apparatus 1 causes the data input unit 21 to obtain the high-resolution image HR received by the communication unit 44 and causes the input buffer 22 to store the high-resolution image HR so that various processes can be performed on the high-resolution image HR.

[0438] The condition determination unit 1 sets, for example, the upper limit and lower limit of the NDVI value as threshold values. For example, the NDVI upper limit is set to 0.85, and the NDVI lower limit is set to 0.7.

[0439] Then, each time a high resolution image HR is transmitted from the aircraft 200, the condition judgment unit 1 reads an image representing the NDVI acquired based on a single high resolution image HR from the calculation result buffer 25, and confirms whether the representative value of the NDVI is within the range between the above upper and lower limits.

[0440] In the case where the representative value falls within the range, it is considered that the plant has no abnormality and the representative value "falls within the specific condition".

[0441] Conditional judgment processing can be performed based on the filtered result image, or can be performed based on the vegetation calculation image or the filtered image. A representative value such as an average value is obtained from these images and compared with a threshold value (i.e., an upper limit and a lower limit) to perform conditional judgment.

[0442] Note that, depending on the vegetation index, some may increase and some may decrease in the case of anomalies. Fig.30 Examples of conceivable setting modes are shown in FIG.

[0443] For example, whether the condition is satisfied is determined based on patterns such as “a specific value or less”, “a specific value or more”, “within a specific range”, and “outside a specific range”.

[0444] In the case of using the photochemical reflectance index (PRI) as the vegetation index, conditional judgment can be made on conditions such as "certain value (0.3) or less" or "certain value (0.3) or more" according to the calculation method. Note that PRI will be described later.

[0445] In the case of using NDVI as the vegetation index, conditional judgment can be made on conditions such as "within a specific range (0.70 to 0.85)" or "outside a specific range (0.70 to 0.85)".

[0446] Fig.31 An example of processing by the information processing device 1 including modification of a flight plan during flight is shown.

[0447] In step S501, the information processing apparatus 1 sets relaxation and enhancement conditions. This is a process of setting a condition determination method to relax or enhance the coverage in response to a user input. That is, a processing method of the condition determination unit 41 is set.

[0448] For example, the setting is constructed as follows. If NDVI is "within a certain range (0.70 to 0.85)", the state is considered appropriate and the coverage is relaxed. Otherwise, the coverage is enhanced.

[0449] Alternatively, it is also conceivable to configure the setting as follows, for example. If the NDVI is not "within the first specific range (e.g., 0.70 to 0.85)", then the coverage is enhanced. If the NDVI is "within the second specific range (a range narrower than the first specific range)", then the coverage is relaxed. In other cases (within the first specific range and outside the second specific range), the coverage is maintained at the current level.

[0450] In step S502, the information processing device 1 performs a process of creating a flight plan. For example, the information processing device 1 refers to Fig.29 Description of the process.

[0451] In step S503, the information processing device 1 detects the start of the flight of the aircraft 200. For example, the information processing device 1 determines that the flight has started by receiving a notification of the start of the flight from the aircraft 200, or by monitoring the start time of the flight according to the flight plan, etc. Alternatively, the start of the flight may be detected as a result of a predetermined operation performed by the user at the timing of the start of the flight.

[0452] After the flight of the aircraft 200 starts, the information processing device 1 repeats the processing of step S504 and subsequent steps until it is determined in step S509 that the flight has been completed.

[0453] In step S504, the information processing device 1 confirms through conditional judgment whether a result falling within the coverage enhancement condition has been acquired. In the case where the result does not fall within the enhancement condition, the process proceeds to step S505.

[0454] In the case where the result falls under the enhanced condition, the information processing device 1 proceeds to step S510 and modifies the flight plan to increase the coverage. For example, the flight plan is modified by reducing the above-mentioned scanning intervals c and f. Then, information about the modified flight plan is sent to the aircraft 200. Then, the information processing device 1 proceeds to step S508.

[0455] In step S505, the information processing device 1 confirms whether the flight is in progress in the state of the coverage enhanced by the modification in the above step S510, and confirms whether the result falling into the condition for recovery from enhancement is obtained through conditional judgment. Note that examples of the condition for recovery from coverage enhancement that can be thought of include the following conditions.

[0456] -The conditional judgment result is restored to the appropriate state.

[0457] - After the condition determination result is restored to the appropriate state, continuously determining that the condition determination result is in the appropriate state a predetermined number of times.

[0458] - Achieving a state in which it is determined that the vegetation index value is in a suitable state based on a criterion stricter than the criterion for the determination of a suitable state based on enhanced conditions (so-called delayed determination).

[0459] In the case where the coverage enhancement flight is not in progress, or in the case where the result does not fall into the recovery condition as described above even during the coverage enhancement flight, the process proceeds to step S506.

[0460] In the case where the coverage enhancement flight is in progress and the result falls under the recovery condition, the information processing device 1 proceeds to step S511 and performs a modification process of recovering the flight plan to a predetermined flight plan (e.g., the flight plan initially created in step S502). For example, an instruction to recover to the original flight plan is sent to the aircraft 200. Then, the information processing device 1 proceeds to step S508.

[0461] In step S506 , the information processing device 1 confirms through conditional judgment whether a result falling within the coverage relaxation condition is obtained.

[0462] In the case where the result does not fall within the relaxation condition, the process proceeds to step S507.

[0463] In the case where the result falls within the relaxation condition, the information processing device 1 proceeds to step S512 and modifies the flight plan to relax the coverage. For example, the flight plan is modified by increasing the above-mentioned scanning intervals c and f. Then, information about the modified flight plan is sent to the aircraft 200. Then, the information processing device 1 proceeds to step S508.

[0464] In step S507, the information processing device 1 confirms whether the flight is in progress in the state of the coverage rate relaxed by the modification in the above step S512, and confirms whether a result falling into the condition for recovery from relaxation is obtained through conditional judgment. Note that examples of the condition for recovery from coverage rate relaxation that can be thought of include the following conditions.

[0465] - The conditional judgment result is not in an appropriate state.

[0466] - Continuously determining that the condition judgment result is not in a suitable state a predetermined number of times.

[0467] A state is reached in which it is determined that the vegetation index value is not in a suitable state based on a criterion stricter than the criterion for the determination of a suitable state based on relaxed conditions (so-called delayed determination).

[0468] In the case where the coverage relaxation flight is not in progress, or in the case where the result does not fall into the recovery condition as described above even during the coverage relaxation flight, the process proceeds to step S508.

[0469] In the case where the coverage relaxation flight is in progress and the result falls under the recovery condition, the information processing device 1 proceeds to step S513 and performs a modification process of restoring the flight plan to a predetermined flight plan (e.g., the flight plan initially created in step S502). For example, an instruction to restore to the original flight plan is sent to the aircraft 200. Then, the information processing device 1 proceeds to step S508.

[0470] In step S508 , the information processing device 1 branches the process according to whether the coverage enhancement flight is in progress. If the coverage enhancement flight is not in progress, the process returns to step S504 via the determination in step S509 .

[0471] In the case where the coverage enhancement flight is in progress, the information processing device 1 proceeds from step S508 to step S514 to confirm the flight strategy and the estimate of the remaining battery usage time, and branches the process according to whether the confirmation result is appropriate.

[0472] The confirmation of the flight strategy is, for example, the confirmation of what should be emphasized when performing the measurement flight. For example, any of the following can be set as the flight strategy.

[0473] (P1) Prioritize measurement of the entire measurement object. When the flight time estimated based on the remaining battery time is insufficient, the measurement is restored to measurement at normal coverage. That is, priority is given to measurement (imaging) of the entire measurement object.

[0474] (P2) Priority is given to the detection of abnormal points, and the measurement is not restored to the measurement at the normal coverage based on the remaining battery time. That is, in the case of enhanced coverage, the focused measurement is prioritized over the measurement of the entire measurement object.

[0475] For example, the specific determination processing of step S514 is performed as follows.

[0476] - In the case where a (sufficient) battery time equal to or greater than a predetermined value remains, it is determined that the current state is appropriate regardless of the flight strategy.

[0477] - In the case where the remaining battery time is less than a predetermined value, if the flight strategy (P2) is applied, it is determined that the current state is appropriate.

[0478] - In the case where the battery usage time less than the predetermined value remains, if the flight strategy (P1) is applied, it is determined that the current state is not appropriate. This is an example. However, in the case where it is determined that the current state is appropriate, the process directly proceeds to step S509.

[0479] Meanwhile, if it is determined that the current state is not appropriate, the information processing device 1 proceeds to step S515 and performs a modification process to restore the flight plan to the predetermined flight plan. For example, an instruction to restore to the original flight plan is sent to the aircraft 200. Then, the information processing device 1 proceeds to step S508.

[0480] When it is detected in step S509 that the flight of the aircraft 200 is completed, the information processing device 1 ends Fig.31 processing.

[0481] When the information processing device 1 recognizes that the flight is completed, for example by receiving a notification of flight completion from the aircraft 200, detecting that a predetermined time (time required to complete the flight) has passed, detecting an end operation by the user, etc., the information processing device 1 ends processing.

[0482] As Fig.31 As a result of performing the above processing as shown in , a flight control (measurement control) is implemented in which the coverage is dynamically changed according to the conditions during the flight. Therefore, for example, if an area to be monitored intensively is found based on, for example, the state of plant growth, measures can be taken, such as increasing the coverage of the field near the area. As a result, more valuable measurements can be made in a single flight. In contrast, for example, in the case where there is no major problem with the state of plant growth, the flight can be completed in a short time by relaxing the coverage. As a result, the efficiency of remote sensing operations can be improved.

[0483] Furthermore, according to the process including the creation of the flight plan described above, the information processing device 1 can automatically generate a flight plan for the aircraft 200 based on the coverage rate at the time of sampling measurement set by the user.

[0484] Regarding the setting of the flight plan, the scanning interval or coverage in the lateral direction and the scanning interval or coverage in the longitudinal direction may be used.

[0485] Note that the scanning interval or coverage in either the longitudinal direction or the lateral direction may be fixed, while the scanning interval or coverage in the other direction may be set by the user.

[0486] Furthermore, in the case where the value acquired as a result of the synthetic calculation or the state of the measurement object falls within a specific condition, the time required to complete the measurement can be shortened by automatically relaxing the coverage during the flight of the aircraft 200 .

[0487] The coverage relaxation condition may be set to, for example, "equal to or lower than," "equal to or greater than," or "range."

[0488] The relaxed rates in the case of relaxed coverage include:

[0489] - the relaxation rate of the scanning interval or coverage in the lateral direction, and

[0490] - Relaxation rate of the scan spacing or coverage in the longitudinal direction. One or both of these can be changed. Alternatively, the relaxation rate can be variable.

[0491] The condition for recovery from relaxation can be set to, for example, "equal to or lower than", "equal to or greater than", or "range".

[0492] Furthermore, during the flight of the aircraft 200 , in the event that a value acquired as a result of synthetic calculation or the state of the measurement object deviates from a specific condition, the coverage can be automatically enhanced and measurement can be performed.

[0493] The coverage enhancement condition may be set to, for example, "equal to or lower than," "equal to or greater than," or "range."

[0494] Relaxed rates in the case of enhanced coverage include:

[0495] - the enhancement rate of scanning interval or coverage in the lateral direction, and

[0496] - The rate of increase in the scanning interval or coverage in the longitudinal direction. One or both of these can be varied. Alternatively, the rate of relaxation can be variable.

[0497] The condition for recovery from enhancement can be set to, for example, "equal to or lower than", "equal to or greater than", or "range".

[0498] In addition, depending on the flight strategy settings, it is possible to fly while taking the remaining battery time into consideration.

[0499] Next, the processing of the function of the second and subsequent flight plan creation unit 43 is described below.

[0500] When the synthesized calculated value or the state of the measurement object deviates from the expected value range as a result of the sampling check during the first flight, the second and subsequent flight plan creation unit 43 can automatically create a flight plan for measuring the area around the portion not measured in the sampling measurement.

[0501] Furthermore, a flight plan may be automatically created to fly only the remaining portion of the path specified in the original flight plan if the flight according to the original flight plan cannot be completed during the first flight.

[0502] Will refer to Fig.32 and 33 An example of focusing on measuring the center of a portion where the measured values ​​deviate based on the first result is described.

[0503] Fig.32 A shows sampling areas #1C to #20C whose high resolution images HR (HR#1 to HR#20) have been captured in the first flight. Here, a case is assumed where the vegetation index values ​​of the black sampling areas #11C and #15C deviate from the expected value range.

[0504] In this case, a second flight plan is created as an additional flight.

[0505] That is, the second flight plan (measurement range) is determined based on the center point of the sampling area as the sampling measurement range so that the surrounding range to be remeasured is remeasured without omission without being measured as sampling measurement.

[0506] For example, Fig.32 B shows a measurement range 700 and a flight range 701 defined as follows. The surrounding range defined by the longitudinal width g and the lateral width h relative to the center point of the sampling area #11C is referred to as the measurement range 700. In addition, the surrounding range defined by the longitudinal width g and the lateral width h relative to the center point of the sampling area #15C is referred to as the flight range 701. Then, a second flight plan is created so that a high-resolution image HR is captured to cover the flight ranges 700 and 701. In this case, all images are captured as high-resolution images HR to perform measurement in the flight range.

[0507] Although the width g and the width h may be fixed values, it is desirable that the user may specify these values.

[0508] Note that, for convenience of description, although the term "sampling area" is also used for the range of imaging to be performed a second time, the sampling area in the second and subsequent flights is not discrete but continuous.

[0509] Fig.33 A processing example of the information processing apparatus 1 is shown.

[0510] In step S601, the information processing apparatus 1 reads the results of the sampling measurement and filtering processing from the calculation result buffer 25. This is to read information on the calculation results obtained as a result of performing the above-mentioned filtering processing and the like based on the high resolution image HR.

[0511] In step S602, the information processing device 1 performs conditional judgment based on the filtering result image based on the single high-resolution image HR, etc. For example, the information processing device 1 performs conditional judgment, such as whether the representative value of the vegetation index extracted from the filtering result image is within a predetermined range, to determine whether the corresponding sampling area is in an abnormal state (for example, in a poor growth state, etc.).

[0512] If an abnormal point is detected as a result of the conditional judgment, the information processing device 1 adds the surrounding range of the position of the center point of the sampling area where the high-resolution image HR is captured to the flight range in step S603. Fig.32 The width g and the width h shown in B may be variable and may be set according to user input when determining the flight range.

[0513] For example, the information processing device 1 uses a method based on Fig.32 The filtering result images of all sampling areas #1C to #20C in A (ie, based on the high resolution images HR (HR#1 to HR#20) acquired by imaging these sampling areas) are subjected to the processing in the above steps S602 and S603.

[0514] For example, no abnormality is found in the filtering result image based on the high-resolution image HR#1. Therefore, step S603 is not performed, and the area around the sampling area #1C is not added to the flight range. At the same time, abnormalities are detected in the filtering result images based on the high-resolution images HR#11C and HR#15C. Therefore, in the processing of step S603, the areas around the sampling areas #11C and #15C are set as flight ranges 700 and 701, respectively.

[0515] After the above processing is performed on all the sampling measurement ranges (all sampling areas #1C to #20C) in the first flight, the information processing device 1 proceeds from step S604 to S605 and confirms whether there is any range to be newly measured (i.e., the flight range set in step S603). If not, the information processing device 1 ends the processing without creating a second flight plan. In other words, this corresponds to a case where it is determined that a flight for the second focused measurement is not required.

[0516] Meanwhile, in the case where the flight ranges 700 , 701 , etc. are set, the information processing device 1 proceeds from step S605 to step S606 to create a reflight plan.

[0517] In this case, the flight path and imaging timing are set so that high-resolution images HR can be captured, covering all set flight ranges 700, 701, etc. Then, information related to the re-flight plan is sent to the aircraft 200. In response thereto, the aircraft 200 performs a second flight.

[0518] Will refer to Figures 34 to 39 Another example of creating second and subsequent flight plans is described.

[0519] In this example, additional measurement is performed with the sampling area where the value deviates as the center until the boundary point between the sampling area and the point where the judgment result indicating normality is obtained. In addition, re-flight is repeated until the range to be re-measured is not detected.

[0520] Fig.34 A shows the result of the sampling measurement in the first flight. Assume a case where an abnormality is detected in the sampling area #11C as a result of the condition judgment.

[0521] The flight range to be set in the second flight plan extends from the sampling area where the anomaly was found to the edge of the adjacent sampling area. Fig.34 As shown in B, the flight range 702 is set to extend from the sampling area #11C where the anomaly is found to the edges of the sampling areas #7C, #6C, #5C, #12C, #13C, #14C, #15C and #10C adjacent to the sampling area #11C.

[0522] Then, a second flight plan is created so that high-resolution images HR are captured in the flight range 702 .

[0523] Note that although the dashed line representing flight range 702 is shown as slightly separated from the adjacent sampling area for clarity, it is assumed that flight range 702 touches the adjacent sampling area. The same applies to flight ranges 703 to 705 in subsequent figures.

[0524] Fig.35 A shows the measurement result in the second flight. In the second flight, the high-resolution image HR is captured in the entire flight range 702. The area where the high-resolution image HR is captured is shown as sampling areas #200C to #208C. Note that, for ease of viewing the figure, the sampling areas #200C to #208C do not overlap. However, in fact, it is assumed that the sampling areas #200C to #208C slightly overlap each other so that the entire flight range 702 is imaged. The same applies to the following figures.

[0525] Note that the sampling area #204C in this case corresponds to Fig.34 Sampling area #11C in B where anomalies have been detected in the first flight.

[0526] Assume that as a result of the second flight, a determination result indicating abnormality has been newly detected in sampling area #208C.

[0527] There is a sampling area that is judged to be abnormal. Therefore, a third flight plan is additionally created.

[0528] The flight range to be set in the third flight plan extends from the sampling area #208 where the new anomaly has been discovered to the edge of the adjacent sampling area. Fig.35 As shown in B, the flight range 703 is set to a range extending from the sampling area #208C to the edge of the adjacent sampling area.

[0529] Then, a third flight plan is created so that high-resolution images HR are captured in the flight range 703 .

[0530] Fig.36 A shows the measurement results in the third flight. In the third flight, high-resolution images HR are captured in sampling areas #300C to #308C in the flight range 703. It is assumed that as a result, judgment results indicating abnormality have been newly detected in sampling areas #306C and #307C.

[0531] Note that sampling area #304C corresponds to sampling area #204C where an abnormality has been detected in the second flight, and sampling area #300C corresponds to sampling area #11C where an abnormality has been detected in the first flight.

[0532] There is a sampling area that is newly judged to be abnormal. Therefore, a fourth flight plan is additionally created.

[0533] For example, Fig.36 As shown in FIG. 2B , the flight range 704 to be set in the fourth flight plan extends from the sampling areas #306C and 307C where the anomaly has been newly discovered to the edge of the adjacent sampling area. Then, the fourth flight plan is created so that the high-resolution image HR is captured in the flight range 704. In this case, the fourth flight range 704 is centered on the two sampling areas #306C and #307C. Therefore, the fourth flight range 704 is wider than the third flight range 703.

[0534] Fig.37 A shows the measurement result in the fourth flight. In the fourth flight, a high-resolution image HR is captured in the sampling areas #400C to #411C in the flight range 704. It is assumed that a judgment result indicating abnormality has been newly detected in the sampling area #408C ​​as a result.

[0535] Note that sampling areas #405C and #406C correspond to sampling areas #306C and #307C where abnormalities have been detected in the third flight. In addition, sampling area #402C corresponds to sampling area #208C where abnormalities have been detected in the second flight.

[0536] There is a sampling area that is newly judged to be abnormal. Therefore, a fifth flight plan is additionally created.

[0537] For example, Fig.37 As shown in B, the flight range 705 to be set in the fifth flight plan extends from the sampling area #408C ​​where the anomaly has been newly discovered to the edge of the adjacent sampling area. Then, the fifth flight plan is created so that the high-resolution image HR is captured in the flight range 705.

[0538] Fig.38 A shows the measurement result in the fifth flight. In the fifth flight, high-resolution images HR are captured in sampling areas #500C to #508C in the flight range 705. It is assumed that there are no sampling areas that are newly judged to be abnormal as a result. Note that sampling area #504C corresponds to sampling area #408C ​​where abnormality has been detected in the fourth flight, and sampling area #502C corresponds to sampling area #306C where abnormality has been detected in the third flight.

[0539] No new abnormalities were found in the sampling area. Therefore, the survey flight was completed.

[0540] Will refer to Fig.39 A processing example is described including creation of flight plans for the second and subsequent flights as described above.

[0541] In step S700, the information processing device 1 performs the processing of the first flight. This corresponds to, for example, Fig.29 processing and flight completion standby processing.

[0542] When the first flight is completed, in step S701 the information processing device 1 reads the results of the sampling measurement and filtering processing from the calculation result buffer 25. That is, the information processing device 1 reads information on the calculation results obtained as a result of the above-mentioned filtering processing and the like based on the high resolution image HR.

[0543] In step S702, the information processing device 1 performs conditional judgment based on the filtering result image based on the single high-resolution image HR, etc. For example, the information processing device 1 performs conditional judgment, such as whether the representative value of the vegetation index extracted from the filtering result image is within a predetermined range, to determine whether the corresponding sampling area is in an abnormal state (for example, in a poor growth state, etc.).

[0544] If an abnormality is detected as a result of the conditional determination, the information processing device 1 adds a range extending from the position of the center point of the sampling area where the high-resolution image HR is captured to the edge of the adjacent sampling area to the flight range in step S703.

[0545] The processing in the above steps S702 and S703 is performed on all sampling regions of the current flight (ie, on each filtering result image based on the high resolution image HR acquired by imaging these sampling regions).

[0546] After performing the processing of steps S702 and S703 on all sampling areas of the immediately preceding flight, the information processing device 1 confirms in step S705 whether there is any range to be newly measured (ie, the flight range set in step S703 ).

[0547] If there is no range to be newly measured (flight range), the flight control process ends in step S705. That is, the next flight is not performed.

[0548] If any of the sampling areas of the immediately previous flight is determined to be abnormal, the information processing device 1 proceeds from step S705 to step S706 to create a next flight plan.

[0549] In this case, the flight path and imaging timing are set so that the high-resolution image HR can be captured, covering the set flight range. Then, information related to the re-flight plan is sent to the aircraft 200. In response to this, the aircraft 200 performs the next flight.

[0550] In step S707 the information processing device 1 detects the start of the flight. After the flight is completed, the information processing device 1 performs the processing in and after step S701 based on the measurement results of the flight.

[0551] As a result, the Figures 34 to 38 The second and subsequent flights are illustrated in FIG.

[0552] The measurements are performed during the second and subsequent flights performed as described above. As a result, a higher accuracy and depth of analysis can be achieved using high resolution images HR without significantly increasing the measurement time or the amount of data.

[0553] Note that, in the case where the flight corresponding to the original flight plan cannot be completed during the first flight, a flight plan for a second flight not only for flying over a range centered on the sampling area determined to be abnormal as described above, but also for flying only over the remaining portion of the route specified in the original flight plan may be created. In this case, it is sufficient to create a re-flight plan after determining a range in which the high-resolution image HR is not acquired at a predetermined coverage rate among the measurement objects in the farmland 300 so that the range is set as the flight range in the re-flight plan.

[0554] Alternatively, as a simpler approach, it may suffice to create a flight plan by subtracting the area covered in flight from the initial flight plan.

[0555] As a result of performing such flight plan creation control, sampling measurement can be performed on the entire measurement object of the farmland 300 without omission.

[0556] With respect to the information processing device 1 that performs the process of creating the above-mentioned second and subsequent flight plans according to the present embodiment, in the event of a deviation in the calculated value synthesized as a result of the sampling inspection or the state of the measured object, a flight plan for measuring an area around a portion that was not measured in the sampling measurement can be automatically created.

[0557] In this case, additional measurements can be performed within a specified range centered on the point where the value deviates. In addition, additional measurements can be performed with the point where the value deviates as the focus point until the boundary point between the point and the point where there is no change.

[0558] Furthermore, a flight plan for flying over the remaining portion of the route may also be automatically created in the event that the flight corresponding to the original flight plan is not completed during the first flight.

[0559] Note that in this embodiment, the second and subsequent flights of aircraft 200 do not have to be performed by the physically same aircraft, but may be performed by other aircraft.

[0560] Incidentally, as Fig.26 and 27 In the configuration of FIG. 1 , an example in which the flight control unit 3 is included in the information processing device 1 is described. However, the configuration is not limited to this.

[0561] For example, Fig.40 An example of the configuration of the aircraft 200 is shown. Fig. 27 The aircraft 200 also has an example of a configuration as a flight plan processing unit 230 in addition to the configuration of FIG. That is, the above-mentioned flight plan creation unit 40, condition judgment unit 41, flight plan modification unit 42, and second and subsequent flight plan creation unit 43 are included in the aircraft 200. It is sufficient to realize the flight plan processing unit 230 as a software function in the control unit 201 or in an independent microcomputer.

[0562] It is possible to adopt a configuration in which the aircraft 200 (control unit 201) can fly while autonomously modifying the flight plan based on the measurement results. In particular, it is preferable that the function of the flight plan modification unit 42 ( Fig.31 ) to modify the flight plan during flight.

[0563] In addition, you can Fig.26 The information processing device 1 is separately provided in the information processing device 1 to perform flight plan creation, etc. For example, this corresponds to Fig.41 An example shown in , in which the flight plan processing unit 230 is included in the information processing device 400 provided separately from the information processing device 1 to perform aircraft control including flight plan generation, modification, etc.

[0564] In addition, as an example, it is also conceivable to set the flight plan processing unit 230 in the information processing device 600 which is a network server, cloud computing server, etc., so that the flight plan created by the information processing device 600 is sent to the aircraft 200 via the information processing device 400.

[0565] <7. Summary and revision>

[0566] The embodiments are described above. Meanwhile, the information processing device 1 according to the embodiment can obtain the following effects.

[0567] The information processing apparatus 1 according to each embodiment includes a calculation unit 2 that performs arithmetic processing on a sample image as a processing object acquired by imaging a part of a measurement object using a multispectral camera, and generates a calculation result image serving as evaluation information of the measurement object.

[0568] As a result, the measurement result of the entire measurement object can be acquired without completely obtaining an image of the entire measurement object.

[0569] In particular, performing sampling imaging prevents efficiency degradation even when obtaining high-resolution images. Thus, high-resolution sampling images (high-resolution images HR) can be used, and higher-precision and in-depth analysis can be achieved without increasing measurement time or data volume.

[0570] Furthermore, since measurement is performed based on sampled images, measurement can be performed efficiently, which is also suitable for situations where it is necessary to search for small objects or slight scratches in a wide range.

[0571] The information processing apparatuses 1 according to the embodiment each include, as the calculation unit 2 , a filter generation unit 24 that generates a filter image that can be used for filtering processing by using a sampled image (high resolution image HR).

[0572] As described above, the filter image is an image corresponding to information that serves as a filter when filtering is performed (an image showing a portion having a specific composition or a specific state).

[0573] In the imaging using the multispectral camera, images of various wavelength bands are acquired. For example, a filtered image can be generated by using one or more of the images to perform appropriate filtering for presenting the measurement results. For example, in the present embodiment, a filtered image FP1 (see FIG. 1 ) for extracting the sun-facing part is generated by using a sampling image with a wavelength of 800 nm. Fig. 9 ).

[0574] Therefore, a filter condition that allows filtering that matches the sampled image may be generated to extract pixels that satisfy the specific filter condition from the image content of the sampled image.

[0575] The information processing apparatuses 1 according to the embodiment each include, as the calculation unit 2 , an evaluation index calculation unit (vegetation index calculation unit 23 ) that generates an evaluation index image by using a sampling image (high resolution image HR).

[0576] Images of various wavelength bands are acquired in the imaging using the multispectral camera. For example, the vegetation index calculation unit 23 generates a vegetation index image by using the images. This vegetation index is appropriate information to be used for evaluating the entire measurement object.

[0577] In particular, in the present embodiment, a vegetation index image representing a vegetation index serves as an evaluation index image.

[0578] Generate an image representing a vegetation index such as NDVI. Images of various wavelength bands are acquired in imaging using a multispectral camera. Vegetation indices such as NDVI can be acquired from such sampled images. In addition, vegetation can be appropriately evaluated by using vegetation index images.

[0579] The information processing apparatus 1 according to the embodiment each includes an evaluation index calculation unit (vegetation index calculation unit 23) and a filter generation unit 24 as the calculation unit 2. The evaluation index calculation unit generates an evaluation index image by using the sampled image (high resolution image HR). The filter generation unit 24 generates a filter image that can be used for filtering processing by using the evaluation index image.

[0580] That is, a filtered image is generated by using an image (evaluation index image) representing an evaluation index (e.g., vegetation index). In the example given in this embodiment, a filtered image is generated by using an NDVI image (see Fig. 9 ) generates a filter image FP2 representing pixels satisfying a condition that NDVI is equal to or greater than a specific value. As a result, a vegetation index such as NDVI can be reflected in the filter condition, thereby generating a filter for acquiring various appropriate measurement result images.

[0581] Note that the filter image to be used for filtering is not limited to the above reference Fig. 9 While the filter images FP1 , FP2 and FP3 are described, there are many more different possible examples of filter images depending on the desired filtering.

[0582] It is conceivable that various filtered images may be generated by, for example, extracting a specific vegetation index based on a numerical value, extracting a vegetation index based on other vegetation index values, extracting a vegetation index based on multiple other vegetation index values, extracting a vegetation index by using brightness, color information, etc.

[0583] The information processing apparatus 1 according to the embodiment each includes a filter generation unit 34 as the calculation unit 2, and the filter generation unit 34 generates a filter image FP3 that can be used for filtering processing by using the filter image FP1 and the filter image FP2. The filter image FP1 is generated by using the sampling image (high resolution image HR). The filter image FP2 is generated by using the evaluation index image.

[0584] That is, a filter image is generated by using a plurality of filter images.

[0585] As a result, a filtered image reflecting multiple filtering conditions can be generated. Fig. 9 In the example of , a filtered image FP3 is generated. The filtered image FP3 is a vegetation image representing a specific portion for extracting a sunny portion. This allows filtering to be performed so that pixels satisfying a plurality of specific filtering conditions are extracted from the image content of the sampled image.

[0586] The information processing apparatuses 1 according to the embodiment each include a filter generation unit 24 and a filter execution unit 26 as the calculation unit 2. The filter generation unit 24 generates a filter image. The filter execution unit 26 generates a filter result image by using the filter image.

[0587] That is, filtering is performed according to the conditions represented by the filter image, so that a filter result image is generated.

[0588] Fig. 9 Each filtered image example is shown. Fig. 9 As shown, examples of filtered images include filtered images generated from a sampled image, filtered images generated from a vegetation index image, filtered images generated from multiple filtered images, and the like. An image of a sunny portion is an example of a filtered image generated from a sampled image. An image generated based on a condition that NDVI is equal to or greater than a specific value is an example of a filtered image generated from a vegetation index image. A vegetation image representing a specific portion for extracting a sunny portion is an example of a filtered image generated from multiple filtered images. Various filtered result images can be generated by using these filtered images.

[0589] The information processing apparatuses 1 according to the embodiment each include an evaluation index calculation unit (vegetation index calculation unit 23) as the calculation unit 2, a filter generation unit 24, and a filter execution unit 26. The filter execution unit 26 generates a filter result image of an evaluation index image (vegetation index image) by using a filter image.

[0590] That is, the evaluation index image (for example, the vegetation index image) is filtered according to the condition represented by the filter image (the image corresponding to the filter condition) to generate a filtering result image.

[0591] like Fig.10 As shown, the NDVI image is filtered by using the filter image. As a result, a filter result image can be generated by extracting pixels satisfying a predetermined filter condition (e.g., vegetation image and sunny part) represented by the filter image from the vegetation index such as NDVI. Therefore, only a part of the vegetation index desired for evaluation can be extracted, and a highly accurate evaluation result can be obtained.

[0592] The information processing apparatuses 1 according to the embodiment each include an evaluation index calculation unit (vegetation index calculation unit 23) as the calculation unit 2, a filter generation unit 24, a filter execution unit 26, and a presentation image generation unit 27. The presentation image generation unit 27 generates a presentation image by using the filter result image.

[0593] The filtering result image is obtained by extracting pixels from the image before filtering according to the conditions represented by the filtering image, and may not be suitable for visual recognition by humans in some cases (evaluation results may be difficult to understand). Therefore, an image suitable for presenting a state to humans is generated.

[0594] As a result, the evaluation result based on the sampled image and the filtered measurement object can be provided to the user in the form of an easily understandable image.

[0595] The rendering image generating unit 27 according to the present embodiment divides the filtering result image into areas and generates a rendering image in which each area is color-mapped (see Fig.11 and 12 ).

[0596] As a result, it is possible to provide the user with an image that enables the user to recognize the evaluation result based on color.

[0597] Furthermore, the rendered image generating unit 27 divides the filtering result image into areas, and generates a rendered image after color mapping by using a representative value of each area (see Fig.11 and 12 ).

[0598] Each field is assigned a color according to its representative value (such as average value, maximum value, minimum value, centroid value, and mode value), thereby providing the user with an image showing the evaluation result of each field that can be identified by color.

[0599] Furthermore, the rendered image generating unit 27 divides the filtered result image into regions, and generates a rendered image by synthesizing the color-mapped image of each region and other images (see Fig.13 , 14 , 15 and 16).

[0600] The color map image and other images are synthesized in the form of, for example, superposition, overwriting, etc. As a result, an image that enables the user to recognize the evaluation result of each field by color while recognizing each field based on the other images can be provided to the user.

[0601] The other image in the above case is any one of the sampling image (high resolution image HR), the evaluation index image (vegetation detection image), the filtered image, or the filtering result image.

[0602] As a result of synthesizing the sampling image and the color mapping image, an image that enables the user to recognize the evaluation result of each field by color while recognizing each field based on the original image of the measurement object can be provided to the user.

[0603] Furthermore, as a result of synthesizing the evaluation index image and the color mapping image, an image that enables the user to recognize the evaluation result of each field by color while checking the evaluation index image of each field can be provided to the user.

[0604] Furthermore, as a result of synthesizing the filtered image and the color mapped image, an image that enables the user to recognize the evaluation result of each field by color while recognizing the filtering condition represented by the filtered image can be provided to the user.

[0605] Furthermore, as a result of synthesizing the filtering result image and the color mapping image, an image enabling the user to recognize the evaluation result of each field by color while recognizing each field as a filtering result can be provided to the user.

[0606] The information processing apparatus 1 according to the embodiment each includes an image synthesis unit 29 as a calculation unit 2, which synthesizes a low-resolution image LR captured as an image of a measurement object having a lower resolution than that of a sampling image (high-resolution image HR) and a presentation image.

[0607] The presentation image is generated based on the filtering result image, and the filtering result image is based on the sampling image. That is, the presentation image is not based on the image of the entire measurement object, but is based on the image of a part of the measurement object. Therefore, the presentation image is synthesized with a low-resolution image obtained by imaging a wider range than the sampling image.

[0608] As a result, the user can be provided with an evaluation result shown with an image in which the range of the measurement object is wider than the range imaged as the sampling image.

[0609] The image synthesis unit 29 performs image synthesis by using the correspondence information associating the presentation image with the low-resolution image acquired by imaging the range including the sampling image as the source of the presentation image (see Fig.17 and 18 ).

[0610] In the synthesis process, the rendered image of the imaging range of the sampled image is synthesized with the low-resolution image obtained by imaging a wider range including the imaging range of the sampled image. Therefore, the rendered image and the low-resolution image are synthesized by using corresponding information indicating a corresponding relationship such as imaging position information, imaging time information, etc.

[0611] As a result, a synthetic image can be generated by accurately associating the rendered image with the low-resolution image of the area containing the imaging range of the sampling image which is the source of the rendered image.

[0612] The image synthesis unit 29 performs synthesis to arrange the presentation image at a position in the low resolution image LR corresponding to the imaging range of the sampling image as the source of the presentation image (see Fig.17 and 18 ).

[0613] For example, in the synthesis process, the sampled image is superimposed or overwritten with the rendered image in the low-resolution image. In this case, the rendered image is arranged at a position in the low-resolution image corresponding to the imaging range of the sampled image. That is, the rendered image and the low-resolution image are aligned and attached together.

[0614] As a result, a synthetic image clearly showing the position of the sample image in the measurement object can be generated and provided to the user.

[0615] exist Fig.17 and 18 In the example of FIG. 1 , a portion of the low-resolution image LR is superimposed or overwritten with the presentation image CP in units of regions (G#1 to G#LN: note that only G#10 to G#18 are shown, and the same applies to the following description). Therefore, it is easy to visually recognize the evaluation result of each region.

[0616] The image synthesis unit 29 performs synthesis in which the representative value of the presented image is reflected in the low-resolution image.

[0617] Therefore, an image is generated in a manner that reflects the representative value of the presented image for each region. Thus, an image that enables the user to easily grasp the state of each region can be provided to the user.

[0618] For example, in Fig.19 , 20In the example of FIG. 21, the low-resolution image is synthesized with the representative value color of the presentation image of each region (G#1 to G#LN). As a result, the evaluation result of each region can be easily visually recognized.

[0619] Furthermore, in Fig. 22 and 23 In the example of , the quality of the evaluation result is determined based on the presented image of each area (G#1 to G#LN), and the determined result is represented in each area. This makes it possible to provide the user with an image that enables the user to recognize the evaluation quality of farmland, etc. at a glance.

[0620] In addition, Fig.24 and 25 In the example of , after aligning and arranging the presentation images for the respective regions (G#1 to G#LN), the low-resolution image is synthesized with the representative value color of the presentation image. As a result, an image showing the evaluation results of each region can be provided to the user, which is easy to visually recognize and also contains detailed information about the imaging range of the sampled image.

[0621] Note that in each example of the embodiment, the presentation image and the low-resolution image LR are synthesized. However, it is also conceivable that the low-resolution image LR is synthesized with the filtering result image before being processed into the presentation image. In addition, it is also conceivable to synthesize the vegetation index image with the low-resolution image LR, or to synthesize the filtered image with the low-resolution image LR. These synthesized images can be output from the image output unit 30 and presented to the user.

[0622] The information processing systems according to the embodiments each include the above-described information processing device 1 and a multispectral camera (imaging device 250 ) that performs imaging to capture images of a plurality of different wavelength bands.

[0623] As a system that calculates the composition or state of a measurement object through optical measurement using a camera, for example, a Normalized Difference Vegetation Index (NDVI) using a near-infrared wavelength and a red wavelength is used for vegetation measurement.

[0624] In the case of imaging a large farmland, a complete measurement of the entire farmland can be performed by capturing images multiple times with these cameras mounted on an aircraft and stitching the images together.

[0625] Furthermore, at this time, a more accurate and in-depth analysis can be performed by using high-resolution information (high-resolution image HR). For example, if imaging is performed at a resolution that enables separation of soil and plants, only the plant part can be extracted from the image by using a specific NDVI threshold in filtering. Then, NDVI is calculated only for this part. As a result, the influence of the soil can be eliminated and the accuracy of NDVI can be improved. Furthermore, the accuracy of NDVI can be improved by calculating NDVI only for the part of the leaf surface in the shade of other leaves.

[0626] However, high-resolution images need to be captured with a zoom lens or at a short distance. Therefore, the measurement range covered in a single imaging process is narrow. Therefore, when attempting to image a wide range, a lot of time is required for measuring the entire range.

[0627] Furthermore, comprehensive imaging of high-resolution images HR increases the size of measurement data, increases the time required to upload data over a network, and also increases storage capacity in the event of an attempt to store data.

[0628] Furthermore, the resolution of a multispectral camera capable of multi-wavelength imaging is generally lower than that of a camera for visible light imaging. Therefore, when attempting to image the same area with the same resolution, the number of imaging operations increases.

[0629] These increases in imaging time and upload time are major issues in the agricultural sector, where rapid responses are required in the case of detected pests and irrigation system failures. Therefore, it is not practical to perform measurements over a wide range with the help of high-resolution imaging and multi-wavelength imaging.

[0630] In view of this, in the technology according to the present embodiment, imaging is performed as sampling imaging. Therefore, although high-resolution imaging is performed for highly accurate analysis, an increase in imaging time and an increase in data size can be prevented. That is, instead of measuring the entire range to be measured with a high-resolution image HR, only a portion thereof is sampled and measured in the inspection of the measurement object. As a result, a measurement object extending over a wide range can also be measured and evaluated with high accuracy in a short time and at low cost.

[0631] The present technology is not limited to the technology illustrated in each embodiment, and there are various possible modifications.

[0632] As an example of vegetation data, NDVI is cited. However, various vegetation indices may be used as vegetation data. Examples of various vegetation indices include the following indices.

[0633] -PRI

[0634] - Chlorophyll fluorescence value

[0635] -State transition reflectivity

[0636] PRI is an index of spectral reflectance that varies with deep oxidation of the xanthophyll cycle. The xanthophyll cycle is a mechanism that releases excess light energy that cannot be used for photosynthesis as heat. Examples of this mechanism include stomatal blockage caused by strong light or water stress.

[0637] Here, PRI is calculated as follows.

[0638] PRI=(R570-R531) / (R570+R531)

[0639] Note that “R570” refers to the reflected light intensity at a wavelength of 570 nm, and “R531” refers to the reflected light intensity at a wavelength of 531 nm.

[0640] Note that PRI can be calculated as follows.

[0641] PRI=(R531-R570) / (R531+R570)

[0642] Fig.30 The definitions shown apply to any of the above calculation methods. Fig.30 The numerator portion of each of the above formulas is shown.

[0643] The amount of chlorophyll fluorescence may be an amount of sunlight induced chlorophyll fluorescence (SIF), or an amount of chlorophyll fluorescence induced by using a laser or LED instead of sunlight.

[0644] Stress detection can also be performed by utilizing chlorophyll fluorescence. Chlorophyll fluorescence is fluorescence to be emitted from plants in association with photosynthesis of plants, and is a phenomenon in which, in the case of higher plants, if energy is not extracted from a reaction center of electrons excited by light within a certain period of time, energy is released as fluorescence with a wavelength of about 680nm to 770nm.

[0645] The energy released is 0.5% to 3% of the energy of the input light and varies with the photosynthetic state of the plant. In the presence of a large excess of light energy that cannot be used for photosynthesis (such as in the case of stomatal occlusion caused by strong light or water stress), the energy released varies.

[0646] In addition, vegetation indices also include the Green Normalized Difference Vegetation Index (GNDVI), the Visible Atmospherically Resistant Index (VARI), etc.

[0647] The Visible Atmospheric Resistivity Index (VARI) is an index that emphasizes vegetation in the visible part of the spectrum while reducing insolation differences and atmospheric effects. The VARI is calculated as (Green-Red) / (Green+Red-Blue).

[0648] Furthermore, a crop water stress index (CWSI) using thermal images and heat can also be applied.

[0649] In each embodiment, the vegetation of the farmland 300 is set as the measurement object. However, this is an example. For example, the present technology can also be applied to the vegetation observation of grass on sports fields (such as football fields and baseball fields), as well as the vegetation observation of natural grasslands, woodlands, virgin forests, etc.

[0650] Furthermore, the technology according to the present embodiment can be applied not only to sensing of vegetation and the like but also to various fields.

[0651] For example, it is conceivable that the technology according to the present embodiment can be applied to a system for measuring cell movement based on sampling measurement in the field of life sciences.

[0652] Furthermore, the present technology can also be applied to capturing images such as endoscopic images, microscope images, and fundus examination images in the field of medical examination.

[0653] In addition, this technology can also be applied to the field of infrastructure inspection.

[0654] For example, as a large-scale solar inspection, a defective portion may be searched in an area where solar panels are located based on remote sensing using the aircraft 200. In this case, sampling measurement based on the high-resolution image HR according to the present embodiment is effective.

[0655] Furthermore, in the case of searching for defective parts in concrete crack inspection of a target area based on remote sensing using the aircraft 200, the sampling measurement based on the high-resolution image HR according to the present embodiment is also effective.

[0656] Furthermore, although the aircraft 200 according to the embodiments are each equipped with a multispectral camera as the imaging device 250 , it is also conceivable to use an RGB camera, a thermal imaging camera, or the like as the imaging device 250 .

[0657] The configurations described in the embodiments are examples.

[0658] For example, the aircraft 200 may be equipped with the computing unit 2 . Also, a microcomputer in the imaging device 250 may function as the computing unit 2 .

[0659] The program according to the embodiment makes Figure 3The CPU 51 of the computer device 100 shown in FIG. 1 performs the following processing ( Figure 6 and 7 ): performing operation processing on a sampling image as a processing object acquired by imaging a part of the range of the measurement object using a multispectral camera, and generating a calculation result image serving as evaluation information of the entire measurement object.

[0660] The information processing apparatus 1 according to the present embodiment can be easily realized by such a program.

[0661] In addition, such a program may be pre-stored in a built-in recording medium of a device such as a computer device, for example, or in a ROM in a microcomputer including a CPU. Alternatively, such a program may be temporarily or permanently stored in a removable recording medium such as a semiconductor memory, a memory card, an optical disk, a magneto-optical disk, or a magnetic disk. In addition, such a removable recording medium may be provided as so-called packaged software.

[0662] Furthermore, besides being installed from a removable recording medium to a personal computer or the like, such a program may be downloaded from a download site via a network such as a LAN or the Internet.

[0663] Note that the effects described in this specification are merely illustrative and not restrictive, and other effects may also be achieved.

[0664] Note that the present technology can also adopt the following configurations.

[0665] (1) An information processing device comprising:

[0666] A calculation unit that performs arithmetic processing on a sample image as a processing object acquired by imaging a part of a measurement object using a multispectral camera, and generates a calculation result image serving as evaluation information of the measurement object.

[0667] (2) The information processing device according to (1) above, wherein

[0668] The computing unit comprises:

[0669] A filter generation unit generates a filter image that can be used for filtering processing by using the sample image.

[0670] (3) The information processing device according to (1) or (2) above, wherein

[0671] The computing unit comprises:

[0672] An evaluation index calculation unit generates an evaluation index image by using the sampled image.

[0673] (4) The information processing device according to (3) above, wherein

[0674] The evaluation index image is a vegetation index image representing a vegetation index.

[0675] (5) The information processing device according to any one of (1) to (4) above, wherein

[0676] The computing unit comprises:

[0677] An evaluation index calculation unit that generates an evaluation index image by using the sampled image; and

[0678] A filter generating unit generates a filter image that can be used for filtering processing by using the evaluation index image.

[0679] (6) The information processing device according to any one of (1) to (5) above, wherein

[0680] The computing unit comprises:

[0681] An evaluation index calculation unit that generates an evaluation index image by using the sampled image; and

[0682] A filter generation unit generates a filter image that can be used for filtering processing by using a filter image generated using the sampling image and a filter image generated using the evaluation index image.

[0683] (7) The information processing device according to any one of (1) to (6) above, wherein

[0684] The computing unit comprises:

[0685] a filter generation unit that generates a filter image that can be used for filtering processing; and

[0686] A filtering execution unit generates a filtering result image by using the filtering image.

[0687] (8) The information processing device according to any one of (1) to (7) above, wherein

[0688] The computing unit comprises:

[0689] an evaluation index calculation unit that generates an evaluation index image by using the sampled image;

[0690] a filter generation unit that generates a filter image that can be used for filtering processing; and

[0691] A filtering execution unit generates a filtering result image of the evaluation index image by using the filtered image.

[0692] (9) The information processing device according to any one of (1) to (8) above, wherein

[0693] The computing unit comprises:

[0694] an evaluation index calculation unit that generates an evaluation index image by using the sampled image;

[0695] A filter generation unit, the filter generation unit generating a filter image that can be used for filtering processing;

[0696] A filtering execution unit that generates a filtering result image of the evaluation index image by using the filtering image; and

[0697] A presentation image generating unit generates a presentation image by using the filtering result image.

[0698] (10) The information processing device according to (9) above, wherein

[0699] The rendered image generating unit divides the filtering result image into regions and generates a rendered image in which each region is color-mapped.

[0700] (11) The information processing device according to (9) or (10) above, wherein

[0701] The rendered image generating unit divides the filtering result image into regions and generates a color-mapped rendered image by using a representative value of each region.

[0702] (12) The information processing device according to any one of (9) to (11) above, wherein

[0703] The rendered image generating unit divides the filtering result image into areas, and generates a rendered image by synthesizing an image in which the color is mapped for each area and other images.

[0704] (13) The information processing device according to (12) above, wherein

[0705] The other image is any one of a sampling image, an evaluation index image, a filtering image, or a filtering result image.

[0706] (14) The information processing device according to any one of (9) to (13) above, wherein

[0707] The computing unit further comprises:

[0708] An image synthesis unit synthesizes a low-resolution image and a presentation image, the low-resolution image being acquired by imaging a measurement object as an image having a lower resolution than the sampling image.

[0709] (15) The information processing device according to (14) above, wherein

[0710] The image synthesis unit performs image synthesis by using correspondence information associating a presentation image with a low-resolution image acquired by imaging a range including a sampling image as a source of the presentation image.

[0711] (16) The information processing device according to (14) or (15), wherein

[0712] The image synthesis unit performs synthesis to arrange the presentation image at a position corresponding to the imaging range of the sampling image as the source of the presentation image in the low-resolution image.

[0713] (17) The information processing device according to any one of (14) to (16) above, wherein

[0714] The image synthesis unit performs synthesis reflecting a representative value of a presentation image in a low-resolution image.

[0715] (18) An information processing method to be performed by an information processing device, the method comprising:

[0716] A sampling image as a processing object acquired by imaging a part of a measurement object using a multispectral camera is subjected to arithmetic processing; and a process of generating a calculation result image serving as evaluation information of the entire measurement object is performed.

[0717] (19) A program for causing an information processing device to perform the following processing:

[0718] arithmetic processing of a sample image as a processing object acquired by imaging a part of the measurement object using a multispectral camera; and

[0719] A process of generating a calculation result image serving as evaluation information of the entire measurement object.

[0720] (20) An information processing system comprising:

[0721] a multispectral camera that performs imaging to capture images in multiple different wavelength bands; and

[0722] An information processing device includes a calculation unit that performs arithmetic processing on a sample image as a processing object acquired by imaging a part of a measurement object using a multispectral camera, and generates a calculation result image serving as evaluation information of the entire measurement object.

[0723] Reference numerals list

[0724] 1 Information processing device

[0725] 2 Computational Unit

[0726] 3 Flight Control Unit

[0727] 21 Data input unit

[0728] 22 Input Buffer

[0729] 32 Vegetation index calculation unit

[0730] 24 Filter generation unit

[0731] 25 Calculation result buffer

[0732] 26 Filter execution unit

[0733] 27 Presentation Image Generation Unit

[0734] 28 Output Buffer

[0735] 29 Image synthesis unit

[0736] 30 Image output unit 30

[0737] 31 User input unit

[0738] 200 Aircraft

[0739] 250 Imaging Device

[0740] 300 Farm

Claims

1. An information processing device, comprising: a calculation unit that performs arithmetic processing on a sample image acquired by imaging a part of the measurement object using a multispectral camera as a processing object and generates a calculation result image serving as evaluation information for the measurement object, Wherein, the computing unit comprises: an evaluation index calculation unit that generates an evaluation index image by using the sampled image; a filter generating unit that generates a first filter image that can be used for filtering processing by using a second filter image generated using the sampling image and a third filter image generated using the evaluation index image; and A filtering execution unit generates a filtering result image of the evaluation index image by executing filtering processing using the first filter image.

2. The information processing device according to claim 1, wherein The evaluation index image is a vegetation index image representing a vegetation index.

3. The information processing device according to claim 1, wherein The computing unit comprises: A presentation image generating unit generates a presentation image by using the filtering result image.

4. The information processing device according to claim 3, wherein The rendered image generating unit divides the filtering result image into regions and generates a rendered image in which each region is color-mapped.

5. The information processing device according to claim 3, wherein The rendered image generating unit divides the filtering result image into regions and generates a color-mapped rendered image by using a representative value of each region.

6. The information processing device according to claim 3, wherein The rendered image generating unit divides the filtering result image into areas, and generates a rendered image by synthesizing an image in which the color is mapped for each area and other images.

7. The information processing device according to claim 6, wherein The other image is any one of a sampling image, an evaluation index image, a filtering image, or a filtering result image.

8. The information processing device according to claim 3, wherein The computing unit further comprises: An image synthesis unit synthesizes a low-resolution image and a presentation image, the low-resolution image being acquired by imaging a measurement object as an image having a lower resolution than the sampling image.

9. The information processing device according to claim 8, wherein The image synthesis unit performs image synthesis by using correspondence information associating a presentation image with a low-resolution image acquired by imaging a range including a sampling image as a source of the presentation image.

10. The information processing device according to claim 8, wherein The image synthesis unit performs synthesis to arrange the presentation image at a position in the low-resolution image corresponding to the imaging range of the sampling image as the source of the presentation image.

11. The information processing device according to claim 8, wherein The image synthesis unit performs synthesis reflecting a representative value of a presentation image in a low-resolution image.

12. An information processing method performed by an information processing device, the method comprising: performing computational processing on a sampling image obtained by imaging a portion of a measurement object using a multispectral camera as a processing object; as well as performing a process of generating a calculation result image serving as evaluation information for the entire measurement object, The method further comprises: generating an evaluation index image by using the sampled image; generating a first filtered image that can be used for filtering processing by using a second filtered image generated using the sampling image and a third filtered image generated using the evaluation index image; and By performing filtering processing using the first filter image, a filtering result image of the evaluation index image is generated.

13. A computer program product storing a program for causing an information processing device to perform the following processing: performing computational processing on a sampling image obtained by imaging a portion of a measurement object using a multispectral camera as a processing object; as well as a process of generating a calculation result image serving as evaluation information for the entire measurement object, The processing also includes: generating an evaluation index image by using the sampled image; generating a first filtered image that can be used for filtering processing by using a second filtered image generated using the sampling image and a third filtered image generated using the evaluation index image; and By performing filtering processing using the first filter image, a filtering result image of the evaluation index image is generated.

14. An information processing system comprising: a multispectral camera that performs imaging to capture images in multiple different wavelength bands; as well as An information processing device including a calculation unit that performs arithmetic processing on a sample image obtained by imaging a part of a measurement object using a multispectral camera as a processing object and generates a calculation result image serving as evaluation information of the entire measurement object, Wherein, the computing unit comprises: an evaluation index calculation unit that generates an evaluation index image by using the sampled image; a filter generating unit that generates a first filter image that can be used for filtering processing by using a second filter image generated using the sampling image and a third filter image generated using the evaluation index image; and A filtering execution unit generates a filtering result image of the evaluation index image by executing filtering processing using the first filter image.

Images