Information processing device, information processing method, and program

Abstract

An information processing apparatus, an information processing method, a non-transitory computer-readable medium, and a storage device for quantitatively analyzing and storing a relationship between an environment of a measurement object and a response of the measurement object to the environment. The information processing apparatus includes a storage circuit and an operation circuit, the operation circuit being configured to acquire a first physical value indicating an environment of a measurement object associated with a first pixel among a plurality of pixels by analyzing captured image information, acquire a second physical value indicating a response of the measurement object to the environment by analyzing the captured image information, and control the storage circuit to store the first physical value and the second physical value in association with each other.

Description

Technical Field

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of Japanese Priority Patent Application JP 2018-133725 filed on Jul. 13, 2018, the entire contents of which are hereby incorporated by reference herein.

[0003] The present disclosure relates to an information processing device, an information processing method, and a program. Background Art

[0004] In recent years, optical measurement technology for obtaining the relationship between an environment in which an arbitrary measurement object is placed and the response of the measurement object to the environment has been widely developed. For example, the measurement object is a plant or the like, and PTL 1 discloses an optical measurement technology that can accurately measure the moisture content of leaves as the response of the measurement object plant to the environment by excluding the influence of scattered light (external light scattering) from nearby leaves even in a leaf group where a plurality of leaves grow densely.

[0005] Citation List

[0006] Patent Literature

[0007] PTL 1: JP 2017-83207A Summary of the invention

[0008] Technical issues

[0009] However, according to the technology described in PTL 1, etc., it is difficult to quantitatively analyze the relationship between the environment in which the measurement object (e.g., plant) is placed and the response of the measurement object to the environment in a more appropriate manner. More specifically, generally, the optical measuring device in the related art performs operations related to the dimensions of the vertical direction and the horizontal direction (e.g., image processing performed in two-dimensional directions (e.g., vertical direction and horizontal direction) in the plane space of the captured image), but it is difficult to perform operations related to the relationship of dimensions of physical values ​​such as the environment and the response. Therefore, the optical measuring device in the related art does not include a mechanism for managing each of the amount of light, reflectance, etc. (or fluorescence intensity, etc.) as physical values, and therefore it is difficult to appropriately express the relationship between physical values, such as the increase and decrease of the reflectance of the measurement object corresponding to the amount of light, or the increase and decrease of the fluorescence intensity.

[0010] Here, the present disclosure has been made in consideration of such circumstances, and it is desirable to provide a new and improved information processing apparatus, information processing method, and program that are capable of quantitatively analyzing the relationship between an environment in which a measurement object is placed and the measurement object's response to the environment in a more appropriate manner.

[0011] Solution to the problem

[0012] According to one aspect of the present disclosure, an information processing device is provided. The information processing device includes a storage circuit and an operation circuit. The operation circuit is configured to acquire a first physical value by analyzing captured image information, the captured image information is based on information from a plurality of pixels, and the first physical value indicates an environment of a measurement object associated with a first pixel of the plurality of pixels. The operation circuit is configured to acquire a second physical value by analyzing the captured image information, the second physical value indicating a response of the measurement object to the environment. The operation circuit is also configured to control the storage circuit to store the first physical value and the second physical value in association with each other.

[0013] In addition, according to another aspect of the present disclosure, an information processing method is provided. The method includes using an electronic processor to acquire a first physical value by analyzing captured image information, the captured image information is based on information from a plurality of pixels, and the first physical value indicates an environment of a measurement object associated with a first pixel of the plurality of pixels. The method includes using the electronic processor to acquire a second physical value by analyzing the captured image information, the second physical value indicating a response of the measurement object to the environment. The method also includes using the electronic processor to control a storage circuit to store the first physical value and the second physical value in association with each other.

[0014] In addition, according to another aspect of the present disclosure, there is provided a non-transitory computer-readable medium including instructions, which, when executed by an electronic processor, cause the electronic processor to perform a set of operations. The set of operations includes acquiring a first physical value by analyzing captured image information, the captured image information being based on information from a plurality of pixels, and the first physical value indicating an environment of a measurement object associated with a first pixel of the plurality of pixels. The set of operations includes acquiring a second physical value by analyzing the captured image information, the second physical value indicating a response of the measurement object to the environment. The set of operations also includes controlling a storage circuit to store the first physical value and the second physical value in association with each other.

[0015] In addition, according to another aspect of the present disclosure, a storage device is provided. The storage device includes a storage circuit and an interface circuit. The interface circuit is configured to receive a first physical value indicating an environment of a measurement object associated with a first pixel among a plurality of pixels. The interface circuit is configured to receive a second physical value indicating a response of the measurement object to the environment. The interface circuit is configured to receive a correlation indicator indicating that the first physical value is correlated with the second physical value. The interface circuit is also configured to output the first physical value and the second physical value to the storage circuit based on the correlation indicator.

[0016] Advantageous Effects of the Invention

[0017] As described above, according to aspects of the present disclosure, the relationship between an environment in which a measurement object is placed and the response of the measurement object to the environment can be quantitatively analyzed in a more appropriate manner.

[0018] Furthermore, the above-described effects are not limited, and any effects described in the present specification or effects that can be understood from the present specification may be provided in combination with or in place of the above-described effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a block diagram illustrating a functional configuration example of the information processing apparatus 100 according to the first embodiment.

[0020] Figure 2 : is a flowchart illustrating an example of the flow of the entirety of processing performed by the information processing apparatus 100 according to the first embodiment.

[0021] Figure 3 is a flowchart illustrating an example of the flow of input processing of captured image information.

[0022] Figure 4 is a view illustrating an example of a data state in a stage where input processing of captured image information is terminated.

[0023] Figure 5 is a flowchart illustrating an example of the flow of multi-wavelength image operation.

[0024] Figure 6 is a flowchart illustrating an example of the flow of generation processing of PRI image information.

[0025] Figure 7 : is a flowchart illustrating an example of the flow of generation processing of NDVI image information.

[0026] Figure 8 is a view illustrating an example of a data state in a stage where a multi-wavelength image operation is terminated.

[0027] Fig. 9 is a flowchart illustrating an example of the flow of an image structure analysis operation.

[0028] Fig.10 is a flowchart illustrating the flow of classification processing of captured image information.

[0029] Fig.11 is a view illustrating a specific example of classification of captured image information.

[0030] Fig.12 is a view illustrating an example of a data state in a stage where the image structure analysis operation is terminated.

[0031] Fig.13 is a view illustrating an example of a data state in a stage where the image structure analysis operation is terminated.

[0032] Fig.14 is a flow chart illustrating an example of the flow of a removal operation.

[0033] Fig.15 is a flowchart illustrating an example of the flow of a correction operation.

[0034] Fig.16 is a view illustrating an example of a data state in a stage where a removal operation and a correction operation are terminated.

[0035] Fig.17 : is a table illustrating the relationship between the conditions of the intensity of light irradiating the leaves and the content of analysis that can be performed.

[0036] Fig.18 is a view illustrating an analysis image of physical values ​​of different individuals measured simultaneously.

[0037] FIG. 19A to FIG. 19C is a view illustrating an analysis image of physical values ​​of the same individual measured at different times.

[0038] Fig. 20A 1 is a view illustrating a case where a normal portion 10 and a portion 11 having an unusual shape state may exist in a leaf.

[0039] Fig. 20B 1 is a view illustrating a case where a normal portion 10 and a portion 11 having an unusual shape state may exist in a leaf.

[0040] Fig.21 is a flowchart illustrating an example of the flow of an analysis operation.

[0041] Fig. 22 is a diagram illustrating the relationship between the conditions of the intensity (PAR) of light irradiating leaves and the PRI.

[0042] Fig.23 is a diagram illustrating the relationship between the conditions of the intensity (PAR) of light irradiating leaves and the PRI.

[0043] Fig.24 It is a diagram illustrating the relationship between the conditions of the intensity (PAR) irradiating leaves and the PRI.

[0044] Fig.25 is a view illustrating an example of a data state in a stage where an analysis operation is completed.

[0045] Fig.26 is a view illustrating an example of an analysis operation.

[0046] Fig. 27 is a view illustrating an example of an analysis operation.

[0047] Fig.28 is a flowchart illustrating an example of the process of color mapping.

[0048] Fig.29 is a view illustrating an example of image information displayed after color mapping.

[0049] Fig.30 is a view illustrating an example of image information displayed after color mapping.

[0050] Fig.31 is a view illustrating an example of image information displayed after color mapping.

[0051] Fig.32A is a view illustrating an example of image information displayed after color mapping.

[0052] Fig.32B is a view illustrating an example of image information displayed after color mapping.

[0053] Fig.33A is a view illustrating an example of image information displayed after color mapping.

[0054] Fig.33B is a view illustrating an example of image information displayed after color mapping.

[0055] Fig.34 is a view illustrating an example of image information displayed after color mapping.

[0056] Fig.35 is a view illustrating an example of image information displayed after color mapping.

[0057] Fig.36 is a view illustrating an example of image information displayed after color mapping.

[0058] Fig.37 is a view illustrating an example of image information displayed after color mapping.

[0059] Fig.38 is a block diagram illustrating a functional configuration example of an information processing apparatus 100 according to the second embodiment.

[0060] Fig.39 is a flowchart illustrating an example of the flow of the entirety of processing performed by the information processing apparatus 100 according to the second embodiment.

[0061] Fig.40is a flowchart illustrating an example of the flow of a correction operation according to the second embodiment.

[0062] Fig.41 : is a view illustrating an example of a data state in a stage where a correction operation according to the second embodiment is terminated.

[0063] Fig.42 is a view illustrating an example of a graph generated by the graph generating unit 163 .

[0064] Fig.43 is a view illustrating a third embodiment.

[0065] Fig.44 is a block diagram illustrating a functional configuration example of an information processing apparatus 100 according to the third embodiment.

[0066] Fig.45 is a flowchart illustrating an example of the flow of the entirety of processing performed by the information processing apparatus 100 according to the third embodiment.

[0067] Fig.46 : is a view illustrating an example of a data state in a stage where a removal operation and a correction operation are terminated according to the third embodiment.

[0068] Fig.47 is a flowchart illustrating an example of the flow of resolution adjustment.

[0069] Fig.48 is a view illustrating resolution adjustment.

[0070] Fig.49 is a view illustrating resolution adjustment.

[0071] Fig.50 is a view illustrating resolution adjustment.

[0072] Fig.51 is a view illustrating resolution adjustment.

[0073] Fig.52 is a view that illustrates the model fit.

[0074] Fig.53 is a block diagram illustrating a functional configuration example of an information processing apparatus 100 according to the fourth embodiment.

[0075] Fig.54 : is a view illustrating an example of a data state in a stage where resolution adjustment according to the fourth embodiment is terminated.

[0076] Fig.55 is a block diagram illustrating a functional configuration example of an information processing apparatus 100 according to the fifth embodiment.

[0077] Fig.56is a block diagram illustrating a hardware configuration example of the information processing apparatus 100 according to each embodiment. DETAILED DESCRIPTION

[0078] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In addition, in this specification and the accompanying drawings, the same reference numerals will be given to constituent elements having substantially the same configuration, and repeated descriptions thereof will be omitted.

[0079] Furthermore, the description will be given in the following order.

[0080] 1. Summary

[0081] 2. First embodiment

[0082] 3. Second embodiment

[0083] 4. Third embodiment

[0084] 5. Fourth embodiment

[0085] 6. Fifth embodiment

[0086] 7. Hardware Configuration Example

[0087] 8. Summary

[0088] <1. Summary>

[0089] First, an outline of the present disclosure will be described.

[0090] The normalized difference vegetation index (NDVI) is widely used as an indicator of the composition (biomass) of plants in remote sensing-based optical measurements. NDVI is an indicator that uses the property that chlorophyll included in chloroplasts absorbs light of red wavelengths well and hardly absorbs light of wavelengths in the infrared region and reflects light of that wavelength. Currently, in large farms, NDVI images acquired from unmanned aerial vehicles (UAVs: including drones, etc.), satellites, etc. are used for farm management, and cameras capable of measuring NDVI are commercially available. In addition, cameras capable of measuring vegetation indices other than NDVI (such as green normalized difference vegetation index (GNDVI), etc.) are also commercially available. However, vegetation indices other than NDVI have not yet been widely popularized in farms.

[0091] The background of the fact that vegetation indices other than NDVI are not widely used is as follows. Changes in composition (biomass) correspond to acclimatization and growth due to the accumulation of photosynthetic products of plants in a constant period of time. Therefore, it is difficult to say that any index can quickly grasp the changes in composition, and vegetation indices are not suitable for use in making improvements corresponding to the changes.

[0092] Furthermore, even if the change in the components is quickly grasped so that the problem is detected early, the determination of the cause must be performed to deal with the problem, but in some cases it may be difficult to determine the cause only by the measured value of the components, etc. For example, even if the "presence" of a component such as chlorophyll is detected, only the photosynthetic ability of the plant as a potential is revealed, and it is not clear whether photosynthesis is actually performed sufficiently and its cause (for example, environmental factors such as light, temperature, humidity, soil moisture, and pests and diseases). Furthermore, even if it is revealed whether photosynthesis is actually performed sufficiently and its cause, from the viewpoint of efficiency and improvement of the farm, as long as an improvement method based on the cause is not proposed, it cannot be said that the analysis is fully utilized.

[0093] Regarding other viewpoints, methods for clarifying the mechanism of the environmental response of plants have been developed. For example, methods for clarifying the mechanism of the environmental response of plants using measurement schemes have been put into practical use. Specifically, examples can be given of methods for measuring the environmental response of plants by gas exchange measurement methods, etc. while changing the environment in which the plants are placed (for example, while changing the light intensity or CO2 concentration) and deriving the parameters of the functional model from the relationship between the environment and the response using model fitting, etc., and the state or characteristics of the plants can be analyzed in detail physically by this method. However, the person who performs the method must measure the environmental response while changing the environment in which the plants are placed, which places a great burden on the person who performs the method.

[0094] In addition, the environmental response of the plant is measured based on the fluctuation of fluorescence or reflected light, and functional parameters are calculated. For example, as a method of irradiating plants with excitation light to measure chlorophyll fluorescence, a method called pulse modulation fluorescence measurement is exemplified, which can obtain physical quantities corresponding to the photosynthesis rate, such as the electron transport rate (ETR) in photosynthesis, the quantum yield ratio (quantum yield ratio, ) etc. An optical measuring device using this method has also been put into practical use, but the optical measuring device requires strong light (artificial light) to saturate photosynthesis, so it is difficult to apply the optical measuring device to remote sensing.

[0095] In view of these circumstances, the discloser of the present disclosure has completed the technology according to the present disclosure. The information processing device 100 according to one aspect of the present disclosure measures the changes in the operational functions of plants in real time by using remote sensing, and can improve the management of plants, especially in large farms. More specifically, the information processing device 100 according to one aspect of the present disclosure analyzes the captured image information of the plant as the measurement object to obtain a first physical value corresponding to the environment in which the plant is left and a second physical value corresponding to the response of the plant to the environment, and stores the first physical value and the second physical value in correlation with each other. In addition, the information processing device 100 performs an analysis operation by using the first physical value and the second physical value to output a third physical value indicating the state or property of the plant.

[0096] Generally, the optical measuring device in the related art performs operations related to the dimensions of the vertical direction and the horizontal direction (for example, image processing performed in the plane space of the captured image in two-dimensional directions (for example, the vertical direction and the horizontal direction), but it is difficult to perform operations related to the relationship of the dimensions of the physical values ​​such as the environment and the response. Therefore, the optical measuring device in the related art does not include a mechanism for managing each of the amount of light, reflectivity, etc. (or fluorescence intensity, etc.) as physical values, and it is difficult to appropriately express the relationship between the physical values, such as the increase and decrease of the reflectivity as the measurement object corresponding to the amount of light, or the increase and decrease of the fluorescence intensity. On the other hand, the information processing device 100 according to one aspect of the present disclosure includes a mechanism for converting the captured image information into physical values ​​and, for example, managing each of the amount of light, reflectivity, etc. (or fluorescence intensity, etc.) in the physical value dimension for each pixel, so that the relationship between the physical values ​​can be appropriately expressed.

[0097] In addition, the intensity of sunlight irradiating each leaf of a plant differs depending on the inclination of the leaves, the shadow state, etc., and with respect to the "environment" in a case where a group of plants is photographed in one captured image information, there are multiple environments in the captured image information (for example, it can be considered that there is a different "environment" for each pixel). Here, the information processing device 100 according to one aspect of the present disclosure obtains a first physical value corresponding to the "environment" and a second physical value corresponding to the "response" at the same granularity (for example, pixel unit), and manages the first physical value and the second physical value in relation to each other. In addition, it should be understood that "a first physical value is related to a second physical value" not only includes the application of information (ID, etc.) that directly links the first physical value to the second physical value, but also includes an indirect method of applying any information that can indicate any relationship between the first physical value and the second physical value.

[0098] In addition, there are limitations to the models that can be used in the operation of physical analysis. For example, when analyzing the environmental response of a plant, if the captured image information includes objects other than the plant, such as soil, the analysis accuracy will be reduced. In addition, with respect to the shadow area in the captured image information, the reflection or absorption of light by the plant becomes complicated, and it may be difficult to analyze through the model. Here, in order to perform the analysis of the environmental response of the plant in a more appropriate manner, the information processing device 100 according to one aspect of the present disclosure has a function of removing data that is not suitable for operation (or a function of extracting data suitable for operation) in the data set obtained as the captured image information (for example, removing optically complex areas such as areas in which images of shadows are captured in the captured image information, or areas where an accurate model of plant physiology does not exist, extracting captured image information captured under environmental conditions (such as strong light, etc.) that provoke the environmental response of the plant, etc.).

[0099] In addition, a description of the background of the present disclosure, an overview of the technology according to the present disclosure, etc. has been given, and the technical scope according to the present disclosure is not limited by this description. For example, the purpose of the present disclosure is not limited to this description, and the technology of the present disclosure is applicable to various devices, methods, systems, and programs that need to analyze the state or nature of plants, etc.

[0100] <2. First embodiment>

[0101] Next, a first embodiment according to the present disclosure will be described. The first embodiment aims to measure stress reactions of plants due to various stress factors (however, the purpose is not limited thereto).

[0102] When receiving various stresses, it is difficult for the plant to fully express the photosynthetic capacity that it could have expressed for the received light, and the plant performs heat radiation. The information processing device 100 according to the present embodiment measures the stress response of the plant to various stress factors based on the degree of heat radiation. Here, as "stress", for example, environmental stresses such as strong light, weak light, low humidity, low temperature, high humidity, high temperature, dryness of the soil and excessive moisture of the soil (anaerobic stress), the occurrence of pests and diseases, changes in soil composition, the influence of pesticides including some herbicides, etc. are assumed, but not limited to this (in the case where the photosynthetic capacity of the plant is reduced, it can be said that the plant receives any stress). In the following, the stress may be referred to as "environmental stress".

[0103] Here, photosynthesis includes a photochemical system that generates adenosine triphosphate (ATP) and reduced nicotinamide adenine dinucleotide phosphate (NADPH) after converting light into electrons, and a Calvin cycle that assimilates CO2 to generate CH2O by using the energy of ATP and NADPH. For example, when a plant faces an environment such as dry soil, the plant closes its stomata to reduce water evaporation. At this time, when the plant cannot absorb enough CO2 from the outside, the efficiency of the Calvin cycle decreases, and the total amount of energy received from the photochemical system at the previous stage is limited. In this case, in order to eliminate excess energy, the photochemical system dissipates excess energy by using a mechanism called the xanthophyll cycle combined with the photochemical system 2.

[0104] When measuring the stress response of plants to various stress factors, the information processing device 100 according to the present embodiment generally uses PRI calculated by the following formula 1. In addition, “λ” in formula 1 531 " represents the captured image information with a wavelength of approximately 531 [nm], and "λ 570 ” indicates captured image information with a wavelength of approximately 570 [nm].

[0105] [Mathematical expression 1]

[0106]

[0107] It is believed that the extent of epoxidation / deepoxidation of the xanthophyll cycle is optically detected by PRI, and therefore considering this mechanism, it is expected that PRI can be used as an indicator for measuring the stress response of plants to various stress factors.

[0108] (2.1. Functional configuration example)

[0109] First, refer to Figure 1 A functional configuration example of the information processing apparatus 100 is described.

[0110] like Figure 1 As shown in , the information processing apparatus 100 includes a multispectral camera 110 , an image input unit 120 , an image processing operation unit 130 , a dimension conversion operation unit 140 , a physical analysis operation unit 150 , a data visualization unit 160 , an image output unit 170 , and a display device 180 .

[0111] The multi-spectral camera 110 includes a plurality of multi-spectral filters through which light beams with different wavelength bands can be transmitted, and is configured to separate incident light into light beams of a plurality of wavelength bands to generate multi-wavelength image information (captured image information) including information of a plurality of wavelength bands through subsequent signal processing. For example, the multi-spectral camera 110 can generate captured image information based on visible light, ultraviolet light, near infrared light, infrared light, etc. Here, it should be understood that "captured image information" includes not only the captured image itself, but also measurement values ​​that are not visualized as images, or image information generated by synthesizing a plurality of captured image information through splicing processing. In addition, as the captured image information in the present embodiment, it is assumed that the information of the image captured from the UAV in flight is, but not limited thereto. For example, the captured image information can be obtained by simply capturing an image of a plant from an arbitrary position. In addition, the types and numbers of wavelength bands that can be grasped by the multi-spectral camera 110 are not particularly limited, and can be flexibly modified according to the analysis content of the later stage. In addition, a plurality of multi-spectral cameras 110 can be set.

[0112] The image input unit 120 is configured to serve as an interface for acquiring captured image information generated by the multispectral camera 110. In addition, in addition to simple acquisition of captured image information, the image input unit 120 may also perform various image processing, such as adjustment of the resolution of the acquired captured image information, pixel matching, and various corrections (e.g., atmospheric correction, geometric correction, ortho-correction) (particularly, for example, in the case where a plurality of multispectral cameras 110 are provided). In addition, the content of the image processing performed by the image input unit 120 is not particularly limited.

[0113] The image processing operation unit 130 is configured to perform operations related to the dimensions of the vertical direction and the horizontal direction (e.g., image processing performed in two-dimensional directions (e.g., vertical direction and horizontal direction) in the plane space of the captured image) on the multi-wavelength image information (captured image information). Figure 1 As shown in , the image processing operation unit 130 includes a multi-wavelength image operation unit 131 , an image buffer 132 , an image structure analysis operation unit 133 , and a structured image buffer 134 .

[0114] The multi-wavelength image operation unit 131 performs an operation using a signal of each wavelength band in the captured image information to generate additional captured image information (this process is referred to as "multi-wavelength image operation"). For example, the multi-wavelength image operation unit 131 generates RGB image information or NIR image information by using the captured image information, or generates captured image information indicating various vegetation indexes (e.g., NDVI image information, PRI image information, etc.). In addition, the type of captured image information generated by the multi-wavelength image operation unit 131 is not particularly limited.

[0115] The image buffer 132 is configured to temporarily store the captured image information generated by the multi-wavelength image operation unit 131 .

[0116] The image structure analysis operation unit 133 classifies the captured image information stored in the image buffer 132 into a plurality of areas (this processing is also referred to as "image structure analysis operation"). For example, the image structure analysis operation unit 133 simply classifies the captured image information into the right side and the left side, or finely classifies the captured image information. In addition, the image structure analysis operation unit 133 may analyze the captured image information by using a predetermined image recognition technology to identify objects included in the captured image information (e.g., plant parts including leaves (upper leaves, middle leaves, lower leaves, etc.), flowers, fruits, stems, etc.), and may classify the captured image information for each object. At this time, the image structure analysis operation unit 133 may perform image recognition (feature extraction) by using the captured image information generated by the multi-wavelength image operation unit 131 (e.g., captured image information indicating a vegetation index, etc.).

[0117] The structured image buffer 134 is configured to temporarily store the captured image information after being classified by the image structure analysis operation unit 133 .

[0118] The dimension conversion operation unit 140 is configured to analyze the information of a specific wavelength band in the captured image information (multi-wavelength image information) stored in the structured image buffer 134 to obtain a first physical value corresponding to the environment in which the plant is placed and a second physical value corresponding to the response of the plant to the environment, and store the first physical value and the second physical value in association with each other. In other words, the dimension conversion operation unit 140 is configured to analyze the dimensional information in the vertical direction and the horizontal direction to convert the dimensional information into dimensional information of the physical values ​​of the environment and the response (this process is referred to as "dimensional conversion operation"). Figure 1 As shown in , the dimension conversion operation unit 140 includes a removal operation unit 141 , a correction operation unit 142 , and a physical value data buffer 143 .

[0119] The removal operation unit 141 is configured to remove the area that is not suitable for operation in the captured image information stored in the structured image buffer 134 (in addition, it should be understood that the removal of the area that is not suitable for operation is equivalent to the extraction of the area suitable for operation). This process is called "removal operation". More specifically, the removal operation unit 141 determines that the predetermined condition area is an area that is not suitable for operation based on the captured image information, and removes the area. Examples of predetermined conditions include the condition that NDVI is a value outside the constant range, the condition that the Nn value (pixel value) is a value outside the constant range, etc. According to the condition that NDVI is a value outside the constant range, the area in which the image of objects other than plants (such as soil, etc.) is captured is removed, and according to the condition that the Nn value (pixel value) is a value outside the constant range, the area in which the image of the shadow is captured is removed. In addition, the area removed by the removal operation unit 141 is not limited to the above-mentioned area. In addition, the analysis method of the removal operation unit 141 is not particularly limited. In addition, the removal operation is performed to remove the information that is not processed by the physical analysis operation unit 150, and the content of the removal operation can be flexibly modified according to the content of the physical analysis operation. For example, the removal operation may not remove the soil image capture area in the event that the physical analysis operation is also able to appropriately process the soil image capture area.

[0120] The correction operation unit 142 is configured to perform conversion or correction of captured image information, etc. For example, in the present embodiment, the correction operation unit 142 converts the Nn value (pixel value) into the PAR of the leaf surface (hereinafter, the operation performed by the correction operation unit 142 is referred to as "correction operation"). In addition, the correction operation unit 142 stores the PAR of the leaf surface as a first physical value and the PRI as a second physical value corresponding to the first physical value in association with each other in the physical value data buffer 143. A detailed description of the relevant specific example will be described.

[0121] The physical value data buffer 143 is configured to temporarily store data output in the correction operation of the correction operation unit 142. More specifically, the physical value data buffer 143 temporarily stores PAR and PRI related to each other by the correction operation unit 142. In addition, the physical value data buffer 143 may store data other than PAR and PRI.

[0122] The physical analysis operation unit 150 is configured to perform analysis operation, model fitting, etc. by using the data (e.g., the first physical value, the second physical value, etc.) stored in the physical value data buffer 143 (this process is referred to as "physical analysis operation"). In addition, the content of the physical analysis operation is not limited thereto. Figure 1 As shown in , the physical analysis operation unit 150 includes an analysis operation unit 151 , an operation control unit 152 , and an analysis data buffer 153 .

[0123] The analysis operation unit 151 is configured to perform physical analysis operations such as analysis operations and model fitting based on control of the operation control unit 152. Details of the physical analysis operations of the analysis operation unit 151 will be described later.

[0124] The operation control unit 152 is configured to generally control the physical analysis operation of the analysis operation unit 151. For example, the operation control unit 152 acquires data used in the physical analysis operation from the physical value data buffer 143 and provides the data to the analysis operation unit 151, or stores data output in the physical analysis operation of the analysis operation unit 151 in the analysis data buffer 153.

[0125] The analysis data buffer 153 is configured to temporarily store data output in the physical analysis operation of the analysis operation unit 151 .

[0126] The data visualization unit 160 is configured to perform various processes for visualization of the data stored in the analysis data buffer 153 (in the present embodiment, it is assumed that the data visualization unit 160 visualizes at least any one of the first physical value, the second physical value, and the third physical value). Figure 1 As shown in , the data visualization unit 160 includes a color mapping unit 161 and an image generation unit 162 .

[0127] The color mapping unit 161 performs mapping of colors to physical values, etc., by using the three primary colors of RGB, for example, for visualization of the data (especially each physical value, etc.) stored in the analysis data buffer 153 (in other words, the color mapping unit 161 performs conversion of physical values ​​to colors, etc. This process is called "color mapping").

[0128] The image generating unit 162 converts the physical values ​​converted into colors by the color mapping unit 161 into image information (imaging), or generates image information superimposed (overlap display) on RGB image information or the like.

[0129] The image output unit 170 is configured to output the image information generated by the image generation unit 162 to the display device 180, or output the image information to a predetermined external device (e.g., an external display device, a storage device, etc.) through a network. In addition, the image output unit 170 can perform predetermined image processing (e.g., archiving processing, etc.) on the image information according to the output.

[0130] The display device 180 is configured to provide the image information output by the image output unit 170 to the user by displaying the image information on a display or the like. In addition, there is no particular limitation on the display aspect of the image information. For example, the display device 180 can be used as a projector to project the image information onto a wall or the like.

[0131] In the above, an example of the functional configuration of the information processing device 100 according to the present embodiment has been described. In addition, with respect to each configuration described above, the image processing operation unit 130, the dimension conversion operation unit 140, the physical analysis operation unit 150, and the data visualization unit 160 can be used individually or in cooperation with each other as an operation unit for performing acquisition of physical values, etc. The physical value data buffer 143 can be used as a storage unit for storing the first physical value and the second physical value in association with each other. In addition, referring to Figure 1 The functional configuration described is merely illustrative, and the functional configuration of the information processing apparatus 100 according to the present embodiment is not limited to this example. For example, all or some of the functional configurations of the information processing apparatus 100 may be provided in an external device. The functional configuration of the information processing apparatus 100 according to the present embodiment may be flexibly modified according to specifications or management.

[0132] (2.2. Example of processing flow)

[0133] The functional configuration example of the information processing apparatus 100 has been described. Next, an example of the flow of processing performed by each functional configuration of the information processing apparatus 100 will be described.

[0134] (Overall Processing Flow by Information Processing Device 100)

[0135] First, refer to Figure 2 An example of the overall flow of processing performed by the information processing apparatus 100 is described.

[0136] In step S1000, input of captured image information is performed. More specifically, the multispectral camera 110 captures an image of a plant to generate multi-wavelength image information (captured image information), and the image input unit 120 acquires the captured image information and performs input of the captured image information.

[0137] In step S1004, the multi-wavelength image operation unit 131 performs multi-wavelength image operation. For example, the multi-wavelength image operation unit 131 generates RGB image information or NIR image information by using the captured image information, or generates captured image information indicating various vegetation indexes (such as NDVI image information, PRI image information, etc.).

[0138] In step S1008, the image structure analysis operation unit 133 performs an image structure analysis operation. For example, the image structure analysis operation unit 133 simply classifies the captured image information into the right side and the left side, or finely classifies the captured image information.

[0139] In step S1012, the removal operation unit 141 performs a removal operation in the dimension conversion operation. For example, the removal operation unit 141 analyzes the captured image information and removes an area of ​​the captured image information that captures an image of an object other than a plant (such as soil, etc.), an area of ​​the captured image that captures a shadow, etc.

[0140] In step S1016, the correction operation unit 142 performs the correction operation in the dimensional conversion operation. For example, the correction operation unit 142 converts the Nn value (pixel value) into the PAR of the leaf surface.

[0141] In step S1020 , the physical analysis operation unit 150 performs physical analysis operation. For example, the physical analysis operation unit 150 performs analysis operation, model fitting, etc. by using data (eg, first physical value, second physical value, etc.) stored in the physical value data buffer 143 .

[0142] In step S1024, the color mapping unit 161 performs color mapping. For example, the color mapping unit 161 maps colors to physical values ​​by using three primary colors of RGB.

[0143] In step S1028, the image generation unit 162 generates image information. For example, the image generation unit 162 generates image information by using the physical values ​​converted into colors by the color mapping unit 161, or generates image information superimposed (overlappingly displayed) on RGB image information or the like.

[0144] In step S1032, the image output unit 170 outputs the image information. For example, the image output unit 170 outputs the image information generated by the image generation unit 162 to the display device 180, and the display device 180 displays the image information on a display etc. As described above, a series of processes is terminated.

[0145] (Flow of input processing of captured image information)

[0146] Next, we will refer to Figure 3 and Figure 4 Description Figure 2 An example of the flow of the input processing of captured image information shown in step S1000 in FIG.

[0147] exist Figure 3 In step S1100, the image input unit 120 assigns an image set ID. Here, the "image set ID" is assumed to be information capable of identifying a plurality of image sets that are considered to be obtained by capturing images of the same part at the same time, but is not limited thereto. The image input unit 120 may assign an image set ID specified by a user, or assign a number that starts counting from zero whenever image capturing is performed as the image set ID.

[0148] In step S1104, a plurality of captured image information of different wavelength bands are input by the multispectral camera 110. In this example, for example, a plurality of captured image information of a near infrared wavelength band (hereinafter referred to as "Nn". The wavelength band is about 850 [nm] to about 870 [nm]), a red wavelength band (hereinafter referred to as "Rn". The wavelength band is about 650 [nm] to about 670 [nm]), a yellow-green wavelength band (hereinafter referred to as "Gr". The wavelength band is about 560 [nm] to about 580 [nm]), a green wavelength band (hereinafter referred to as "Gp". The wavelength band is about 525 [nm] to about 545 [nm]), and a blue wavelength band (hereinafter referred to as "B". The wavelength band is about 400 [nm] to about 440 [nm]) are sequentially input.

[0149] In step S1108, the image input unit 120 assigns an image ID to the captured image information of each wavelength band. The "image ID" is assumed to be information capable of identifying the captured image information of each wavelength band. The image IDs of the captured image information that are frequently used are predefined, and the image input unit 120 automatically assigns the image IDs based on the definition. In addition, the image input unit 120 can assign the image ID by performing a predetermined operation. When the operations in steps S1104 and S1108 are performed for multiple pieces of captured image information of all wavelength bands that are the subject (refer to step S1112), the input processing of a series of captured image information is terminated.

[0150] Figure 4 The data state in the stage where the input processing of captured image information is terminated is illustrated. "#0001" is assigned to the image set ID, and "#86020", "#66020", "#57020", "#53520" and "#42040" are assigned to the image IDs of the respective captured image information of Nn, Rn, Gr, Gp and B.

[0151] (Process of multi-wavelength image calculation)

[0152] Next, we will refer to Figures 5 to 8 Described in Figure 2 An example of the flow of the multi-wavelength image operation shown in step S1004.

[0153] exist Figure 5In step S1200, the multi-wavelength image operation unit 131 generates RGB image information by using the input captured image information. In step S1204, the multi-wavelength image operation unit 131 assigns an image ID to the generated RGB image information. In step S1208, the multi-wavelength image operation unit 131 generates leaf light intensity image information by using the input captured image information. Here, in this example, it is assumed that the captured image information of Nn is used as the leaf light intensity image information as it is, and therefore no image ID is newly assigned (the image ID utilizing the captured image information of Nn). In addition, the light of Nn is less likely to be absorbed by the leaves but is greatly reflected by the leaves, and therefore the captured image information of Nn is used as the leaf light intensity image information. However, other information (for example, captured image information of visible light such as green light) may also be used as the leaf light intensity image information.

[0154] In step S1212, the multi-wavelength image operation unit 131 generates PRI image information by using the input captured image information. The generation process of the PRI image information will be described in detail at a later stage. In step S1216, the multi-wavelength image operation unit 131 assigns an image ID to the generated PRI image information.

[0155] In step S1220, the multi-wavelength image operation unit 131 generates NDVI image information by using the input captured image information. The generation process of the NDVI image information will be described in detail at a later stage. In step S1224, the multi-wavelength image operation unit 131 assigns an image ID to the generated NDVI image information, and a series of multi-wavelength image operations are terminated.

[0156] Here, we will refer to Figure 6 Description Figure 5 The process of generating the PRI image information shown in step S1212 in FIG. Figure 6 In step S1300, the multi-wavelength image operation unit 131 reads one pixel of the Gr image information (image ID: #57020). In step S1304, the multi-wavelength image operation unit 131 reads one pixel (one pixel corresponding to one pixel of the Gr image information) of the Gp image information (image ID: #53520). In step S1308, the multi-wavelength image operation unit 131 calculates the PRI by inputting the information of each pixel read by the multi-wavelength image operation unit 131 into the following Formula 2. In addition, a typical PRI is calculated by (Gp-Gr) / (Gp+Gr) as in Formula 1. However, in this example, the PRI is defined so that the PRI increases when the stress response of the plant to various stress factors is large, and therefore the following Formula 2 is used.

[0157] [Mathematical expression 2]

[0158]

[0159]

[0160] When the operations in step S1300 to step S1308 are performed for all pixels in the captured image information (refer to step S1312 ), a series of generation processing of PRI image information is terminated.

[0161] Here, we will refer to Figure 7 Description Figure 5 The process of generating the NDVI image information shown in step S1220 in FIG. Figure 7 In step S1400, the multi-wavelength image operation unit 131 reads one pixel of the Nn image information (image ID: #86020). In step S1404, the multi-wavelength image operation unit 131 reads one pixel (one pixel corresponding to one pixel of the Nn image information) of the Rn image information (image ID: #66020). In step S1408, the multi-wavelength image operation unit 131 calculates the NDVI by inputting the information of each pixel read by the multi-wavelength image operation unit 131 into the following formula 3.

[0162] [Mathematical expression 3]

[0163]

[0164] When the operations in step S1400 to step S1408 are performed for all pixels in the captured image information (refer to step S1412 ), a series of generation processing of NDVI image information is terminated.

[0165] Figure 8 The data state in the stage where the multi-wavelength image operation is terminated is illustrated. "#001", "#003", and "#002" are the assigned image IDs of the newly generated RGB image information, PRI image information, and NDVI image information. In addition, a description of the main points of generating the PRI image information and the NDVI image information has been given, but the image information generated according to the purpose of the physical analysis can be flexibly modified.

[0166] (Process of image structure analysis operation)

[0167] Next, we will refer to Figures 9 to 13 Description Figure 2 An example of the flow of the image structure analysis operation shown in step S1008 in FIG. 1004. The image structure analysis operation unit 133 classifies the captured image information into a plurality of regions through the image structure analysis operation.

[0168] More specifically, in Fig. 9 In step S1500, the image structure analysis operation unit 133 classifies the leaf light intensity image information. The details of the classification processing of the image information will be described in detail at a later stage. In addition, as in the leaf light intensity image information, the image structure analysis operation unit 133 classifies the PRI image information in step S1504 and classifies the NDVI image information in step S1508. Accordingly, a series of image structure analysis operations are terminated.

[0169] Here, we will refer to Fig.10 Describe the process of image information classification. Fig.10 In step S1600, the image structure analysis operation unit 133 reads each piece of image information. In step S1604, the image structure analysis operation unit 133 cuts out a portion of the image information. Fig.11 As shown in , the image structure analysis operation unit 133 cuts off the left 40[%] of the image information. In step S1608, the image structure analysis operation unit 133 assigns a classification ID to the cut-out portion. The "classification ID" is information that can identify the cut-out portion and is assumed to be a subclassification of the image ID. In step S1612, the image structure analysis operation unit 133 cuts out another portion of the image information. For example, Fig.11 As shown in , the image structure analysis operation unit 133 cuts out the right 40[%] of the image information. In step S1616, the image structure analysis operation unit 133 assigns a classification ID to the cut-out portion. According to this, a series of classification processing of the image information is terminated.

[0170] In addition, the method of assigning the classification ID is not limited to the above configuration. For example, in the case where the image structure analysis operation unit 133 simply divides the image information into 64 pieces in the vertical and horizontal directions, etc., a classification ID can be automatically generated for each divided area. In addition, in the case where the image structure analysis operation unit 133 recognizes a part specified by an image recognition technology (such as a part of a plant, etc.), an ID corresponding to the recognized part can be assigned to the classification ID (for example, in the case of recognizing a leaf, "Leaf" etc. can be assigned to the classification ID). In addition, the image structure analysis operation unit 133 can set the area of ​​the classification ID based on the resolution of the display to which the image information after color mapping is finally output. For example, the image structure analysis operation unit 133 can set the classification ID for each area having a size corresponding to the resolution of the display. In addition, the image structure analysis operation unit 133 can set the area of ​​the classification ID based on the physical value average unit in the color mapping (which will be described in detail at a later stage). For example, in the case where the image information is classified in units of several pixels in the vertical and horizontal directions during color mapping, the image structure analysis operation unit 133 can set the classification ID for each area having a size corresponding to the classification unit. In addition, in the case where defects are included in the data of the area of ​​the classification ID set for the first time, the image structure analysis operation unit 133 can dynamically set the classification ID to compensate for the influence of the defects. For example, in the case where all data (all pixel data) of the area of ​​the classification ID have defects, the image structure analysis operation unit 133 can set the area of ​​the classification ID so that defect-free data is included in the area. Here, for example, "defect" means deficiencies such as lack of data, so-called "halo" caused by the imaging environment, but defects are not limited to this. In addition, the image structure analysis operation unit 133 can set the area of ​​the classification ID so as to include normal data of the amount required for evaluation in the physical analysis at the later stage. Here, for example, "normal" means that the leaf surface light intensity is equal to or greater than a constant value, etc., but is not limited to this. In addition, a user-specified ID can be assigned to the classification ID.

[0171] In addition, the processing of cutting the right local part and the left local part in the captured image information makes it easy to compare the stress responses of different individuals in the measurement at the same time, so it should be understood that this processing is the most simplified processing example. In addition, in the research experiment, different measurement objects are respectively deployed in the left part and the right part in the captured image information (for example, the case where the plant treated with environmental stress is deployed on one side and the plant not treated with environmental stress is deployed on the other side as a comparison object, etc.), and the measurement and comparison of physical values ​​are performed. Here, the ratio of the cut-out part to the entire image information is not limited to 40 [%]. In addition, as described above, the image structure analysis operation unit 133 can analyze the captured image information by using a predetermined image recognition technology to identify the objects included in the captured image information (for example, plant parts including leaves (upper leaves, middle leaves, lower leaves, etc.), flowers, fruits, stems, etc.), and can classify the captured image information for each object. In addition, the image structure analysis operation unit 133 can classify the captured image information in three dimensions according to the image recognition technology (for example, the known three-dimensional image information analysis technology, etc.). In addition, the above-mentioned various classification methods can be specified by the user.

[0172] Fig.12 The diagram shows the data status at the stage where the image structure analysis operation is terminated. Fig.12 In the example of , the classification ID is assigned in units of pixels of image information and is assigned to each pixel. Fig.13 As shown in , files can be classified for each classification ID (in Fig.13 In the example of , "L" represents the left 40[%] in the image information, and "R" represents the right 40[%] in the image information). As described above, the image structure analysis operation unit 133 divides the captured image information, so that processing corresponding to the classification can be performed in a later stage. For example, the image structure analysis operation unit 133 classifies the upper leaves and the lower leaves from each other, so that parameters used in operations related to the classification corresponding to the upper leaves and the classification corresponding to the lower leaves are changed to different parameters in a later stage. In addition, the image structure analysis operation unit 133 classifies the types of plants, so in a case where the presence of multiple plants in the captured image information is revealed, the algorithm used in the operation in the later stage can be changed.

[0173] (Removal operation process)

[0174] Next, we will refer to Fig.14 Described in Figure 2 An example of the flow of the removal operation in the dimensional conversion operation shown in step S1012 of FIG. The removal operation unit 141 removes an area in which the analysis accuracy is predicted to be lower than a predetermined value in the area of ​​captured image information from the operation object.

[0175] More specifically, in Fig.14 In step S1700, the removal operation unit 141 reads a pixel of the NDVI image information (image ID: #002). In step S1704, the removal operation unit 141 confirms whether the NDVI in the pixel is a value within a constant range. Here, the "constant range" means a range that can determine whether an area is an area including plants in the NDVI image information, and when the NDVI is a value within a constant range (such as NDVI>0.5, etc.), it can be said that the possibility of including plants in the area is high.

[0176] In the case where NDVI is a value within a constant range (step S1704 / yes), in step S1708, the removal operation unit 141 reads one pixel of the Nn image information (image ID: #86020) (one pixel corresponding to one pixel of the NDVI image information). In step S1712, the removal operation unit 141 confirms whether the Nn value (pixel value) in the pixel is a value within a constant range. Here, the "constant range" means a range that can determine whether an area is an area in the Nn image information in which an image of a shadow is captured, and in the case where the Nn value is a value within a constant range, it can be said that there is a high possibility that the shadow is not included in the area. The shadow of the leaves of a plant or the scattered reflection of the leaves has a unique spectral characteristic (optical complexity), and it is difficult to appropriately evaluate in which way the irradiation of light is performed. However, when confirming whether the Nn value is a value within a constant range, the removal operation unit 141 can set a part with small optical complexity caused by the shape of the plant as an analysis object.

[0177] In the case where the Nn value is a value within a constant range (step S1712 / Yes), in step S1716, the removal operation unit 141 determines that the Nn value and the PRI value of the corresponding pixel represent appropriate leaf light intensity and PRI value. In step S1720, the removal operation unit 141 assigns a physical value ID to the Nn value and the PRI value, and stores the physical value ID. In addition, in the case where it is determined in step S1704 that the NDVI is not a value within a constant range (step S1704 / No) or in the case where it is determined in step S1712 that the Nn value is not a value within a constant range (step S1712 / No), the removal operation unit 141 does not perform the assignment of the physical value ID to the Nn value and the PRI value, etc. Accordingly, the removal operation unit 141 can remove from the operation object an area in the image information where the analysis accuracy is predicted to be lower than a predetermined value. When the operations in steps S1700 to S1720 are performed for all pixels (refer to step S1724), a series of removal operations are terminated.

[0178] (Calibration calculation process)

[0179] Next, we will refer to Fig.15 Description Figure 2 An example of the flow of the correction operation in the dimensional conversion operation shown in step S1016 in FIG. The correction operation unit 142 performs a process of converting the Nn value (pixel value) into leaf surface light intensity (also referred to as photosynthetically active radiation (PAR)).

[0180] More specifically, in step S1800, the correction operation unit 142 converts the Nn value (pixel value) into the actual NIR reflection intensity of the leaf surface by using the relationship between the incident light intensity and the pixel value determined by the sensitivity or gain setting of the imager of the multispectral camera 110. In step S1804, the correction operation unit 142 converts the NIR reflection intensity of the leaf surface into the NIR intensity of the light source by using the reflectivity k of the NIR of a group of vegetation (for example, about 0.8). In step S1808, the correction operation unit 142 converts the NIR intensity of the light source into the PAR intensity of the light source by using the ratio of the PAR intensity of the light source to the NIR intensity. Accordingly, a series of processing is terminated. In addition, the correction operation unit 142 obtains the ratio of the PAR intensity of the light source to the NIR intensity from the sensor, or uses a representative value as the ratio. In addition, in a case where the information processing device 100 calculates relative values ​​of a plant's stress response to various stress factors (for example, in a case where knowing the differences in physical values ​​in each area is advantageous, such as in a case where the information processing device 100 expects to find an individual receiving relatively strong stress in captured image information), the conversion processing performed by the correction operation unit 142 can be omitted.

[0181] Fig.16 The diagram shows the data status at the stage where the removal operation and the correction operation are terminated. Fig.16 In the example, "physical value 1 (first physical value)" represents the leaf light intensity (PAR), and the ID identifying the leaf light intensity (PAR) (in this example, #1000) is set to "physical value ID1". In addition, "physical value 2 (second physical value)" represents PRI, and the ID identifying PRI (in this example, #1100) is set to "physical value ID2". In addition, Fig.16 The "physical set ID" in is information that can identify a data set of physical values, and plays a role in associating physical value 1 (first physical value) and physical value 2 (second physical value) with each other. In this way, the dimension conversion operation unit 140 converts the dimensional information of the vertical and horizontal directions in the captured image information into dimensional information of the physical values ​​of the environment and the response, and can store the first physical value corresponding to the environment and the second physical value corresponding to the response in association with each other. In addition, as Fig.16As shown in , the data output to the physical value data buffer 143 includes an image set ID, a classification ID, and an image position ID capable of identifying a position in captured image information, but the content of the data is not limited thereto.

[0182] (Analysis of the operation process and processing details)

[0183] Next, we will describe Figure 2 The analysis operation shown in step S1020 in . The analysis operation unit 151 outputs a third physical value indicating the stress response (state) of the plant caused by various stress factors based on the leaf light intensity (PAR) as a first physical value corresponding to the environment and the PRI as a second physical value corresponding to the response. For example, in the case of environmental pressure in which stomatal closure occurs due to soil drying, the magnitude of the pressure actually received by the plant is greatly affected by the intensity of light received by the plant. The reason is as follows. Since the stomata are closed, the efficiency of the Calvin cycle is reduced, and the total amount of energy received by the photochemical system 2 that converts light into electrons is reduced, but the photochemical system 2 continues to convert light into electrons.

[0184] In this case, as the intensity of light received by the plant increases, the amount of excess energy also increases, the photosynthesis rate decreases, and, for example, hydrogen peroxide is produced inside the plant depending on the situation. That is, it is also considered that the plant is directly damaged. On the contrary, even in the presence of environmental stress, when the intensity of light received by the plant is weak, the impact is small. In addition, if the value related to the measured pressure does not correspond to the same light intensity, it is difficult to perform comparison of the states of different individuals or comparison when the environmental pressure changes.

[0185] Here, the analysis operation unit 151 extracts the physical value when the intensity of the light irradiating at least the leaves is equal to or greater than a constant value as a value corresponding to the environmental pressure of the plant. Fig.17 As shown in the table of , the physical values ​​of different individuals measured under the same measurement conditions (same time) can be analyzed by comparing the relative differences of the physical values. Fig.18 As shown in , analysis of the physical values ​​(pressure) of individual A and individual B measured at the same time can be performed by calculating the relative difference between the physical values. In addition, the physical value when the intensity of the light irradiating the leaves is equal to or greater than a constant value can be used in the processing, so the analysis operation unit 151 can output the environmental pressure with respect to the part where photosynthesis is not fully performed and the excess energy is large. In addition, in the case where the physical value when the intensity of the light irradiating the leaves is equal to or greater than a constant value is used in the analysis, the number of samples of the analysis object (the number of physical set IDs) increases, compared with the physical value when the intensity of the light irradiating the leaves is within a constant range as described later in the analysis, so it may be possible to make a more accurate comparison.

[0186] In addition, the analysis operation unit 151 extracts the physical value when the intensity of the light irradiating the leaves is within a constant range as a value corresponding to the environmental pressure of the plant as needed. Fig.17 As shown in the table, not only the analysis of physical values ​​of different individuals measured under the same measurement conditions (same time) is possible, but also the analysis of physical values ​​of the same individual or different individuals measured under different measurement conditions (different time) is possible. Fig.19A , Fig.19B and Fig.19C As shown in Fig.19B In the case of Fig.19C In the case where the sun's altitude changes, the analysis operation unit 151 can implement the analysis by using the physical value of the part of the light intensity that is within a constant range.

[0187] In addition, if Fig. 20A and Fig. 20B As shown in , in the leaf, there may be a normal part 10, and there may be a part 11 with an unusual shape state (such as withered). In the part 11 with the unusual shape state, the distribution of the leaf surface light intensity is more likely to change compared with the normal state. Therefore, even in the analysis of the physical values ​​of different individuals measured under the same measurement conditions (same time), it is more preferable to use the physical values ​​when the intensity of the light irradiating the leaf is within a constant range. Here, Figures 18 to 20B The illustration shows a single leaf from the plant, but Figures 18 to 20B The individual leaves shown in the figure can be treated as individual entities of the plant. More specifically, after the inclination of the individual leaves is analyzed in detail, the inclination distribution of the leaves when the individual entity is replaced by one leaf can be obtained based on the inclination trend of the individual leaves included in the individual entity. In one aspect of the present disclosure, it is not necessary to obtain image resolution at the level of individual leaves, and for example, the resolution in parts such as upper leaves that are considered to be illuminated approximately uniformly with light does not need to be high. That is, when Figures 18 to 20B Optimization of treatment is possible when the individual leaves shown in are treated as models to indicate the individual whole of the plant.

[0188] Next, we will refer to Fig.21 An example describing the flow of analytical operations. Fig.21 An example of the flow of analysis operation in the case where the analysis operation unit 151 extracts a physical value when the intensity of light irradiating the leaves is equal to or greater than a constant value and within a constant range as a value corresponding to the environmental stress of the plant is illustrated.

[0189] In step S1900, the analysis operation unit 151 reads a physical set ID of the image set ID (e.g., #0001) that is the target. In step S1904, the analysis operation unit 151 confirms whether the physical value 1 (leaf light intensity (PAR)) of the physical set ID is equal to or greater than a constant value and is within a constant range. In the case where the physical value 1 (leaf light intensity (PAR)) is equal to or greater than a constant value and is within a constant range (step S1904 / Yes), in step S1908, the analysis operation unit 151 copies the value of PRI (physical value 2 (second physical value)) in the physical set ID as a pressure measurement value (physical value 3 (third physical value)). In addition, in the case where the physical value 1 (leaf light intensity (PAR)) is lower than a constant value or is not within a constant range (step S1904 / No), the processing in step S1908 is not performed. When the operations in steps S1900 to S1908 are performed for all physical set IDs in the image set ID that has become the target (refer to step S1912), a series of analysis operations are terminated.

[0190] Here, we will refer to Figure 22 to Figure 24 A description is given of the relationship between the conditions of the intensity of light irradiating the leaves (PAR) and the PRI. Fig. 22 The PRI group 20 is illustrated as being replicated when the intensity of light (PAR) illuminating the leaves is equal to or greater than a constant value (equal to or greater than PAR_TH_MIN in the figure). Fig.23 The PRI group 21 is illustrated as being replicated when the intensity of light irradiating the leaves (PAR) is equal to or greater than a constant value and is within a constant range (PAR_TH_MIN to PAR_TH_MAX in the drawing).

[0191] In addition, the analysis operation unit 151 can analyze the data in the stage where the removal operation and the correction operation are terminated (refer to Fig.16 ) extracts the minimum value (hereinafter referred to as "PAR_RSLT_MIN") and the maximum value (hereinafter referred to as "PAR_RSLT_MAX") of PAR, and these values ​​can be input into Formula 4 and Formula 5 to be described below, as shown in Fig.24 Set the constant range (the range defined by PAR_TH_MIN and PAR_TH_MAX) as shown in . In addition, Fig.24 The PRI group 22 is shown in the figure in the constant range set by Formula 4 and Formula 5. In addition, as shown from Figure 22 to Figure 24 It can be seen that, in the case where PAR is above PAR_TH_MIN, PRI tends to increase as PAR becomes higher.

[0192] PAR_TH_MAX=PAR_RSLT_MAX-m(PAR_RSLT_MAX-PAR_RSLT_MIN)...(Formula 4)

[0193] m: Predetermined coefficient (can be specified by the user)

[0194] PAR_TH_MIN=PAR_RSLT_MIN+n(PRA_RSLT_MAX-PAR_RSLT_MIN)...(Formula 5)

[0195] n: Predetermined coefficient (can be specified by the user)

[0196] Fig.25 The diagram shows the state of the data at the stage where the analysis operation is terminated. Fig.25 In , the pressure measurement value (third physical value) is added as "physical value 3", and an ID (in this example, #2000) that can identify the pressure measurement value is set as "physical value ID3". Fig.25 In the example of , the analysis operation is performed for physical set ID#00000001, and as described above, the value of PRI (physical value 2) is copied as the pressure measurement value (physical value 3).

[0197] also, Fig.26 The figure shows the result of simulating the relationship between PAR and ETR using the Farquhar model for each value of stomatal conductance (Gs) by the analysis operation unit 151. It can be seen that as Gs decreases, ETR is limited to the high PAR region.

[0198] also, Fig. 27 The simulation results of the stress amount of the plant when Gs is 0.1 or 1.0 are shown in the figure. In addition, it is assumed that the initial gradient of photosynthesis is set to The amount of stress (Stress) in the case is defined by the following formula 6.

[0199] [Mathematical expression 4]

[0200]

[0201] Fig. 27The broken line 30 in indicates a PAR where no pressure response occurs even when Gs is 0.1 (stomatal closure), and indicates that it is difficult to evaluate the pressure response by using PAR even when the plant makes a pressure response such as stomatal closure to an environmental pressure such as soil drying. On the other hand, the arrow 31 indicates the width of the PAR where the pressure response is the same when Gs is 0.1 and when Gs is 1.0, and this indicates that it is difficult to compare the pressure response values ​​under conditions where PAR is different. When PAR_TH_MAX and PAR_TH_MIN are obtained, in addition to performing the operations in Formula 4 and Formula 5, the analysis operation unit 151 can also automatically calculate the optimal setting value by performing simulation. In addition, the content of the analysis operation performed by the analysis operation unit 151 using each physical value is not limited to the above content. For example, a study has been conducted to calculate ETR by using two physical values, PRI and SIF, and the analysis operation unit 151 can perform an operation to obtain ETR by obtaining PRI and SIF.

[0202] (Color mapping process)

[0203] Next, we will refer to Figures 28 to 37 Described in Figure 2 The example of the process of color mapping shown in step S1024 of . The color mapping unit 161 associates at least any one of the first physical value, the second physical value, and the third physical value outputted in the previous stage with a color. The color mapping unit 161 can associate all physical values ​​(the first physical value, the second physical value, and the third physical value) with a color. In addition, hereinafter, a method of performing color mapping for each classification ID will be described as an example (in this example, the image structure analysis operation unit 133 performs classification of the captured image information into the left side (classification ID: #0) and the right side (classification ID: #1)). For example, the color mapping unit 161 calculates the average value of the physical value for each classification ID, and performs color mapping corresponding to the average value. In addition, the image structure analysis operation unit 133 can identify objects included in the captured image information (for example, plant parts including leaves (upper leaves, middle leaves, lower leaves, etc.), flowers, fruits, stems, etc., etc.), and a classification ID can be assigned to each object. Based on this, a physical value can be calculated for each object.

[0204] exist Fig.28In step S2000, the color mapping unit 161 sets "#0 (left side)" as the classification ID. In step S2004, the color mapping unit 161 specifies the image set ID #0001 and the classification ID, and retrieves the physical set ID from the analysis data buffer 153. In step S2008, the color mapping unit 161 temporarily stores the value (set to "x") of the pressure measurement value (physical value 3 (third physical value)) of the detected physical set ID and the number of detected physical set IDs (set to "y"). In addition, until the retrieval for one classification ID is terminated (step S2012 / No), the color mapping unit 161 repeatedly performs the processing in steps S2004 and S2008. In the case where the retrieval for one classification ID is terminated (step S2012 / Yes), in step S2016, the color mapping unit 161 obtains the average value of the pressure measurement values ​​(physical value 3) in the classification ID by using x and y. More specifically, the color mapping unit 161 divides the total value of x by y to obtain the average value of the pressure measurement value (physical value 3) (in other words, it can be said that the color mapping unit 161 outputs the third physical value for each classification (region of the classification)).

[0205] In step S2020, the color mapping unit 161 performs color mapping on the average value, and the image generation unit 162 generates image information based on the color mapping result. In addition, regarding color mapping, the user can specify the minimum and maximum values ​​(of physical values) corresponding to each color, or the minimum and maximum values ​​of the physical values ​​in the image set ID determined in the above process can be associated with each color.

[0206] Furthermore, when the search for all the classification IDs has not been terminated (step S2024 / No), in step S2028, the color mapping unit 161 adds "1" to the classification ID and performs the processing in steps S2004 to S2024 again. When the search for all the classification IDs has been terminated (step S2024 / Yes), a series of processing related to color mapping is terminated.

[0207] Figures 29 to 37 An example of image information displayed after color mapping is shown. For example, Fig.29 The diagram shows image information in the case where the color mapping unit 161 performs color mapping of the pressure measurement value (physical value 3) to the area corresponding to the classification ID #0 (the left area in the screen information) and the area corresponding to the classification ID #1 (the right area in the screen information). In the case where rough accuracy and display are sufficient, the following may be adopted: Fig.29 A simplified display method in .

[0208] Fig.30The image generation unit 162 makes the parts other than the plants transparent and converts the color mapping result ( Fig.29 In this case, the image generation unit 162 can realize display by reading out the NDVI image information (image ID: #002) from the structured image buffer 134 and making the image position where the NDVI is lower than a constant value transparent. Fig.29 Compared with , a more detailed display is achieved.

[0209] Fig.31 The image generation unit 162 is shown in FIG. Fig.28 The area of ​​the image position ID not detected in the search process of step S2004 (that is, the area where the pressure measurement value (physical value 3) is not set) is transparent and the color mapping result ( Fig.29 According to this, only the portion where the stress response is evaluated (can be evaluated) is color-mapped, so the user can confirm the magnitude of the stress response expressed by the color in relation to the image position (actual position).

[0210] Fig.32A RGB image information (image ID: #001) is shown, and Fig.32B The image generation unit 162 is shown in FIG. Fig.31 The image information obtained after the image information in is superimposed on the RGB image information. According to this, the user can more easily identify the part of the plant with a high (or low) pressure measurement value (physical value 3).

[0211] Fig.33A The image information is shown in the case where the image position ID is actively used and the color mapping unit 161 does not calculate the average value of the physical value for each classification ID but performs color mapping for each image position ID. Fig.33B The image generation unit 162 is shown in FIG. Fig.33A Image information when the image information in is overlapped on the RGB image information.

[0212] In addition, the color mapping unit 161 can calculate the average value of the pressure measurement value (physical value 3) for each constant area (for example, an area classified in units of several pixels in the vertical and horizontal directions in the image information. Hereinafter, the area is referred to as a "block") by using the image position ID instead of the classification ID, and color mapping can be performed for each block. Fig.34An example of image information generated by this method is illustrated. A description has been given of the main points that the captured image information is classified and assigned a classification ID by the image structure analysis operation unit 133, but the color mapping unit 161 may perform color mapping for each block without using a classification ID (this is equivalent to a configuration in which the color mapping unit 161 classifies the captured image information for each block). The advantages of the configuration in which the color mapping unit 161 classifies the captured image information for each block are as follows. According to the result of the removal operation or the analysis operation, when the amount of data that can be evaluated is very small, an image that is easy for the user to see may not be generated in some cases, such as a case where the color mapping area is very small (for example, the color mapping area has a dot shape). In this case, when the color mapping unit 161 classifies the captured image information in units of appropriate blocks corresponding to display convenience (a unit including at least one physical value is preferred), generation of an image that is easy for the user to see can be achieved. In the captured image information, in the case where measurement is performed by sampling, a particularly advantageous effect is obtained (there is no limitation to the case where measurement is performed by sampling).

[0213] Fig.35 The image generation unit 162 makes the parts other than the plants transparent and converts the color mapping result ( Fig.34 The image information in the case where the content in () is superimposed on the obtained image. Fig.34 Compared with , a more detailed display is achieved.

[0214] Fig.36 The image generation unit 16 is shown in FIG. Fig.28 The area of ​​the image position ID not detected in the search process of step S2004 (that is, the area where the pressure measurement value (physical value 3) is not set) is transparent and the color mapping result ( Fig.34 According to this, only the portion where the stress response is evaluated (can be evaluated) is color-mapped, so the user can confirm the magnitude of the stress response expressed by the color in relation to the image position (actual position).

[0215] Fig.37 The image generation unit 162 shows the color mapping result ( Fig.34 The image information is as follows: the image information in which the content in the image is superimposed on the RGB image information. Accordingly, the user can more easily identify the part of the plant where the pressure measurement value (physical value 3) is high (or low).

[0216] About Reference Figures 29 to 37In terms of the display described, it can be said that the image generation unit 162 generates image information in which the third physical value (pressure measurement value) is related to the original spatial position (e.g., the position of the plant, etc.). In addition, the image generation unit 162 can generate image information in which not only the third physical value (pressure measurement value) is related to the original spatial position but also the first physical value or the second physical value is related to the original spatial position. In addition, the color mapping method and the image generation method (image synthesis method) can be flexibly combined with each other.

[0217] <3. Second embodiment>

[0218] A description has been given of the first embodiment according to the present disclosure. Next, a description will be given of the second embodiment according to the present disclosure.

[0219] Unlike the first embodiment, the information processing apparatus 100 according to the second embodiment can perform various processes or can correct various data (such as leaf surface light intensity, etc.) by using data acquired by various sensors or various measuring devices. In addition, the information processing apparatus 100 according to the second embodiment performs color mapping by using physical values, and can also express physical values ​​using graphics.

[0220] (3.1. Functional configuration example)

[0221] First, refer to Fig.38 A functional configuration example of the information processing apparatus 100 according to the second embodiment is described. In addition, for contents overlapping with the functional configuration example of the information processing apparatus 100 according to the first embodiment, description thereof will be omitted.

[0222] like Fig.38 As shown in , the information processing apparatus 100 according to the second embodiment additionally includes a sensor and measuring device 190 , a sensor data input unit 200 , and a graph generating unit 163 .

[0223] The sensor and measuring device 190 includes various sensors and various measuring devices, and is configured to obtain various data (hereinafter referred to as "sensor data") by using the various sensors and various measuring devices. For example, the sensor and measuring device 190 includes an ambient light sensor, a temperature sensor, a humidity sensor, a CO2 concentration sensor, etc., and obtains sensor data by using these sensors. In addition, the types of sensors and measuring devices provided in the sensor and measuring device 190 are not particularly limited.

[0224] The sensor data input unit 200 is configured to serve as an interface for acquiring sensor data from the sensor and measurement device 190. In addition, the sensor data input unit 200 may perform various processes such as filtering and conversion of sensor data in addition to simply acquiring sensor data. In addition, the content of the processing performed by the sensor data input unit 200 is not particularly limited.

[0225] The correction operation unit 142 according to the second embodiment corrects various data by using the sensor data input to the sensor data input unit 200. For example, the correction operation unit 142 corrects the physical value 1 (leaf surface light intensity (PAR)) by using the sensor data (sensor data related to the environment) sent from the ambient light sensor. According to this, the accuracy of physical analysis and the like in the later stage is improved. The correction processing of the physical value 1 (leaf surface light intensity (PAR)) will be described in detail in the later stage.

[0226] The graph generation unit 163 is configured to visualize various physical values ​​by using various graphs (in the present embodiment, it is assumed that the graph generation unit 163 visualizes at least any one of the first physical value, the second physical value, and the third physical value by using a graph). Accordingly, the user can more easily understand the relationship between the physical values, etc. In addition, the type of graph generated by the graph generation unit 163 is not particularly limited. For example, the graph generation unit 163 can generate a line graph, a scatter graph, etc. The graph generation process will be described in detail in the later stage.

[0227] (3.2. Example of process flow and process details)

[0228] Hereinabove, the functional configuration example of the information processing apparatus 100 according to the second embodiment has been described. Next, an example of a process flow performed by each functional configuration of the information processing apparatus 100 according to the second embodiment will be described, and details of the process will be described.

[0229] First, refer to Fig.39 An example of the overall flow of processing performed by the information processing apparatus 100 according to the second embodiment is described. The second embodiment is different from the first embodiment in that Fig.39 Step S2116 and step S2124 in the embodiment 1 are omitted, and the other steps may be the same as those in the first embodiment. Hereinafter, the differences between the first embodiment and the second embodiment will be described.

[0230] (Calibration calculation process)

[0231] exist Fig.39In step S2116, the correction operation unit 142 converts the Nn value (pixel value) into the leaf light intensity (PAR) by using the sensor data sent from the ambient light sensor. Fig.40 The correction operation in the dimensionality conversion operation performed in step S2116 is described.

[0232] exist Fig.40 In step S2200, the correction operation unit 142 retrieves the maximum value of Nn pixel values ​​in the physical value of the image set ID having the operation object, and sets the value to M. In step S2204, the correction operation unit 142 obtains the PAR intensity (sensor data related to the environment) of the environmental sensor obtained by the sensor and measurement device 190 and input to the sensor data input unit 200, and sets the value to P. In step S2208, the correction operation unit 142 obtains the ratio of the PAR intensity to the NIR intensity of the ambient light sensor from the sensor and measurement device 190 and the like, and sets the value to K. In step S2212, the correction operation unit 142 calculates the correction coefficient C expressed by Formula 7.

[0233] [Mathematical expression 5]

[0234]

[0235] In step S2216, the correction operation unit 142 retrieves the physical value 1 from the physical values ​​of the image set ID having the operation object. Furthermore, in the case where the physical value 1 is found (step S2220 / Yes), in step S2224, the correction operation unit 142 multiplies the physical value 1 by the correction coefficient C to calculate a new physical value 1 (in other words, correction of the physical value 1 is performed by using the correction coefficient C). In the case where the physical value 1 is not found (step S2220 / No), a series of correction operations are terminated.

[0236] In addition, although Fig.40 Although not shown in the figure, the correction operation unit 142 may temporarily store various sensor data in the physical value data buffer 143 for processing in a later stage. For example, the correction operation unit 142 may temporarily store sensor data (a kind of the first physical value) of the temperature sensor in the physical value data buffer 143. The details will be described later.

[0237] Fig.41 The data state at the stage where the correction operation is terminated is shown in the figure. As described above, the physical value 1 is corrected using the correction coefficient C. In addition, as described above, Fig.41, the sensor data of the temperature sensor (a kind of the first physical value. In this example, 20.2 [°C]) is added as "physical value 3", and the ID for identifying the sensor data of the temperature sensor (in this example, #1010) is set to "physical ID3". In addition, in the case where measurement (image capture) is performed on the same object at a timing where the temperature conditions are different from each other, the sensor data (temperature) at the time of measurement is reflected as the physical value 3 of each image set ID. For example, in the case where the temperature at the time of the second measurement is 25.0 [°C], 25.0 [°C] is reflected as the physical value 3 of the image set ID (e.g., #0002) at the time of measurement.

[0238] (Details of the graphics generation process)

[0239] exist Fig.39 In step S2124, the graph generation unit 163 can generate various graphs (e.g., line graphs, scatter graphs, etc.) by using various physical values ​​(first physical value, second physical value, etc.). In addition, the content (type, etc.) of the graph generated by the graph generation unit 163 is not particularly limited. In addition, the content (type, etc.) of the graph generated by the graph generation unit 163 can be determined based on input from the user.

[0240] As an example of a graph generated by the graph generation unit 163, Fig.42 A view is shown in which physical values ​​corresponding to a plurality of temperatures (in addition, since the temperature is a value corresponding to the environment, it is one of the first physical values, and in this example, 20 [° C.], 25 [° C.], and 30 [° C.]) are plotted as a scatter plot with leaf light intensity (PAR) (first physical value) set on the vertical axis and PRI (second physical value) set on the horizontal axis. Fig.42 In , for convenience, an approximate curve based on the scatter plot output is shown. Fig.42 , an approximate curve based on the PRI defined by Equation 2 is shown (in other words, the higher the pressure response, the higher the PRI is plotted).

[0241] Accordingly, the user can quantitatively identify the environmental response of the plant according to the temperature change. In addition, in the second embodiment, as an example, the third physical value representing the state or property of the plant can be Fig.42 That is, the information processing device 100 according to the second embodiment can output a graph (or an approximate curve) (third physical value) representing the state or property of the plant by analyzing the relationship between the leaf surface light intensity (PAR) (first physical value) and the PRI (second physical value) on the vertical axis.

[0242] <4. Third embodiment>

[0243] Hereinabove, the second embodiment according to the present disclosure has been described. Next, the third embodiment according to the present disclosure will be described.

[0244] The information processing device 100 according to the third embodiment obtains a leaf surface light intensity (PAR)-photosynthesis rate (ETR) curve and calculates Jmax (maximum electron transfer rate, a third physical value) by model fitting. In the related art, a gas exchange method or the like is used in the measurement of Jmax, and the photosynthesis rate is measured by a physical measuring device, but in the third embodiment, this method is replaced by optical measurement.

[0245] In the case of solving by model fitting, multiple data sets ("environment (input)" and "response (output)") are necessary for the operation. In addition, for example, in the case of performing model fitting of the data set in units of classification ID, as a result, a physical value is output for each classification ID. At this time, in the experimental environment (indoors, etc.), multiple measurements are performed while changing environmental conditions (such as light intensity, CO2 concentration, etc.). However, in the case of performing image capture using a camera, the intensity of light irradiating the plant is different for each part due to the tilt of the leaves, shadows, etc., so multiple environments (physical values ​​corresponding to the environments) are included in one image set. At this time, as in the first and second embodiments, an appropriate environment (physical values ​​corresponding to the environment) is extracted from the image set, so that correct measurement can be achieved. Hereinafter, the configuration in which the user performs measurements multiple times while changing the environmental conditions as described above is referred to as the "first conception."

[0246] On the other hand, as another concept (hereinafter referred to as "second concept"), in the case of performing image capture using a camera, the intensity of light irradiating the plant is different for each part due to the inclination of leaves, shadows, etc., and thus a plurality of environments (physical values ​​corresponding to the environments) are included in one image set. However, this situation can also be utilized. For example, in Fig.43 In the point a, point b, and point c in FIG, when the light beam emitted from the light source is set as a parallel beam, the leaf surface light intensity (PAR) is different for each point due to the different inclinations of the leaves. When these three points are used, even if the intensity of the light source is not changed, a set of physical values ​​for different environmental conditions can be obtained.

[0247] In addition, environmental responses of different locations are used, but it is considered that there is a certain local homogeneity regarding the state of the plant. The information processing device 100 according to one aspect of the present disclosure includes a function of classifying or categorizing captured image information into specific units or specific parts by the image structure analysis operation unit 133, thereby considering the guarantee of local homogeneity.

[0248] Hereinafter, as a third embodiment, the second concept, ie, a method of performing model fitting by acquiring a plurality of environmental conditions without controlling the environmental conditions, will be described in detail. In addition, the first concept may be performed.

[0249] (4.1. Functional configuration example)

[0250] First, refer to Fig.44 A functional configuration example of the information processing apparatus 100 according to the third embodiment is described. In addition, for contents overlapping with the functional configuration example of the information processing apparatus 100 according to the first embodiment or the second embodiment, description thereof will be omitted.

[0251] like Fig.44 As shown in the figure, the information processing device 100 according to the third embodiment additionally includes a manual operation input unit 210, a program and data input unit 220, various program and data retention units (in the accompanying drawings, image processing operation program and data retention unit 135, dimension conversion operation program and data retention unit 145 and physical analysis operation program and data retention unit 155), a resolution adjustment unit 144, a model fitting operation unit 154, a data output unit 230, a network 240 (in the accompanying drawings, networks 240a to 240e), a storage device 250 (in the accompanying drawings, storage devices 250a to 250e) and an environmental control device 260.

[0252] The manual operation input unit 210 is configured to receive input of programs, data, etc. based on the user's operation. More specifically, the manual operation input unit 210 can receive input of various programs (including scripts), selected wavelength bands of captured image information generated by the multi-wavelength image operation unit 131, various setting values ​​including definitions of various IDs, physical models used in various operations, parameters constituting the physical models, etc. based on the user's operation.

[0253] The program and data input unit 220 is configured to receive input of a program, data, etc. from an external device, etc. The content of data input to the program and data input unit 220 may be similar to that in the manual operation input unit 210 .

[0254] The various program and data retention units (in the accompanying drawings, the image processing operation program and data retention unit 135, the dimension conversion operation program and data retention unit 145, and the physical analysis operation program and data retention unit 155) are configured to temporarily store programs, data, etc. input from the manual operation input unit 210 or the program and data input unit 220.

[0255] The resolution adjustment unit 144 is configured to adjust the resolution of the physical value to an appropriate value. More specifically, the resolution adjustment unit 144 can reduce the resolution of the physical value by performing calculation of an average value of a plurality of physical values ​​included in an arbitrary range, etc. The details of the resolution adjustment will be described later.

[0256] The model fitting operation unit 154 is configured to perform model fitting by a predetermined method. For example, the model fitting operation unit 154 calculates various parameters (such as a third physical value, etc.) by fitting a set of multiple physical values ​​(such as a first physical value, a second physical value, etc.) using a Farquhar model, etc. The details of the model fitting will be described later.

[0257] The data output unit 230 has a configuration that can be used in the case of outputting data without imaging the data. The data output unit 230 outputs the data stored in the analysis data buffer 153 to an external device or a storage device 250e connected thereto through a network 240e.

[0258] The network 240 is an electrical communication line connecting various configurations of the information processing apparatus 100 and external devices. In addition, the storage device 250 is a storage medium connected to various configurations of the information processing apparatus 100. Various data (such as captured image information, sensor data, etc.), programs, etc. used in the above-mentioned various processes may be provided from an external device through the network 240, or may be provided from the storage device 250. In addition, various data output through various processes may be output to an external device or the storage device 250 connected through the network 240. In addition, the types of the network 240 and the storage device 250 are not particularly limited.

[0259] The environment control device 260 is configured to control changes in environmental conditions (e.g., light intensity or CO2 concentration). The environment control device 260 can change the environmental conditions through user operations, etc. In addition, the environment control device 260 is used in the first concept (a case where the user performs measurement multiple times while changing the environmental conditions), but not in the second concept implemented by the present embodiment.

[0260] (4.2. Example of process flow and process details)

[0261] A description has been given of the functional configuration example of the information processing apparatus 100 according to the third embodiment. Next, a description will be given of an example of the flow of processing performed by each functional configuration of the information processing apparatus 100 according to the third embodiment or processing details.

[0262] First, refer to Fig.45 An example of the overall flow of processing performed by the information processing apparatus 100 according to the third embodiment is described. The third embodiment is different from the second embodiment in that Fig.45 Step S2320, step S2324 and step S2336 in the second embodiment, and the other steps may be the same as those in the second embodiment. Hereinafter, the differences between the second embodiment and the third embodiment will be described.

[0263] (Resolution adjustment process and processing details)

[0264] The resolution adjustment unit 144 of the information processing apparatus 100 according to the third embodiment Fig.45 The resolution of the physical value is adjusted to a more appropriate value in step S2320. Generally, data of multiple points is obtained from captured image information, and the resolution is adjusted from the perspective of spatial resolution (for example, the focus is intentionally blurred in the image capture process, etc. Excluding the case where no spatial separation is set between adjacent pixels), but the resolution of the physical value generated by the conversion of the captured image information is not appropriately adjusted.

[0265] For example, in the case where the information processing device 100 calculates a physical value called leaf light intensity (PAR) by using Nn captured image information, part of the data is removed in a removal operation or the like, but the data size may still be large in some cases. Fig.46 FIG. 1 shows a portion of data output to the physical value data buffer 143 in the stage where the removal operation and the correction operation are terminated, and FIG. Fig.46 In the example, you can see that the number of physical set IDs is greater than 1700000.

[0266] However, when the measurement accuracy is considered, for example, in the measurement result of physical value 1 (foliar light intensity (PAR)), a difference of about ±50 can be within the range of measurement error, so Fig.46 Processing the physical value 1 by distinguishing up to the first decimal place as shown is not effective in principle. In addition, when the scale of data to be processed is large, the amount of calculation of model fitting, etc. or the storage capacity or communication capacity that can be used for model fitting or data storage is tight, thus causing a problem. In consideration of such a situation, the resolution adjustment unit 144 of the information processing device 100 according to the third embodiment adjusts the resolution of the physical value to a more appropriate value.

[0267] Reference Fig.47 Described in Fig.45 Resolution adjustment in the dimensional conversion operation performed in step S2320. Fig.47 The resolution of the physical value 1 (foliar light intensity (PAR)) is 100 [μmol / m 2 Flowchart of the process of adjusting the resolution when the resolution is adjusted by 1 / s.

[0268] exist Fig.47In step S2400, the resolution adjustment unit 144 sets the search initial value of the physical value 1 (leaf light intensity (PAR)) (for example, 150 to 250). In step S2404, the resolution adjustment unit 144 specifies the image set ID #0001, the classification ID, and the search initial value of the physical value 1, and searches for the physical set ID.

[0269] In step S2408, the measured value (set to x) of the ETR (physical quantity corresponding to the photosynthesis rate) of the detected physical set ID and the number of detected physical set IDs (set to y) are temporarily stored. In addition, until the retrieval for all multiple physical values ​​1 is terminated (step S2412 / No), the resolution adjustment unit 144 repeatedly performs the processing in steps S2404 and S2408. In the case where the retrieval for all physical values ​​1 is terminated (step S2412 / Yes), in step S2416, the resolution adjustment unit 144 obtains the average value of the ETR in the classification ID by using x and y. More specifically, the resolution adjustment unit 144 divides the total value of x by y to obtain the average value of the ETR.

[0270] In step S2420, the resolution adjustment unit 144 removes all data of the detected physical set ID, and stores the combination of the average value of the leaf light intensity and the ETR as one physical set ID in the physical value data buffer 143. In addition, in a case where the entire range search of the physical value 1 is not terminated (step S2424 / No), in step S2428, the resolution adjustment unit 144 sets the search value (for example, 250 to 350) of the next physical value 1 (leaf light intensity (PAR)), and performs the processing in steps S2404 to S2424 again. In a case where the entire range search of the physical value 1 is terminated (step S2424 / Yes), a series of processing related to the resolution adjustment is terminated.

[0271] Fig.48 A graph showing data (measurements of leaf light intensity (PAR) and ETR) before resolution adjustment is shown. Fig.49 The resolution of the PAR based on the leaf surface light intensity (100 [μmol / m 2 / s]) When 150 to 250, 250 to 350, 350 to 450, 450 to 550, and 550 to 650 are set as the search value. In addition, Fig.50 The average value of the ETR calculated by the resolution adjustment unit 144 for each search value is shown. Fig.51 The state of the data stored in the physical value data buffer 143 after the resolution adjustment is illustrated (in addition, Fig.51 Only part of the data stored in the physical value data buffer 143 is illustrated).

[0272] Regarding the above-mentioned resolution adjustment, it can be said that the resolution adjustment unit 144 adjusts the resolution (100 [μmol / m 2 / s] to adjust the resolution of the first physical value (PAR) and the second physical value (ETR). In addition, the resolution adjustment unit 144 can adjust the resolution of the first physical value and the second physical value based on the resolution of the second physical value. In addition, in the above embodiment, Fig.16 As shown in the data shown in , the color mapping unit 161 can perform color mapping based on the image position ID, but in the case of performing resolution adjustment in the present embodiment, the information of the image position ID is lost. Here, in the present embodiment, the management table manages the classification ID and the image position corresponding to each classification ID in association with each other. Accordingly, in the case of performing resolution adjustment of the physical value, the color mapping unit 161 can specify the image position corresponding to each physical value based on the classification ID, so that color mapping can be appropriately performed. Here, as the format of the area defining the classification ID, "raster (bitmap)", "vector", etc. can be used.

[0273] (Details of the model fitting process)

[0274] exist Fig.45 In step S2324 of FIG. 1 , the model fitting operation unit 154 performs model fitting by using the physical values ​​(such as the first physical value, the second physical value, etc.) after the resolution adjustment, and outputs the third physical value. For example, the model fitting operation unit 154 uses the Farquhar model to fit Fig.51 to obtain the physical set ID#00000001 to physical set ID#00000005 shown in Jmax (maximum electron transfer rate) and θj. Fig.52 The diagram illustrates an operation in which the model fitting operation unit 154 changes the leaf surface light intensity (PAR)-photosynthesis rate (ETR) curve by using a graph showing the leaf surface light intensity (PAR)-photosynthesis rate (ETR) curve. Jmax and θj are simultaneously simulated by the Farquhar model expressed by the following formula 8, and fitting with actual measurement data (physical set ID#00000001 to physical set ID#00000005) is performed. The model fitting operation unit 154 performs model fitting by using the least squares method, the maximum likelihood estimation method, etc. In addition, the model fitting method is not limited to these methods.

[0275] [Mathematical expression 6]

[0276]

[0277] Jmax calculated as described above is output from the data output unit 230 , a graph showing a leaf surface light intensity (PAR)-photosynthesis rate (ETR) curve is generated by the graph generation unit 163 , and the graph is output to the image output unit 170 . Fig.52 An example of a case where model fitting of a data set is performed in units of classification IDs is illustrated, and in this case, a physical value is calculated for each classification ID.

[0278] <5. Fourth embodiment>

[0279] A description has been given of the third embodiment according to the present disclosure. Next, a description will be given of the fourth embodiment according to the present disclosure.

[0280] According to the fourth embodiment of the present disclosure and the fifth embodiment to be described in a later stage, a combination of appropriate devices is provided in the case of performing measurements according to one aspect of the present disclosure by using a UAV (drone, etc.), and the fourth embodiment relates to an information processing device mounted on the UAV (the description related to the information processing device (such as a PC, server, etc.) set on the outside of the UAV will be described in the fifth embodiment).

[0281] Generally, the data scale of the multi-wavelength image information generated by the multispectral camera 110 is larger than that of the RGB image information, and the data storage capacity or the communication capacity is likely to be tight. In addition, the data scale of the operation result of the stress or photosynthesis rate of the plant is small. However, when only the operation result is stored and the original data used in the operation is deleted, it is difficult to perform other analysis later, and thus lacks flexibility.

[0282] The fourth embodiment (and the fifth embodiment) are made in consideration of such a situation. The information processing apparatus 100 according to the present embodiment mounted on the UAV provides a mechanism capable of realizing other analysis by subsequent processing while greatly reducing the data size by recording data in an appropriate format.

[0283] Here, we will refer to Fig.53 A functional configuration example of the information processing apparatus 100 according to the fourth embodiment is described. Fig.53 As shown in , the information processing apparatus 100 does not include a partial configuration located at a later stage compared to the dimension conversion operation unit 140 (e.g., the physical analysis operation unit 150, the data visualization unit 160, the display device 180, etc.). The data stored in the physical value data buffer 143 is output to the storage device 250e through the data output unit 230. Then, the data output to the storage device 250e is provided to the information processing apparatus (e.g., a PC, a server, etc.), and is used for processing such as physical analysis operation and color mapping performed by the apparatus.

[0284] Fig.54 is a diagram illustrating an example of data stored in the physical value data buffer 143 (in this example, Fig.16 In this way, the physical value corresponding to the environment (first physical value) and the physical value corresponding to the response (second physical value) are retained. Thus, when these data are provided, in the processing performed by the information processing device (e.g., PC, server, etc.) at the later stage, calculation of the third physical value indicating the state or property of the plant and the like can be performed.

[0285] In addition, various image information (e.g., RGB image information, NDVI image information, etc.) stored in the image buffer 132 can be output to the storage device 250e through the image output unit 170. Accordingly, in the processing performed by the information processing apparatus (e.g., PC, server, etc.) at the later stage, for example, the output of image information obtained by synthesizing the third physical value (pressure measurement value, etc.) and the RGB image information can be performed.

[0286] In addition, the functions and processing flows of the various configurations of the information processing device 100 according to the fourth embodiment may be similar to the above-described functions and processing flows, and thus their detailed descriptions will be omitted. In addition, the functional configuration of the information processing device 100 mounted on the UAV is not limited to Fig.53 For example, the information processing device 100 may include a communication unit, and may provide various data (e.g., data) to other information processing devices (e.g., PCs, servers, etc.) in real time through communication (e.g., wireless communication) performed via the communication unit. Fig.54 In addition, the information processing device 100 may include a functional configuration for performing the later stage processing of the dimension conversion operation (e.g., a physical analysis operation unit 150, a data visualization unit 160, etc.), and may provide the data after the physical analysis operation (a third physical value indicating the stress response (state) of the plant to various stress factors, etc.) or the generated image information to other information processing devices (e.g., a PC, a server, etc.). In this way, the functional configuration of the information processing device 100 installed in the UAV can be flexibly modified.

[0287] <6. Fifth embodiment>

[0288] A description of the fourth embodiment according to the present disclosure has been given. Next, a fifth embodiment according to the present disclosure will be described. As described above, the fifth embodiment relates to an information processing device (e.g., a PC, a server, etc.) that performs processing by using data provided from a UAV (drone, etc.) according to the fourth embodiment.

[0289] Here, we will refer to Fig.55A functional configuration example of the information processing apparatus 100 according to the fifth embodiment will be described. Fig.55 As shown in , the information processing device 100 does not include some of the configurations located in the previous stage (such as the multispectral camera 110, a part of the image processing operation unit 130, a part of the dimension conversion operation unit 140, etc.). The data processed on the UAV side (resolution-adjusted data, various image information (such as RGB image information, NDVI image information, etc.)) is stored in the storage device 250a and input to the information processing device 100 through the image input unit 120 and the data input unit 200. Then, analysis operations and model fitting by the physical analysis operation unit 150, color mapping or image information generation by the data visualization unit 160, image information output by the image output unit 170, etc. are performed.

[0290] Furthermore, the functional configuration of the information processing apparatus 100 (eg, PC, server, etc.) according to the fifth embodiment is not limited to Fig.55 The example shown in . For example, the information processing device 100 may include a communication unit, and various data may be received from the UAV in real time through the communication unit. In addition, the information processing device 100 may include a functional configuration for performing the processing of the previous stage of the physical analysis operation (such as an image processing operation unit 130, a dimension conversion operation unit 140, etc.), and may perform various image processing, dimension conversion operations, etc. (in addition, in this case, the data before various image processing or dimension conversion operations are provided from the UAV to the information processing device 100). As described above, the functional configuration of the information processing device 100 (such as a PC, server, etc.) can be flexibly modified.

[0291] <7. Hardware Configuration Example>

[0292] A description has been given of the fifth embodiment according to the present disclosure. Fig.56 A hardware configuration example of the information processing apparatus 100 according to each embodiment is described.

[0293] Fig.56 900 is a diagram illustrating a hardware configuration of the information processing apparatus 100. Each information processing apparatus 100 includes a central processing unit (CPU) 901, a read-only memory (ROM) 902, a random access memory (RAM) 903, a host bus 904, a bridge 905, an external bus 906, an interface 907, an input device 908, an output device 909, a storage device (HDD) 910, a drive 911, and a communication device 912.

[0294] The CPU 901 is used as an operation processing device and a control device, and controls the entire operation in the information processing apparatus 100 according to various programs. In addition, the CPU 901 may be a microprocessor. The ROM 902 stores programs and operation parameters used by the CPU 901. The RAM 903 temporarily stores programs in execution of the CPU 901, parameters that change appropriately during execution, and the like. These constituent elements are connected to each other via a host bus 904 constituted by a CPU bus and the like. The image processing operation unit 130, the dimension conversion operation unit 140, the physical analysis operation unit 150, and parts of the data visualization unit 160, etc. can be realized by the cooperation of the CPU 901, the ROM 902, and the RAM 903. In addition, the configuration that can be realized by the cooperation of the CPU 901, the ROM 902, and the RAM 903 is not limited thereto.

[0295] The host bus 904 is connected to the external bus 906 such as a peripheral component interconnect / interface (PCI) bus through the bridge 905. In addition, the host bus 904, the bridge 905, and the external bus 906 do not necessarily have to be constructed in a separate state, and their functions may be embedded in one bus.

[0296] The input device 908 includes an input unit such as a mouse, a keyboard, a touch panel, a button, a microphone, a switch, and a joystick, which are used by the user to input information, and an input control circuit that generates an input signal based on the user's input and outputs the input signal to the CPU 901, etc. By operating the input device 908, the user can input various information or can make instructions for processing operations. In addition, with respect to the input device 908, the input does not have to be performed by the user. The multispectral camera 110, the image input unit 120, the sensor and measurement device 190, the sensor data input unit 200, the manual operation input unit 210, the program and data input unit 220, the environmental control device 260, etc. can be implemented by the input device 908.

[0297] For example, the output device 909 includes display devices such as cathode ray tube (CRT) display devices, liquid crystal display (LCD) devices, organic light emitting diode (OLED) devices, and lamps. In addition, the output device 909 includes voice output devices such as speakers and headphones. The output device 909 performs display of image information, etc. On the other hand, the voice output device performs voice output of image information (numerical value), etc. The image output unit 170, the display device 180, the data output unit 230, etc. can be implemented by the output device 909.

[0298] The storage device 910 is a device for storing data. The storage device 910 may include a storage medium, a recording device for recording data on the storage medium, a reading device for reading data from the storage medium, a deleting device for deleting data recorded in the storage medium, etc. For example, the storage device 910 is composed of a hard disk drive (HDD). The storage device 910 drives the hard disk and stores programs executed by the CPU 901 and various data. Various buffers (such as the structured image buffer 134, etc.), various retention units (such as the image processing operation program and the data retention unit 135, etc.), various storage devices 250, etc. can be implemented by the storage device 910.

[0299] The drive 911 is a reader / writer for storage media, and is embedded in the information processing apparatus 100 or externally mounted thereon. The drive 911 reads out information recorded on a removable storage medium 913 such as a mounted magnetic disk, an optical disk, a magneto-optical disk, and a semiconductor memory, and outputs the information to the RAM 903. In addition, the drive 911 can write information on the removable storage medium 913.

[0300] For example, the communication device 912 is a communication interface constituted by a communication device or the like connected to the communication network 914. Access to each network 240 is achieved through the communication device 912.

[0301] <8. Conclusion>

[0302] As described above, each of the information processing devices 100 according to one aspect of the present disclosure analyzes captured image information of a plant as a measurement object to obtain a first physical value corresponding to an environment in which the plant is left and a second physical value corresponding to a response of the plant to the environment, and stores the first physical value and the second physical value in association with each other. In addition, the information processing device 100 performs an analysis operation by using the first physical value and the second physical value, and outputs a third physical value directly indicating a state or property of the plant.

[0303] According to this, the information processing device 100 can measure the change of the operational function (response) of the plant corresponding to the change of the environmental factor in real time, and thus can detect the problem (such as strong light, weak light, low humidity, low temperature, high humidity, high temperature, soil drying, excessive soil moisture (anaerobic pressure), the occurrence of pests and diseases, changes in soil components, pesticides with adverse effects, etc.) for which no countermeasures are taken in time using vegetation indices such as NDVI at an early stage. In the related art, for example, the change of the operational function (response) of the plant is measured by performing model fitting while changing the environment in which the plant is left. In contrast, the information processing device 100 can measure the change of the operational function (response) of the plant by using the captured image information generated by the image capture processing performed once, so the time required on the measuring device side can be greatly shortened. That is, it should be understood that "real time" means a concept having a time interval corresponding to approximately the time required for the analysis processing of the captured image information after the image capture processing is performed once. Aerial image capture by UAV requires a lot of effort or time, so it may be difficult to perform aerial image capture at a desired timing (desired environment) depending on the influence such as weather, and is therefore not suitable for performing multiple aerial image captures. On the other hand, the information processing device 100 can perform measurement in real time. Thus, for example, a person performing measurement can obtain measurement results during the flight of the UAV, and therefore does not have to perform aerial image capture multiple times.

[0304] From a different point of view, the information processing device 100 can efficiently identify situations where no problems have occurred through real-time measurement. More specifically, when all conditions (all conditions that have an impact on the photosynthesis rate) are satisfied at the same time (AND condition), the photosynthesis rate becomes the maximum. Thus, when the measured photosynthesis rate shows a normal value, the information processing device 100 can identify that no problems have occurred that require countermeasures. In other words, the photosynthesis rate is a sufficient condition for the problem not to occur, and therefore, when the measured photosynthesis rate is a normal value, it can be said that no problem has occurred. On the other hand, static information related to plants such as NDVI, such as tissue and composition, is not a necessary condition for the problem not to occur, and therefore it is difficult to perform the above analysis through NDVI, etc.

[0305] Furthermore, from a different point of view, the information processing device 100 can appropriately perform analysis of the cause of the problem by real-time measurement. More specifically, when the information processing device 100 can identify the state (symptoms) of the plant in real time, it is likely that the cause of the problem has also occurred during the identification, so it is likely that the cause has been specified. In addition, the user can assume the cause of the problem (such as a reduction in irrigation, etc.) based on the output from the information processing device 100, and can perform verification by immediately confirming whether the problem is solved by re-measurement after taking countermeasures (such as increasing irrigation, etc.). In addition, in the case where the content of the problem or its degree changes due to changes in the environment that do not follow the artificial countermeasures taken by the user (such as the case where a low temperature problem is naturally solved when the temperature rises, etc.), even in this case, the user can estimate the cause of the problem based on the output from the information processing device 100. For example, when the information processing device 100 performs measurement at a timing before or after the content of the problem or its degree changes, the user can estimate the cause of the problem based on the change in the measurement result and the condition generated at the timing when the change occurs (for example, the user can perform estimation of the cause of the problem based on the weather conditions at the timing when the measurement result changes, etc.). On the other hand, in the case of performing cause analysis by using NDVI or the like, the problem is detected several days later from the time when the problem occurs, so it is difficult for the user to perform the above-mentioned verification or the like according to changes in environmental conditions such as weather conditions.

[0306] Furthermore, for example, in irrigation control of a large farm in the related art, the necessary irrigation amount is calculated by estimating the evaporation rate from weather conditions, the size of the plant, etc., but sufficient accuracy is not achieved in many cases due to reasons such as differences in the estimation accuracy of the evaporation rate, differences in the characteristics of the soil of the farm and each location. In addition, countermeasures for measuring the evaporation amount from leaves have also been taken by using thermal imaging technology, but heat transfer is easily affected by wind or temperature, so the measurement of the evaporation amount using thermal imaging technology has not yet been put into practical use. Here, using the information processing device 100 as described above, in the case where a cycle of measurement of the operating function (response) of the plant, detection of problems, countermeasures (control), and verification of the effects of the countermeasures can be repeated, automation of farm management becomes possible, and thus irrigation control of the farm, etc. can be achieved more efficiently.

[0307] In the above, the preferred embodiments of the present disclosure have been described in detail with reference to the accompanying drawings, but the technical scope of the present disclosure is not limited to these examples. It is clear that those skilled in the art of the present disclosure can conceive various modified examples or variant examples within the scope of the technical spirit described in the attached claims, and it should be understood that these modified examples and variant examples belong to the technical scope of the present disclosure.

[0308] In addition, the effects described in this specification are merely illustrative and not limiting. That is, the technology according to the present disclosure can combine the above effects or replace these effects to obtain other effects that are clear to those skilled in the art from the description of this specification.

[0309] In addition, the following configurations also belong to the technical scope of the present disclosure. (1)

[0311] An information processing device, comprising:

[0312] an operation unit that analyzes captured image information of the measurement object to obtain a first physical value corresponding to an environment in which the measurement object is placed and a second physical value corresponding to a response of the measurement object to the environment; and

[0313] A storage unit stores the first physical value and the second physical value in association with each other. (2)

[0315] The information processing device according to (1),

[0316] wherein the operation unit acquires the first physical value and the second physical value by performing analysis in a two-dimensional direction in a plane space of the captured image information. (3)

[0318] The information processing device according to (2),

[0319] The operation unit removes the area in the captured image information that is not suitable for operation from the operation object. (4)

[0321] The information processing device according to (3),

[0322] The region not suitable for calculation includes a region in the captured image information where an image of an object other than the measurement target is captured or a region where an image of a shadow is captured. (5)

[0324] The information processing device according to any one of (2) to (4),

[0325] The captured image information is multi-wavelength band image information including information of a plurality of wavelength bands, and the operation unit acquires the first physical value and the second physical value by analyzing information of a specific wavelength band in the multi-wavelength band image information. (6)

[0327] The information processing device according to any one of (2) to (5),

[0328] The computing unit acquires sensor data related to the environment and corrects the first physical value by using the sensor data. (7)

[0330] The information processing device according to any one of (2) to (6),

[0331] The operation unit adjusts the resolutions of the first physical value and the second physical value based on the resolution of at least any one of the first physical value and the second physical value. (8)

[0333] The information processing device according to any one of (2) to (7),

[0334] wherein the measurement object is a plant, and

[0335] The operation unit outputs a third physical value indicating a state or property of the plant by using the first physical value and the second physical value. (9)

[0337] The information processing device according to (8),

[0338] wherein the operation unit outputs the third physical value by performing model fitting using the first physical value and the second physical value. (10)

[0340] The information processing device according to (8) or (9),

[0341] The third physical value includes at least any one of a value corresponding to the pressure received by the plant and a value corresponding to the photosynthesis rate in the plant. (11)

[0343] The information processing device according to (10),

[0344] The first physical value includes at least one of leaf surface light intensity and temperature. (12)

[0346] The information processing device according to (11),

[0347] Wherein, in a case where the first physical value is equal to or greater than a constant value or is within a constant range, the operation unit outputs the third physical value by using the second physical value. (13)

[0349] The information processing device according to any one of (8) to (12),

[0350] The operation unit classifies the captured image information into a plurality of regions and outputs the third physical value for each region. (14)

[0352] The information processing device according to any one of (8) to (13),

[0353] The computing unit further includes a data visualization unit for visualizing at least any one of the first physical value, the second physical value, and the third physical value. (15)

[0355] The information processing device according to (14),

[0356] The data visualization unit associates at least any one of the first physical value, the second physical value, and the third physical value with a color. (16)

[0358] The information processing device according to (15),

[0359] The data visualization unit generates image information in which at least any one of the first physical value, the second physical value, and the third physical value associated with the color is associated with an original spatial position. (17)

[0361] The information processing device according to (14),

[0362] wherein the data visualization unit visualizes at least one of the first physical value, the second physical value, and the third physical value by using a graph. (18)

[0364] The information processing device according to any one of (2) to (14),

[0365] The captured image information is information of images captured from an unmanned aerial vehicle (UAV) in flight. (19)

[0367] An information processing method executed by a computer, the method comprising:

[0368] analyzing captured image information of the measurement object to obtain a first physical value corresponding to an environment in which the measurement object is placed and a second physical value corresponding to a response of the measurement object to the environment; and

[0369] The first physical value and the second physical value are stored in association with each other. (20)

[0371] A program that causes a computer to perform the following operations: analyze captured image information of a measurement object to obtain a first physical value corresponding to an environment in which the measurement object is placed and a second physical value corresponding to a response of the measurement object to the environment; and store the first physical value and the second physical value in association with each other. (twenty one)

[0373] An information processing device, comprising:

[0374] storage circuit; and

[0375] An operation circuit, the operation circuit being configured as:

[0376] acquiring a first physical value by analyzing captured image information, the captured image information being based on information from a plurality of pixels, and the first physical value being indicative of an environment of a measurement object associated with a first pixel of the plurality of pixels,

[0377] acquiring a second physical value by analyzing the captured image information, the second physical value indicating a response of the measurement object to the environment, and

[0378] The storage circuit is controlled to store the first physical value and the second physical value in association with each other. (twenty two)

[0380] The information processing device according to (21), wherein the operation circuit is further configured to acquire the first physical value and the second physical value by performing the analysis in a two-dimensional direction in a plane space of the captured image information. (twenty three)

[0382] The information processing device according to (22), wherein the operation circuit is further configured to:

[0383] determining whether the region of the captured image information is unsuitable for analysis,

[0384] determining whether the first physical value and the second physical value originate from an area that is not suitable for analysis, and

[0385] In response to determining that the first physical value and the second physical value originate from a region that is not suitable for analysis, the storage circuit is controlled to exclude or delete the first physical value and the second physical value. (twenty four)

[0387] The information processing device according to (23), wherein, in order to determine whether the area of ​​the captured image information is not suitable for analysis, the operation circuit is further configured to:

[0388] determining whether the area of ​​the captured image information includes an object other than the measurement object, and

[0389] A determination is made as to whether the area of ​​the captured image information includes a shadow. (25)

[0391] The information processing device according to any one of (22) to (24), wherein the captured image information is multi-wavelength image information including information of a plurality of wavelength bands, and

[0392] The operation circuit is further configured to acquire the first physical value and the second physical value by analyzing information of a specific wavelength band in the multi-wavelength image information. (26)

[0394] The information processing device according to any one of (22) to (25), wherein the arithmetic circuit is further configured to:

[0395] acquiring sensor data related to the environment, and

[0396] The first physical value is corrected based on the sensor data. (27)

[0398] An information processing device according to any one of (22) to (26), wherein the operation circuit is further configured to adjust the resolution of the first physical value and the second physical value based on the resolution of at least one of the first physical value or the second physical value. (28)

[0400] The information processing device according to any one of (22) to (27), wherein the measurement object is a part of a plant, and

[0401] The operation circuit is further configured to output a third physical value indicating a state or property of the part of the plant based on the first physical value and the second physical value. (29)

[0403] An information processing device according to (28), wherein, in order to output the third physical value indicating the state or property of the plant based on the first physical value and the second physical value, the operation circuit is also configured to perform model fitting using the first physical value and the second physical value. (30)

[0405] The information processing device according to any one of (28) to (29), wherein the third physical value includes at least one of a value corresponding to the pressure received by the plant or a value corresponding to the photosynthesis rate in the plant. (31)

[0407] The information processing device according to (30), wherein the first physical value includes at least one of leaf surface light intensity or temperature. (32)

[0409] An information processing device according to any one of (28) to (31), wherein the operation circuit also includes a data visualization circuit, which is configured to generate data visualization of at least one of the first physical value, the second physical value or the third physical value. (33)

[0411] An information processing device according to (32), wherein, in order to generate the data visualization, the data visualization circuit is also configured to associate color with at least one of the first physical value, the second physical value, or the third physical value. (34)

[0413] The information processing device according to (33), wherein, in order to generate the data visualization, the data visualization circuit is further configured to:

[0414] associating an original spatial position with at least one of the first physical value, the second physical value, or the third physical value associated with the color, and

[0415] Image information is generated based on the at least one of the first physical value, the second physical value, or the third physical value associated with the color and the original spatial position. (35)

[0417] An information processing device according to any one of (32) to (34), wherein, in order to generate the data visualization, the data visualization circuit is also configured to generate a graphic that visualizes the first physical value, the second physical value, or at least one of the third physical values. (36)

[0419] The information processing device according to any one of (22) to (35), wherein the measurement object is a part of a plant,

[0420] The first physical value includes at least one of leaf light intensity or temperature, and

[0421] The operation circuit is further configured to output a third physical value indicating a state or property of the part of the plant based on the second physical value when the first physical value is equal to or greater than a constant value or within a constant range. (37)

[0423] The information processing device according to any one of (22) to (36), wherein the captured image information is information of an image captured from an unmanned aerial vehicle (UAV) in flight. (38)

[0425] The information processing device according to any one of (21) to (37), wherein the arithmetic circuit is further configured to:

[0426] classifying the captured image information into a plurality of regions,

[0427] acquiring one or more fourth physical values ​​for each of the plurality of regions by analyzing the plurality of regions, the one or more fourth physical values ​​indicating one or more environments of one or more measurement objects associated with one or more pixels of the plurality of pixels, and

[0428] acquiring one or more fifth physical values ​​for each of the plurality of regions by analyzing the plurality of regions, the one or more fifth physical values ​​indicating one or more responses of the one or more measurement objects to the one or more environments, and

[0429] A sixth physical value for each of the plurality of regions is output based on the one or more fourth physical values ​​for each of the regions and the one or more fifth physical values ​​for each of the regions. (39)

[0431] An information processing method, the method comprising:

[0432] acquiring, with an electronic processor, a first physical value by analyzing captured image information, the captured image information being based on information from a plurality of pixels, and the first physical value being indicative of an environment of a measurement object associated with a first pixel of the plurality of pixels;

[0433] acquiring, by the electronic processor, a second physical value by analyzing the captured image information, the second physical value indicating a response of the measurement object to the environment; and

[0434] The electronic processor controls a storage circuit to store the first physical value and the second physical value in association with each other. (40)

[0436] A non-transitory computer-readable medium comprising instructions that, when executed by an electronic processor, cause the electronic processor to perform a set of operations comprising:

[0437] acquiring a first physical value by analyzing captured image information, the captured image information being based on information from a plurality of pixels, and the first physical value being indicative of an environment of a measurement object associated with a first pixel of the plurality of pixels;

[0438] acquiring a second physical value by analyzing the captured image information, the second physical value indicating a response of the measurement object to the environment; and

[0439] The storage circuit is controlled to store the first physical value and the second physical value in association with each other. (41)

[0441] A storage device, comprising:

[0442] storage circuit; and

[0443] An interface circuit, the interface circuit being configured to:

[0444] receiving a first physical value indicative of an environment of a measurement object associated with a first pixel of the plurality of pixels,

[0445] receiving a second physical value indicating a response of the measurement object to the environment,

[0446] receiving a correlation indicator indicating that the first physical value is correlated with the second physical value, and

[0447] The first physical value and the second physical value are output to the storage circuit based on the correlation indicator.

[0448] It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and alterations may occur depending on design requirements and other factors insofar as they are within the scope of the appended claims or the equivalents thereof.

[0449] Reference numerals list

[0450] 100 Information processing device

[0451] 110 Multispectral Camera

[0452] 120 Image input unit

[0453] 130 Image processing unit

[0454] 131 Multi-wavelength image processing unit

[0455] 132 Image Buffer

[0456] 133 Image structure analysis computing unit

[0457] 134 Structured Image Buffer

[0458] 135 Image processing operation program and data storage unit

[0459] 140 Dimension conversion unit

[0460] 141 Remove the operation unit

[0461] 142 Correction calculation unit

[0462] 143 Physical value data buffer

[0463] 144 Resolution adjustment unit

[0464] 145 Dimension conversion operation program and data retention unit

[0465] 150 Physical Analysis Operation Unit

[0466] 151 Analysis and Operation Unit

[0467] 152 Operation control unit

[0468] 153 Analysis Data Buffer

[0469] 154 Model fitting operation unit

[0470] 155 Physical analysis calculation program and data retention unit

[0471] 160 Data Visualization Unit

[0472] 161 Color Mapping Unit

[0473] 162 Image Generation Unit

[0474] 163 Graphics Generation Unit

[0475] 170 Image output unit

[0476] 180 Display devices

[0477] 190 Sensors and measuring devices

[0478] 200 Sensor data input unit

[0479] 210 Manual operation input unit

[0480] 220 Program and data entry unit

[0481] 230 Data output unit

[0482] 240 Network

[0483] 250 storage devices

[0484] 260 Environmental Control Equipment

Claims

1. An information processing device, comprising: Storage circuit; as well as An operation circuit, the operation circuit being configured as: classifying captured image information into a plurality of areas, the captured image information being based on information from a plurality of pixels and being multi-wavelength image information including information of a plurality of wavelength bands, determining that a predetermined condition area among the plurality of areas of the captured image information is an area unsuitable for operation, and removing the area, acquiring a first physical value and a second physical value for each of the plurality of regions by analyzing the plurality of regions of the captured image information, the first physical value indicating an environment of a measurement object associated with a first pixel of the plurality of pixels, the second physical value indicating a response of the measurement object to the environment, the measurement object being a part of a plant, controlling the storage circuit to store the first physical value and the second physical value in association with each other, and outputting a third physical value indicating a state or property of the part of the plant based on the first physical value and the second physical value, wherein the first physical value and the second physical value are obtained at the same granularity, and The operation circuit is further configured to convert the dimensional information into dimensional information of the first physical value and the second physical value by analyzing dimensional information in the vertical direction and the horizontal direction in the plane space of the captured image information.

2. The information processing device according to claim 1, wherein: In order to determine that the predetermined condition area among the plurality of areas of the captured image information is an area that is not suitable for operation, the operation circuit is further configured to: determining whether the predetermined condition area includes an object other than the measurement object, and It is determined whether the predetermined condition area includes a shadow.

3. The information processing device according to claim 1, wherein: The operation circuit is further configured to acquire the first physical value and the second physical value by analyzing information of a specific wavelength band in the multi-wavelength image information.

4. The information processing device according to claim 1, wherein: The operation circuit is further configured to: acquiring sensor data related to the environment, and The first physical value is corrected based on the sensor data.

5. The information processing device according to claim 1, wherein: The operation circuit is further configured to adjust resolutions of the first physical value and the second physical value based on a resolution of at least one of the first physical value or the second physical value.

6. The information processing device according to claim 1, wherein: In order to output the third physical value indicating the state or property of the plant based on the first physical value and the second physical value, the operation circuit is further configured to perform model fitting using the first physical value and the second physical value.

7. The information processing device according to claim 1, wherein: The third physical value includes at least one of a value corresponding to a pressure received by the plant or a value corresponding to a photosynthesis rate in the plant.

8. The information processing device according to claim 7, wherein: The first physical value includes at least one of leaf surface light intensity or temperature.

9. The information processing device according to claim 1, wherein: The computational circuitry further includes a data visualization circuit configured to generate a data visualization of at least one of the first physical value, the second physical value, or the third physical value.

10. The information processing device according to claim 9, wherein: To generate the data visualization, the data visualization circuit is further configured to associate a color with the at least one of the first physical value, the second physical value, or the third physical value.

11. The information processing device according to claim 10, wherein: To generate the data visualization, the data visualization circuit is further configured to: associating an original spatial position with at least one of the first physical value, the second physical value, or the third physical value associated with the color, and Image information is generated based on the at least one of the first physical value, the second physical value, or the third physical value associated with the color and the original spatial position.

12. The information processing device according to claim 9, wherein: To generate the data visualization, the data visualization circuit is further configured to generate a graphic that visualizes the at least one of the first physical value, the second physical value, or the third physical value.

13. The information processing device according to claim 1, in, The first physical value includes at least one of leaf light intensity or temperature, and The operation circuit is further configured to output a third physical value indicating a state or property of the part of the plant based on the second physical value when the first physical value is equal to or greater than a constant value or within a constant range.

14. The information processing device according to claim 1, wherein: The captured image information is information of images captured from an unmanned aerial vehicle (UAV) in flight.

15. An information processing method, the method comprising: Using electronic processors: classifying captured image information into a plurality of areas, the captured image information being based on information from a plurality of pixels and being multi-wavelength image information including information of a plurality of wavelength bands, determining that a predetermined condition area among the plurality of areas of the captured image information is an area unsuitable for operation, and removing the area, acquiring a first physical value and a second physical value for each of the plurality of regions by analyzing the plurality of regions of the captured image information, the first physical value indicating an environment of a measurement object associated with a first pixel of the plurality of pixels, the second physical value indicating a response of the measurement object to the environment, the measurement object being a part of a plant, controlling a storage circuit to store the first physical value and the second physical value in association with each other, and outputting a third physical value indicating a state or property of the part of the plant based on the first physical value and the second physical value, wherein the first physical value and the second physical value are obtained at the same granularity, and The method further includes using the electronic processor to convert the dimensional information into dimensional information of the first physical value and the second physical value by analyzing the dimensional information in the vertical direction and the horizontal direction in the plane space of the captured image information.

16. A non-transitory computer readable medium comprising instructions that, when executed by an electronic processor, cause the electronic processor to perform a set of operations comprising: classifying captured image information into a plurality of regions, the captured image information being based on information from a plurality of pixels and being multi-wavelength image information including information of a plurality of wavelength bands; determining that a predetermined condition area among the plurality of areas of the captured image information is an area unsuitable for operation, and removing the area; acquiring a first physical value and a second physical value for each of the plurality of regions by analyzing the plurality of regions of the captured image information, the first physical value indicating an environment of a measurement object associated with a first pixel of the plurality of pixels, the second physical value indicating a response of the measurement object to the environment, the measurement object being a part of a plant; controlling a storage circuit to store the first physical value and the second physical value in association with each other; as well as outputting a third physical value indicating a state or property of the part of the plant based on the first physical value and the second physical value, wherein the first physical value and the second physical value are obtained at the same granularity, and The set of operations further includes converting the dimensional information into dimensional information of the first physical value and the second physical value by analyzing dimensional information in the vertical direction and the horizontal direction in the plane space of the captured image information.

Images