Tree power estimation method, tree power estimation system, computer program, and optical simulation method

Abstract

To relax a restriction on an image used for estimating a tree vigor.SOLUTION: A tree vigor estimation method executed by one or more computers includes acquiring one or more pieces of image data indicating a scene including a tree in a field in a real space, and estimating an index value indicating a vigor of the tree based on the image data. When estimating the index value, a relationship between the image of the scene and the total leaf area for the tree is used.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a method, system, and computer program executed by one or more computers for tree vigor estimation. The present disclosure also relates to an optical simulation method for determining a relationship between an image of a tree and the total leaf area of ​​the tree. [Background technology]

[0002] Research and development is underway into smart agriculture, which utilizes ICT (Information and Communication Technology) and IoT (Internet of Things) as the next generation of agriculture. Smart agriculture aims to improve productivity, alleviate labor shortages, and reduce environmental impact.

[0003] Non-Patent Document 1 describes software that detects leaf and branch areas in an image based on an image captured with a monocular camera and estimates the leaf area index of a tree. The leaf area index is the total leaf area per unit land area (total leaf area / unit land area). The leaf area index is one of the index values ​​that indicates the vigor of plants such as trees, and is an important parameter for estimating the photosynthetic capacity, transpiration rate, etc. of plants. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] De Bei, R., et al. “VitiCanopy: A free computer App to estimate canopy vigor and porosity for grapevine.” Sensors, 2016, Vol. 16, No. 4, p. 585 Summary of the Invention [Problem to be solved by the invention]

[0005] The above-mentioned conventional technology requires the use of an image captured by placing a monocular camera on the ground or a predetermined distance below the tree canopy, looking up at the tree canopy with the sky as the background.

[0006] The present disclosure provides a method, system, and computer program for estimating tree vigor, as well as an optical simulation method, that relaxes constraints on images used for tree vigor estimation. [Means for solving the problem]

[0007] The present invention has the configurations described in the following items.

[0008] [Item 1] A tree vigor estimation method executed by one or more computers, the method comprising: acquiring one or more image data representing a scene including trees in a farm field in real space; and estimating an index value indicating the tree vigor of the trees based on the image data; and estimating the index value uses a relationship between an image of the scene and the total leaf area for the trees.

[0009] [Item 2] Item 2. A tree vigor estimation method according to item 1, further comprising storing the index value as digital data associated with the scene in a memory within the computer or transmitting the index value to a device connected to the computer.

[0010] [Item 3] 3. The tree vigor estimation method according to item 1 or 2, wherein the relationship is defined by an equation that associates a feature amount of the tree in the image of the scene with the total leaf area.

[0011] [Item 4] Item 4. The tree vigor estimation method according to Item 3, wherein the formula is determined based on a three-dimensional model of the tree in the simulation space.

[0012] [Item 5] The relationship is: determining a total leaf area of ​​the tree based on the three-dimensional model; generating a plurality of images of the tree based on the three-dimensional model; extracting features from the plurality of images and determining a relationship between the features and the total leaf area; 5. The tree vigor estimation method according to item 4, wherein the tree vigor is determined by a process including:

[0013] [Item 6] 6. The tree vigor estimation method according to item 5, wherein the feature amount includes pixel information of a leaf area in each of the plurality of images.

[0014] [Item 7] 7. The tree vigor estimation method according to item 6, wherein the feature amount includes the number of leaf pixels in each of the plurality of images, or the ratio of the number of leaf pixels to the total number of pixels in each of the plurality of images.

[0015] [Item 8] 8. The tree vigor estimation method according to item 7, wherein the feature amount includes a brightness histogram of leaves in each of the plurality of images.

[0016] [Item 9] 7. The tree vigor estimation method according to item 6, wherein the feature amount includes at least one of leaf size, tree height, and number of branches in each of the plurality of images.

[0017] [Item 10] 10. The tree vigor estimation method according to any one of items 1 to 9, wherein the index value includes at least one of the total leaf area, leaf area index, canopy volume, canopy height, average leaf size, variance of leaf size, leaf count density, average branch thickness or length, variance of branch thickness or length, and number of branches for the tree.

[0018] [Item 11] processing the image data to estimate a depth for each pixel of the scene; selecting leaf pixels in a predetermined depth range based on the depth and color of each pixel; extracting features related to total leaf area from the leaf pixels; 11. The tree vigor estimation method according to any one of items 1 to 10, comprising:

[0019] [Item 12] acquiring the image data photographing the tree using one or more imaging devices; or receiving digital data acquired by photographing the tree using one or more imaging devices; 12. The tree vigor estimation method according to any one of items 1 to 11, comprising:

[0020] [Item 13] 13. The tree vigor estimation method according to any one of items 1 to 12, wherein the image data includes a side image of the tree.

[0021] [Item 14] the image data includes a near-infrared image of the scene; 9. The tree vigor estimation method according to any one of items 1 to 8, further comprising extracting a brightness histogram of leaves in the near-infrared image as the feature amount.

[0022] [Item 15] Item 15. The tree vigor estimation method according to item 14, wherein the image data includes a visible light image taken from a camera viewpoint different from that of the near-infrared image.

[0023] [Item 16] 1. An optical simulation method for determining a relationship between an image of a tree and a total leaf area of ​​said tree, comprising: establishing a three-dimensional model of a tree in a simulation space; generating a plurality of images of the tree based on the three-dimensional model; determining a total leaf area of ​​the tree based on the three-dimensional model; extracting features from the plurality of images and determining a relationship between the features and the total leaf area; An optical simulation method comprising:

[0024] [Item 17] Item 17. The optical simulation method according to item 16, wherein the feature amount is based on pixel information of a leaf region in each of the plurality of images.

[0025] [Item 18] Item 18. The optical simulation method according to item 16 or 17, wherein the three-dimensional model is defined by leaf optical model parameters including at least one of leaf refractive index, surface roughness, density, size, absorbance, spectral transmittance, and spectral reflectance.

[0026] [Item 19] a storage device that stores one or more image data representing a scene including trees in a field in real space; a processing device that estimates an index value indicating tree vigor based on the image data; and Equipped with A tree vigor estimation system, wherein the processing device uses a relationship between the image of the scene and the total leaf area for the tree when estimating the index value.

[0027] [Item 20] the relationship is defined by an equation that associates a feature amount of the tree in the image of the scene with the total leaf area; the processing device includes a processor and a memory; The memory may include: the processing device reads out the image data stored in the storage device and extracts the feature amount of the tree from the image of the scene; and applying the features to the formula to determine the total leaf area; Item 20. The tree vigor estimation system according to Item 19, wherein the system stores a computer program for executing the above.

[0028] [Item 21] On one or more computers, acquiring one or more image data representing a scene including a tree in a field in real space; a processing step of estimating an index value indicating tree vigor based on the image data; Execute A computer program that uses a relationship between an image of the scene and total leaf area for the trees when estimating the index value.

[0029] [Item 22] A recording medium on which the computer program according to item 21 is stored.

[0030] [Item 23] means for acquiring one or more image data representing a scene including trees in a field in real space; means for estimating an index value indicating tree vigor based on the image data; Equipped with A tree vigor estimation system that uses a relationship between the image of the scene and the total leaf area of ​​the tree when estimating the index value. [Effects of the Invention]

[0031] According to an embodiment of the present disclosure, a method, system, computer program, and optical simulation method for estimating tree vigor are provided, in which constraints on images used for estimating tree vigor are relaxed. [Brief explanation of the drawings]

[0032] [Figure 1] FIG. 1 is a flowchart illustrating processing steps of a tree vigor estimation method according to an exemplary embodiment of the present disclosure. [Figure 2] FIG. 2 is a flowchart showing the processing steps of the optical simulation in this embodiment. [Figure 3] FIG. 3 is a diagram showing two tree models (model 1 and model 2) at different growth stages. [Figure 4] FIG. 4 is a scatter plot that schematically shows the relationship between the image features obtained for Model 1 and Model 2 and the total leaf area. [Figure 5] Figure 5 is a scatter plot that shows the relationship between the image features obtained for multiple models and the total leaf area. [Figure 6] FIG. 6 is a flowchart showing a process for extracting feature amounts from an image showing a scene including trees in a farm field in real space. [Figure 7] FIG. 7 is a diagram schematically illustrating a part of a row of fruit-bearing trees in an orchard in real space. [Figure 8] FIG. 8 is a diagram showing an example of a scene image acquired by capturing an image of a tree in real space with an imaging device. [Figure 9] FIG. 9 is a diagram schematically showing a leaf area selected from the image of FIG. [Figure 10] FIG. 10 is a diagram of a depth image (depth map) showing depth information for each pixel in grayscale. [Figure 11] FIG. 11 is a diagram schematically illustrating an example of a depth histogram in a scene image of a tree. [Figure 12] FIG. 12 is a diagram showing an example of a binary image acquired by selecting pixels that indicate depths within a predetermined range from among the pixels that make up the leaf region shown in FIG. [Figure 13] FIG. 13 is a diagram showing the position and orientation of an imaging device that captures an image of a tree bathed in sunlight from the sun. [Figure 14] FIG. 14 is a diagram schematically showing a leaf region made up of pixels of three overlapping leaves in a captured image. [Figure 15] FIG. 15 is a diagram schematically illustrating a leaf region made up of pixels of three overlapping leaves. [Figure 16] FIG. 16 is a diagram schematically illustrating an example of a brightness histogram of a near-infrared image obtained by photographing a tree with a near-infrared camera. [Figure 17] FIG. 17 is a diagram showing an example of a luminance histogram after normalization. [Figure 18] FIG. 18 is a diagram schematically showing the position and orientation (posture) of the near-infrared camera. [Figure 19] FIG. 19 is a diagram showing the positions and orientations of the near-infrared camera and the visible light camera that photograph the tree. [Figure 20] FIG. 20 is a block diagram showing an example of the configuration of a tree vigor estimation system according to this embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0033] <Terminology> "Tree" refers to a woody plant (standing tree), including tall trees and shrubs.

[0034] "Tree vigor" is an index that indicates the growth and health of a tree. Tree vigor indicates the growth potential, vitality, and resistance of a target tree or group of trees. Tree vigor can be evaluated based on the tree's appearance, physiological function, and ability to adapt to the environment.

[0035] A "tree vigor index" is a numerical value or measurable characteristic value used to quantitatively express tree vigor. These index values ​​are quantified versions of various morphological characteristics and / or physiological parameters of a tree and can be used to objectively evaluate the growth and health of the tree. In the present disclosure, the "tree vigor index" may include at least one of the following for a tree or group of trees of interest: total leaf area, leaf area index, canopy volume, canopy height, mean leaf size, leaf size variance, leaf count density, mean branch thickness or length, variance of branch thickness or length, and number of branches. These index values ​​may be used individually or in combination. By observing changes in tree vigor index values ​​over time, it is possible to track the progression of a tree's growth and health.

[0036] The term "one or more computers" includes desktop computers, laptops, tablets, mobile devices such as smartphones, wearable devices, embedded systems, and server computers interconnected via a network. These computers may operate independently or may be configured as multiple computer systems performing distributed processing in cloud computing or edge computing environments. Furthermore, computer systems equipped with specialized computing devices such as dedicated hardware accelerators, graphics processing units (GPUs), and field programmable gate arrays (FPGAs) are also included. In this disclosure, a "computer" typically includes a processor such as a GPU or central processing unit (CPU), and a memory device for storing computer programs and data. Any device or system equipped with these components and capable of executing the tree vigor estimation method of this disclosure is included in the "computer" of this disclosure, regardless of its name. Electronic control units (ECUs) installed in and used on work vehicles such as tractors are also included in the "computer."

[0037] <Embodiment> Hereinafter, a method, system, and computer program for estimating tree vigor, as well as an optical simulation method, according to embodiments of the present disclosure will be described with reference to the drawings. Parts that appear in multiple drawings with the same reference numerals indicate the same or equivalent parts.

[0038] The embodiments described below are examples for embodying the technical concept of the present invention, and do not limit the present invention. Furthermore, descriptions of the size, material, shape, relative arrangement, etc. of components are intended for illustration purposes only, and are not intended to limit the scope of the present invention. The size and positional relationship of components shown in each drawing may be exaggerated to facilitate understanding.

[0039] (tree vigor estimation method) The tree vigor estimation method in this embodiment is a tree vigor estimation method executed by one or more computers.

[0040] First, the tree vigor estimation method of this embodiment will be described with reference to Fig. 1. Fig. 1 is a flowchart showing an example of basic processing steps of the tree vigor estimation method of this embodiment.

[0041] As shown in Figure 1, the tree vigor estimation method of this embodiment includes step S10 of acquiring one or more image data showing a scene including trees in a field in real space, and step S20 of estimating an index value indicating the tree vigor based on the image data.

[0042] [Image data acquisition] The acquisition of image data in step S10 can be performed by photographing the tree using one or more image capture devices. The image capture devices can acquire images using one or a combination of the following methods: Portable imaging devices such as cameras mounted on handheld smartphones Imaging devices mounted on agricultural machinery such as tractors -Image capture device mounted on an autonomous mobile robot Imaging devices mounted on unmanned aerial vehicles such as drones Fixed-mounted surveillance cameras or time-lapse cameras

[0043] Acquisition of image data is not limited to the above method, but can also be performed by receiving digital data acquired by photographing trees using one or more imaging devices. Such digital data can be received via a communication line such as the Internet, a dedicated line, or a wireless communication network.

[0044] Image data is not limited to visible light images, but may also include special images such as near-infrared images, thermal images, multispectral images, and hyperspectral images. These special images are obtained by detecting electromagnetic waves in different wavelength bands. For example, near-infrared images detect near-infrared light (wavelength: 800 nm to 2500 nm), which has a longer wavelength than visible light, while thermal images detect near-mid-infrared light (wavelength: 800 nm to 14 μm) emitted from objects. Multispectral images and hyperspectral images capture a wider range of wavelength bands, dividing them into multiple bands.

[0045] Furthermore, the image data may include not only still images but also moving images (multiple time-series images). In the case of moving images, each frame is treated as an independent image, making it possible to capture changes over time.

[0046] In this embodiment, to acquire color image data, an imaging device is used that includes an image sensor in which each pixel has R (red), G (green), and B (blue) subpixels. Such imaging devices typically use a color filter array, such as a Bayer filter, placed in front of the image sensor. This allows the amount of R (red), G (green), and B (blue) radiation contained in the light beam incident on each pixel of the image sensor to be detected. Therefore, the image data acquired in step S10 includes color information (R value, G value, B value) for each pixel. These values ​​are typically recorded with 8-bit or 10-bit precision and expressed as integer values ​​ranging from 0 to 255 (for 8-bit) or 0 to 1023 (for 10-bit).

[0047] To acquire multispectral image data, an image sensor is used in which each pixel has four or more sub-pixels, each of which is provided with a color filter that transmits different wavelength ranges. The wavelength range is not limited to the visible light range, but may also be in the near-infrared range.

[0048] As described above, the conventional technology described in Non-Patent Document 1 requires a monocular camera to be placed on the ground or a predetermined distance below the tree canopy, and images taken by looking up at the tree canopy with the sky as the background must be used. In contrast, this embodiment alleviates such constraints. For example, it becomes possible to estimate index values ​​indicating tree vigor using color images (side views) taken from the side of a tree. The reason why index values ​​indicating tree vigor can be estimated using such images will be described later.

[0049] [Estimation of index values ​​showing tree vigor] In step S20, when estimating an index value indicating tree vigor, a relationship between the image of the scene and the total leaf area of ​​the tree is used (step S25). This "relationship" can be defined, for example, by an equation (relational expression) that associates the "feature amount" of the tree in the image of the scene with the "total leaf area." The feature amount can include pixel information of the leaf area in each of the multiple images. For example, the feature amount is the number of pixels of the leaves in each of the multiple images. When the number of pixels is used as the feature amount, the total number of pixels included in the images to be used needs to be standardized to a predetermined value (e.g., 200,000 pixels). Another example of a feature amount is the ratio of the number of pixels of the leaves to the total number of pixels in each of the multiple images. Such a feature amount is not limited to the number of pixels of the leaf area, and may be a feature amount extracted from the image by deep learning. In this embodiment, the number of pixels of the leaves in each of the multiple images is used as the feature. Note that the total number of pixels included in each image is standardized to 200,000 pixels. In this way, when using images with a predetermined number of pixels, the number of pixels in the leaves can be used as a feature, but if the number of pixels in the images used can vary from image to image, it is preferable to use the ratio of the number of pixels in the leaves to the total number of pixels in each image.

[0050] [Formula relating leaf pixel count to total leaf area] The formula relating the number of pixels in a leaf to the total leaf area can be determined, for example, by the following process. -Several trees in a real-world field are photographed from various angles, and multiple images of scenes containing trees are obtained. Count the feature value (here, the number of leaf pixels) in each of the above images. Measure the area of ​​each leaf (single sided area) of the tree being photographed and collect measurements of the total leaf area. The relationship between the above features and the total leaf area is derived using a statistical method, such as regression analysis.

[0051] However, with the above method, it takes a lot of time to acquire a large number of images for each of a large number of trees and measure the total leaf area.

[0052] In this embodiment, the above formula is determined based on a three-dimensional model of a tree in a simulation space. For example, an optical simulation including the process illustrated in Fig. 2 is executed. An example of an optical simulation method in this embodiment will be described below.

[0053] First, in step S40 shown in FIG. 2, a three-dimensional model of a target tree is created. The three-dimensional tree model used in this embodiment may specifically include elements such as the trunk, branches, leaves, flowers, and fruit (in the case of fruit trees). These elements may be modeled, including their respective shapes, sizes, textures, and relative positions, depending on the type of tree, growth stage, etc. The three-dimensional model is also defined by leaf optical model parameters, including at least one of the leaf refractive index, surface roughness, density, size, absorbance, spectral transmittance, and spectral reflectance.

[0054] In this embodiment, after a three-dimensional model is set, in step S42, the total leaf area of ​​the tree is calculated based on the three-dimensional model. Once the three-dimensional model is set, the shape and size of all leaves of the tree in the three-dimensional model are known, so the total leaf area of ​​the tree can be calculated from the specific three-dimensional model. For example, if each leaf is represented by a polygon mesh such as a triangular polygon, the leaf area of ​​each leaf can be calculated by calculating the area of ​​the polygon mesh that defines one side of each leaf. The total leaf area can be obtained by calculating the leaf area of ​​each leaf of the tree of interest and then calculating the sum of these leaf areas.

[0055] By performing such calculations, it is also possible to calculate the leaf area index, canopy volume, canopy height, mean leaf size, leaf size variance, leaf count density, mean branch thickness or length, variance of branch thickness or length, and number of branches for any tree in the three-dimensional model.When calculating the total area index, a step of obtaining information about the arrangement of tree rows in the field (such as tree spacing or density) may be performed.

[0056] Next, in step S44, multiple images of the tree are generated based on the set 3D model. Specifically, physically based rendering technology in computer graphics is applied to the 3D tree model to generate images from a virtual camera viewpoint. For example, image generation is performed using the following procedure. Place one or more 3D tree models in the simulation space. Set the virtual camera's position, orientation, and optical properties (focal length, angle of view, etc.). Set the position of the light source, such as the sun. A physically based rendering algorithm is used to generate a 2D image of the tree from a given camera viewpoint.

[0057] By freely changing the "position" and "orientation (posture)" of the camera (image capture device), images from various viewpoints can be generated from the same 3D model. This allows for the generation of multiple images (free-viewpoint images) of trees of any type and at any stage of growth, captured from any viewpoint. A feature indicating tree vigor can be determined from each of the multiple images. In the present embodiment, when the feature is the "number of leaf pixels in an image," pixels classified as leaves are first selected from all pixels constituting each image. For images obtained by applying physically based rendering technology to a 3D tree model, the correspondence of each pixel constituting the image to an object surface in the 3D model can be determined, for example, by geometrical optical ray tracing. The number of leaf pixels can be determined for each generated image by counting the number of pixels classified as leaves.

[0058] The order of steps S42 and S44 is not limited to the above example, and may be arbitrary.

[0059] In step S46, it is determined whether or not the data necessary to derive the relationship between the aforementioned feature values ​​and the total leaf area by statistical methods has been collected. If it is determined that the data has not been collected (No), the process returns to step S40 and a new three-dimensional model is generated. The new three-dimensional model can be created, for example, by changing model parameters such as the shape, size, and number of at least one of the trunk, branches, and leaves of the tree. After the new three-dimensional model is set in step S40, the above-mentioned processing steps are repeated and new data is acquired.

[0060] If it is determined in step S46 that the necessary data has been collected (Yes), the process proceeds to step S48. In step S48, feature amounts are extracted from a plurality of images, and the relationship between the feature amounts and the total leaf area is determined.

[0061] Figure 3 is a diagram that shows two tree models (Model 1 and Model 2) at different growth stages. The total leaf area of ​​the tree in Model 1 is "A1," and the total leaf area of ​​the tree in Model 2 is "A2." As described above, after setting up Model 1, a large number of images of the tree in Model 1 are taken with a virtual camera and generated. Similarly, after setting up Model 2, a large number of images of the tree in Model 2 are taken with a virtual camera and generated. Figure 4 shows an example of a dataset determined by the features extracted from each image and the total leaf area, using black dots.

[0062] As shown in Figure 4, the value of total leaf area is constant for each tree model. Because the features extracted from the images may differ for each image, the data obtained for each model is scattered along the horizontal axis. By performing similar processing for multiple models, a scatter plot such as that shown in Figure 5 can be obtained. In the example of Figure 5, the total leaf area is linearly dependent on the image features. Figure 5 also shows an example of a straight line K that approximately indicates the relationship between the image features and the total leaf area. The relationship between the image features and the total leaf area cannot always be approximated linearly.

[0063] Based on the dataset showing the relationship between features and total leaf area shown in the scatter plot above, various analyses can be performed to estimate the total leaf area of ​​trees from image features, using linear regression, polynomial regression, or machine learning algorithms (e.g., random forests, support vector machines, etc.).

[0064] Based on the results of the regression analysis, a mathematical model (function) can be constructed that estimates and outputs the total leaf area from the input features. This mathematical model can be expressed as the following equation: Total leaf area = f (feature) Here, f(·) represents a function determined by, for example, regression analysis. The function f(·) may be a multivariate function containing two or more feature quantities as variables, or may be a deep learning model. This function f(·) may differ depending on the type of tree being studied. To estimate the total leaf area for many types of trees from image features, it is preferable to prepare a function f(·) for each type of tree.

[0065] To verify the estimation accuracy of the constructed mathematical model, it is preferable to test it using actual images of trees with known total leaf area. If necessary, adjust the parameters of the mathematical model and optimize the selection of features.

[0066] Note that limiting the camera height, orientation, and camera-to-tree distance to a certain range reduces the variance of features across multiple images acquired from the same tree model, improving the accuracy of estimations using regression equations, etc.

[0067] An example of a feature extracted from an image is the number of leaf pixels in the image. The feature may be the ratio of the number of leaf pixels to the total number of pixels in each of the multiple images, the size of the leaves in each of the multiple images, the height of the trees, or the number of branches. The number of feature types (independent variables) used as independent variables in the regression equation is not limited to one. Different regression equations may be used depending on information (metadata) related to the date and time of photography, weather conditions, the distance from the trees to the camera, the height of the camera, the orientation of the camera, the angle of view, etc.

[0068] The mathematical model (e.g., function) obtained in this embodiment makes it possible to estimate the total leaf area of ​​a tree using features extracted from an actual tree image as input. This method provides an efficient and non-invasive method for leaf area estimation, replacing traditional destructive sampling techniques and labor-intensive manual measurements.

[0069] This embodiment, compared to the technology described in Patent Document 1, makes it possible to use a variety of images as real-space tree images required for estimation. For example, it is possible to use an image of a tree photographed from the side. Such images contain information about the vertical leaf arrangement (three-dimensional structure), enabling more accurate estimation of the total leaf area. For low-growing shrubs (e.g., grapevines), it may be difficult to photograph the canopy of the tree from directly below the canopy using a handheld camera. In particular, in vineyards, grapevines (shrubs) are arranged in rows along wires, making it easier to photograph the side of the grapevine from an area between the rows of grapevines than to photograph the canopy from directly below. Furthermore, a scene image of a shrub such as a grapevine photographed from the side contains information about the three-dimensional structure of the canopy, including its numerous leaves.

[0070] Referring again to FIG. 1 , in step S30, the estimated index value is stored in the computer's memory as digital data associated with the scene. This index value may also be transmitted to an external device connected to the computer. For example, if a computer server that provides users with the tree vigor estimation method of this embodiment is located on the cloud, the computer server can execute the processing steps of FIG. 1 based on image data uploaded by the user to estimate an index value indicating tree vigor, and then notify the user of the index value, for example, in response to a user request.

[0071] The index values ​​estimated by the above method may be stored in a storage device in association with an identification number that identifies the tree or the geographic coordinates of the tree. This makes it possible to display index values ​​indicating tree vigor for each tree, or for each position or segmented area within the field. The display format of the index values ​​is not limited to numerical values, and may include various formats such as the class to which the numerical value belongs, or colors or symbols that indicate whether the numerical value is high or low. The index values ​​may also be displayed in association with the positions of the trees on a map of the field.

[0072] [Feature extraction] Next, an example of a process for extracting a feature from an image showing a scene including trees in a field in real space will be described with reference to Fig. 6. In this example, the feature is the number of pixels in the leaves. Hereinafter, a "scene including trees in a field in real space" will be simply referred to as a "tree scene," and an image of the tree scene will be simply referred to as a "scene image."

[0073] First, in step S52, the image data acquired in step S10 of Fig. 1 is processed to estimate the depth of each pixel in the tree scene. Depth information can be acquired by the following method.

[0074] Using a monocular camera and deep learning models: From 2D images taken with a conventional monocular camera, it is possible to use deep learning models to estimate the "depth" of each pixel or segment classified as a leaf. This method has the advantage of not requiring additional hardware and can be applied to existing image data.

[0075] Other methods include the following techniques:

[0076] Using the depth camera: Depth cameras, such as ToF cameras and stereo cameras, can directly capture information about the depth of each pixel, as they use the time of flight and parallax of light to measure distance.

[0077] Stereo Vision: A method of calculating depth using parallax information from images taken with multiple cameras.

[0078] Structured Light: A technology that projects a known pattern and estimates depth from the distortion of that pattern.

[0079] These methods each have different characteristics, and it is possible to select one depending on the usage environment and the required accuracy.

[0080] By using the depth information estimated by the above method, leaf pixels within a predetermined depth range are selected, which makes it easy to select only the leaf pixels of the target tree even if other trees are captured in the scene image.

[0081] FIG. 7 is a diagram showing a portion of a row of fruit-bearing trees (fruit trees) 10 in an orchard in real space. FIG. 7 also shows a schematic representation of an imaging device 110, such as a camera. The imaging device 110 is located at a height H relative to the ground G. The line of sight of the imaging device 110 forms an angle θ with respect to the horizontal plane. The angle θ may be defined as having a "positive" value when the line of sight of the imaging device 110 is tilted in an elevation angle direction relative to the horizontal plane, and as having a "negative" value when it is tilted in a depression angle direction. This definition may also be reversed.

[0082] The position and orientation (attitude) of the image capture device 110 can be expressed by six parameters. Specifically, three parameters represent the position of the image capture device 110, and the other three parameters (e.g., Euler angles) represent the attitude of the image capture device 110. The attitude of the image capture device 110 may be represented by a quaternion.

[0083] To acquire a scene image from which effective features for estimating the total leaf area can be extracted, it is undesirable for the distance L from the image capture device 110 to the subject tree 10 to be too close or too far. Therefore, the distance L may be set, for example, so that the crown of the target tree 10 occupies 10% or more of the image and so that the image includes 50% or more of the area from the bottom to the top of the crown. Specifically, the distance L may be set, for example, between 50 cm and 500 m. The height H of the image capture device 110 is, for example, between 30 cm and 250 cm. When the image capture device 110 acquires a side view of the tree 10, the angle θ between the line of sight of the image capture device 110 and the horizontal plane is, for example, between −70° and +70°. As described above, a scene image (side view) of the tree 10 captured from the side contains information about the arrangement of leaves in the vertical direction (three-dimensional structure), enabling more accurate estimation of the total leaf area. In particular, when the tree is a grapevine, since grapevines are long vertically, observing the tree from the side, i.e., from the side, has the advantage that the structure of the tree can be more easily recognized.

[0084] When capturing a scene image from which useful features can be extracted to estimate the total leaf area in this manner, it is preferable that the position and orientation of the virtual imaging device that defines the viewpoint of the image generated by simulation to obtain the "formula relating the number of leaf pixels to the total leaf area" is also set within a similar range.

[0085] If the distance from the imaging device to the tree varies significantly, the ratio of the number of leaf pixels to the total number of pixels in each image will vary significantly, even among multiple images of the same tree. Therefore, different "formulas relating the number of leaf pixels to the total leaf area" may be prepared for different distances or distance ranges from the imaging device to the tree. In this case, an appropriate formula can be selected based on the distance from the imaging device to the tree when the tree is photographed in real space. The same applies to the angle of view of the imaging device. The metadata about the imaging conditions described above is preferably used to select an appropriate "formula relating the number of leaf pixels to the total leaf area."

[0086] Specifically, the system allows a user to input several parameters that define the shooting conditions into a computer. For example, if a user is required to photograph a tree from directly to the side, the range of shooting conditions is limited, so it is sufficient to input only a few parameters, such as the distance from the image capture device to the tree. On the other hand, if the user is also allowed to photograph the tree canopy from diagonally below or above, it is preferable that the system be configured to have the user input information regarding the camera position and select an appropriate "formula relating the number of leaf pixels to the total leaf area" based on that information.

[0087] Fruit trees, such as grapes, are often planted in multiple rows. When trees in one row are photographed from the side, there is a high possibility that trees in other rows will be reflected in the scene image. In this embodiment, depth information for each pixel in the scene image is used to distinguish between trees in the row of interest and trees in other rows that are reflected in the scene image. When trees are photographed from the side, there is a possibility that trees in other rows will be reflected, but by using the depth information, it is easy to identify the trees in the row of interest in the image.

[0088] Fig. 8 is a diagram showing an example of a scene image acquired by capturing an image of trees in real space with the image capturing device 110. The scene image captured by the image capturing device 110 is itself a color image, with each pixel having a value of R (red), G (green), and B (blue). However, Fig. 8 shows a monochrome (grayscale) image generated from this color image. In Fig. 8, some trees belonging to another row are reflected behind the row of trees of interest.

[0089] Referring again to Figure 6, in this embodiment, in step S54, leaf pixels within a predetermined depth range are selected based on the depth and color of each pixel in the scene image acquired by the image capture device 110. Specifically, after selecting the leaf pixels, leaf pixels within the predetermined depth range are selected.

[0090] As described above, in this embodiment, color image data is acquired, and by using color information for each pixel, it is possible to classify whether each pixel belongs to a tree branch, fruit, flower, or leaf, or to the "background" such as the ground or sky. To detect leaf regions, a segmentation technique based on deep learning can be used. When using segmentation techniques, not only color information but also texture information of pixels can be used.

[0091] Fig. 9 is a diagram schematically showing a leaf area 12 selected from the image of Fig. 8. Leaf area 12 is surrounded by a solid line 12L. Leaf area 12 in the image of Fig. 9 includes not only the leaf pixels of trees in the row to which the tree of interest belongs, but also the leaf pixels of trees in the row behind it.

[0092] After selecting the leaf pixels in this way, by masking the pixels outside a predetermined depth range, it is possible to select and detect the leaves of the tree of interest from the scene image.

[0093] FIG. 10 is a diagram of a depth image showing the depth information of each pixel in grayscale. A depth image is sometimes called a depth map. In FIG. 10, the shorter the distance between the object surface position corresponding to each pixel and the imaging device (the shallower the depth), the brighter the pixel is displayed. As is clear from the depth image in FIG. 10, the brightness of the pixels of the leaves of the trees in the back row is low, and they are black or a gray close to black. A depth histogram can be created from the depth image in FIG. 10. A depth histogram is a graph in which the horizontal axis represents bins, which are depths divided by a predetermined width, and the vertical axis represents the frequency of pixels included in each bin. Hereinafter, the depth bins will be simply referred to as "depth," and the frequency of pixels included in each bin will be referred to as "pixel count."

[0094] 11 is a diagram illustrating an example of a depth histogram for a tree scene image. In the example of FIG. 11, there is a first peak in the number of pixels in an area that is shallower than a certain depth TH, and there are other peaks in areas that are deeper than the depth TH.

[0095] When trees belonging to multiple tree rows are captured in the same image, the trees in different rows are located in different depth ranges, resulting in multiple peaks in the depth histogram. The pixels of the trees in the front row have low depths, forming the first peak in the depth histogram. The pixels of the trees in the rear row have successively higher depths, forming the second peak. From the depth histogram, a depth midway between adjacent peaks can be selected as a threshold. Pixels with depths equal to or greater than this threshold are pixels of the background or trees in the rear row. In the example of Figure 11, for example, depth TH can be selected as the threshold. Pixel group R1 with a depth lower than the threshold depth TH can be determined to belong to the trees in the front row. Pixel group R2 with a depth higher than depth TH can be determined to belong to the trees in the rear row or to other objects between the tree rows.

[0096] Fig. 12 shows an example of a binary image obtained by selecting pixels showing depths in a predetermined range (for example, depths lower than depth TH) from among the pixels constituting the leaf region 12 shown in Fig. 9. In the binary image of Fig. 12, the white pixels are composed of the selected pixels and are pixels of the leaves of the trees belonging to the column of interest.

[0097] 9 may include areas other than leaves, such as branches, fruits, flowers, etc. In such cases, it is preferable to exclude pixels belonging to areas other than leaves based on color information of the image, etc.

[0098] Next, in step S56 shown in FIG. 6, a feature related to the total leaf area is extracted from the selected leaf pixels. In the present embodiment, when the feature is the "number of leaf pixels in the image," the total number of selected pixels is counted. In the example of FIG. 12, the total number of pixels included in the white region of the binary image is used as the feature. As mentioned above, the feature is not limited to this example. For example, for the pixels included in the white region of the binary image of FIG. 12, the feature may be extracted from various information, such as the ratio of the total number of pixels in the white region to the total number of pixels included in the image, or the shape of pixel group R1 in the depth histogram.

[0099] (Variation 1 of the tree vigor estimation method) Next, a modified example of the tree vigor estimation method will be described. In this modified example, a near-infrared image of a tree scene is used as image data. By extracting the brightness histogram of leaves in the near-infrared image as a feature, it becomes possible to detect and evaluate overlapping leaves. The brightness histogram is a graph in which the brightness (near-infrared irradiation amount) in each near-infrared image is divided into classes with a predetermined width on the horizontal axis and the frequency of pixels included in each class on the vertical axis. Hereinafter, the brightness classes in the brightness histogram will be simply referred to as "brightness," and the frequency of pixels included in each class will be referred to as "pixel count."

[0100] 13 is a diagram showing the position and orientation of an imaging device capturing an image of a tree 10 bathed in sunlight from the sun 20. The diagram shows an XYZ coordinate system (global coordinate system) fixed to the ground, an X1-Y1-Z1 coordinate system (camera coordinate system) fixed to the near-infrared camera, and an X1-Y1 plane 21 parallel to the image plane of the near-infrared camera. The Z1 axis indicates the line of sight of the near-infrared camera.

[0101] 13 has leaves 14a, 14b, 14c, 14d, 14e, and 14f. When the tree 10 is photographed with a near-infrared camera in the position and orientation shown in the figure, the other leaves 14c, 14d, 14e, and 14f may only be partially captured due to the leaves 14a and 14b in the foreground.

[0102] 14 is a diagram illustrating a leaf region 12A in a captured image, which is composed of pixels of three overlapping leaves 14a, 14c, and 14e. This leaf region 12A does not contain information about the overlapping areas of the leaves 14a, 14c, and 14e. The number of pixels in the leaf region 12A in FIG. 14 lacks useful information about the overlapping areas for estimating the sum of the areas of the leaves 14a, 14c, and 14e in three-dimensional space.

[0103] 15 is a diagram illustrating a leaf region 12B composed of pixels of three overlapping leaves 14a, 14c, and 14e. In FIG. 14, the brightness of pixels belonging to the overlapping portion of two of leaves 14a, 14c, and 14e is lower than the brightness of pixels belonging to leaves with no overlap. Also, the brightness of pixels belonging to the overlapping portion of three of leaves 14a, 14c, and 14e is lower than the brightness of pixels belonging to the overlapping portion of two leaves.

[0104] If the pixels can be classified according to the degree of overlap of the leaves, it becomes possible to obtain information that correlates with the total area of ​​the leaves 14a, 14c, and 14e.

[0105] Grape leaves are known to transmit near-infrared light with wavelengths above 800 nm, which is also contained in sunlight. In the case of grapes, the near-infrared transmittance of a single leaf is approximately 40% at a wavelength of 800 nm. Therefore, the transmittance in the area where two leaves overlap is approximately 16%, and in the area where three leaves overlap is approximately 6%. Similar properties can be observed in leaves other than grape leaves.

[0106] Therefore, if you photograph a tree with a camera that can generate near-infrared images and create a brightness histogram of the acquired near-infrared image, the brightness (brightness) of pixels belonging to the leaves will show different values ​​depending on the degree of leaf overlap.

[0107] FIG. 16 is a diagram showing an example of a brightness histogram of a near-infrared image of a tree captured with a near-infrared camera. As mentioned above, the brightness histogram has peaks reflecting the degree of leaf overlap. Because branches transmit almost no near-infrared light, the brightness of pixels belonging to the branches is extremely low. Furthermore, if a background such as the sky is captured in part of the near-infrared image, some pixels may exhibit very high brightness. Furthermore, because the brightness of a near-infrared image depends on the intensity of sunlight reaching the tree from the sun, the average brightness and brightness distribution range in a near-infrared image may vary depending on weather conditions. Therefore, when creating a brightness histogram, it is preferable to normalize the brightness of pixels belonging to the background in each of the multiple acquired near-infrared images by setting the maximum brightness and the minimum brightness to the brightness of pixels belonging to the tree branches. This normalization allows for a brightness histogram that is easy to evaluate the degree of leaf overlap, regardless of the brightness value itself, which varies depending on the shooting conditions. An example brightness histogram after such normalization is shown in FIG. 17.

[0108] 15, leaf region 12B is divided into a group of pixels with the highest brightness, a group of pixels with intermediate brightness, and a group of pixels with the lowest brightness. If the number of pixels in the group of pixels with the highest brightness is N1, the number of pixels in the group of pixels with intermediate brightness is N2, and the number of pixels in the group of pixels with the lowest brightness is N3, the number of pixels belonging to leaf region 12B can be calculated by, for example, N1+2·N2+3·N3, rather than N1+N2+N3.

[0109] Note that because the proportion of near-infrared light that passes through the overlapping areas of the three leaves is low, there is a possibility that the pixels will not be accurately detected as leaf pixels. For this reason, the division of pixel groups using the brightness histogram of the near-infrared image may be performed by dividing the pixel groups into overlapping and non-overlapping areas.

[0110] The higher the intensity of near-infrared light incident on tree leaves, the higher the average brightness of the brightness histogram, making it easier to obtain accurate information about leaf overlap. For this reason, the position and orientation of the near-infrared camera are preferably such that the camera optical axis (Z1 axis) is directed toward the sun 20 or its vicinity, as shown in the X1-Y1-Z1 coordinate system (camera coordinate system) in Figure 18. Furthermore, it is preferable to acquire near-infrared images when the sun 20 is high in the sky and the weather is clear.

[0111] The amount of near-infrared radiation varies depending on the altitude of the sun 20 and weather conditions, but the effect of such variations can be reduced by using the normalized brightness histogram described above.

[0112] In this modified example, when deriving the function (relationship) of total leaf area = f (feature) through optical simulation, the refractive index, absorbance, spectral transmittance, and spectral reflectance of the leaf at near-infrared wavelengths are used as important model parameters.

[0113] As mentioned above, because grapevines are vertically long, it is easier to recognize the structure of the tree by observing it from the side, i.e., from the side. For this reason, it is preferable to use an image of the tree photographed from the side in addition to a near-infrared image acquired by a near-infrared camera positioned and oriented as shown in Fig. 18. The image of the tree 10 photographed from the side is preferably a visible light image containing color information, but may also be a near-infrared image.

[0114] Figure 19 is a schematic diagram illustrating how a tree 10 is photographed with a near-infrared camera and also photographed with a visible light camera at a camera viewpoint different from the camera viewpoint of the near-infrared image. Figure 19 schematically illustrates an X1-Y1-Z1 coordinate system and an X1-Y1 plane 21 fixed to the near-infrared camera, as well as an X2-Y2-Z2 coordinate system and an X2-Y2 plane 22 fixed to the visible light camera. The Z2 axis indicates the line of sight of the visible light camera.

[0115] By using a visible light image from a camera viewpoint different from that of the near-infrared image, there is an advantage that three-dimensional information about the leaves that reflects the three-dimensional structure of the crown of the tree 10 can be obtained.

[0116] (Variation 2 of the tree vigor estimation method) As shown in Figure 19, when using image data including a visible light image and a near-infrared image, an equation that associates tree feature 1 in the visible light image and tree feature 2 in the near-infrared image with the total leaf area can be created using the optical simulation method described above, and the index value can be estimated using this equation.

[0117] This formula is generally expressed as follows, for example: Total leaf area = f(feature 1, feature 2) where f(·) represents the function determined by regression analysis.

[0118] Feature 1 is, for example, the total number of pixels belonging to leaves. Feature 2 is a feature extracted from a brightness histogram, and is, for example, a set (multivariate or vector) of counts of pixels belonging to each of multiple brightness groups with different brightness peaks. For example, if, among the pixels included in leaf region 12, X1 represents the number of pixels in a pixel group whose brightness is equal to or greater than a threshold, and X2 represents a prime number of pixels in a pixel group whose brightness is less than the threshold, then vector (X1, X2) can be used as feature 2. Such a feature may contain information that contributes to estimating the total leaf area, in that it contains information about leaf overlap rather than the number of leaf pixels in the visible light image.

[0119] In Variation 2, optical simulation is performed in the same manner as for visible light images to generate multiple near-infrared images, and a brightness histogram is generated for each generated near-infrared image. To generate such near-infrared images using optical simulation, the three-dimensional leaf model used must have model parameters related to the transmittance and reflectance of near-infrared light. For example, an optical leaf model with transmittance and reflectance at a wavelength of 800 nm is established to create a three-dimensional tree structure within the simulation space. Using the simulation method described with reference to Figure 2, near-infrared images captured from various camera viewpoints are generated for each of the multiple three-dimensional models using physically based rendering. Because the near-infrared images obtained using this simulation method are based on the optical properties of leaves, they exhibit a brightness distribution that reflects the degree of leaf overlap, which varies depending on the leaf arrangement (canopy structure) of the tree. As a result, it becomes possible to derive with greater accuracy an equation relating the above-mentioned Feature 2 to the total leaf area.

[0120] When a near-infrared camera used to photograph trees in a real-world field is equipped with a filter that transmits near-infrared light in a specific wavelength band, it is preferable to use a function f (feature 1, feature 2) created by performing an optical simulation tailored to that wavelength band.

[0121] (Tree vigor estimation method variation 3) Visible light images from multiple viewpoints may be used as image data. In this case, the camera position (e.g., height) and camera attitude (e.g., the tilt angle of the camera's line of sight relative to the horizontal plane) may be used as metadata for each image data. If different functions f(·) are prepared according to such metadata, then by inputting information about the camera's position and attitude when photographing a tree in real space into a computer, an appropriate function f(·) can be selected, enabling more accurate estimation of tree vigor index values.

[0122] When performing optical simulation, if images are generated from individual 3D models by physically based rendering, a scatter plot with low variance may be obtained by restricting the camera position (e.g., height) and camera attitude (e.g., tilt angle of the camera's line of sight relative to the horizontal plane) to a specific range. In this case, when acquiring images of trees in a farm field in real space, it is preferable to align the camera position (e.g., height) and camera attitude (e.g., tilt angle of the camera's line of sight relative to the horizontal plane) within the above-mentioned specific range.

[0123] (Tree vigor estimation method variation 4) Near-infrared images from multiple viewpoints may be used as image data. In this case, the position (e.g., height) and orientation of the near-infrared camera (e.g., the tilt angle of the camera's line of sight relative to the horizontal plane) may be used as metadata for each image data. If different functions f(·) are prepared according to such metadata, then by inputting information about the camera's position and orientation when photographing a tree in real space into a computer, an appropriate function can be selected from the multiple functions f(·) prepared in advance, enabling more accurate estimation of tree vigor index values.

[0124] As mentioned above, near-infrared images contain information about leaf overlap, so information about leaf overlap can be extracted as a feature based on the brightness histogram. By using near-infrared images from different camera viewpoints, it is possible to obtain information that reflects the three-dimensional structure of the tree crown.

[0125] (Tree vigor estimation system) 20 is a block diagram showing an example configuration of a tree vigor estimation system 1000 according to this embodiment. The tree vigor estimation system 1000 includes a computer 100, and an imaging device 110, a communication device 140, and a display device 150 connected to the computer 100. The computer 100 includes a storage device 120 and a processing device 130. These components are connected to each other via a bus, for example. The computer 100 can also be realized by a computing device installed in a mobile terminal device such as a smartphone owned by a user who receives the services of the tree vigor estimation method.

[0126] The tree vigor estimation system 1000 in this embodiment further includes another computer 200. The computer 200 is connected to a communication device 240, which is connected to the communication device 140 via wired or wireless network communication. The computer 200 includes a storage device 220 and a processing device 230. The storage device 220 and the processing device 230 are connected to each other, for example, via a bus. The computer 200 may be a personal computer that communicates with the computer 100 via the Internet. The computer 200 may also be realized by a cloud server. Such a cloud server may be configured to download a computer program (application) for executing the tree vigor estimation method to the storage device 120 in the mobile terminal device owned by the user.

[0127] In this embodiment, the imaging device 110 is used to photograph trees in a field in real space. For example, an image (actually a color image) such as that shown in FIG. 8 is captured by the imaging device 110. The imaging device 110 may be a camera mounted on the mobile terminal device described above. A user may capture a tree selected as a subject in the field using the imaging device 110, and process the image data using a computing device mounted on the mobile terminal device owned by the user, thereby implementing the tree vigor estimation method described above.

[0128] The storage device 120 in this embodiment includes volatile and nonvolatile memory, and stores computer programs and data that cause the processing device 130 and the like to execute the processing steps shown in FIG. 1 . The storage device 120 also stores one or more image data representing a scene including trees in a field in real space. Volatile memory is, for example, semiconductor memory such as DRAM (Dynamic Random Access Memory) and SRAM (Static RAM). Nonvolatile memory is, for example, semiconductor memory such as ROM (Read Only Memory) and flash memory, optical disk drives, and hard disk drives.

[0129] The processing device 130 may be configured with a processor such as a GPU or a CPU. The processor is a hardware device realized by a semiconductor integrated circuit. Some or all of the processor may be a field programmable gate array (FPGA) equipped with a CPU, an application specific integrated circuit (ASIC), or an application specific standard product (ASSP). The processing device 130 operates according to a computer program recorded in the storage device 120. The processing device 130, either independently or in cooperation with the processing device 230 of the computer 200, estimates an index value indicating the tree vigor based on image data acquired by the imaging device 110.

[0130] The communication devices 140 and 240 are interfaces for performing data communication between the computer 100 and the computer 200, or between the computer 100 and the computer 200 and an external computing device. The communication devices 140 and 240 can perform wired communication using a controller area network (CAN) or the like, or wireless communication conforming to the Bluetooth (registered trademark) standard and / or the Wi-Fi (registered trademark) standard.

[0131] In this embodiment, the processing device 130 uses a relationship between the image of the scene and the total leaf area of ​​the trees when estimating the index value. In this embodiment, this relationship is determined by the computer 200 executing the simulation described above. The storage device (recording medium) 220 of the computer 200 stores a computer program that causes the processing device 230 to execute the following processing steps. Setting up a three-dimensional model of the tree in the simulation space; generating multiple images of a tree based on a three-dimensional model; determining the total leaf area of ​​said trees based on a three-dimensional model; and Extracting features from multiple images and determining the relationship between the features and total leaf area.

[0132] As described above, the three-dimensional model is defined by optical model parameters of the leaf, including at least one of the leaf's refractive index, surface roughness, density, size, absorbance, spectral transmittance, and spectral reflectance.

[0133] The processing unit 130 can present the estimated index value to the user via the display unit 150 .

[0134] Note that all or part of the components and functions of computer 100 may be implemented by a cloud server. In this case, image data captured by a user is uploaded to the cloud server, and computer 100 in the cloud server may execute each process of the tree vigor estimation method. [Industrial Applicability]

[0135] The tree vigor estimation method, tree vigor estimation system, computer program, and optical simulation method of the present disclosure can be widely used in the agricultural field where trees are cultivated. [Explanation of symbols]

[0136] 10... Tree, 12... Leaf area, 14... Leaf, 20... Sun, 100... Computer, 110... Imaging device, 120... Storage device, 130... Processing device, 140... Communication device, 150... Display device, 200... Computer, 220... Storage device, 230... Processing device, 240... Communication device, 1000... Tree vigor estimation system

Claims

1. A tree vigor estimation method executed by one or more computers, comprising: acquiring one or more image data representing a scene including a tree in a field in real space; estimating an index value indicating tree vigor based on the image data; Including, A method for estimating tree vigor, wherein a relationship between an image of the scene and a total leaf area for the tree is used when estimating the index value.

2. The tree vigor estimation method according to claim 1 , further comprising storing the index value as digital data associated with the scene in a memory within the computer or transmitting the index value to a device connected to the computer.

3. The tree vigor estimation method according to claim 1 , wherein the relationship is defined by an equation that associates a feature amount of the tree in the image of the scene with the total leaf area.

4. The tree vigor estimation method according to claim 3 , wherein the formula is determined based on a three-dimensional model of the tree in a simulation space.

5. The relationship is: determining a total leaf area of ​​the tree based on the three-dimensional model; generating a plurality of images of the tree based on the three-dimensional model; extracting features from the plurality of images and determining a relationship between the features and the total leaf area; The method for estimating tree vigor according to claim 4, wherein the tree vigor is determined by a process including the steps of:

6. The tree vigor estimation method according to claim 5 , wherein the feature amount includes pixel information of a leaf area in each of the plurality of images.

7. The tree vigor estimation method according to claim 6 , wherein the feature amount includes the number of leaf pixels in each of the plurality of images, or a ratio of the number of leaf pixels to the total number of pixels in each of the plurality of images.

8. The tree vigor estimation method according to claim 7 , wherein the feature amount includes a brightness histogram of leaves in each of the plurality of images.

9. The tree vigor estimation method according to claim 6 , wherein the feature amounts include at least one of leaf size, tree height, and number of branches in each of the plurality of images.

10. 2. The tree vigor estimation method according to claim 1, wherein the index values ​​include at least one of the total leaf area, leaf area index, canopy volume, canopy height, average leaf size, variance of leaf size, leaf count density, average branch thickness or length, variance of branch thickness or length, and number of branches for the tree.

11. processing the image data to estimate a depth for each pixel of the scene; selecting leaf pixels in a predetermined depth range based on the depth and color of each pixel; extracting features related to total leaf area from the leaf pixels; The tree vigor estimation method according to claim 1, comprising:

12. acquiring the image data photographing the tree using one or more imaging devices; or receiving digital data acquired by photographing the tree using one or more imaging devices; The tree vigor estimation method according to claim 1, comprising:

13. The tree vigor estimation method according to claim 1 , wherein the image data includes a side image of the tree.

14. the image data includes a near-infrared image of the scene; The tree vigor estimation method according to claim 3 , further comprising extracting a brightness histogram of leaves in the near-infrared image as the feature amount.

15. The tree vigor estimation method according to claim 14 , wherein the image data includes a visible light image taken from a camera viewpoint different from that of the near-infrared image.

16. 1. An optical simulation method for determining a relationship between an image of a tree and a total leaf area of ​​said tree, comprising: establishing a three-dimensional model of a tree in a simulation space; generating a plurality of images of the tree based on the three-dimensional model; determining a total leaf area of ​​the tree based on the three-dimensional model; extracting features from the plurality of images and determining a relationship between the features and the total leaf area; An optical simulation method comprising:

17. The optical simulation method according to claim 16 , wherein the feature amount is based on pixel information of a leaf region in each of the plurality of images.

18. 18. The optical simulation method of claim 17, wherein the three-dimensional model is defined by leaf optical model parameters including at least one of leaf refractive index, surface roughness, density, size, absorbance, spectral transmittance, and spectral reflectance.

19. a storage device that stores one or more image data representing a scene including trees in a field in real space; a processing device that estimates an index value indicating tree vigor based on the image data; and Equipped with A tree vigor estimation system, wherein the processing unit uses a relationship between the image of the scene and the total leaf area for the tree when estimating the index value.

20. On one or more computers, acquiring one or more image data representing a scene including a tree in a field in real space; a processing step of estimating an index value indicating tree vigor based on the image data; Execute A computer program that uses a relationship between an image of the scene and total leaf area for the trees when estimating the index value.

Images