Information processing device, information processing method, and program

Abstract

To streamline and accelerate breeding and trait surveys by imputing missing or outlier data in 3D data according to the plant's growth stage. [Solution] The information processing device of the embodiment includes a generation unit that generates a plant shape hypothesis in time series using a 4D growth model that adds a time dimension to the 3D shape of the plant, based on time-series 3D data of the plant, and an adjustment unit that adjusts the parameters of the 4D growth model so that the 3D data and the generated shape hypothesis match, wherein the generation unit outputs the plant shape hypothesis at the time the parameter adjustment is completed as complementary data in which at least one of the missing values ​​and outliers of the 3D data is filled in.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] Conventionally, techniques for complementing missing values and outliers in three-dimensional shape data of a photographed plant have been known. For example, Non-Patent Document 1 describes a technique for estimating phenotype values from 3D plant phytomers by formulating a phenotyping problem as an optimization problem in a plant model parameter space.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the conventional techniques described above can only handle point cloud data with missing or outlier values ​​at specific points in time for plants, and do not compensate for missing or outlier values ​​in 3D data according to the growth stage of the plant. Furthermore, in recent years, there has been a demand for labor-saving and speedy breeding and trait surveys, and many methods for measuring traits from 3D plant data have been studied. However, these methods assume that there are few missing or outlier values ​​in the 3D plant data, and may not be applicable to field environments such as outdoors where missing or outlier values ​​are likely to occur in 3D data.

[0005] This invention has been made in consideration of these circumstances, and one of its objectives is to provide an information processing device, an information processing method, and a program that can streamline and speed up breeding and trait surveys by imputing missing or outlier data in three-dimensional data according to the growth stage of plants. [Means for solving the problem]

[0006] An information processing device according to a first aspect of the present invention comprises: a generation unit that generates a plant shape hypothesis in time series using a 4D growth model that adds a time dimension to the 3D shape of the plant, based on time-series 3D data of the plant; and an adjustment unit that adjusts the parameters of the 4D growth model so that the 3D data and the generated shape hypothesis match, wherein the generation unit outputs the plant shape hypothesis at the time the parameter adjustment is completed as complementary data in which at least one of the missing values ​​and outliers of the 3D data is filled in.

[0007] The generation unit may probabilistically generate individual parameters for each plant species according to a predetermined probability distribution characterized by common parameters common to each plant species, and generate a morphological hypothesis of the plant in time series based on the individual parameters.

[0008] The individual parameters are parameters that define a logistic function representing the traits of one or more phytomers contained in the plant, and the generation unit may calculate the traits of the phytomers by inputting each point in the time series into the logistic function defined by the individual parameters, and generate a morphological hypothesis of the plant in the time series from the calculated traits.

[0009] The phytomer trait may include at least one of the following: leaf length, leaf width, petiole length, petiole width, internode length, stem diameter, and leaf inclination angle.

[0010] The adjustment unit may adjust the common parameters and individual parameters so that the three-dimensional data of the time series matches the hypothesis of the shape of the time series.

[0011] The adjustment unit may calculate the degree of agreement between the depth image of the time-series 3D data and the depth image of the time-series shape hypothesis as the likelihood, and adjust the common parameters and individual parameters using the Markov chain Monte Carlo method so that the product of the likelihood and the probability of occurrence of the common parameters and individual parameters increases.

[0012] Another aspect of the present invention relates to an information processing method in which a computer generates a morphological hypothesis of a plant in time series using a 4D growth model of the plant based on time-series three-dimensional data of the plant, adjusts the parameters of the 4D growth model so that the three-dimensional data and the generated morphological hypothesis match, and outputs the morphological hypothesis of the plant at the time the parameter adjustment is completed as complementary data in which at least one of the missing values ​​and outliers of the three-dimensional data is filled in.

[0013] Another aspect of the present invention involves a program that causes a computer to generate a morphological hypothesis of a plant in time series using a 4D growth model of the plant based on time-series three-dimensional data of the plant, adjust the parameters of the 4D growth model so that the three-dimensional data and the generated morphological hypothesis match, and output the morphological hypothesis of the plant at the time the parameter adjustment is completed as complementary data in which at least one of the missing values ​​and outliers of the three-dimensional data is filled in. [Effects of the Invention]

[0014] According to each of the above embodiments, by imputing missing or outlier data in 3D data according to the growth stage of the plant, it is possible to reduce the effort and speed up breeding and trait surveys. [Brief explanation of the drawing]

[0015] [Figure 1] This figure shows an example of the operating environment for the information processing device 100. [Figure 2] This figure shows another example of how to image plants using camera 10. [Figure 3] This figure shows an example of acquired image data 152. [Figure 4] This is a diagram illustrating the parameters Xi of the 4D growth model. [Figure 5] This figure shows the logistic function used to calculate traits using the parameter Xi of the 4D growth model. [Figure 6] This diagram illustrates a method for calculating the likelihood of agreement between 3D data Pt,i and 3D shape hypotheses Yt,i. [Figure 7] This figure shows an example of the supplementary image data 156. [Figure 8] This figure shows an overview of parameter tuning for a 4D growth model. [Figure 9] This flowchart shows an example of the processing flow executed by the information processing device 100. [Modes for carrying out the invention]

[0016] [overview] The following describes an information processing device 100 according to an embodiment of the present invention, with reference to the drawings. In the following embodiment, as an example of a plant, we will describe how to compensate for missing data (for example, when a point cloud that should exist does not exist) and outliers (for example, when a point cloud deviates from the position where it should exist) in 3D data for cabbage. However, the invention is not limited to cabbage, but can be broadly applied to any plant (including crops, etc.) grown in a field.

[0017] Figure 1 shows an example of the operating environment and configuration of the information processing device 100. The information processing device 100 operates in cooperation with, for example, a camera (an example of an imaging device) 10 and a terminal device 20. The camera 10 is an RGB-D camera installed in a field that takes images of cabbage, a plant grown in the field, in a time series at predetermined intervals. In other words, in this embodiment, the camera 10 is a sensor that can acquire not only a color image (RGB image) but also the distance (D) to the target plant, and the image captured by the camera 10 represents the three-dimensional data of the plant. The camera 10 transmits the captured image data to the information processing device 100 via a wireless or wired network NW.

[0018] In another configuration, the camera 10 may be manually carried by the user of the information processing device 100. In this case, the user of the information processing device 100 manually uses the camera 10 to image plants, and the captured image data is stored in a USB (universal serial bus) memory or the like. Subsequently, the user transfers and stores the image data to the information processing device 100 by connecting the USB memory or the like to the information processing device 100.

[0019] Furthermore, in another embodiment, the camera 10 may consist of one or more cameras mounted on a mobile device (e.g., a cart, tractor, drone, etc.), which is moved along a route in the field while photographing plants. Figure 2 shows another example of a method for imaging plants using the camera 10. In Figure 2, (a) shows an example where the camera 10 is mounted on a cart C, and (b) shows an example where the camera 10 is mounted on a tractor T. As shown in Figure 2, the camera 10 may be mounted on a cart C or a tractor T, and the cart C or tractor T may be moved along a route in the field while the camera 10 photographs plants. This allows for stable acquisition of images of the same plant over time, without varying conditions such as the shooting angle and timing, compared to manual shooting. It also allows for efficient acquisition of images of plants even when there is a large quantity of plants to be photographed. In Figure 2, multiple cameras 10 are installed on the trolley C or tractor T, but the present invention is not limited to such a configuration, and only one camera 10 may be installed on the trolley C or tractor T.

[0020] The terminal device 20 is a computer device such as a personal computer, smartphone, or tablet terminal. The terminal device 20 communicates with the information processing device 100 via a network NW and outputs the supplemental image data 156 received from the information processing device 100 to a screen such as a display. As described later, the supplemental image data 156 is the 3D data represented by the image data captured by the camera 10, with missing or outlier values ​​filled in. As a result, even if the image data captured by the camera 10 contains missing or outlier values, users of the information processing device 100 can use the supplemental image data 156 as a reference to streamline and speed up breeding and trait surveys.

[0021] The information processing device 100 is a server device such as a web server. The information processing device 100 includes, for example, an acquisition unit 110, a generation unit 120, and an adjustment unit 130. Each of the acquisition unit 110, the generation unit 120, and the adjustment unit 130 is realized by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or GPU (Graphics Processing Unit), or by the cooperation of software and hardware. The program may be stored in advance in a storage device such as an HDD (Hard Disk Drive) or flash memory (a storage device with a non-transient storage medium), or it may be stored in a removable storage medium such as a DVD or CD-ROM (a non-transient storage medium) and installed when the storage medium is inserted into a drive device. The storage unit 150 is implemented using a storage device such as an HDD, flash memory, or RAM (Random Access Memory). The storage unit 150 stores, for example, acquired image data 152, 4D growth model parameters 154, and interpolated image data 156.

[0022] [Information Processing Device] The acquisition unit 110 acquires three-dimensional data of plants captured in time series by the camera 10 and stores it in the storage unit 150 as acquired image data 152. Figure 3 shows an example of acquired image data 152. Figure 3 shows the time-series three-dimensional data obtained by the camera 10 capturing images of cabbage growing in the field from time t=0 to T. As shown in Figure 3, for example, the image at time t=T-2 shows that there is a defect in the leaves. For convenience, in the following explanation, it will be assumed that the storage unit 150 stores three-dimensional data from time t=0 to T for N (N is an integer greater than 1) plants.

[0023] The generation unit 120 generates a hypothesis of the three-dimensional shape (three-dimensional model) of the plant using a four-dimensional growth model of the plant, based on the acquired three-dimensional data. Here, a four-dimensional growth model of a plant is a growth model that adds a time dimension to the three-dimensional shape of the plant and simulates the three-dimensional shape of the plant over time. In this embodiment, a four-dimensional growth model that takes observations into account is obtained by using actual three-dimensional data of the plant (acquired image data 152) as observation data and applying Bayes' theorem. More specifically, the four-dimensional growth model includes parameter A, which defines a common probability distribution for each plant variety or lineage, and parameter X, which relates to the shape unique to each individual 1 to N belonging to the variety or lineage, as parameters to be adjusted (estimated). i (where i is any number from 1 to N) is included. The 4D growth model includes information about the 3D shape of each individual as a random variable, so that the observed 4D growth model can complement the 3D data. Parameter A is an example of a “common parameter” in the claims, and parameter X i This is an example of an "individual parameter" in the patent claims.

[0024] Parameter A is equal to each parameter X iRepresents hyperparameters that characterize the probability distribution used as the generator of (where \(i\) is any number from 1 to \(N\)). More specifically, in this embodiment, for each plant variety and strain, a logistic function is prepared to represent each trait (e.g., leaf length, leaf width, petiole length, petiole width, internode length, stem diameter, leaf tilt angle, etc.) of a phytomer. Here, a phytomer means the repeating structure of the unit composed of leaves, stems, and buds that make up a plant. The logistic function is characterized by parameters such as the terminal value, slope, and inflection point time, and these parameters are assumed to be similar for each common plant variety and strain. Therefore, in this embodiment, it is assumed that these parameters such as the terminal value, slope, and inflection point time follow a predetermined probability distribution (e.g., multivariate normal distribution), and parameter \(A\) is set as the hyperparameter that characterizes this probability distribution. For example, when the probability distribution is a multivariate normal distribution, the hyperparameters are the mean and variance. Parameter \(A\) itself is also a random variable that follows a probability distribution such as a multivariate normal distribution or a LKJ distribution.

[0025] Parameter \(X\) i (where \(i\) is any number from 1 to \(N\)) is a parameter that characterizes the logistic function representing each trait of the phytomer, generated according to the probability distribution characterized by parameter \(A\) for each individual \(i\) of the plant. That is, parameter \(X\) i includes, as its components, values such as the terminal value, slope, and inflection point time for each trait of each phytomer. Parameter \(X\) i (where \(i\) is any number from 1 to \(N\)) is vector \(X\) i = T [x i,1 , x i,2 , ···, x i,j ···, x i,K and is expressed as, and each element \(x\) i of vector \(X\) i,j is a row vector representing the \(j\) -th phytomer in the \(i\) -th individual.

[0026] Figure 4 shows parameter \(X\) of the 4D growth model iThis is a diagram to explain the content. As shown in Figure 4, element x represents the j-th Fighter. i,j This includes parameters (i.e., terminal value, slope, and inflection point time) that characterize the logistic function representing each trait of the phytomer, which is generated according to a probability distribution characterized by parameter A. Furthermore, each element x i,j This includes values ​​such as the time at which the j-th phytomer appears and the relative orientation of the j-th and j-1th phytomers (for example, the difference in the center of gravity position and rotation angle when each phytomer is considered as a rigid body), and these values ​​are also generated according to a probability distribution characterized by parameter A. Furthermore, in Figure 4, the leaf length is specified as a parameter relating to the leaf diameter, including the petiole length, but the present invention is not limited to such a configuration, and the leaf length may be defined as the value obtained by subtracting the petiole length from the leaf length in Figure 4.

[0027] Figure 5 shows the parameter X of the 4D growth model. i This figure shows the logistic function used to calculate traits. As mentioned above, the parameter X of the 4D growth model i Each element x i,j This includes parameters that characterize the logistic function representing each trait of the j-th phytomer. Therefore, the generator 120 generates each element x i,j By constructing a logistic function from the parameters included and substituting time t into the constructed logistic function, the values ​​of each trait of each phytomer can be calculated as deterministically determined fixed values.

[0028] The generation unit 120 calculates the values ​​of each trait represented by the 4D growth model at time points t=0,1,2...,T-1,T, and then uses the calculated values ​​of each trait to determine the 3D shape hypothesis Y of the plant for each time point t. t,iThe generation unit 120 calculates the values ​​of each trait (e.g., leaf length, leaf width, petiole length, petiole width, internode length, stem diameter, leaf inclination angle, etc.) of each individual plant i by substituting each time point t into a logistic function that represents each trait of the phytomer. Therefore, the shape of each individual i at time point t can be reproduced from the calculated trait values.

[0029] 3D shape hypothesis Y from the calculated phytomer traits t,i Regarding the method for generating the phytomer, known techniques can be applied. More specifically, a single phytomer consists of three parts: stem, petiole, and leaf blade. For the stem, the generation unit 120 can approximate it using the stem diameter and internode length from the calculated phytomer traits, with a cylinder whose diameter is the stem diameter and height is the internode length. Furthermore, for the petiole, the generation unit 120 can approximate it using the petiole width and petiole length from the calculated phytomer traits, with a cylinder whose diameter is the petiole width and height is the petiole length. Furthermore, for the leaf blade, the generation unit 120 can approximate it using the leaf length and leaf width from the calculated phytomer traits, with an ellipse whose longer of the leaf length and leaf width is the major axis and whose shorter of the two is the minor axis. In this way, the generation unit 120 uses the values ​​of each calculated phytomer trait to represent each part of the phytomer using mathematical models such as cylinders and ellipses, thereby generating the plant's three-dimensional shape hypothesis Y at each time point t. t,i It can generate [this].

[0030] The adjustment unit 130 adjusts the 3D data P of each individual i at each time point t included in the acquired image data 152. t,i And the 3D shape hypothesis Y generated by the generation unit 120 t,i Parameters A and X of the 4D growth model should match. i Adjusts the 3D data P. More specifically, the adjustment unit 130 adjusts the 3D data P. t,i and 3D shape hypothesis Y t,i The degree of agreement with the given parameters A and X is calculated as the likelihood, and the calculated likelihood is used in relation to the parameters A and X. iTo increase the product of the probability of occurrence (the probability of the prior distribution), for example, using a Markov chain Monte Carlo (MCMC) method, parameters A and X i The parameters A and X are sampled. The adjustment unit 130 performs sampling, for example, when the calculated product value exceeds a predetermined value, or when sampling has been performed a predetermined number of times or more. i These parameters are finalized and stored in the memory unit 150 as 4D growth model parameters 154.

[0031] Figure 6 shows the 3D data P t,i and 3D shape hypothesis Y t,i This diagram illustrates a method for calculating the degree of agreement with the hypothesis Y as the likelihood. First, the adjustment unit 130 calculates the 3D shape hypothesis Y of each individual i at each time point t. t,i By taking a virtual camera image from the same position in the field where the plant was photographed with camera 10, the 3D shape hypothesis Y is established. t,i The depth image is acquired. Next, the adjustment unit 130 obtains the 3D data P of each individual i at each time point t. t,i The image is converted into a depth image. Next, the adjustment unit 130 performs the 3D shape hypothesis Y t,i Depth image and 3D data P t,i The degree of agreement with the depth image is calculated using a known method (e.g., the difference in depth values ​​for each pixel of the image), and this is expressed as the likelihood p(P t,i |A,Y t,i ) The likelihood p is given by the sampled parameter A and the 3D shape hypothesis Y. t,i Based on this premise, 3D data P t,i This expresses the likelihood that it will occur.

[0032] In this way, the adjustment unit 130 calculates t × i likelihoods p(P) for each individual i. t,i |A,Y t,i After calculating the (A) value, the overall likelihood is calculated according to the following equation (1). Equation (1) is given by the sampled parameters A and X i Based on this premise, N 3D data points P from time 0 to T. t,i This expresses the likelihood that it will occur.

[0033]

number

[0034] This results in 3D data P t,i The prior distribution before observation (4D growth model) is p(A,X 1:N ) can be expressed as 3D data P t,i The posterior distribution after observation (4D growth model) is p(A,X 1:N |P 0:T,1:N ) can be expressed by the following equation (2).

[0035]

number

[0036] Here, in the right-hand side of equation (2), the denominator p(P 0:T , 1:N ) is a constant, but it is a value that is difficult to calculate. Therefore, in this embodiment, the posterior distribution p(A,X) that we originally wanted to obtain is not used. 1:N |P 0:T,1:N ) and the numerator on the right side is p(P 0:T,1:N |A,X 1:N )p(A,X 1:N Focusing on the fact that ) are proportional, parameters A and X i The adjustment is performed. More specifically, the adjustment unit 130 uses the MCMC method to adjust parameters A and X i Sample the initial values ​​of and evaluate the numerator value on the right-hand side of equation (2), and parameter A and X i The process of resampling (searching) the values ​​of is repeated. The adjustment unit 130 adjusts the parameters A and X when, for example, the value of the numerator on the right side of equation (2) becomes greater than or equal to a predetermined value, or when sampling has been performed more than a predetermined number of times. i This will be adopted as the final parameter.

[0037] The generation unit 120 adjusts parameters A and X by the adjustment unit 130. i Once the adjustments are complete, the hypothesis Y of the plant's three-dimensional shape at the time the parameter adjustments are completed will be determined. t,iThe 3D data P included in the acquired image data 152 t,i The data is output as interpolated data in which at least one of the missing data and outliers has been filled in, and stored in the storage unit 150 as interpolated image data 156. Filling in at least one of the missing data and outliers means that if there are missing parts in the data, those parts are filled in, and if there are outliers, that data is deleted. The generation unit 120 may output the interpolated image data 156 to the terminal device 20 via the network NW.

[0038] Figure 7 shows an example of the interpolated image data 156. As an example, Figure 7 shows the 3D shape hypothesis Y generated by the generation unit 120 based on the acquired image data 152 shown in Figure 3. t,i And the 3D shape hypothesis Y after adjustment by the adjustment unit 130 has been completed. t,i This represents the complementary image data 156 shown in Figure 7, which is not limited to images at a specific point in time, but is data in which at least one of missing or outlier values ​​of the image (3D data) has been filled in over time. In addition to the complementary image data 156, the generation unit 120 may also output the trait values ​​of each individual i at the time when parameter adjustment is completed to the terminal device 20. As a result, users of the terminal device 20 can reduce labor and speed up breeding and trait surveys by referring to the complementary image data 156.

[0039] Figure 8 shows an overview of the parameter tuning process for the 4D growth model. Figure 8 outlines the process described in detail through Figures 4 to 7. First, the parameter X of each individual i is adjusted according to a probability distribution characterized by parameter A, which is common to each plant variety and lineage. i It generates probabilistically. Parameter X i Since this includes the parameters of the logistic function that define each trait of the plant (i.e., terminal value, slope, and time of inflection), the logistic function defined by these parameters is defined from time t0 to t n By substituting these values, we obtain the values ​​for each trait of individual i at each time point.

[0040] Next, from the values ​​of each trait of individual i, we hypothesize the three-dimensional shape Y of that individual i. t0,i ~Y tn,i This generates the 3D data P of individual i included in the acquired image data 152. t0,i ~P tn,i The likelihood is calculated by comparing each of these. Next, the likelihood function that takes into account the overall likelihood of each individual i is calculated (equation (1) above), and the parameters A and X i The product of the probability of occurrence (the probability of the prior distribution) is taken (equation (2) above), and the MCMC method is performed so that the calculated product increases, and parameters A and X i Adjust parameters A and X. i The hypothesis Y of the 3D shape of individual i at the time the adjustment is completed. t,i However, 3D data P t0,i ~P tn,i The data will have at least one of the missing values ​​or outliers filled in.

[0041] [Process Flow] Next, with reference to Figure 9, the flow of processing performed by the information processing device 100 will be described. Figure 9 is a flowchart showing an example of the flow of processing performed by the information processing device 100.

[0042] First, the acquisition unit 110 acquires 3D data of the plant in a time series (step S100). Next, the generation unit 120 generates the shape of the plant in a time series using a 4D growth model of the plant based on the acquired 3D data (step S102). Next, the adjustment unit 130 adjusts the parameters of the 4D growth model so that the acquired 3D data and the generated shape match (step S104). Finally, the generation unit 120 outputs the shape of the plant at the time the parameter adjustment is completed as interpolated data in which at least one of the missing or outlier values ​​in the 3D data is filled in (step S106). This completes the processing of this flowchart.

[0043] As described above, according to this embodiment, 3D data of a plant is acquired, the shape of the plant is generated using a 4D growth model of the plant based on the acquired 3D data, the parameters of the 4D growth model are adjusted so that the 3D data and the generated shape match, and the shape of the plant at the time the parameter adjustment is completed is output as complementary data in which at least one of the missing values ​​and outliers of the 3D data is filled in. This makes it possible to reduce labor and speed up breeding and trait surveys by filling in missing values ​​and outliers of the 3D data according to the growth stage of the plant.

[0044] In this embodiment, the acquisition unit 110 acquires 3D data of plants from the camera 10 via the network NW in a time series, and the generation unit 120 generates the shape of the plants in a time series using a 4D growth model of the plants based on the acquired 3D data. However, the present invention is not limited to such a configuration, and the generation unit 120 may generate the shape of the plants in a time series using time-series 3D data of plants stored in advance in the storage unit 150 or external storage.

[0045] [Other applications] As described above, the present invention can reduce labor and speed up breeding and trait surveys. However, the present invention is not limited to such applications and can be applied to a variety of other uses. For example, when a machine such as a robot approaches a specific part of a plant, missing values ​​or outliers may occur in the 3D data recognized by the mounted camera, resulting in the machine being unable to approach that part normally. In contrast, according to the present invention, by providing the machine such as a robot with image data in which missing values ​​or outliers have been filled in, it is possible to approach the part where the missing values ​​or outliers occurred normally.

[0046] Furthermore, in order to simulate the deformation of plants due to interactions with natural phenomena such as wind or robots, it is necessary to accurately understand the three-dimensional shape of the plant. Therefore, according to the present invention, by providing image data in which missing or outlier values ​​have been imputed, the deformation of plants can be accurately simulated. Also, as described above, in this embodiment, the characteristics of the plant's phytomers (such as leaf width and leaf length) are estimated in the process of imputing missing or outlier values ​​in the three-dimensional data. Therefore, according to the present invention, the leaf area and leaf position and orientation can be calculated from the estimated characteristics of the phytomers and used in the photosynthesis simulation of the plant. Also, according to the present invention, for example, a user of the information processing device 100 can retrospectively check the changes in the three-dimensional shape of the plant over time by referring to the three-dimensional shape hypotheses generated in a time series.

[0047] Although embodiments for carrying out the present invention have been described above using examples, the present invention is not limited in any way to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention. [Explanation of symbols]

[0048] 100 Information Processing Devices 110 Acquisition Department 120 Generation part 130 Adjustment section 150 Storage section 152 Acquired image data 154 4D Growth Model Parameters 156 Complementary image data

Claims

1. A generation unit generates a hypothesis about the shape of the plant in the time series using a 4D growth model that adds a time dimension to the three-dimensional shape of the plant, based on the time-series three-dimensional data of the plant. The system includes an adjustment unit that adjusts the parameters of the 4D growth model so that the three-dimensional data and the generated shape hypothesis match, The generation unit outputs the hypothesis regarding the shape of the plant at the time the parameter adjustment is completed as complementary data in which at least one of the missing values ​​and outliers of the three-dimensional data is filled in. Information processing device.

2. The generation unit probabilistically generates individual parameters for each plant species according to a predetermined probability distribution characterized by common parameters common to each plant species, and generates a morphological hypothesis of the plant in a time series based on the individual parameters. The information processing apparatus according to claim 1.

3. The aforementioned individual parameter is a parameter that defines a logistic function representing the trait of one or more phytomes contained in the plant, The generation unit calculates the traits of the phytomer by inputting each point in the time series into a logistic function defined by the individual parameters, and generates a hypothesis of the plant's shape in the time series from the calculated traits. The information processing apparatus according to claim 2.

4. The phytomer traits include at least one of the following: leaf length, leaf width, petiole length, petiole width, internode length, stem diameter, and leaf inclination angle. The information processing apparatus according to claim 3.

5. The adjustment unit adjusts the common parameters and individual parameters so that the three-dimensional data of the time series matches the hypothesis of the shape of the time series. The information processing apparatus according to claim 2.

6. The adjustment unit calculates the degree of agreement between the depth image of the time-series three-dimensional data and the depth image of the time-series shape hypothesis as the likelihood, and adjusts the common parameter and the individual parameter using the Markov chain Monte Carlo method so that the product of the likelihood and the probability of occurrence of the common parameter and the individual parameter increases. The information processing apparatus according to claim 5.

7. Computers Based on time-series 3D data of the plant, a 4D growth model is used, which adds a time dimension to the 3D shape of the plant, to generate a hypothesis about the shape of the plant in the time series. The parameters of the 4D growth model are adjusted so that the three-dimensional data and the generated shape hypothesis match. The hypothesis regarding the shape of the plant at the time the adjustment of the parameters is completed is output as complementary data in which at least one of the missing or outlier values ​​of the three-dimensional data is filled in. Information processing methods.

8. On the computer, Based on time-series 3D data of plants, a 4D growth model is used, which adds a time dimension to the 3D shape of the plants, to generate a hypothesis about the shape of the plants in the time series. The parameters of the 4D growth model are adjusted so that the three-dimensional data and the generated shape hypothesis match. The hypothesis regarding the shape of the plant at the time the adjustment of the aforementioned parameters is completed is output as complementary data in which at least one of the missing values ​​and outliers of the three-dimensional data is filled in. program.

Images