Generating method, determining method, computer program, generating device and determining device
By generating a discriminative model and using chloroplast density images for machine learning, the problem of difficulty in distinguishing the causes of plant chlorosis in existing technologies has been solved, and efficient and accurate identification of plant growth status has been achieved.
Patent Information
- Application Number
- CN202410346847.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2023-11-24
- Filing Date
- 2024-03-26
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-03-26
AI Technical Summary
Existing technologies are insufficient for efficiently observing and judging plant growth status, especially in distinguishing whether yellowing is caused by disease or other factors, leading to a high misjudgment rate.
By generating a discriminative model, machine learning is performed using chloroplast density images to analyze the differences in absorption spectra of chlorophyll and carotenoids, and a discriminative model is generated to distinguish between healthy and yellowed parts. Supervised or unsupervised learning methods are combined to improve the discrimination accuracy.
It enables efficient and accurate identification of plant growth status, reduces the false judgment rate, and improves the accuracy of identifying the causes of chlorosis.
Smart Images

Figure CN118135572B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a method and device for generating a discrimination model for discriminating the growth state of a plant, a method and device for discriminating the growth state of a plant, and a computer program for executing these methods. Background Art
[0002] Sometimes the growth state of a plant is observed from its fresh leaves. For example, Patent Document 1 describes a device for diagnosing whether the plant is infected with pathogens by irradiating the leaves of the plant with excitation light.
[0003] Patent Literature
[0004] Patent Document 1: Japanese Patent Application Publication No. 2013-36889
[0005] However, there is still room for improvement in terms of easily observing the growth state of plants and observing the growth state of plants with high accuracy. Summary of the invention
[0006] Therefore, the purpose of the present disclosure is to provide a method and a device for generating a discrimination model for discriminating the growth state of plants, a method and a device for discriminating the growth state of plants, and a computer program for executing these methods, which can easily and / or highly accurately observe the growth state of plants.
[0007] The generation method disclosed in the present invention generates a discriminant model for discriminating the growth state of a plant, and the discriminant model is executed by a computer including an operation circuit capable of accessing a storage device. The storage device stores a chloroplast density image, which is an image obtained from images of leaves of a plurality of plants for learning and reflects the difference in absorption spectra of chlorophyll and carotenoids included in the leaves of the plants. The generation method is executed by the operation circuit, the chloroplast density image is read from the storage device, and the chloroplast density image is used as learning data for machine learning, thereby generating a discriminant model for the growth state of a plant.
[0008] Such general and specific aspects may also be implemented by systems, methods, computer programs, and combinations thereof.
[0009] Effects of the Invention
[0010] The method and device for generating a discrimination model for discriminating the growth status of a plant, the method and device for discriminating the growth status of a plant, and the computer program for executing these methods disclosed in the present invention provide easy and / or high-precision observation of the growth status of a plant. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] Figure 1AA model that represents the emission spectrum of a white LED (Light Emitting Diode).
[0012] Figure 1B Model representing the absorption spectrum of chlorophyll.
[0013] Figure 1C Model representing the absorption spectrum of carotenoids.
[0014] Figure 1D Model representing the action spectrum of chlorophyll.
[0015] Figure 2A Model representing the healthy portion of the spectrum.
[0016] Figure 2B A model representing the yellowed part of the spectrum.
[0017] Figure 2C A model representing the spectrum of a leaf vein.
[0018] Figure 3A This is a model showing the spectral sensitivity curve of R (red) of a digital camera.
[0019] Figure 3B This is a model showing the spectral sensitivity curve of G (green) of a digital camera.
[0020] Figure 3C This is a model showing the spectral sensitivity curve of B (blue) of a digital camera.
[0021] Figure 4 Represents the difference between the model spectrum of the yellowed part and the model spectrum of the healthy part.
[0022] Figure 5A This is an image obtained by photographing the surface of a leaf.
[0023] Figure 5B Yes Figure 2A The image is a binary image after the image is converted to monochrome.
[0024] Figure 5C Is to use from Figure 2A The R image and B image obtained from the image are used to obtain a binary image of the chloroplast density image.
[0025] Fig. 6A This is a schematic diagram for explaining the generation of a discriminant model.
[0026] Figure 6B This is a schematic diagram for explaining discrimination using a discrimination model.
[0027] Figure 7This is a block diagram showing an example of a generating device according to the first embodiment.
[0028] Figure 8 This is a block diagram showing an example of a determination device according to the first embodiment.
[0029] Fig. 9 This is a flowchart showing an example of a process of generating a discriminant model executed by the generating device according to the first embodiment.
[0030] Fig.10 This is a flowchart showing an example of the determination process executed by the determination device according to the first embodiment.
[0031] Fig.11 This is a block diagram showing an example of a generating device according to the second embodiment.
[0032] Fig. 12A This is a schematic diagram for explaining clustering data.
[0033] Fig. 12B This is a schematic diagram that explains the clustering results.
[0034] Fig. 12C This is a schematic diagram illustrating labels for receiving clustering results.
[0035] Fig.13 This is a flowchart showing an example of a process of generating clustering and discriminant models executed by the generating device according to the second embodiment.
[0036] Fig.14 This is a schematic diagram showing an example of data used in Experimental Example 1.
[0037] Fig.15 It means according to Fig.14 This is a schematic diagram showing an example of the learning data and evaluation data selected from the first to ninth times for the group.
[0038] Fig.16 3 is a graph showing the results of Experimental Example 3.
[0039] Description of reference numerals:
[0040] 1: Generating device;
[0041] 2: Identification device;
[0042] 11, 21: Operation circuit;
[0043] 12, 22: input device;
[0044] 13, 23: output device;
[0045] 14, 24: Communication circuit;
[0046] 15, 25: storage device;
[0047] D11: Shooting data;
[0048] D12: light intensity data;
[0049] D13: Data for learning;
[0050] D21: Shooting data;
[0051] D22: light intensity data;
[0052] D23: data for discrimination;
[0053] M: discriminant model;
[0054] P1: Generate program;
[0055] P2: Discrimination procedure. DETAILED DESCRIPTION
[0056] Hereinafter, the embodiments of the present disclosure will be described using the accompanying drawings, with appropriate reference to the accompanying drawings. However, in the detailed description, unnecessary parts may be omitted from the description of the prior art and substantially the same structure. This is to simplify the description. In addition, the following description and drawings are disclosed to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter of the claims.
[0057] The present disclosure relates to a method for generating a discrimination model for discriminating the growth state of a plant and a discrimination method. Hereinafter, as the growth state of a plant, the generation and use of a discrimination model for discriminating the healthy growth state of a plant is described as an example. More specifically, an example in which a disease of a plant manifests itself as yellowing of leaves is described.
[0058] 〈Growth Status of Plants〉
[0059] The growth state of a plant may be reflected in its leaves. For example, the appearance of the leaves may indicate whether the plant is healthy or infected with a disease. On the other hand, sometimes, even if the plant is healthy, it may appear to be infected with a disease at first glance.
[0060] For example, when a plant is infected with a disease, the leaves of the plant may turn yellow. Therefore, plant growers may sometimes determine that the plant is infected with a disease when the leaves of the plant turn yellow. On the other hand, even when the plant is not infected with a disease, the leaves of the plant may turn yellow. For example, when the plant is under conditions such as insufficient nutrition and insufficient sunlight, the leaves may turn yellow even when the plant is not infected with a disease. In this case, even experts may find it difficult to determine whether the plant is infected with a disease or not.
[0061] The countermeasures to be taken for a plant are different when the plant is not infected with a disease and when the plant is infected with a disease. Therefore, it is desirable to improve the accuracy of determining whether a plant is infected with a disease.
[0062] As the amount of chlorophyll changes, the density of carotenoids becomes relatively high, causing the leaves of plants to turn yellow. Yellowing caused by nutrient deficiency and disease may have differences in shape, distribution, and intensity. In some cases, only skilled agricultural workers can recognize the difference in the yellowed parts.
[0063] Plants carry out photosynthesis, so the mesophyll of leaves contains photosynthetic pigments such as chlorophyll and carotenoids. Chlorophyll in chloroplasts is one of the main photosynthetic pigments, and carotenoids are one of the auxiliary photosynthetic pigments.
[0064] When a plant is not infected with a virus or the like and grows well, that is, when the plant is healthy, the mesophyll of the leaves contains sufficient chlorophyll and carotenoids. Hereinafter, such a portion containing sufficient chlorophyll and carotenoids is also referred to as a "healthy portion".
[0065] When a plant is suffering from a disease or lack of nutrition, the development of the chloroplasts in the mesophyll deteriorates, the chlorophyll decreases, and the leaves turn yellow. Hereinafter, such a part is referred to as a "yellowing part". That is, there are "cases where the yellowing part is caused by the disease" and "cases where the yellowing part is caused by reasons other than the disease".
[0066] It should be noted that the veins of leaves usually contain little or no chlorophyll and carotenoids.
[0067] Therefore, the amounts of chlorophyll and carotenoids in each part are shown in Table 1 below.
[0068] (Table 1)
[0069] Chlorophyll content Carotenoid content Health section many many Yellowing part few many vein Less / Not Containing Less / Not Containing
[0070] Here, as described above, there are two types of yellowing: "cause of disease" and "cause of other reasons". The distribution of chlorophyll on the surface of the leaf including the yellowing portion differs depending on the cause. In other words, the shape, density, and distribution of the yellowing portion due to a decrease in the amount of chlorophyll differ. If the distribution of chlorophyll that cannot be easily distinguished from the appearance can be distinguished, it can be distinguished whether the cause of yellowing is "disease" or "other than disease". For example, if the shape of the yellowing portion, the distribution in the leaf, and the density can be easily recognized, the accuracy of the distinction can be improved.
[0071] 〈Light Transmission in Chloroplasts〉
[0072] Next, the light transmission of chlorophyll and carotenoids will be described. As an example, a case where a white LED is used as a light source is considered. Figure 1A Model representing the emission spectrum of a white LED. Figure 1B Model representing the absorption spectrum of chlorophyll. Figure 1C Model representing the absorption spectrum of carotenoids.
[0073] However, Figure 1B and Figure 1C The absorbance shown in is ideally measured in a solution. In reality, light entering a leaf is internally scattered in the leaf, so even green light may be absorbed by chloroplasts or other substances in the leaf, contributing to photosynthesis. Figure 1D As shown, its effect can be observed in the form of the action spectrum of photosynthesis.
[0074] As described above, the absorption effect of green light is greatly affected by the internal scattering in the leaves. Therefore, it is known that thin plants such as Ulvaceae have photosynthesis spectra close to the ideal chlorophyll absorption spectrum. Regardless of the substance that absorbs green light, the absorption effect of green light is greatly affected by the increased probability of encounter between incident light and absorbing substances due to the internal scattering effect caused by the presence of chloroplasts. Therefore, if the chloroplasts disappear due to yellowing of the leaves, the absorption effect of green light will also weaken. In this sense, for convenience, the term "green" is used here. Figure 1D As for carotenoids, Figure 1 was used as it is.
[0075] According to Table 1, the absorption spectrum was applied to the light from the light source under various conditions. Figure 1B The absorption spectrum of chlorophyll shown and Figure 1C The absorption spectra of carotenoids shown are for healthy parts. Figure 1CThe absorption spectrum of carotenoids shown acts on the yellowed part, so that neither the absorption spectrum of chlorophyll nor the absorption spectrum of carotenoids acts on the vein part. Figure 2A represents the result of the healthy part (model of the spectrum of the healthy part), Figure 2B The result of the yellowed part (the model of the spectrum of the yellowed part) is shown. Figure 2C The results for the leaf vein portion (model of the spectrum of the leaf vein portion) are shown.
[0076] Next, the spectral sensitivity curve of the digital camera is applied to these spectral groups. That is, although the three colors of RGB are distinguished, there is no wavelength information when the spectral separation is performed, and the accumulated pixel value is retained. It should be noted that the smallest element that is arranged in a grid shape and constitutes an image is called a pixel. Each pixel represents the intensity and color of light by a numerical value. In addition, the value representing the intensity and color of light of each pixel is called a pixel value. Figure 3A Model representing the spectral sensitivity curve of R. Figure 3B Model representing the spectral sensitivity curve of G. Figure 3C Model representing the spectral sensitivity curve of B.
[0077] Table 2.1 shows an example of pixel values indicating the intensity of each light in a healthy part, a yellowed part, and a leaf vein part when each light of R, G, and B is applied. As can be seen from Table 2.1, in any case of R, G, and B, the highest value is in the leaf vein part, and the yellowed part cannot be highlighted. In addition, when the pixel values are compared, it can be seen that the leaf vein part has a high value in all bands of the R image, the G image, and the B image, while in the healthy part and the yellowed part, the B image has a lower value than the R image and the G image.
[0078] (Table 2.1)
[0079] Health section Yellowing part Leaf vein part R 1.67 2.61 2.71 G 1.81 3.02 3.64 B 0.43 0.81 2.24
[0080] Figure 4 represents the difference between the model spectrum of the yellowed part and the model spectrum of the healthy part. Figure 2B The model spectrum is subtracted Figure 2A The values obtained from the model spectrum are plotted. Figure 4In the above matrix, it can be seen that the difference is larger in the medium wavelength (e.g., about 500nm to 600nm) and long wavelength (e.g., about 600nm to 780nm) than in the short wavelength (e.g., wavelength about 400nm to 500nm). That is, there is a large difference in the wavelength portion that spans the distribution of G (green light: for example, medium wavelength) and R (red light: for example, long wavelength), and the difference between yellowing and health is relatively small in the band where B (blue light: for example, short wavelength) is counted. In this matrix, even if the difference between the values of the yellowing part and the health part is calculated for each of the pixel values of R, G, and B, it is obvious.
[0081] Based on the above content, we can draw the following two points:
[0082] The difference between the yellowed part and the healthy part appears in the pixel values across the R and G bands.
[0083] The leaf vein part has high pixel values in all RGB bands.
[0084] Therefore, in order to maximize the "difference between the yellowed part and the healthy part", that is, the "density of chloroplasts", it can be said that formula (1.1) is the best calculation formula.
[0085] (Formula for matching the R image and the G image with arbitrary weights) - (Formula for using the B image) (1.1)
[0086] Here, (the formula for matching the R image and the G image with arbitrary weights) includes the arithmetic mean, the geometric mean, and other formulas that can achieve weighting of the two elements. In the (formula using the B image), for example, coefficients and correction items can also be used to improve the effect of vein removal. With respect to the (formula for matching the R image and the G image with arbitrary weights) and the (formula using the B image), in order to ultimately output the chloroplast density image as a monochrome image, standardization can also be performed to make the maximum and minimum ranges of the two the same.
[0087] In the first term of the above formula (1.1), the difference between the yellowed part and the healthy part becomes clear. In the second term of the formula (1.1), the information of the leaf vein part is cut. Here, it is necessary to make the maximum and minimum ranges of the first and second terms of the formula (1.1) the same. Generally speaking, the pixel values of RGB each take values from 0 to 255. Therefore, the first term of the formula (1.1) is the formula (1.2).
[0088] (R image × x + G image × y) / (x + -y) (1.2)
[0089] (where 0≤x, 0≤y)
[0090] Here, the values of x and y cannot be uniquely determined because the action spectrum of photosynthesis differs among plant leaves. Depending on the plant species, which of R and G is multiplied by the specific gravity to highlight the difference between the yellowed part and the healthy part differs.
[0091] Table 2.2 shows an example of the accumulated pixel values of the healthy part, the yellowed part, and the leaf vein part when the x and y values of the following formula (1.3) are set to (0, 1), (1, 0), and (1, 1), respectively. It can be seen that in any case, the yellowed part has the largest pixel value and is the most prominent in the monochrome image.
[0092] (R image × x + G image × y) / (x + y) - B image (1.3)
[0093] (where 0≤x, 0≤y)
[0094] The image obtained by equation (1.3) is an example of the “chloroplast density image” in the present disclosure.
[0095] (Table 2.2)
[0096] x,y Formula (1.3) Health section Yellowing part Leaf vein part 0,1 G picture-B picture 1.38 2.21 1.40 1,0 R picture - B picture 1.24 1.80 0.47 1,1 (R image + G image) / 2-B image 1.31 2.00 0.94
[0097] Chloroplast density image
[0098] Here, use FIG. 5A to FIG. 5C , the chloroplast density image used in the discrimination model disclosed in the present invention is explained. Figure 5A This is an image obtained by photographing the surface of a leaf. Figure 5A Represented as a monochrome image, but actually a color image.
[0099] Figure 5B Yes Figure 5A The image after the monochromatic image is binarized with a specified threshold. The threshold is determined while adjusting so that the yellowed part can be extracted. Figure 5B In the image, the black part is mainly the yellow part. Figure 5B In the image, the non-yellowing leaf veins and the like are also detected as black.
[0100] Figure 5C An example of an image obtained by adjusting the threshold and binarizing the chloroplast density image so that the yellowed part can be extracted is shown. The chloroplast density image is obtained by using Figure 5A Specifically, Figure 5C The image is the image obtained by binarizing the chloroplast density image when the values of x and y are set to (1, 0) respectively in the following formula (1.3).
[0101] (R image × x + G image × y) / (x + y) - B image (1.3)
[0102] exist Figure 5C In the image, the black part is also mainly the yellowed part. Figure 5B The binary image and Figure 5C Compare the chloroplast density image with the binary image, then Figure 5C The image obtained by binarizing the chloroplast density image can reduce the situation where the leaf vein part is extracted as yellow. This shows the leaf vein reduction effect (the effect of cutting the information of the leaf vein part) as described above.
[0103] 〈Discrimination Model〉
[0104] use Fig. 6A and Figure 6B , briefly explain the discriminant model M for discriminating the growth status of plants. Fig. 6A As shown, the learning data D13 used in generating the discriminant model includes a plurality of sets of image data and labels assigned to the images. Each image data is a "chloroplast density image of a plant leaf". In addition, the label is the "growth state of the plant" corresponding to each image data. Specifically, the growth state of the plant is at least one of a healthy state and a disease state that may affect the color of the leaves, a malnutrition / overnutrition state, a poor growth environment state (sunlight, temperature, humidity, improper watering), and an insect pest state.
[0105] In the learner, the relationship between each image data included in the learning data D13 and the growth state of the plant is learned through machine learning, thereby generating a discriminant model M. When a new image representing the density of the plant is input, the discriminant model M outputs the growth state of the plant as a discrimination result. Assume that when the growth state of the plant as a label is "good" and "bad", the discriminant model M, for example, outputs the possibility that the plant is "good" or "bad" for the input image representing the density of the plant. It should be noted that the labels representing the growth state of the plant are not limited to "good" and "bad". For example, three or more labels can also be used to represent the growth state of the plant. By using the discriminant model M, it is easy to discriminate plants that are difficult to perform by visual inspection alone.
[0106] It should be noted that each image data (chloroplast density image) is, for example, data generated based on image data obtained by photographing the surface of a plant leaf. It should be noted that, preferably, the image data included in a learning data set is photographed and generated under the same conditions. For example, the image data (chloroplast density image) included in a learning data set can be generated by image data photographed under the same or equivalent light irradiation conditions. More specifically, it can be image data obtained by photographing using a transmission optical system with a built-in surface light source whose brightness is roughly uniform relative to the surface. In addition, the image data (chloroplast density image) included in a learning data set is generated by the same method (calculation formula) as the photographed data. In this way, by photographing the same or equivalent learning data and discrimination data under the same conditions, accurate discrimination can be performed.
[0107] Assuming that images are taken under multiple shooting conditions with different light irradiation conditions, the light source spectrum and spectral sensitivity curve may be different. According to the above discussion of the model spectrum, when the light irradiation conditions are different, it is impossible to uniquely determine the optimal x and y values, which is inefficient. Therefore, by setting the learning data and the discrimination data to the same or equivalent shooting conditions, more accurate discrimination can be performed.
[0108] In addition, the above-mentioned model spectrum discussion is based on the premise that the healthy (mesophyll) part, the yellowed part, and the vein part of the leaf can be clearly distinguished. The present invention is a technology that further improves the discrimination accuracy of the discrimination model M based on the use of high-quality plant leaf imaging data. For example, by adopting a high-precision imaging environment such as a transmission optical system, high-quality plant leaf images can be captured.
[0109] 〈Construction of the Generating Device〉
[0110] use Figure 7 , the structure of the generating device 1 for generating the discriminant model M is described. Figure 7 As shown, the generating device 1 includes a calculation circuit 11, an input device 12, an output device 13, a communication circuit 14, a storage device 15, etc. The generating device 1 is, for example, a personal computer or an information processing device.
[0111] The operation circuit 11 is a controller responsible for controlling the entire generation device 1. For example, the operation circuit 11 realizes various processes related to the generation of the discriminant model M by reading and executing the generation program P1 stored in the storage device 15. The operation circuit 11 can be various processors such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphic Processing Unit), an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), or a specially designed hardware circuit.
[0112] The input device 12 is used for user operation and data input. The input device 12 may be, for example, an operation button, a keyboard, a mouse, a touch panel, a microphone, etc. The output device 13 is used for outputting processing results and data. The output device 13 may be, for example, an output unit such as a display and a speaker.
[0113] The communication circuit 14 performs data communication with other devices. Data communication can be performed by wire and / or wireless, for example, according to a known communication standard. For example, wired data communication can be performed by using a communication controller of a semiconductor integrated circuit that operates according to the Ethernet (registered trademark) standard and / or the USB (registered trademark) standard as the communication circuit 14. In addition, wireless data communication can be performed by using a communication controller of a semiconductor integrated circuit that operates according to the IEEE802.11 standard related to wireless LAN (Local Area Network) and / or the fourth / fifth / sixth generation mobile communication system called 4G / 5G / 6G related to mobile communication as the communication circuit 14.
[0114] The storage device 15 is a recording medium for recording various information. The storage device 15 is implemented by, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an SSD (Solid State Drive), a hard disk drive, other storage devices, or a suitable combination thereof. The storage device 15 stores the generation program P1, the shooting data D11, the light intensity data D12, the learning data D13, the discrimination model M, etc. The generation program P1 is a computer program executed by the operation circuit 11. Various processes for generating the discrimination model M are executed by executing the generation program P1.
[0115] The image data D11 is image data obtained by photographing leaves of a plant.
[0116] The light intensity data D12 is data of an R image, a G image, and a B image generated based on the imaging data D11.
[0117] Use as Fig. 6A As described above, the learning data D13 is data including a plurality of sets of chloroplast density images and labels of corresponding plant growth states. The chloroplast density images are generated based on the light intensity data D12.
[0118] If used Fig. 6A and Figure 6B As briefly described, the discriminant model M is generated in the generating device 1 using the learning data D13.
[0119] Figure 7 The example in which the generating device is implemented by one information processing device is shown. However, the generating device 1 may also be implemented by multiple information processing devices. For example, at least a part of the multiple processes executed may be executed by a computing circuit. In addition, for example, at least a part of the multiple data may be stored in different storage devices.
[0120] The generating device 1 can generate a discrimination model M that distinguishes and discriminates the above-mentioned "chlorosis due to disease" and "chlorosis due to causes other than disease" associated with Table 1 by learning using a chloroplast density image labeled with labels indicating each state. That is, it is difficult to distinguish "chlorosis due to disease" and "chlorosis due to causes other than disease" by visual inspection of a color image alone. However, by using a chloroplast density image indicating the density of chloroplasts obtained based on the amount of chlorophyll as learning data, it is possible to distinguish "the degree of chlorosis due to disease" and "the degree of chlorosis due to causes other than disease" which are difficult to distinguish visually.
[0121] <Composition of the discrimination device>
[0122] use Figure 8 , the structure of the discriminating device 2 for discriminating the growth state of a plant using the discriminating model M is described. Figure 8 As shown, the determination device 2 includes a calculation circuit 21, an input device 22, an output device 23, a communication circuit 24, a storage device 25, etc. For example, the determination device 2 is a personal computer or an information processing device.
[0123] The operation circuit 21, the input device 22, the output device 23, the communication circuit 24 and the storage device 25 are respectively connected to the Figure 8 The above-mentioned operation circuit 11, input device 12, output device 13, communication circuit 14 and storage device 15 are implemented by the same specific means.
[0124] The storage device 25 stores the determination program P2 , the determination model M, the imaging data D21 , the light intensity data D22 , the determination data D23 and the like.
[0125] The determination program P2 is a computer program executed by the calculation circuit 21. Various processes for determination are performed by executing the determination program P2.
[0126] The discriminant model M is a learned model generated by the above-mentioned generation device 1.
[0127] The image data D21 is image data of leaves of plants in a discriminated growth state.
[0128] The light intensity data D22 is data of an R image, a G image, and a B image generated based on the imaging data D21.
[0129] The discrimination data D23 is a chloroplast density image obtained from the imaging data D21.
[0130] Figure 8 The example in which the determination device 2 is implemented by one information processing device is shown, but the determination device 2 may be implemented by a plurality of information processing devices.
[0131] It should be noted that Figure 8 The identification device 2 shown can also be used by using Figure 7 That is, the generation of the discrimination model M and the discrimination of the growth state of the plant using the discrimination model M can also be performed by the same information processing device.
[0132] <Processing of generating a discriminant model>
[0133] Use as Fig. 9 The flowchart shown in FIG. 1 illustrates the process of generating the discriminant model M executed by the generating device 1 .
[0134] First, the operation circuit 11 obtains the photographic data D11 of leaves of multiple plants (S01). The operation circuit 11 stores the obtained multiple photographic data D11 in the storage device 15. Here, the label representing the growth status of the plant can be pre-associated with each photographic data D11. For example, labels such as "healthy", "early stage of disease", "mid-stage of disease", and "terminal stage of disease" are associated with each photographic data D11. In addition, each photographic data D11 is associated with a label representing the growth status of the plant and stored in the storage device 15. It should be noted that when the label representing the growth status of the plant is not associated with the photographic data D11 obtained in step S01, the operation circuit 11 performs processing on the marking label of each photographic data D11.
[0135] The operation circuit 11 generates light intensity data D12 using the captured data D11 acquired in step S01 (S02). Specifically, the operation circuit 11 decomposes each captured data D11 into an R component, a G component, and a B component, thereby generating an R image representing the intensity of red light, a G image representing the intensity of green light, and a B image representing the intensity of blue light, thereby generating a plurality of light intensity data D12. In addition, the operation circuit 11 stores the generated light intensity data D12 in the storage device 15.
[0136] The calculation circuit 11 uses the light intensity data D12 generated in step S02 to generate a chloroplast density image (S03). The calculation circuit 11 applies the following formula (1.3) to each corresponding pixel of the image, thereby generating a chloroplast density image. The processing from step S02 to step S03 is the so-called preprocessing. Here, in the case where other preprocessing is required before or after the processing of step S02 or step S03, other preprocessing can also be performed. For example, as an example of preprocessing, processing such as image cropping and image rotation can be performed.
[0137] (R image × x + G image × y) / (x + y) - B image (1.3)
[0138] (where 0≤x, 0≤y)
[0139] In one discrimination model, the same calculation formula is used to obtain the chloroplast density image. Here, in the formula (1.3) or the above-mentioned formula (1.2), the combinations of x and y that become the same formula after reduction are defined as all the same. In other words, when the ratio of x and y is the same, it is defined as the same. For example, the combinations of x:y of 1:2, 2:4, and 3:6 are defined as the same.
[0140] The calculation circuit 11 associates the corresponding label with each chloroplast density image generated in step S03 to generate learning data D13 ( S04 ). The calculation circuit 11 stores the generated learning data D13 in the storage device 15 .
[0141] The calculation circuit 11 uses the learning data D13 generated in step S04 to generate a discrimination model M (S05). Specifically, the calculation circuit 11 learns the relationship between the image representing the density of chloroplasts included in the learning data D13 and the growth state of the plant to generate the discrimination model M. In addition, the calculation circuit 11 stores the generated discrimination model M in the storage device 15.
[0142] In this way, the generating device 1 can generate the discrimination model M for discriminating the growth state of the leaves of the plant based on the image of the leaves of the plant.
[0143] <Discrimination processing using a discriminant model>
[0144] use Fig.10 The flowchart shown in FIG. 1 illustrates a process of discriminating the growth state of a plant using the discrimination model M executed by the discrimination device 2 .
[0145] First, the calculation circuit 21 acquires image data of leaves of a plant whose growth state is to be determined ( S11 ) and stores the acquired image data D21 in the storage device 25 .
[0146] The calculation circuit 21 generates light intensity data D22 using the imaging data D21 acquired in step S11 ( S12 ). The light intensity data D22 can be generated by the same process as step S02 described above using FIG. 6 . The calculation circuit 21 stores the generated light intensity data D22 in the storage device 25 .
[0147] The calculation circuit 21 generates a chloroplast density image using the light intensity data D22 generated in step S12 (S13). The chloroplast density image can also be generated by using Fig. 9 The same processing as the above-mentioned step S03 is performed. In addition, the calculation circuit 21 stores the generated chloroplast density image in the storage device 25 as the determination data D23.
[0148] The calculation circuit 21 uses the determination data D23 generated in step S13 as an input of the determination model M to determine the growth state of the target plant (S14).
[0149] The calculation circuit 21 outputs the determination result in step S14 to the output device 23 ( S15 ).
[0150] In this way, the discrimination device 2 can use the discrimination model M to discriminate the growth state of the leaves of the plant based on the image of the leaves of the plant.
[0151] (Determination of x and y used in equation (1.3))
[0152] By feeding back and optimizing the discrimination accuracy obtained at that time, the optimal values of x and y used in the formula (1.3) can be determined. For example, the determination of the values of x and y used in the formula (1.3) can also be performed in the generating device 1. At this time, the operation circuit 11 can determine the values of x and y using the method used in general optimization problems. The data used to determine the values of x and y is data associated with the "shooting data" labeled by the expert, such as "healthy", "early stage of disease", "mid-stage of disease", and "late stage of disease".
[0153] <Second Embodiment>
[0154] use Figure 11 to Figure 13 , a generating device 1A of the second embodiment is described. In the above-mentioned first embodiment, an example in which the generating device 1 uses supervised learning to generate the discriminant model M is described. Specifically, the generating device 1 uses learning data D13 including images pre-labeled by preprocessing to generate the discriminant model M. In contrast, the generating device 1A of the second embodiment uses unsupervised learning to cluster images that are not assigned labels. In addition, when the generating device 1A receives the labels of each cluster cluster generated by clustering, it generates learning data to generate the discriminant model M. In embodiment 2, the same structure and the same processing as the generating device 1 of embodiment 1 are sometimes marked with the same figure marks and the description is omitted.
[0155] 〈Construction of the Generating Device〉
[0156] like Fig.11 As shown, in the generating device 1A of the second embodiment, the operation circuit 11 is implemented by an information processing device including an input device 12, an output device 13, a communication circuit 14, and a storage device 15. Fig.14 As shown, the storage device 15 of the generation device 1A of the second embodiment stores the clustering program P3, the generation program P1, the imaging data D11, the light intensity data D12, the clustering data D31, the clustering result data D32, the learning data D13, and the discrimination model M.
[0157] The clustering program P3 is a computer program executed by the operation circuit 11. By executing the clustering program P3, the generation device 1A performs a clustering process. The clustering program P3 generates clusters using hard clustering such as hierarchical clustering and non-hierarchical clustering, and soft clustering.
[0158] The clustering data D31 is as follows Fig. 12A As shown in FIG. 1 , the data includes simple images without labels. The clustering data may be, for example, chloroplast density images generated from the plurality of imaging data D11 .
[0159] The clustering result data D32 is the result of clustering the clustering data D31. Fig. 12B As shown, through unsupervised learning, multiple clusters are obtained from the clustering data D31, which become clustering result data D32. For example, Fig. 12B An example of clusters A to E obtained by hard clustering in which one data belongs to only one cluster is shown. It should be noted that when soft clustering is used, which allows one data to belong to multiple clusters, multiple labels such as "disease and malnutrition" can also be assigned. In addition, the relative allocation of multiple labels can also be evaluated. In general, natural phenomena including yellowing of leaves are manifested due to complex factors.
[0160] The learning data D13 is data generated by the user by labeling each cluster A to E of the clustering result data D32. The learning data D13 is used to generate the discriminant model M. Fig. 12C Yes Fig. 12B The clusters A to C obtained in the example are labeled "disease", "health", and "malnutrition" respectively. It should be noted that there are cases where it is impossible to determine the labels of all clusters. In such cases, Fig. 12C As shown, the label "unclear" can also be marked. In addition, for example, like citrus Huanglongbing, there may be a situation where there are multiple modes of disease manifestation and multiple genetic systems of pathogens. In this case, for example, the clusters of the "disease" label can also be subdivided and hierarchical into "disease pattern A" and "disease pattern B". It should be noted that the subdivision and hierarchy of labels are not limited to the "disease" label.
[0161] 〈Processing of clustering and discriminant model generation〉
[0162] use Fig.13 The flowchart shown in FIG. 1 illustrates the clustering and generation of the discriminant model M performed by the generation device 1A. Specifically, Fig.13 The process within the dotted line ( S21 to S23 ) of the flowchart shown is clustering processing.
[0163] The calculation circuit 11 first acquires the image data D11 of leaves of a plurality of plants (S01). The calculation circuit 11 stores the acquired image data D11 in the storage device 15. It should be noted that in the generation device 1A, labels are added after clustering, so labels are not added in step S01.
[0164] The calculation circuit 11 generates light intensity data D12 using the imaging data D11 acquired in step S01 ( S02 ). The method of generating the light intensity data D12 is the same as that of the generating device 1 of the first embodiment. The calculation circuit 11 stores the generated light intensity data D12 in the storage device 15 .
[0165] The calculation circuit 11 generates a chloroplast density image using the light intensity data D12 generated in step S02 (S03). The method of generating the chloroplast density image is the same as the generation method of the generation device 1 of the first embodiment. In addition, the calculation circuit 11 stores the generated multiple chloroplast density images as clustering data D31 in the storage device 15.
[0166] The calculation circuit 11 performs clustering by unsupervised learning using the clustering data D31 stored in step S03 ( S21 ). The calculation circuit 11 stores the clustering result in the storage device 15 as clustering result data D32 .
[0167] The calculation circuit 11 receives the label of each cluster obtained in step S21 ( S22 ). For example, the label of each cluster is input by the user via the input device 12 .
[0168] The calculation circuit 11 associates the corresponding label received in step S22 with each chloroplast density image in the clustering data D31 to generate the learning data D13 ( S23 ). The calculation circuit 11 stores the generated learning data D13 in the storage device 15 .
[0169] The calculation circuit 11 generates a discriminant model M using the learning data D13 generated in step S04 ( S05 ). The method of generating the discriminant model M is the same as that of the generating device 1 of the first embodiment. The calculation circuit 11 stores the generated discriminant model M in the storage device 15 .
[0170] In this way, the generation device 1A performs clustering based on the image of the plant leaves. In addition, the generation device 1A can generate a discrimination model M for discriminating the growth state of the plant leaves using the clustering result.
[0171] It should be noted that in the above description, the example in which the generation device 1A performs the generation of the chloroplast density image, clustering, and the generation of the discriminant model M is described. However, a part of the processing can also be performed by other information processing devices. In addition, each process can also be performed by different information processing devices. For example, the chloroplast density image generated by other information processing devices can also be used to perform clustering. In addition, clustering and the generation of the discriminant model M can also be performed by different information processing devices.
[0172] As described in the second embodiment, the discrimination by the generated discrimination model M is performed in the same manner as described above in the first embodiment. Fig. 12C In the example, the labels "disease", "health", "malnutrition", "unclear 1", and "unclear 2" are respectively marked for each cluster. In the discriminant model M generated by the learning data D13 generated in this way, the discrimination result can be "unclear" in addition to "disease", "health" or "malnutrition". In addition, when the label to be marked for the "unclear" cluster is known later, the correct label can be marked. In this way, the discriminant model M can make more accurate discriminations.
[0173] <Variant 1>
[0174] In the first embodiment described above, the example of distinguishing whether a person is healthy or has a disease with chlorosis is used for explanation. In addition, in the case of a disease, a label indicating whether the disease is in the early stage, the middle stage, or the late stage is used for learning, and the learned discrimination model is used for discrimination. However, of course, the discrimination model may also use learning data marked with labels other than these. For example, in addition to or instead of the above labels, the discrimination model may be generated using learning data marked with labels such as "lack of nutrition" or "overwatering".
[0175] <Variant 2>
[0176] In the above example, the orientation of the leaves in the captured data is not mentioned. On the other hand, there are plants such as bamboo leaves where the orientation of the veins is easy to distinguish. In this case, by using learning data and discrimination data to adjust the orientation of the leaves to a predetermined orientation, the discrimination accuracy can be improved.
[0177] <Variant 3>
[0178] In variant example 2, the orientation of the leaves is adjusted for plants whose vein orientation is easy to distinguish. However, in the case of leaves whose vein orientation is uneven, one shot data may be rotated to increase the amount of learning data. In this way, by learning image data after one shot data is rotated, it is possible to make a distinction without relying on the shooting direction of the leaves, and it is possible to easily shoot the shooting data for distinction. In addition, it is assumed that even if there is a gradient in the image, learning can be performed without relying on the directionality of the gradient. Therefore, over-learning caused by the gradient of the learning data can be prevented, and the accuracy of distinction can be improved.
[0179] <Variant 4>
[0180] In the above example, an example of shooting the shooting data D11 using a "transmissive optical system with roughly uniform brightness" is described. However, the shooting optical system used for shooting the shooting data D11 is not limited to the transmissive optical system. Specifically, the shooting data D11 may be data that can separate the yellowed part, the healthy part, and the vein part. In addition, the shooting optical system used for shooting the shooting data D11 may be a shooting optical system that can shoot under uniform conditions in each sample. For example, in addition to the transmissive optical system that can be carried outdoors, it may also be a shooting optical system that has a uniform and complete shooting environment indoors.
[0181] <Other Modifications>
[0182] In the above example, an example of discerning the growth status of a plant through machine learning is used to illustrate, but it is also possible to implement generative AI (Artificial Intelligence) based on the same learning data, for example, using a generative adversarial network. Specifically, the chloroplast density image is learned to generate a non-actual chloroplast density image under arbitrary conditions, for example, by coloring from yellow to green from parts with high pixel values to parts with low pixel values, to simulate and restore the RGB image of the leaf. Such generative AI can generate desired conditional images such as virtual images of "healthy" leaves and virtual images of "early stage of disease + mid-stage of malnutrition" leaves. By using such generative AI, for example, in the case of citrus Huanglongbing, it is possible to predict what kind of yellowing state citrus in areas where the disease has not yet spread will become when it is infected with the disease, which can help with early detection.
[0183] <Experimental Example 1>
[0184] The following is an explanation of the experimental results of the generation of the above-mentioned discrimination model M. In the experiment, 841 shooting data including leaves of healthy plants and leaves of diseased plants were prepared. Each shooting data is an image shot using a transmission optical system with a built-in surface light source with a brightness roughly uniform relative to the surface. The plants are leaves of citrus fruits such as flat lemon, and the disease is citrus Huanglongbing. Fig.14 As shown in the figure, each photographic data is labeled with one of the labels "healthy", "early stage of disease", "mid-stage of disease", and "late stage of disease". In addition, the photographic data is divided into group 1 (early stage of shooting), group 2 (mid-stage of shooting), and group 3 (late stage of shooting) according to each shooting time, and learning data and evaluation data are selected from each group for evaluation. It should be noted that the groups are created in such a way that the images of the same leaves are not placed in different groups.
[0185] like Fig.15As shown, learning and evaluation were performed three times with different combinations and repeated three times. Specifically, in the first learning, learning was performed based on the shooting data of group 2 and group 3, and evaluation was performed based on the shooting data of group 1. In addition, in the second learning, learning was performed based on the shooting data of group 1 and group 3, and evaluation was performed based on the shooting data of group 2. In the third learning, learning was performed based on the shooting data of group 1 and group 2, and evaluation was performed based on the shooting data of group 3. The fourth and seventh times were evaluated with the same groups as the first time. In the fifth and eighth times, the evaluation was performed with the same groups as the second time. In the sixth and ninth times, the evaluation was performed with the same groups as the first time.
[0186] Specifically, the discriminant model was generated and verified using images obtained from various processing or calculation results. Here, in addition to the captured data itself (Table 3.1), the results with high evaluation accuracy are also shown. Specifically, the discriminant model was generated and verified by using an image calculated from an R image and a B image (R image - B image) (Table 3.2), an image calculated from a G image and a B image (G image - B image) (Table 3.3), an image obtained from (G image + R image) / 2 - B image (Table 3.4), and an image in Table 3.3 that was increased 8 times by rotating the image by 45 degrees each time (Table 3.5).
[0187] In the following tables, “early stage of disease” is simply referred to as “early stage”, “middle stage of disease” is simply referred to as “middle stage”, and “late stage of disease” is simply referred to as “late stage”.
[0188] In addition, below, "accuracy" is the accuracy of the classification results classified into each label from the early stage to the middle stage of the disease. In addition, "accuracy of diseased leaves" is the accuracy of the classification results of health or disease. For example, the boundary between the early and middle stages of a disease is vague. In addition, the boundary between the middle and late stages of a disease is vague. Therefore, if the early to late stages of the disease are regarded as a discrimination result of "disease", if it is classified as any one, it is the accuracy in the positive case. Below, for the sake of simplicity of explanation, the "accuracy of diseased leaves" is used as the center, and the "accuracy of diseased leaves" and the "accuracy" both tend to increase. The "improvement in the accuracy of evaluation results" is explained.
[0189] Furthermore, the following results show that Fig.15 The total of the results of learning and evaluation from the first to the ninth time performed by the combination.
[0190] (Table 3.1)
[0191]
[0192] Table 3.2 shows an example of learning and evaluation using an R image and a B image obtained from the captured data and an image obtained by applying the following equation (2.1) to each pixel.
[0193] R picture - B picture (2.1)
[0194] Compared with the case of the photographic data in Table 3.1, the accuracy of the evaluation results is improved.
[0195] (Table 3.2)
[0196]
[0197] Table 3.3 shows an example of learning and evaluation using a G image and an R image obtained from the captured data and an image obtained by applying the following equation (2.2) to each pixel.
[0198] G image - B image (2.2)
[0199] Compared with the case of the photographic data in Table 3.1, the accuracy of the evaluation results is improved.
[0200] (Table 3.3)
[0201]
[0202] Table 3.4 shows an example of learning and evaluation using the G image and R image obtained from the captured data and the image obtained by applying the following equation (2.3) to each pixel.
[0203] (G image + R image) / 2-B image (2.3)
[0204] The equation (2.3) is the case where the values of x and y in the above equation (1.3) are set to (1, 1).
[0205] Compared with the case of the photographic data in Table 3.1, the accuracy of the evaluation results is improved.
[0206] (Table 3.4)
[0207]
[0208] Table 3.5 shows an example of learning and evaluation using an image obtained by the above-mentioned formula (2.2) which is increased eightfold by rotating the image by 45° at a time.
[0209] The accuracy of the evaluation results is improved compared to the case of the photographic data in Table 3.1. In addition, the evaluation results shown in Table 3.5 are also found to be more accurate than all the other evaluation results from Table 3.2 to Table 3.4 described above.
[0210] (Table 3.5)
[0211]
[0212] <Experimental Example 2>
[0213] In Experimental Example 2, images of hue (H), saturation (S), brightness (V), R, G, and B were used. Fig.14 In the group shown in Fig.15 As shown, learning and evaluation were repeated a total of nine times, and the results were totaled. In Experiment 2, the results of Table 3.5, which had the highest evaluation results in Experiment 1, were compared. The evaluation results of any of the H image, S image, V image, R image, G image, and B image were all less accurate than the results of Table 3.5. As an example, the following describes only the evaluation results of the H image.
[0214] Table 4 shows an example of learning and evaluating an image by converting the captured data of a color image into a hue image. Compared with the case of Table 3.5, the accuracy of the evaluation result is lower. In addition, compared with the result of Table 3.1 as the original image, the correct rate of diseased leaves is also worse overall.
[0215] (Table 4)
[0216]
[0217] It is believed that the evaluation result is poor because the H image treats the information of the dead part of the leaf in the same way as the other yellowed parts and healthy parts. From this, it can be seen that although the yellowing has different tones, the discrimination accuracy is not high if the discrimination model using the H image that shows different tones is used. In addition, it can be seen that the discrimination model using the image obtained by formula (1.3) is useful.
[0218] <Experimental Example 3>
[0219] In Experimental Example 3, the change in discrimination accuracy when the values of x and y in Formula (1.3) were changed was evaluated. Experimental Example 3 is the discrimination accuracy when a discrimination model is generated based on the image used in Experimental Example 1 and the image obtained by Formula (1.3). Table 5 shows the results of Experimental Example 3. Table 5 shows the typical values of the diseased leaf accuracy when the ratio of R in R:G (x / (x+y)) is at various values. The typical value of the diseased leaf accuracy is the total value of the diseased leaf accuracy.
[0220] (Table 5)
[0221] x / (x+y) Typical values of correctness of diseased leaves 0.0 88.5% 0.1 87.8% 0.2 87.9% 0.3 87.5% 0.4 86.8% 0.5 86.3% 0.6 86.1% 0.7 86.0% 0.8 85.9% 0.9 85.9% 1.0 85.6%
[0222] also, Fig.16 The results are shown in Table 5. Fig.16 The tendency of the graph shown shows that the smaller the R ratio is, the higher the accuracy of discrimination is. It is also possible to use x=0, y=1, which has the highest accuracy of discrimination. For example, the values of x and y may be x=0.15, y=0.85, which has an average value of 87% or more and an R ratio of 0.0 to 0.3.
[0223] (1) The generation method disclosed herein is a method for generating a discrimination model for discriminating the growth state of a plant, the discrimination model being executed by a computer including a computing circuit capable of accessing a storage device, wherein:
[0224] The storage device stores a chloroplast density image, which is an image obtained from images of leaves of a plurality of plants for learning and reflects the difference in absorption spectra of chlorophyll and carotenoid included in the plants.
[0225] The generating method is executed by the operation circuit.
[0226] The arithmetic circuit reads the chloroplast density image from the storage device,
[0227] The chloroplast density image is used as learning data for machine learning to generate a discrimination model for the growth state of the plant.
[0228] (2) In the generation method of (1), it is also possible that
[0229] The chloroplast density image is an image obtained by using two or more of the R image, G image and B image obtained from the image of the plant leaf, and by using the following formula (1) for each corresponding pixel value:
[0230] (Formula for matching the R image and the G image with arbitrary weights) - (Formula for using the B image) (1)
[0231] In the above equation (1), the values of pixels of the unused images among the R image, the G image, and the B image are 0.
[0232] (3) In the generation method of (2), it is also possible that
[0233] The above-mentioned equation (1) is expressed as the following equation (2).
[0234] (R image × x + G image × y) / (x + y) - B image (2)
[0235] Among them, 0≤x, 0≤y.
[0236] (4) In the generation methods of (1) to (3), it is also possible that:
[0237] The growth state of the plant includes at least one selected from the group consisting of a healthy state, a diseased state, a malnutrition state or an overnutrition state, a bad growth environment state, and an insect pest state.
[0238] (5) In the generation methods of (1) to (4), it is also possible that:
[0239] The growth state includes a selected level among levels of varying degrees.
[0240] (6) In the generation method of (3), the following steps may also be included:
[0241] The values of x and y in the equation (2) are optimized to a combination with the highest discrimination accuracy.
[0242] (7) In the generation methods of (1) to (6), it is also possible that:
[0243] To acquire the image of the leaves of the plant, a transmission optical system is used that includes a surface light source having substantially uniform brightness with respect to the surface.
[0244] (8) In the generation methods of (1) to (7), it is also possible that:
[0245] The machine learning includes at least one of supervised learning and unsupervised learning.
[0246] (9) In the generation methods of (1) to (8), it is also possible that:
[0247] The storage device stores the label indicating the growth state of the plant in association with each of the chloroplast density images.
[0248] The arithmetic circuit reads the chloroplast density image together with the label from the storage device as learning data for supervised learning,
[0249] The operation circuit learns the relationship between the multiple chloroplast density images included in the learning data and the corresponding labels through machine learning, and generates a discrimination model for the chloroplast density image of a new plant, which outputs the growth status of the new plant as a discrimination result.
[0250] (10) In the generation methods of (1) to (9), it is also possible that:
[0251] The arithmetic circuit reads a plurality of the chloroplast density images from the storage device as learning data for unsupervised learning.
[0252] The calculation circuit generates a plurality of clusters for each feature of the growth state of the plant based on a result of analyzing the features of the plurality of chloroplast density images of the learning data,
[0253] When each cluster is assigned a label representing the growth status of the plant, the operation circuit uses the multiple chloroplast density images included in the cluster and the labels assigned to the cluster as learning data for supervised learning, learns the relationship between the multiple chloroplast density images and the corresponding labels, and generates a discrimination model for the chloroplast density image of the new plant, which outputs the growth status of the new plant as a discrimination result.
[0254] (11) The determination method disclosed herein is a method for determining the growth state of a plant, and is executed by a computer including a computing circuit capable of accessing a storage device, wherein:
[0255] The determination method is performed by the operation circuit.
[0256] reading an image of a plant leaf for identification from the storage device,
[0257] A chloroplast density image representing the density of chloroplasts of the plant is calculated based on the image.
[0258] The calculated chloroplast density image is used as an input to the discrimination model generated by the generation method of claim 1 to discriminate the growth state of the plant.
[0259] (12) The computer program of the present disclosure executes the generation method of any one of (1) to (10).
[0260] (13) The computer program of the present disclosure executes the determination method of (11).
[0261] (14) The generating device disclosed in the present invention is a generating device for a discriminant model for discriminating the growth state of a plant, comprising a computing circuit capable of accessing a storage device, wherein:
[0262] The storage device stores a chloroplast density image, which is an image obtained from images of leaves of a plurality of plants for learning and reflects the difference in absorption spectra of chlorophyll and carotenoid included in the plants.
[0263] The chloroplast density image is read from the storage device by the calculation circuit.
[0264] The calculation circuit uses the chloroplast density image as learning data for machine learning, thereby generating a discrimination model for the growth state of the plant.
Claims
1. A method for generating a discriminant model, which is a method for generating a discriminant model for discriminating the growth state of a plant, wherein the discriminant model is executed by a computer including an arithmetic circuit capable of accessing a storage device, wherein: The storage device stores a plurality of chloroplast density images, which are images of leaves of a plurality of plants, and are obtained by using the following formula (1) for each pixel of the image: (Intensity of red light × x + Intensity of green light × y) / (x + y) - Intensity of blue light (1) Except for the case where 0≤x, 0≤y, and x=y=0, In the generating method, The arithmetic circuit reads a plurality of chloroplast density images from the storage device, The computing circuit uses the plurality of chloroplast density images as learning data for machine learning, thereby generating a discrimination model for the growth state of the plant. The generation method further comprises the following steps: The calculation circuit optimizes the values of x and y in the equation (1) to a combination with the highest discrimination accuracy.
2. The generation method according to claim 1, wherein: The growth state of the plant includes at least one selected from the group consisting of a healthy state, a diseased state, a malnutrition state or an overnutrition state, a bad growth environment state, and an insect pest state.
3. The generation method according to claim 1, wherein: The growth state includes a level selected from levels of varying degrees.
4. The generation method according to claim 1, wherein: To acquire the image of the leaves of the plant, a transmission optical system is used that includes a surface light source having substantially uniform brightness with respect to the surface.
5. The generation method according to claim 1, wherein: The machine learning includes at least one of supervised learning and unsupervised learning.
6. The generation method according to claim 1, wherein: The storage device stores the label indicating the growth state of the plant in association with each of the chloroplast density images. In the generating method, The arithmetic circuit reads the chloroplast density image together with the label from the storage device as learning data for supervised learning, The operation circuit learns the relationship between the multiple chloroplast density images included in the learning data and the corresponding labels through machine learning, and generates a discrimination model for the chloroplast density image of a new plant, which outputs the growth status of the new plant as a discrimination result.
7. The generation method according to claim 1, wherein: The arithmetic circuit reads a plurality of the chloroplast density images from the storage device as learning data for unsupervised learning. The calculation circuit generates a plurality of clusters for each feature of the growth state of the plant based on a result of analyzing the features of the plurality of chloroplast density images of the learning data, When each cluster is assigned a label representing the growth status of the plant, the operation circuit uses the multiple chloroplast density images included in the cluster and the labels assigned to the cluster as learning data for supervised learning, learns the relationship between the multiple chloroplast density images and the corresponding labels, and generates a discrimination model for the chloroplast density image of the new plant, which outputs the growth status of the new plant as a discrimination result.
8. A method for determining the growth state of a plant, which is executed by a computer including an operation circuit, wherein: The computing circuit receives an image of a leaf of a plant for identification. The calculation circuit calculates a chloroplast density image representing the density of chloroplasts of the plant based on the image. The calculation circuit uses the calculated chloroplast density image as an input to the discrimination model generated by the generation method of claim 1 to discriminate the growth state of the plant.
9. A computer program causing a computer to execute the generation method according to any one of claims 1 to 7.
10. A computer program causing a computer to execute the determination method according to claim 8.
11. A generating device, which is a generating device of a discriminant model for discriminating the growth state of a plant, comprising a computing circuit capable of accessing a storage device, wherein: The storage device stores a plurality of chloroplast density images, which are images of leaves of a plurality of plants, and are obtained by using the following formula (1) for each pixel of the image: (Intensity of red light × x + Intensity of green light × y) / (x + y) - Intensity of blue light (1) Except for the case where 0≤x, 0≤y, and x=y=0, The arithmetic circuit reads a plurality of chloroplast density images from the storage device, The computing circuit uses the plurality of chloroplast density images as learning data for machine learning, thereby generating a discrimination model for the growth state of the plant. The operation circuit also optimizes the values of x and y in the equation (1) to a combination with the highest discrimination accuracy.
