Branch and leaf age estimation system, branch and leaf age estimation method, and computer program

Abstract

To enable estimation of the age of a branch or its leaves of a citrus fruit tree with improved reliability compared to conventional methods, thereby reducing a mental burden on the worker and securing pruning accuracy.SOLUTION: A branch and leaf age estimation device 1 for estimating the age of a branch of a citrus fruit tree or the age of leaves on the branch is provided with a SPAD value calculation unit 504 and an age estimation unit 505. The SPAD value calculation unit 504 calculates a SPAD value representing a chlorophyll level of the leaf. The age estimation unit 505 estimates the age on the basis of the calculated SPAD value.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a technique for estimating the age of the branches and leaves of citrus fruit trees. [Background technology]

[0002] Accurate pruning of fruit trees is important to harvest the planned amount of fruit from the trees. Therefore, pruners are required to accurately distinguish which branches to cut off and which to leave. If pruning is not performed accurately, alternate bearing is likely to occur. In addition, since there have been many abnormal weather events in recent years, more accurate pruning is required to more reliably harvest the planned amount of fruit.

[0003] For example, when pruning Satsuma mandarin trees, workers distinguish between one-year-old branches and branches that are two or more years old, and generally cut off the branches that are two or more years old.

[0004] However, even experienced workers have difficulty distinguishing between the two, so this places a great physical and mental strain on workers.

[0005] Incidentally, a device described in Patent Document 1 has been proposed as a technique for reducing the burden on workers.

[0006] This device is a fruit tree cultivation support device that comprises an acquisition unit that acquires fruit tree condition information including the condition of the fruit tree to be supported at the flowering period of the fruit tree, and a generation processing unit that generates work information for the fruit tree by inputting the fruit tree condition information for the fruit tree to be supported acquired by the acquisition unit into a learning model that has been trained to have a correlation between the fruit tree condition information including the condition of the fruit tree to be studied at the flowering period of the fruit tree and work information including the work to be done on the fruit tree until the flowering period of the next year.

[0007] The work information includes the pruning points in the pruning work for the fruit tree to be assisted, so that the worker can perform the pruning work while referring to the pruning points inferred by the fruit tree cultivation assistance device. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Patent No. 7113555 Summary of the Invention [Problem to be solved by the invention]

[0009] However, it is unclear what kind of information is specifically included in the work information used by the fruit tree cultivation support device described in Patent Document 1 during learning. Therefore, the accuracy of the pruning points inferred by the fruit tree cultivation support device is questionable. Moreover, the age of the branches cannot be inferred.

[0010] In view of the above problems, the present invention aims to estimate the age of fruit tree branches or the age of their leaves for pruning more reliably than ever before. [Means for solving the problem]

[0011] A branch and leaf age estimation system according to one embodiment of the present invention is a branch and leaf age estimation system that estimates the age of a branch of a citrus tree or a leaf of the branch, and has a chlorophyll amount acquisition means for acquiring the chlorophyll amount of the leaf, and an estimation means for estimating the age based on the chlorophyll amount.

[0012] Preferably, the method further comprises: image acquisition means for acquiring an image of the leaf; and the chlorophyll amount acquisition means acquires the chlorophyll amount based on the image.

[0013] Alternatively, the chlorophyll amount acquisition means acquires the chlorophyll amount based on at least a difference between a value of G and a value of B among RGB (Red, Green, Blue) of the image, or further based on a difference between a value of G and a value of R of the image.

[0014] Alternatively, the estimation means estimates that the age is one year old (fruiting branch) if the amount of chlorophyll is above a threshold, and estimates that the age is two years old or older if the amount is below a threshold. More specifically, the estimation means estimates that the age is two years old (fruiting branch) if the amount of chlorophyll is below the threshold and above a second threshold, and estimates that the age is three years old if the amount is below the second threshold. Effect of the Invention

[0015] According to the present invention, the age of a branch of a fruit tree or the age of the leaves on that branch can be estimated more reliably than in the past for pruning purposes. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 is a diagram illustrating an example of the overall configuration of a branch and leaf pruning support system. [Diagram 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a branch and leaf age estimation device. [Diagram 3] FIG. 2 is a diagram illustrating an example of a functional configuration of a branch and leaf age estimation device. [Figure 4] FIG. 13 is a diagram showing an example of the transition of the average SPAD values ​​for each of the first to third graders. [Diagram 5] FIG. 1 shows an example of the relationship between SPAD values ​​and chlorophyll content. [Figure 6] FIG. 1 is a diagram showing an example of the relationship between RGB values ​​of a digital camera and RGB values ​​of a color difference meter. [Figure 7] FIG. 11 is a diagram illustrating an example of noise removal processing. [Figure 8] FIG. 13 is a diagram illustrating an example of a relationship between a feature amount and a SPAD value. [Figure 9] 13 is a flowchart illustrating an example of an overall processing flow of a branch and leaf age estimation program. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] 1. Overall Structure Fig. 1 is a diagram showing an example of the overall configuration of a branch and leaf pruning support system 4. Fig. 2 is a diagram showing an example of the hardware configuration of a branch and leaf age estimation device 1. Fig. 3 is a diagram showing an example of the functional configuration of the branch and leaf age estimation device 1.

[0018] The branch and leaf pruning support system 4 is a system that supports the pruning of branches and leaves of citrus trees grown in orchards, and is composed of a branch and leaf age estimation device 1, one or more terminal devices 2, and a communication line 3, as shown in Figure 1.

[0019] The branch and leaf age estimation device 1 and the terminal device 2 can exchange data with each other via a communication line 3. As the communication line 3, the Internet, a LAN (Local Area Network) line, a public line, a short-distance wireless, a USB (Universal Serial Bus) cable, or the like is used.

[0020] The terminal device 2 is a device for an orchard worker to photograph the leaves of fruit trees. As the terminal device 2, a device equipped with a digital camera and a communication device, such as a smartphone, a tablet computer, or wearable glasses, may be used. Alternatively, a set combining a digital camera and a computer such as a personal computer may be used as the terminal device 2. Alternatively, a digital camera alone may be used as the terminal device 2.

[0021] The branch and leaf age estimation device 1 estimates the age of a leaf photographed by a terminal device 2 and the branch to which the leaf is attached. A personal computer or a cloud server is used as the branch and leaf age estimation device 1. The following describes an example in which a personal computer is used as the branch and leaf age estimation device 1.

[0022] As shown in Figure 2, the branch and leaf age estimation device 1 is composed of a main processor 10, a RAM (Random Access Memory) 11, a ROM (Read Only Memory) 12, an auxiliary storage device 13, a network interface 14, a serial interface 15, a wireless communication device 16, a display 17, a keyboard 18, and a pointing device 19.

[0023] In addition to an operating system, computer programs such as a branch and leaf age estimation program 50 are installed in the ROM 12 or the auxiliary storage device 13. As the auxiliary storage device 13, a hard disk or an SSD (Solid State Drive) is used.

[0024] According to the branch and leaf age estimation program 50, functions such as a correction data storage unit 501, an image correction unit 502, a noise removal unit 503, a SPAD value calculation unit 504, an age estimation unit 505, and an estimation result output unit 506 shown in Fig. 3 are realized. The functions of each unit will be described later in order.

[0025] The RAM 11 is a main memory of the branch and leaf age estimation device 1. Computer programs stored in the ROM 12 or the auxiliary storage device 13 are loaded into the RAM 11 as appropriate.

[0026] The main processor 10 executes a computer program loaded into a RAM 11. As the main processor 10, a GPU (Graphics Processing Unit), a CPU, or the like is used.

[0027] The network interface 14 communicates with other devices such as the terminal device 2 by a protocol such as TCP / IP (Transmission Control Protocol / Internet Protocol). As the network interface 14, a NIC (Network Interface Card) or a communication device for Wi-Fi is used.

[0028] The serial interface 15 communicates with peripheral devices by a serial communication method. A board conforming to a standard such as USB is used as the serial interface 15.

[0029] The wireless communication device 16 communicates with peripheral devices by short-range wireless communication. A board conforming to standards such as Bluetooth is used as the wireless communication device 16.

[0030] Depending on the communication interface of the terminal device 2, the network interface 14, the serial interface 15, and the wireless communication device 16 are used. For example, if the terminal device 2 only has Bluetooth as a communication interface, the wireless communication device 16 is used. Alternatively, if the terminal device 2 is a digital camera alone, the serial interface 15 is used. Any of these devices is used to receive image data and the like from the terminal device 2, as described below.

[0031] The display 17 displays a screen for inputting commands or information, a screen showing the results of calculations performed by the main processor 10, and the like.

[0032] The keyboard 18 and the pointing device 19 are input devices for an operator to input commands, information, and the like.

[0033] Hereinafter, the principle of estimating the age of a branch or leaf and the functions of each part of the branch or leaf age estimating device 1 shown in FIG. 3 will be described using an example of supporting pruning of the branches and leaves of a Satsuma mandarin fruit tree.

[0034] [2. Principles of estimating the age of branches and leaves] Fig. 4 is a diagram showing an example of transition of the average SPAD value for each of the first to third graders, and Fig. 5 is a diagram showing an example of the relationship between the SPAD value and the chlorophyll content.

[0035] Like other citrus trees, Satsuma mandarin trees have a fruiting habit of forming fruiting branches. More specifically, in spring, first-year fruiting branches sprout, but they do not flower that year, so no fruit is produced. After winter, in spring, when these fruiting branches become second-year, fruiting branches sprout, flowers bloom on the fruiting branches, and fruit is produced.

[0036] Then, in autumn, the mature fruits are harvested. After the fruits are harvested, the stalk hardens and turns into a sclerenchyma branch. This branch will not bear fruit the following year. Therefore, it is cut off during pruning, which is done after the harvest and before spring.

[0037] However, if the first-year result of the main branch is cut off during pruning, no fruit will be obtained. Therefore, workers must determine the age of the branch. In particular, in the case of Unshu mandarin trees, it is necessary to distinguish in winter whether the branch is first year old or two years or older. However, this distinction is difficult even for experienced workers.

[0038] Therefore, the inventors focused on the leaves attached to the fruiting mother branch and the fruiting branch. Furthermore, they paid particular attention to the SPAD (Soil & Plant Analyzer Development) value of the leaves. The "SPAD value" is a value that represents the amount of chlorophyll. The SPAD values ​​of the leaves on the branches of one-year-old, two-year-old, and three-year-old plants were measured on several days from late October 2022 to late May 2023, and the average values ​​for each age were calculated, resulting in the results shown in FIG. 4. According to these results, it can be seen that the leaves can be classified as one-year-old or two-year-old or older depending on whether the SPAD value is equal to or greater than the threshold value "80".

[0039] When measuring the SPAD value of Satsuma mandarin leaves, the inventor used one of the most popular chlorophyll meters, the Konica Minolta SPAD-502Plus. However, according to the company's webpage (https: / / www.konicaminolta.jp / instruments / products / color / chlorophyll / index.html), this chlorophyll meter is primarily used to measure the SPAD value of live rice leaves. Therefore, the inventor checked whether this chlorophyll meter was suitable for measuring Satsuma mandarin leaves as follows.

[0040] The inventors measured the SPAD values ​​and chlorophyll concentrations of approximately 45 Satsuma mandarin leaves using this chlorophyll meter and a chlorophyll quantification method using an extraction method with 80% acetone, respectively, and obtained a correlation as shown in Figure 5. In view of this correlation, it can be seen that the SPAD values ​​of Satsuma mandarin leaves measured using this chlorophyll meter are sufficiently reliable.

[0041] Therefore, when pruning, workers can use this chlorophyll meter to measure the SPAD value of the leaves of the fruiting branches and estimate whether the branches are one year old or two years old or older.

[0042] However, with this chlorophyll meter, the leaf must be clamped with a clip-like head about the size of a thumb, so it takes a lot of time and effort to measure the SPAD value.

[0043] Therefore, the inventor came up with a method of taking a picture of the leaf with a digital camera and calculating the SPAD value based on the image of the leaf. Furthermore, in order to reduce errors due to the characteristics of the light receiving sensor of the digital camera, a method of correcting the image according to the digital camera before calculating the SPAD value is adopted.

[0044] [3. Functions of each part] [3.1 Data preparation] FIG. 6 is a diagram showing an example of the relationship between the RGB values ​​of a digital camera and the RGB values ​​of a color difference meter.

[0045] In the correction data storage unit 501 (see FIG. 3), correction data 61 is stored in advance for each model of digital camera in association with the model name. The correction data 61 indicates a correction formula for correcting (calibrating) each RGB (Red, Green, Blue) value (brightness) of an image captured by the digital camera. A method for generating the correction data 61 will be described below by taking a specific model as an example.

[0046] A person in charge at the manufacturer of the branch and leaf age estimation device 1 takes pictures of each color of the color palette several times under the same environment using a color difference meter and a specific model of digital camera (hereinafter referred to as the "target digital camera"). The color difference meter measures in the XYZ color system and converts the measured values ​​into RGB.

[0047] The average value of each RGB value measured by the colorimeter for each color in the color palette and the average value of each RGB value of the image captured by the target digital camera are calculated. Hereinafter, the average value of each RGB value measured by the colorimeter for the i-th color in the color palette is referred to as "U Ri ", "U Gi ", "U Bi " and the average value of each RGB of the image taken with the target digital camera is "V Ri ", "V Gi ", "V Bi " should be written.

[0048] (V Ri ,U Ri ), (V Gi ,U Gi ), and (V Bi ,U Bi ) can be represented on a plane as shown in Figures 6(A), (B), and (C), respectively.

[0049] (V Ri ,U Ri) are considered to be on the same quadratic function. Then, a correction formula that approximates this quadratic function (especially in the range 0≦x≦255) to a straight line that is directly proportional to y=x is calculated using a known method. (V Gi ,U Gi ) and (V Bi ,U Bi ) is also calculated in the same way.

[0050] These correction formulas are used to correct the RGB values ​​of the image captured by the target digital camera.

[0051] Alternatively, the correction data 61 may be generated by commercially available application software together with a color palette for calibration.

[0052] Data indicating the correction formulas for each of RGB is stored in the correction data storage unit 501 as the correction data 61 for the target digital camera.

[0053] Note that a registration application may be provided in the branch and leaf age estimation device 1 so that a user (operator) can generate the correction data 61 and register it in the correction data storage unit 501.

[0054] [3.2 Pruning Treatment] Fig. 7 is a diagram showing an example of a noise removal process, and Fig. 8 is a diagram showing an example of a relationship between a feature amount and a SPAD value.

[0055] The worker selects one leaf on the branch whose age he / she wants to know as a subject, and takes a picture of it with the digital camera of the terminal device 2. The terminal device 2 then transmits an image 62 including the subject to the branch / leaf age estimation device 1. At this time, the model name of the digital camera of the terminal device 2 is transmitted to the branch / leaf age estimation device 1.

[0056] When the image correction unit 502 receives the image 62 and the model name, it corrects the image 62 using the correction data 61 corresponding to the digital camera of the terminal device 2. Specifically, the correction is performed by substituting each of the RGB values ​​of the image 62 into the correction formulas for each of the RGB values ​​shown in the correction data 61. As a result, a corrected image 63 is obtained.

[0057] The noise removal unit 503 retains a portion of the corrected image 63 that can be used to calculate the SPAD value, and removes the other portion as noise, thereby obtaining a suitable image 64. The removal process will be described below using as an example a case in which the image shown in FIG. 7(A) is obtained as the corrected image 63 by the image correction unit 502.

[0058] The noise removal unit 503 performs segmentation processing. That is, as shown in FIG. 7B, the subject is left and the other parts, that is, the background parts, are deleted from the corrected image 63. The hatched parts are the deleted parts. The same applies below. The subject may be recognized by a known object detection method.

[0059] Furthermore, the noise removal unit 503 removes shadow areas and whiteout (halation) areas from the image of the subject, as shown in FIG. 7C. Shadow areas and whiteout areas can be detected by known detection methods. For example, threshold processing can be performed. That is, the HSV (Hue, Saturation, Value) value of each pixel in the image of the subject is calculated, and pixels that do not fall within a predetermined range are detected as shadow areas or whiteout areas.

[0060] The SPAD value calculation unit 504 calculates the SPAD value of the object, for example, as follows: A feature value P of the vegetation index of the object is calculated based on the following equation (1). P=M B -M G …… (1) "M B " and "M G" are the average values ​​of B and G among RGB of each pixel of the preferred image 64. Note that if the values ​​represent the characteristics of B and G in the preferred image 64, M B , M G Instead of the average value, a representative value or a median value may be used.

[0061] Then, the SPAD value calculation unit 504 calculates the feature amount P based on the calibration curve of the following equation (2). S=αP+β …… (2) α and β are constants that are calculated in advance as follows. The SPAD value and RGB value of each of the many leaves are measured. The SPAD value and feature value (in this example, (V B -V G )) shows the correlation shown in Figure 8. B , V G are the B (Blue) value and G (Green) value, respectively.

[0062] A linear function expressing this correlation is determined. In this case, the linear function may be obtained by regression analysis such as that described in JP 2023-7457 A. In the example of FIG. 8, α and β in (2) are calculated as “0.8838” and “102.88”, respectively.

[0063] The age estimation unit 505 estimates whether the subject leaf and the branch to which it is attached are first-year or second-year or older based on the SPAD value S calculated by the SPAD value calculation unit 504. That is, if the SPAD value S is a threshold value, i.e., "80," or greater, it is estimated to be first-year, and if it is less than 80, it is estimated to be second-year or older.

[0064] The estimation result output unit 506 outputs the estimation result 65 (whether the child is a first grader or a second grader or above) by the age estimation unit 505. For example, the estimation result 65 may be displayed on the display 17 of the branch and leaf age estimation device 1. Alternatively, the estimation result 65 may be transmitted to the terminal device 2. In this case, the terminal device 2 displays the estimation result 65 or outputs it by voice. Then, the worker performs pruning work while referring to the output estimation result 65.

[0065] 4. Overall Processing Flow and Effects of This Embodiment FIG. 9 is a flowchart illustrating an example of the overall processing flow of the branch and leaf age estimation program 50.

[0066] Next, the overall process flow of the branch and leaf age estimation device 1 will be described with reference to the flowchart of FIG.

[0067] The branch and leaf age estimation device 1 executes the process of estimating the age of a branch based on the learning and inference program 40 in the procedure shown in FIG.

[0068] When the branch and leaf age estimation device 1 receives an image 62 from the terminal device 2 (#101), it generates a corrected image 63 by correcting the image 62 based on the correction data 61 of the digital camera of the terminal device 2 (#102). Furthermore, it generates a preferred image 64 by removing parts other than the subject in the corrected image 63 and removing shadows and blown-out highlights in the subject (#103). It calculates the SPAD value S based on the preferred image 64 and the above formulas (1) and (2) (#104).

[0069] If the SPAD value S is equal to or greater than the threshold, the branch and leaf age estimation device 1 estimates that the child is in the first grade, and if it is less than the threshold, the child is in the second grade or above (#105), and outputs the estimation result (#106).

[0070] According to the present embodiment, the age of the branches of Satsuma mandarin trees can be estimated more reliably than in the past. In addition, the mental burden on the worker can be reduced regardless of the worker's experience. Furthermore, the risk of an inexperienced worker misjudging the age can be reduced.

[0071] Furthermore, according to this embodiment, estimation can be performed based on images of leaves without using a chlorophyll meter, which reduces the physical burden on the worker.

[0072] This will also enable visualization of the growth status, making it possible to improve the efficiency of cultivation and the feasibility of plans in citrus farming.

[0073] 5. Modifications and Applications In this embodiment, the branch and leaf age estimation device 1 estimates whether the branch and its leaves are first-year or second-year or older, but it may estimate their age.

[0074] As shown in FIG. 4, in early November, there may not be much difference between the SPAD values ​​of second-year and third-year branches. However, pruning is performed from January until spring. During this period, there is a sufficient difference between the two. Therefore, the second threshold for distinguishing between second-year and third-year branches may be set to, for example, "65." In this case, the branch and leaf age estimation device 1 estimates that a branch with a SPAD value of 80 or more is a first-year branch, that a branch with a SPAD value of less than 80 and greater than or equal to 65 is a second-year branch, and that a branch with a SPAD value of less than 65 is a third-year branch.

[0075] In this embodiment, as shown in formula (1), the branch and leaf age estimation device 1 calculates the value obtained by subtracting the value of G from the value of B as the characteristic amount P (vegetation index), but it may also calculate the value obtained by subtracting the value of B from the value of G. Alternatively, the characteristic amount P may be calculated based on the following formula (3) or (4). R " is the average value of R among RGB of each pixel of the preferred image 64. P=2M G -M R -M B ... (3) P=(M G-M B ) / (M G +M B ) …… (4)

[0076] The right-hand side of equation (3) is (M G -M B )+(M G -M R ), and is transformed into the following. In other words, even in the case of formula (3), the feature amount P is (M G -M B ) is calculated based on the coefficient of determination R 2 As shown in Figure 8, the coefficient of determination R 2 The result is 0.5411, and 0.4616 is obtained using equation (4). The same result can be obtained by inverting the signs of the terms on the right-hand side of equations (3) and (4), and these can be used in place of equation (1).

[0077] In this embodiment, the correction data 61 is prepared for each model of digital camera, but the correction data 61 may be prepared for each combination of a model of digital camera and a shooting mode (scene selection).

[0078] In this embodiment, a method for estimating the age of branches and their leaves has been described using Satsuma mandarins as an example, but a similar method can also be used to estimate the age of fruit trees of varieties other than Satsuma mandarins, such as Iyokan, Villafranca, Sudachi, and Kabosu.

[0079] Furthermore, the formula (1), the formula (2), the threshold value, the second threshold value, and the like may be prepared in advance in the branch and leaf age estimation device 1 for each of these varieties, so that estimation according to the variety can be performed.

[0080] In this embodiment, the components shown in FIG. 3 are integrated into the branch and leaf age estimation device 1, but may be distributed among a plurality of computers.

[0081] In addition, the configuration of the whole or each part of the branch and leaf pruning support system 4 and the branch and leaf age estimation device 1, the processing contents, the processing order, etc. can be appropriately changed in accordance with the spirit of the present invention. [Explanation of symbols]

[0082] 4. Branch and leaf pruning support system (age estimation system) 1. Branch and leaf age estimation device (age estimation system) 2. Terminal device (image acquisition means) 14 Network interface (image acquisition means) 15 Serial interface (image acquisition means) 16 Wireless communication device (image acquisition means) 504 SPAD value calculation unit (chlorophyll amount acquisition means) 505 Age Estimation Unit (Estimation Means) 62 images

Claims

1. A branch and leaf age estimation system for estimating the age of a branch or a leaf of a citrus fruit tree, comprising: A chlorophyll amount acquisition means for acquiring the chlorophyll amount of the leaf; An estimation means for estimating the age based on the amount of chlorophyll; A branch and leaf age estimation system comprising:

2. image capture means for capturing an image of the leaf; The chlorophyll amount acquisition means acquires the chlorophyll amount based on the image. The branch and leaf age estimation system according to claim 1 .

3. The chlorophyll amount acquisition means acquires the chlorophyll amount based on a difference between at least a G value and a B value among RGB (Red, Green, Blue) of the image. The branch and leaf age estimation system according to claim 2 .

4. The chlorophyll amount acquisition means further acquires the chlorophyll amount based on a difference between a G value and an R value of the image. The branch and leaf age estimation system according to claim 3 .

5. The estimation means estimates that the age is a first year if the amount of chlorophyll is above a threshold, and estimates that the age is a second year or older if the amount of chlorophyll is below a threshold. The branch and leaf age estimation system according to any one of claims 2 to 4.

6. The estimation means estimates that the age is a second grader if the amount of chlorophyll is below the threshold and above a second threshold, and estimates that the age is a third grader if the amount of chlorophyll is below the second threshold. The branch and leaf age estimation system according to claim 5 .

7. A method for estimating the age of a branch or a leaf of a citrus fruit tree, comprising: A process of acquiring the amount of chlorophyll in the leaves is performed by a computer. causing a computer to execute a process of estimating the age based on the amount of chlorophyll; A method for estimating branch and leaf age.

8. 1. A computer program for use in a computer to estimate the age of a branch of a citrus fruit tree or a leaf of said branch, comprising: The computer includes: Executing a process of acquiring a chlorophyll amount of the leaf; executing a process of estimating the age based on the amount of chlorophyll; A computer program comprising:

Images