Strawberry plant information acquisition system

Abstract

To accurately calculate a petiole length or the leaf area of each strawberry plant.SOLUTION: An information processor obtains a difference (horizontal projection movement amount MP) between a distance Dn between the center of a strawberry plant and a petiole tip of a predetermined leaf, which is obtained from a plant image captured from above by not blowing onto the strawberry plant or blowing at first speed (weak wind), and a second distance Dm from the plant center to the petiole tip, which is obtained from a plant image captured from above by blowing at second speed (middle wind or strong wind). Then, the information processor calculates a value of a petiole length on the basis of a value of the leaf area, wind speed when the images are captured, and a horizontal projection movement amount MP.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a strawberry plant information acquisition system. [Background technology]

[0002] Image measurement is known as an effective method for measuring the biometrics of plants, as it is non-destructive, allows for wide-area measurement, and is labor-saving. When the plant being measured is a strawberry, the measurement targets include petiole length and leaf area. These measurement targets are important factors in growth diagnosis and yield prediction, so it is desirable to be able to measure them accurately.

[0003] Conventionally, there is known a technique for determining the growth state of a plant (stem thickness) by capturing an image of the plant using a photographing unit while a fan is blowing air onto the plant and processing the image (see, for example, Patent Document 1). There is also known a technique for monitoring the growth state of a plant (such as harvest time) based on the time-series transition of images (see, for example, Patent Document 2). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-29940 [Patent Document 2] Japanese Patent Application Publication No. 2019-37225 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the above-mentioned petiole length and leaf area cannot be measured even using the above-mentioned Patent Documents 1 and 2. For example, since the petiole is located below the leaf blade (leaflet) and is difficult to see from the outside, it is difficult to measure the petiole length using an image. Furthermore, since leaf blades often overlap each other, it is also difficult to measure the leaf area with high accuracy.

[0006] Therefore, an object of the present invention is to provide a strawberry plant information acquisition system that can accurately calculate strawberry plant information. [Means for solving the problem]

[0007] The strawberry plant information acquisition system of the present invention includes a blowing device that blows wind onto strawberry plants from above, a photographing device that takes images of the plants from above, a control device that controls the photographing device to take a first image of the plants from above in a first state where no wind is blown onto the plants from the blowing device or where wind is blown onto the plants from the blowing device at a first speed, and controls the photographing device to take a second image of the plants from above in a second state where wind is blown onto the plants from the blowing device at a second speed, and a control device that determines the leaf surface of a predetermined leaf of the strawberry plant from the images taken by the photographing device. and a calculation device that calculates one of the values ​​of the leaf area and the petiole length of the specified leaf, wherein the calculation device determines the difference between a first distance from the center of the plant to the petiole tip of the specified leaf in the first image and a second distance from the center of the plant to the petiole tip of the specified leaf in the second image, and calculates the value of the other of the leaf area and the petiole length of the specified leaf based on the value of the other of the leaf area and the petiole length of the specified leaf, the speed of the wind blown from the spraying device to the plant in each of the first state and the second state, and the difference between the first distance and the second distance. [Effects of the Invention]

[0008] The strawberry plant information acquisition system of the present invention has the effect of being able to accurately calculate strawberry plant information. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram illustrating a schematic configuration of an information acquisition system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating a hardware configuration of a control device and an information processing device. [Figure 3]FIG. 2 is a functional block diagram of a control device and an information processing device. [Figure 4] Figures 4(a) and 4(b) show the experimental setup. [Figure 5] Figure 5(a) is an image of a strawberry plant taken by a camera in the experimental apparatus of Figure 4(a) when the blower was not operating and the wind speed was 0 m, and Figure 5(b) is an image of a strawberry plant taken by a camera when wind was blown from the blower at a specified wind speed. [Figure 6] FIG. 6(a) is a diagram schematically showing the state of the petiole and leaf blade in the first and second states, and FIG. 6(b) is a diagram for explaining the position of the tip of the petiole. [Figure 7] FIG. 7(a) is a graph showing the relationship between wind speed and wind load per unit leaf area, and FIG. 7(b) is a graph showing the relationship between the drag coefficient of the leaf blade and wind speed. [Figure 8] Figures 8(a) to 8(e) are graphs showing the results of actual measurements of the horizontal projection movement Mp of the petiole of each of the first to fifth leaves of a strawberry plant against wind speed v, obtained using the experimental apparatus 200 of Figure 4(a). [Figure 9] 9(a) to 9(e) are diagrams showing examples of approximations of the actual measurement values ​​of FIG. 8(a) to FIG. 8(e). [Figure 10] FIG. 10 is a schematic diagram for explaining approximate formula 2. [Figure 11] FIG. 10 is a schematic diagram for explaining approximate formula 3. [Figure 12] 10 is a flowchart showing the processing of the control device. [Figure 13] 10 is a flowchart showing processing by the information processing device. [Figure 14] 14(a) to 14(c) are diagrams for explaining a method for detecting the center of a stock. [Figure 15] 15(a) to 15(g) are diagrams (part 1) for explaining an example of calculating the petiole length L using approximation formula 1. FIG. [Figure 16]16(a) to 16(g) are diagrams (part 2) for explaining an example of calculating the petiole length L using approximation formula 1. FIG. [Figure 17] 17(a) to 17(g) are diagrams (part 1) for explaining an example of calculating the leaf area A using approximate formula 1. FIG. [Figure 18] 18(a) to 18(g) are diagrams (part 2) for explaining an example of calculating the leaf area A using approximate formula 1. [Figure 19] 19(a) to 19(g) are diagrams for explaining an example of determining the petiole length L using approximation formula 2. FIG. [Figure 20] 20(a) to 20(g) are diagrams for explaining an example of determining the petiole length L using approximation formula 3. FIG. [Figure 21] Figure 21(a) is a graph showing the relationship between the measured petiole length and the petiole length calculated using approximation formula 1 (predicted petiole length), and Figure 21(b) is a graph showing the relationship between the measured leaf area and the leaf area calculated using approximation formula 1 (predicted leaf area). DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, a strawberry plant information acquisition system according to one embodiment will be described in detail.

[0011] FIG. 1 shows a schematic configuration of an information acquisition system 100 according to one embodiment. In this embodiment, the information acquisition system 100 is a system that calculates and records the petiole length and leaf area of ​​each strawberry plant cultivated in an elevated cultivation bed 12 as shown in FIG. 1. The distance (height) from the ground to the upper surface of the cultivation bed 12 is H1 (H1 is, for example, 1000 mm). The strawberry plants do not have to be cultivated in the cultivation bed 12, but may be planted in the ground in a ridge, for example.

[0012] The information acquisition system 100 includes a monitoring device 10 and an information processing device 90 as a calculation device. The monitoring device 10 moves near a cultivation bed 12 and acquires images of strawberry plants photographed from above. The information processing device 90 is communicably connected to the monitoring device 10 and calculates the petiole length and leaf area of ​​each strawberry plant planted in the cultivation bed 12 based on the images and position information acquired from the monitoring device 10. The information processing device 90 and the monitoring device 10 may be connected via a wired LAN (Local Area Network) or the like, or wirelessly via Wi-Fi or the like. The information processing device 90 may also be mounted on the monitoring device 10. In this embodiment, the extension direction of the cultivation bed 12 is defined as the X-axis direction, the direction perpendicular to the X-axis direction in a horizontal plane is defined as the Y-axis direction, and the vertical direction is defined as the Z-axis direction.

[0013] 1, the monitoring device 10 is installed on a rail 14 laid along the X-axis direction below (on the ground) the cultivation bed 12. The monitoring device 10 is movable along the rail 14 in the X-axis direction.

[0014] The monitoring device 10 includes a housing 20, wheels 22, a camera 24 as a photographing device, a blower 26 as a spraying device, a motor 28, a position detection device 29, and a control device 30. The housing 20 has a substantially rectangular frame shape. A ceiling panel 20a of the housing 20 extends above the cultivation beds 12 and the strawberry plants, and the lower surface (-Z surface) of the ceiling panel 20a and the strawberry plants are vertically opposed to each other.

[0015] The wheels 22 are driven to rotate on the rails 14 by a motor 28. The rotation of the motor 28 is controlled by a control device 30. That is, in this embodiment, a configuration including the wheels 22 that run on the rails 14 and the motor 28 is adopted as a moving mechanism for moving the monitoring device 10. However, without being limited to this, the moving mechanism may also be configured to include a running unit such as a wheel or crawler that runs on the ground, and a drive unit (such as a motor) that drives the running unit.

[0016] The camera 24 and the blower 26 are mounted on the underside of the ceiling panel 20a of the housing 20. The camera 24 is positioned at a distance (height) H2 (e.g., 1000 mm) from the cultivation bed 12. The camera 24 photographs the strawberry plants from above. The blower 26 has the function of blowing air downward onto the strawberry plants. The speed (wind speed) of the air blown from the blower 26 is changeable and controlled by the control device 30. The control device 30 photographs the plants from above using the camera 24 with or without the blower 26 blowing air onto the plants. The images captured by the camera 24 are transmitted to the control device 30, which then transmits the photographed images to the information processing device 90. The control device 30 also transmits information on the speed (wind speed) of the air blown onto the plants from the blower 26 during photography to the information processing device 90.

[0017] The position detection device 29 may be, for example, a position detection device having an RFID reader capable of communicating with multiple RFID (radio frequency identifier) ​​tags installed near the cultivation bed 12, a position detection device having a camera capable of photographing markers installed near the cultivation bed 12, a position detection device that detects position from the amount of rotation of the wheels 22 or the motor 28, or an RTK-GNSS (Real Time Kinematic-Global Navigation Satellite System). The position detection device 29 detects the XY position of the camera 24. From the XY position of the camera 24, the XY position of each point in the captured image (capture range) can be identified. In addition, the detection result of the position detection device 29 is transmitted to the control device 30.

[0018] The control device 30 controls the motor 28 based on the detection result of the position detection device 29 to adjust the position of the monitoring device 10 (the shooting location of the camera 24). The control device 30 also transmits to the information processing device 90 the image captured by the camera 24, its position information (XY position), and information on the speed of the wind blown from the blower 26 when the image was captured. FIG. 2 shows the hardware configuration of the control device 30. As shown in FIG. 2, the control device 30 includes a CPU (Central Processing Unit) 190, a ROM (Read Only Memory) 192, a RAM (Random Access Memory) 194, a storage unit (such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive)) 196, a network interface 197, and a portable storage medium drive 199. These components of the control device 30 are connected to a bus 198. In the control device 30, the CPU 190 executes a program stored in the ROM 192 or the storage unit 196, or a program read by the portable storage medium drive 199 from the portable storage medium 191, thereby realizing the functions of the units shown in Fig. 3. Note that the functions of the units shown in Fig. 3 may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array), for example.

[0019] The information processing device 90 uses the location information transmitted from the control device 30 to identify the plant photographed in the image. The information processing device 90 also calculates the petiole length and leaf area of ​​each plant using the image and wind speed information. The information processing device 90 has the same hardware configuration as the control device 30 (see FIG. 2). In the information processing device 90, the CPU 190 executes a program to realize the functions of each unit shown in FIG. 3.

[0020] (Calculation formula for petiole length and leaf area) Here, we will explain the mathematical formulas used by the information processing device 90 (calculation unit 56 in FIG. 3, which will be described later) when calculating the petiole length and leaf area. In this embodiment, one of three types of formulas (approximation formulas 1 to 3), which will be described later, is used, but this is not limiting and other formulas can also be used.

[0021] 4(a) and 4(b) show schematic diagrams of an experimental apparatus 200 used to derive the formula. As shown in FIG. 4(a), a cultivation pot 202 containing a strawberry plant is installed in the experimental apparatus 200. The cultivation pot 202 is installed so that its upper end is positioned at a height H0 (e.g., 500 mm) above the ground. The experimental apparatus 200 also includes a hollow cylinder 204 with a height H2 (e.g., 1000 mm) that covers the strawberry plant. The diameter of the hollow cylinder 204 is, for example, 500 mm. In the experimental apparatus shown in FIGS. 4(a) and 4(b), the height H0 of the cultivation pot 202 is set equal to the diameter of the hollow cylinder 204 so that air passes through the hollow cylinder 204 (to straighten the airflow). Furthermore, a camera 24 and a blower 26 are installed at the upper end of the hollow cylinder 204. The camera 24 and the blower 26 in FIG. 4(a) are the same as the camera 24 and the blower 26 in FIG.

[0022] FIG. 5(a) is an image of a strawberry plant taken by camera 24 when blower 26 is not operating and the wind speed is 0 m. FIG. 5(b) is an image of a strawberry plant taken by camera 24 when wind is blown from blower 26 at a predetermined wind speed. As shown in FIG. 5(b), when wind is blown from above, the leaf blades are subjected to a force (wind load), and as can be seen by comparing with FIG. 5(a), the leaf petioles fall outward around the base (center of the plant (crown)). The leaf blades also fall outward, and the entire leaf moves outward.

[0023] In Fig. 6(a), the state of the entire leaf in a state where the wind speed is 0 m or a wind of a first speed (weak wind) is blowing against the leaf blade (first state) as in Fig. 5(a) is shown by a thick solid line. Also, in Fig. 6(a), the state of the entire leaf in a state where the wind of a second speed (medium wind or strong wind) is blowing against the leaf blade (second state) as in Fig. 5(b) is shown by a thick dashed line. In Fig. 6(a), the horizontal difference M between the position of the tip of the petiole in the first state and the position of the tip of the petiole in the second state is P is the horizontal projection movement of the petiole M p The position of the tip of the petiole shown in Figure 6(a) is actually the base (see symbol S) of the petiole (the thin petiole connecting the petiole and the leaf blade) shown in Figure 6(b).

[0024] Next, an experimental apparatus 200′ shown in FIG. 4(b) will be described. The experimental apparatus 200′ in FIG. 4(b) is an improved version of the experimental apparatus 200 in FIG. 4(a). In the experimental apparatus 200′ in FIG. 4(b), a load cell 206 is installed at a height H1 (e.g., 500 mm) from the ground, and a leaf blade 210 is installed on a fixture 208 connected to the load cell 206. The height of the upper surface of the leaf blade 210 is set to approximately the same height as the height of the leaf blade of the strawberry plant in FIG. 4(a) (e.g., 581 mm from the ground). In addition, an anemometer 212 is installed near the leaf blade 210. This experimental apparatus 200′ can measure the wind speed near the leaf blade and the wind force (wind load) acting on the leaf blade when the propeller of the blower 26 is rotated at a predetermined rotation speed.

[0025] FIG. 7(a) shows the relationship between the wind velocity (Air velocity) v (m / s) and the wind load per unit leaf area (Load / Leaf area) (N / m 2 ) and the relationship between the drag coefficient C of the leaf blade. d 10 is a graph showing the relationship between wind speed v (m / s) and

[0026] 8(a) to 8(e) show the horizontal projection movement M of the petiole of each of the first to fifth leaves of a strawberry plant relative to the wind speed v, obtained using the experimental device 200. p The results of actual measurements are shown.

[0027] (About approximation formula 1) This approximation formula 1 is based on the actual measurements in Fig. 8(a) to Fig. 8(e) and is calculated by the wind speed v and horizontal projection displacement M P This is an example in which the relationship between is approximated as shown by the broken lines in FIGS. 9(a) to 9(e).

[0028] Here, the following formula (1) is known as a general formula for wind load.

[0029]

number

[0030] In the above equation (1), P (N) is the wind load, ρ (kg / m 3 ) is the air density, A(m 2 ) is the pressure-receiving area (leaf area), v (m / s) is the wind speed, C d is the drag coefficient.

[0031] Here, the drag coefficient of the leaf blade, C d is expressed as the following equation (2) using wind speed v.

[0032]

number

[0033] Note that m and n are coefficients.

[0034] Also, the horizontal projection movement amount M p The proportionality of is expressed by the following equation (3) using the proportionality coefficient k to the wind load P, and the petiole length L (mm) is expressed by the following equation (4) using the proportionality coefficient k.

[0035]

number

[0036] The above equation (4) is a modified equation of k=aL+b, and a and b in equation (4) are coefficients that need to be calculated in advance.

[0037] In this approximation formula 1, the petiole length L can be calculated from the leaf area A by using the above formulas (1) to (4).

[0038] On the other hand, when calculating leaf area A from petiole length L, k can be obtained by substituting petiole length L into the following equation (5), and leaf area A can be obtained by substituting k and other values ​​into equation (6). Note that equation (6) is obtained from equations (1) and (3) above.

[0039]

number

[0040] (Approximate formula 2) Next, we will explain Approximation Formula 2. Approximation Formula 2 is calculated based on the actual measurement values ​​of Fig. 8(a) to Fig. 8(e) by calculating the wind speed v and the horizontal projection movement amount M P This is an example in which the relationship between is approximated as shown by the solid lines in FIGS. 9(a) to 9(e).

[0041] FIG. 10 is a schematic diagram for explaining approximation formula 2. In FIG. 10, the state of the entire leaf in the first state (when the wind speed is 0 m or the first wind speed (weak wind)) is shown by a thick solid line, and the state of the entire leaf in the second state (when the second wind speed (medium wind or strong wind)) is shown by a thick dashed line. In FIG. 10, the petiole angle (the angle between the direction in which the petiole extends and the horizontal plane) in the first state is θ (rad), and the petiole angle in the second state is δ (rad). Note that θ can be said to be the value of the petiole angle when the leaf is attached. Furthermore, the actual movement amount of the petiole tip between the first state and the second state is M (mm), and the horizontally projected movement amount is M P (mm).

[0042] Based on the knowledge that the wind load on the leaf blade is proportional to the square of the wind speed (v) and the pressure-receiving area (A), we decided to use the following equation (7), where l is the proportionality coefficient.

[0043]

number

[0044] In addition, we assumed that when the petiole falls outward from the base of the plant, the tip of the petiole moves in a circular motion around the base, and defined the inclination angle δ of the petiole with respect to the medium (horizontal surface) when subjected to wind pressure as follows:

[0045]

number

[0046] Furthermore, the formula for converting the length of petiole inclination into a projection value was defined as follows: (9)

[0047]

number

[0048] In this approximation formula 2, the petiole length L can be calculated from the leaf area A by using the above formulas (7) to (9).

[0049] (Approximate formula 3) Next, we will explain Approximation Formula 3. Approximation Formula 3 is calculated based on the actual measurement values ​​of Fig. 8(a) to Fig. 8(e) by calculating the wind speed v and the horizontal projection movement amount M P This is an example of the case where the relationship between the angle δ and the leaf stem is approximated as shown by the thin solid lines in Figures 9(a) to 9(e). Figure 11 is a schematic diagram for explaining approximation formula 3. Approximation formula 3 expresses that the angle δ returns from the state shown by the dashed line due to bending of the petiole, as shown in Figure 11.

[0050] Specifically, the following equation (10) was used.

[0051]

number

[0052] In addition, (θ-M / L) in the above equation (10) means almost the same as the right side of the above equation (8). Also, p is the advance angle coefficient, and (v 4 / p) expresses the fact that the angle returns to its original position due to the bending of the petiole.

[0053] In this approximation formula 3, the petiole length L can be calculated from the leaf area A by using formulas (7), (9), and (10).

[0054] (Functions of the control device 30 and the information processing device 90) Returning to FIG. 3, the functions of the control device 30 and the information processing device 90 will be described.

[0055] (Functions of the control device 30) As shown in FIG. 3, the control device 30 has the functions of a position control unit 40, a wind speed control unit 41, an imaging control unit 42, and an image acquisition and transmission unit 44.

[0056] The position control unit 40 controls the motor 28 based on the detection result of the position detection device 29 to adjust the position of the monitoring device 10 (the shooting location of the camera 24). For example, the position control unit 40 moves the monitoring device 10 so that each plant (a range including the center of at least one plant and the target leaf for which the petiole length L and leaf area A of that plant are to be calculated) sequentially falls within the shooting range of the camera 24.

[0057] When an area including the center of at least one plant and the target leaves for which the petiole length L and leaf area A of the plant are to be calculated falls within the imaging range of the camera 24, the wind speed control unit 41 controls the blower 26 to put the plant into a first state. Here, the first state is a state in which no wind is blown from the blower 26. However, the first state may also be a state in which the blower 26 is blowing wind at a first speed (weak wind). Alternatively, when an area including the center of at least one plant and the target leaves for which the petiole length L and leaf area A of the plant are to be calculated falls within the imaging range of the camera 24, the wind speed control unit 41 controls the blower 26 to put the plant into a second state. Here, the second state is a state in which the blower 26 is blowing wind at a second speed (moderate wind or strong wind) that is faster than the first speed.

[0058] The photography control unit 42 photographs the plant in the first state from above using the camera 24. The photography control unit 42 also photographs the plant in the second state from above using the camera 24. After photographing the plant in the first state and the plant in the second state, the photography control unit 42 notifies the position control unit 40 of this. Upon receiving this notification, the position control unit 40 adjusts the position of the monitoring device 10 so that the next plant falls within the photography range of the camera 24.

[0059] The image acquisition / transmission unit 44 acquires the image captured by the camera 24 and transmits the acquired image to the information processing device 90 together with the position information (XY position) at the time of image capture obtained from the position control unit 40 and the wind speed information at the time of image capture obtained from the wind speed control unit 41.

[0060] (Functions of information processing device 90) As shown in FIG. 3, the information processing device 90 includes an image acquisition unit 52, an image analysis unit 54, a calculation unit 56, a data management unit 58, and an output unit 60.

[0061] The image acquisition unit 52 acquires an image of the stock taken in a first state (first image), information on the shooting location of the first image and information on the wind speed, an image of the stock taken in a second state (second image), and information on the shooting location of the second image and information on the wind speed.

[0062] The image analysis unit 54 analyzes the first and second images captured by the image acquisition unit 52 and passes the analysis results to the calculation unit 56.

[0063] The calculation unit 56 calculates the petiole length and leaf area of ​​the strawberry based on the analysis results of the image analysis unit 54, predetermined coefficients, and information input by the user.

[0064] The data management unit 58 acquires the calculation results (petiole length and leaf area) from the calculation unit 56 and manages them in association with the location information of the plant.

[0065] The output unit 60 outputs the information managed by the data management unit 58 .

[0066] (Regarding the processing of the control device 30) FIG. 12 is a flowchart showing the processing of the control device 30.

[0067] 12 starts, first, in step S10, the position control unit 40 controls the motor 28 based on the detection result of the position detection device 29, and moves the monitoring device 10 to a position where it can photograph the strawberry plant (an area including at least the center of the plant and the leaves to be measured) using the camera 24. Note that the approximate position of each plant on the cultivation bed 12 is assumed to be known in advance.

[0068] Next, in step S12, the wind speed control unit 41 turns off the blower 26 or controls the wind speed from the blower 26 to be a first wind speed (weak wind) (i.e., so that the stumps are in a first state). Here, the first state is assumed to be a windless state. When step S12 is performed, if the blower 26 is in the OFF state, the wind speed control unit 41 does not control the blower 26 and maintains the state as it is.

[0069] Next, in step S14, the photography control unit 42 uses the camera 24 to photograph the strawberry plant from above.

[0070] In step S16, the image acquisition / transmission unit 44 acquires the image (first image) taken in step S14 from the camera 24, and transmits the first image, information on the position when the first image was taken, which is obtained from the position control unit 40, and information on the wind speed of the blower 26 when the first image was taken, which is obtained from the wind speed control unit 41, to the information processing device 90.

[0071] Next, in step S18, the wind speed control section 41 controls the speed of the wind from the blower 26 to be a second wind speed (medium wind or strong wind) (that is, so that the stalks are in a second state).

[0072] Next, in step S20, the photography control unit 42 uses the camera 24 to photograph the strawberry plant from above.

[0073] Next, in step S22, the image acquisition / transmission unit 44 acquires the image (second image) taken in step S20 from the camera 24, and transmits the second image, information on the position when the second image was taken, which is obtained from the position control unit 40, and information on the wind speed of the blower 26 when the second image was taken, which is obtained from the wind speed control unit 41, to the information processing device 90.

[0074] Thereafter, the process returns to step S10. Note that the wind speed control unit 41 may stop the operation of the blower 26 before returning to step S10. When the process returns to step S10, the position control unit 40 moves to photograph the next stalk. Then, in steps S12 to S22, the control device 30 performs processes such as photographing the first and second images of the next stalk.

[0075] Although details will be described later, steps S18, S20, and S22 in FIG. 12 may be repeated multiple times with different second wind speeds.

[0076] 12, the monitoring device 10 is moved (S10), a first image is captured at a first wind speed (S12, S14), a second image is captured at a second wind speed (S18, S20), and the monitoring device 10 is moved again (S10). However, the present invention is not limited to this. For example, the monitoring device 10 may be moved with the blower 26 turned off or maintained at a first wind speed, and images of multiple plants (first images) may be captured. Thereafter, the monitoring device 10 may be moved with the blower 26 maintained at a second wind speed, and images of multiple plants (second images) may be captured. The image acquisition / transmission unit 44 may then associate the first and second images captured at the same location (i.e., the first and second images of the same plant) and transmit them to the information processing device 90 together with location information and wind speed information.

[0077] (Regarding the processing of the information processing device 90) Next, the processing of the information processing device 90 will be described with reference to the flowchart of FIG.

[0078] When the processing of FIG. 13 starts, first in step S30, the image acquisition unit 52 acquires the image, position information, and wind speed information transmitted from the image acquisition / transmission unit 44 of the control device 30.

[0079] Next, in step S32, the image analysis unit 54 detects the center of the plant from either the image in the first state or the image in the second state. Specifically, the image analysis unit 54 performs deep learning using, for example, a large number of images showing the shape and center of the plant as training data, and can detect the center of the plant present in the acquired image (first image or second image) using the resulting training model. The image analysis unit 54 can also detect the center of the plant using a method other than deep learning. For example, as shown in FIG. 14(a), the image analysis unit 54 can detect multiple petioles from the second image and detect the intersection of their extensions as the center of the plant. By using this method, the center of the plant can be accurately detected even when it is hidden by leaves of other plants, as shown in FIG. 14(b). Furthermore, as shown in Figure 14(c), strawberries typically have three leaflets connected to one petiole, so the image analysis unit 54 may, for example, perform a process of drawing a straight line (see the dashed line in Figure 14(c)) multiple times in the longitudinal direction of the middle leaflet out of the three leaflets, and determine the intersection of these lines as the center of the plant. Note that the image analysis unit 54 may preferentially perform detection of the plant center using deep learning, and if the plant center cannot be detected by deep learning, may detect the plant center using the method of Figure 14(a), Figure 14(b), or Figure 14(c).

[0080] Next, in step S34, the image analysis unit 54 detects petioles from the image in the first state. Here, the image analysis unit 54 detects leaf blades in the order of the first leaf, second leaf, third leaf, and so on, and detects the petioles of each leaf. For example, the image analysis unit 54 performs deep learning using a large amount of learning data correlating leaf images with the degree of unexpandedness to obtain a learning model. Then, the image analysis unit 54 uses the obtained learning model to detect unexpanded leaves of the plant from within the image and designate the detected unexpanded leaf as the first leaf. Alternatively, the image analysis unit 54 may detect leaves from the image and determine the smallest leaf among the detected leaves as the first leaf. Alternatively, the image analysis unit 54 may detect leaves from the image and determine the lightest leaf among the detected leaves as the first leaf. Furthermore, the image analysis unit 54 may detect leaves from the image and designate the leaf closest to the center of the plant among the detected leaves as the first leaf. The image analysis unit 54 may preferentially apply a method for identifying the first leaf using deep learning, and if the first leaf cannot be identified using this method, it may identify the first leaf using one of the other methods described above. Alternatively, the image analysis unit 54 may identify the first leaf using multiple of the methods described above and combine the results of the multiple identifications to identify the first leaf. Furthermore, the image analysis unit 54 detects the angle of the line connecting the first leaf and the center of the plant relative to a reference line, and identifies the leaf located approximately 144° from the center of the plant relative to the direction of the first leaf's growth as the second leaf, the leaf located approximately 144° from the center of the plant relative to the direction of the second leaf's growth (approximately 288° from the direction of the first leaf's growth) as the third leaf, and so on. The image analysis unit 54 then detects the petioles connected to the identified first leaf, second leaf, and so on. Deep learning may be used as a method for detecting petioles. However, the present invention is not limited to this, and the connecting point between the petiole and one leaf blade may be detected, or the position where the longitudinal directions of three leaf blades intersect may be regarded as the petiole.

[0081] Next, in step S36, the image analysis unit 54 uses the first image to measure the distance Dn from the center of the plant to the petiole (see FIG. 6(a)). Note that the distance Dn may be measured for each of the first leaf, second leaf, ..., or may be measured only for a specified leaf that requires measurement.

[0082] Next, in step S38, the image analysis unit 54 detects petioles from the image in the second state. The process in step S38 is the same as that in step S34 described above.

[0083] Next, in step S40, the image analysis unit 54 uses the second image to measure the distance Dm (see FIG. 6(a)) from the center of the plant to the petiole. Note that the distance Dm may be measured for each of the first leaf, second leaf, ..., or may be measured only for a specified leaf that requires measurement.

[0084] Next, in step S42, the calculation unit 56 calculates the horizontal projection movement amount M P Specifically, the horizontal projection movement amount M P Calculate. M P =Dm-Dn …(11)

[0085] Next, in step S44, the calculation unit 56 uses one of the approximation formulas 1 to 3 to calculate the petiole length L of each leaf.

[0086] Next, in step S46, the data management unit 58 manages the petiole length L in association with the position.

[0087] Next, in step S48, the output unit 60 determines whether an output request has been input. If the determination in step S48 is negative, the process returns to step S30, but if the determination is affirmative, the process proceeds to step S50.

[0088] When the process proceeds to step S50, the output unit 60 outputs the data for which an output request has been made. After that, the process returns to step S30, and the above processing is repeated.

[0089] (Example) Next, the processing in step S44 in FIG. 13 will be described using an example.

[0090] (Example of calculating petiole length L using approximation formula 1) 15(a) to 15(g) are diagrams for explaining an example of determining the petiole length L using approximation formula 1. In this example, as described above, the following formulas (1) to (4) are used.

[0091]

number

[0092] 15(a) shows an example of coefficients m, n, a, and b that need to be calculated in advance when using approximate formula 1. m and n are coefficients of formula (2), and a and b are coefficients of formula (4).

[0093] 15(b) shows an example of a wind speed (first wind speed) v1 in a first state and a wind speed (second wind speed) v2 in a second state. The difference between the second wind speed v2 and the first wind speed v1 is v in equations (1) and (2).

[0094] Fig. 15(c) shows an example of the distance D1 from the center of the plant to the petiole obtained from the first image, and the distance D2 from the center of the plant to the petiole obtained from the second image. Fig. 15(d) shows the horizontal projection shift M obtained from the difference between the distance D2 and the distance D1 in Fig. 15(c) (see equation (11) above). P The values ​​shown are in pixels. The values ​​shown in units of pixels represent the dimensions in the image, and the values ​​shown in units of mm represent the actual dimensions converted from the distance between the camera and the culture medium.

[0095] The calculation unit 56 calculates the drag coefficient Cd by substituting the coefficients m (= 1.34) and n (= -0.34) in Figure 15(a) and the value v (= v2 - v1 = 1.5 m / s) calculated from Figure 15(b) into equation (2). The value of Cd calculated by this calculation is shown in Figure 15(e).

[0096] Next, the calculation unit 56 calculates the leaf area A value input by the user (see FIG. 15(f)) and the air density ρ (= 1.2 kg / m 3 ), and the values ​​of v and Cd are substituted into equation (1) to calculate the wind load P on the leaf. Figure 15(g) shows the calculation results of the wind load P.

[0097] Next, the calculation unit 56 adds M P The coefficient k is calculated by substituting the values ​​of P (Fig. 15(g)) and P (Fig. 15(d)). The calculation result of the coefficient k is shown in Fig. 15(g).

[0098] Furthermore, the calculation unit 56 calculates the petiole length L by substituting the coefficient k and the coefficients a and b (see Figure 15(a)) into equation (4). Figure 15(g) shows the calculation results for the petiole length L. In the example of Figures 15(a) to 15(g), the petiole length L is calculated to be 139.394 mm, and it was found that a value close to the true value of 135 mm could be derived.

[0099] Figures 16(a) to 16(g) show the calculation results when the second wind speed v2 was set to 2.3 m / s, 3.0 m / s, and 3.8 m / s. When the second wind speed v2 was set to 2.3 m / s, the petiole length L was calculated to be 126.727 mm. When the second wind speed v2 was set to 3.0 m / s, the petiole length L was calculated to be 110.614 mm. When the second wind speed v2 was set to 3.8 m / s, the petiole length L was calculated to be 142.059 mm. These calculated petiole lengths L were close to the true value of 135 mm.

[0100] (Example of calculating leaf area A using approximation formula 1) 17(a) to 17(g) are diagrams for explaining an example of finding the leaf area A using approximate formula 1. In this example, as described above, the following formulas (5) and (6) are used.

[0101]

number

[0102] 17(a) to 17(e) are similar to the cases where petiole length L is determined (FIGS. 15(a) to 15(e)).

[0103] The calculation unit 56 calculates the coefficient k by substituting the value of the petiole length L input by the user (see FIG. 17(f)) and the coefficients a and b (see FIG. 17(a)) into equation (5). FIG. 17(g) shows the calculation result of the coefficient k.

[0104] Furthermore, the calculation unit 56 calculates the calculated coefficient k, the air density ρ (= 1.2 kg / m 3 ), v, Cd (Fig. 17(e)), M P (Fig. 17(d)) ​​is substituted into equation (6) to calculate the leaf area A. Fig. 17(g) shows the calculation result of the leaf area A. In the example of Fig. 17(a) to Fig. 17(g), the leaf area A = 7730.57 mm 2 The true value is calculated as 7484 mm. 2 It was found that it is possible to derive a value that is close to

[0105] 18(a) to 18(g) show the calculation results when the second wind speed v2 is set to 2.3 m / s, 3.0 m / s, and 3.8 m / s. When the second wind speed v2 is set to 2.3 m / s, the leaf area A is 7019.76 mm 2 In addition, when the second wind speed v2 is set to 3.0 m / s, the leaf area A is 6115.64 mm 2 In addition, if the second wind speed v2 is 3.8 m / s, the leaf area A is 7880.08 mm 2 The calculated leaf area A is 7484 mm 2 It was found that the value was close to

[0106] (Example of calculating petiole length L using approximation formula 2) 19(a) to 19(g) are diagrams for explaining an example of determining the petiole length L using approximation formula 2. In this example, as described above, the following formulas (7) to (9) are used.

[0107]

number

[0108] FIG. 19(a) shows an example of a coefficient l that needs to be calculated in advance when using approximation formula 2. l is a coefficient in formula (7). FIG. 19(b) shows an example of a value of cos θ calculated from the leaf area A, petiole angle θ (see FIG. 10), and θ, which are values ​​input by the user. The petiole angle θ may be measured in advance, or if the relationship between the leaf position and the petiole angle θ is known in advance, the petiole angle θ may be calculated using that relationship.

[0109] 19(c) shows an example of wind speed (first wind speed) v1 in the first state and wind speeds (second wind speeds) v2 to v8 in the second state. The difference between these second wind speeds v2 to v8 and the first wind speed v1 is v in equation (7). In this embodiment, two or more second images (seven in FIG. 19(c)) are taken while varying the speed of the wind blown from blower 26.

[0110] FIG. 19(d) shows the horizontal projection movement amount M calculated from the image captured in the first state (wind speed v1) and the images captured in the second state (wind speeds v2 to v8). P The values ​​of M P 1 is the difference between the distance D2 from the center of the plant to the petiole obtained at wind speed v2 and the distance D1 from the center of the plant to the petiole obtained at wind speed v1. P 2 is the difference between the distance D2 from the center of the plant to the petiole obtained at wind speed v3 and the distance D1 from the center of the plant to the petiole obtained at wind speed v1. M P 3~M P7. In this way, in this embodiment, for each combination of the first image and the second image, the horizontal projection shift amount M P Calculate.

[0111] The calculation unit 56 calculates the amount of movement M of the petiole at each of the wind speeds v2 to v8 by substituting the value of the leaf area A in Figure 19(b), one of the wind speeds v2 to v8, and the value of the coefficient l in Figure 19(a) into equation (7). Figure 15(e) shows values ​​M1 to M7 of the amount of movement M of the petiole (see Figure 10) calculated from the image captured in the first state (wind speed v1) and the image captured in the second state (wind speeds v2 to v8).

[0112] The calculation unit 56 substitutes a provisional value for the value L (petiole length) in the formulas (8) and (9), and calculates the tilt angle δ of the petiole when subjected to wind pressure and the horizontal projection movement amount M P ' is calculated (see Fig. 19(f) and Fig. 19(g)). Then, the horizontal projection movement amount M P ' (Fig. 19(g)) and the horizontal projection movement amount M P The value of the petiole length L is optimized so that the slope of the relationship between the petiole length L and the horizontal projection distance M (Fig. 19(d)) approaches 1. In the example of Fig. 19(g), when the petiole length L is set to 132.286 mm, the horizontal projection distance M P 1'~M P 4' and horizontal projection movement amount M P 1~M P It was found that the slope of the relationship with 4 approached 1. Therefore, in this example, the petiole length L was set to 132.286 mm. This value was close to the true value of 161.3158 mm.

[0113] In the examples of FIGS. 19(a) to 19(g), a plurality of horizontal projection movement amounts M P and M P Although the case of optimizing the petiole length L from the horizontal projection distance M has been described, the present invention is not limited to this. For example, by expanding cosine using a Fourier transform or the like, one horizontal projection shift amount M P The petiole length L may be calculated using the following formula:

[0114] (Example of calculating petiole length L using approximation formula 3) Figures 20(a) to 20(g) are diagrams for explaining an example of calculating the petiole length L using approximation formula 3. Note that this example is a value for the third leaf of a strawberry plant. In this example, as described above, the following formulas (7), (9), and (10) are used.

[0115]

number

[0116] Fig. 20(a) shows an example of coefficients l and p (advance angle coefficients) that need to be calculated in advance when using approximate formula 3. l is a coefficient in formula (7), and p is a coefficient in formula (10).

[0117] 20(b) shows an example of the value of cos θ calculated from the leaf area A, petiole angle θ (see FIG. 11), and θ, which are values ​​input by the user. The petiole angle θ may be measured in advance, or if the relationship between the leaf position and the petiole angle θ is known in advance, the petiole angle θ may be calculated using that relationship.

[0118] 20(c) shows an example of wind speed (first wind speed) v1 in the first state and wind speeds (second wind speeds) v2 to v8 in the second state. The difference between these second wind speeds v2 to v8 and the first wind speed v1 is v in equations (7) and (10). In this embodiment, two or more second images (seven in FIG. 20(c)) are taken while varying the speed of the wind blown from blower 26.

[0119] FIG. 20(d) shows the horizontal projection movement amount M calculated from the image captured in the first state (wind speed v1) and the images captured in the second state (wind speeds v2 to v8). P The values ​​of M P 1 is the difference between the distance D2 from the center of the plant to the petiole obtained at wind speed v2 and the distance D1 from the center of the plant to the petiole obtained at wind speed v1. P2 is the difference between the distance D2 from the center of the plant to the petiole obtained at wind speed v3 and the distance D1 from the center of the plant to the petiole obtained at wind speed v1. M P 3~M P 7. In this way, in this embodiment, for each combination of the first image and the second image, the horizontal projection shift amount M P Calculate.

[0120] The calculation unit 56 calculates the amount of movement M of the petiole at each of the wind speeds v2 to v8 by substituting the value of the leaf area A in Figure 20(b), one of the wind speeds v2 to v8, and the value of the coefficient l in Figure 20(a) into equation (7). Figure 20(e) shows values ​​M1 to M7 of the amount of movement M of the petiole (see Figure 11) calculated from the image captured in the first state (wind speed v1) and the image captured in the second state (wind speeds v2 to v8).

[0121] The calculation unit 56 also assigns a temporary value to the value L (petiole length) in equations (9) and (10), and calculates the tilt angle δ of the petiole when subjected to wind pressure and the horizontal projection movement amount M P ' is calculated (see Fig. 20(f) and Fig. 20(g)). Then, the horizontal projection movement amount M P ' (Fig. 20(g)) and the horizontal projection movement amount M P The value of the petiole length L is optimized so that the slope of the relationship between the petiole length L and the horizontal projection distance M (Fig. 20(d)) approaches 1. In the example of Fig. 20(g), when the petiole length L is set to 163.2854362 mm, P 1'~M P 7' and horizontal projection movement amount M P 1~M P It was found that the slope of the relationship with 7 approached 1. Therefore, in this example, the petiole length L was set to 163.2854362 mm. This value was close to the true value of 161.3158 mm.

[0122] In the examples of FIGS. 20(a) to 20(g), a plurality of horizontal projection movement amounts M P and M P Although the case of optimizing the petiole length L from the horizontal projection distance M has been described, the present invention is not limited to this. For example, by expanding cosine using a Fourier transform or the like, one horizontal projection shift amount MP The petiole length L may be calculated using the following formula:

[0123] As described above in detail, according to this embodiment, the control device 30 controls the camera 24 to take a first image of the plant from above in a first state in which no wind is blown from the blower 26 onto the plant or in which wind is blown from the blower 26 at a first speed, and controls the camera 24 to take a second image of the plant from above in a second state in which wind is blown from the blower 26 at a second speed (FIG. 12). Furthermore, the information processing device 90 calculates the value of the petiole length of a predetermined leaf of the strawberry plant from the image taken by the camera 24. In this calculation, the information processing device 90 calculates the difference (horizontal projection movement amount M) between the distance Dn from the center of the plant to the petiole tip of the predetermined leaf in the first image and the second distance Dm from the center of the plant to the petiole tip of the predetermined leaf in the second image. P ) is calculated. The information processing device 90 then calculates the leaf area value of a predetermined leaf, the speed of the wind blown onto the stump from the blower 26 in each of the first state and the second state, and the horizontal projection movement amount M P Based on this, the petiole length value of a given leaf is calculated. As a result, even if the petiole is located below the leaf blade (leaflet) and is difficult to see from the outside, the petiole length can be calculated accurately by using the first and second images. Figure 21(a) shows a graph of the relationship between the actually measured petiole length and the petiole length calculated using approximation formula 1 (predicted petiole length). The relationship between the actually measured petiole length (x-axis) and the predicted petiole length (y-axis) can be approximated as y = 1.00x, and R 2 is as high as 0.95, it can be seen that this embodiment can calculate the petiole length with high accuracy.

[0124] In this embodiment, the information processing device 90 also calculates the value of the petiole length of a predetermined leaf, the speed of the wind blown onto the plant from the blower 26 in each of the first state and the second state, and the horizontal projection movement amount M PBased on this, the value of the leaf area of ​​a predetermined leaf is calculated. As a result, even when leaf blades overlap, the leaf area can be calculated with high accuracy by using the first and second images. Figure 21(b) shows a graph of the relationship between the actually measured leaf area and the leaf area calculated using approximation formula 1 (predicted leaf area). The relationship between the actually measured leaf area (x-axis) and the predicted leaf area (y-axis) can be approximated as y = 1.02x, and R 2 is as high as 0.87, it can be seen that this embodiment can calculate the leaf area with high accuracy.

[0125] In this embodiment, the information processing device 90 uses approximation formulas 2 and 3 to calculate the leaf area A, the petiole angle θ at the time of leaf attachment, the wind speed at the time of photographing, and the horizontal projection movement amount M P The petiole length L is calculated based on the above formula: In this way, the petiole length can be calculated with high accuracy.

[0126] 19 and 20, the control device 30 captures two or more second images while varying the speed of the air blown from the blower 26. Then, the information processing device 90 calculates the horizontal projection movement amount M P Calculate the horizontal projection movement amount M P Using these and the wind speed at the time of capturing each image, the petiole length L is calculated. In this way, the petiole length L can be calculated with high accuracy.

[0127] The wind speed distribution of the blower 26 is strongest directly below the blower 26 and becomes weaker at positions farther away from the blower 26. Therefore, when the monitoring device 10 is moved while the blower 26 is maintained in a constant state, the plants transition from a state in which they are exposed to weak wind (first state) to a state in which they are exposed to strong wind (second state). Therefore, in the above embodiment, images may be taken when the plants are located away from directly below the blower 26 (first state) and when they are located close to directly below the blower 26 (second state), and the processing of FIG. 13 may be performed using each image. In this case, the monitoring device 10 may have a first camera disposed away from the blower 26 in the X-axis direction to photograph the plants in the first state, and a second camera disposed near the blower 26 to photograph the plants in the second state. Alternatively, a single camera may be used to photograph the stock in the first state and the stock in the second state at different angles, and the images may be corrected to appear as if they were photographed from the same angle before being used in the processing shown in FIG. 13 above.

[0128] In the above embodiment, the monitoring device 10 moves on the ground, but this is not limiting. For example, an unmanned aerial vehicle (drone or multicopter) equipped with a camera 24, blower 26, position detection device 29, and control device 30 may be used as the monitoring device. In this case, the strength of the wind blowing on the plants can be adjusted by adjusting the height of the unmanned aerial vehicle and the relative positions of the unmanned aerial vehicle and the observation target. In this case, the blower 26 may be omitted. Note that adjusting the height of the unmanned aerial vehicle and the relative positions of the unmanned aerial vehicle and the observation target changes the relative positions of the camera 24 and the observation target, which changes the shooting distance and shooting angle. However, the above processing can be performed taking into account the relative positions of the unmanned aerial vehicle and the observation target.

[0129] The above-described embodiment is a preferred example of the present invention, but the present invention is not limited to this and can be modified in various ways without departing from the spirit of the present invention. [Explanation of symbols]

[0130] 24 Camera (photographic device) 26 Blower (spraying device) 30 Control device 90 Information processing device (calculating device) 100 Information Acquisition System

Claims

1. A blowing device that blows air onto strawberry plants from above, a photographing device for photographing an image of the stock from above; a control device that controls the photographing device to photograph a first image of the plant from above in a first state where no wind is blown from the blowing device to the plant or where wind is blown from the blowing device at a first speed, and that controls the photographing device to photograph a second image of the plant from above in a second state where wind is blown from the blowing device at a second speed; a calculation device that calculates one of the leaf area and the petiole length of a predetermined leaf of the strawberry plant from the image captured by the photographing device; Equipped with The calculation device determining a difference between a first distance from the center of the plant to the tip of the petiole of the specified leaf in the first image and a second distance from the center of the plant to the tip of the petiole of the specified leaf in the second image; calculating one of the leaf area and the petiole length of the specified leaf based on the other of the leaf area and the petiole length of the specified leaf, the speed of the wind blown from the blowing device to the plant in each of the first state and the second state, and the difference between the first distance and the second distance; A strawberry plant information acquisition system characterized by:

2. The calculation device The strawberry plant information acquisition system of claim 1, wherein in the calculation process, the value of the petiole length is calculated based on the value of the leaf area, the value of the petiole angle at the time of attachment of the specified leaf, the speed of the wind blown from the spraying device to the plant in each of the first state and the second state, and the difference between the first distance and the second distance.

3. the control device executes the process of capturing the second image a plurality of times while varying the speed of the air blown by the blowing device; The calculation device In the process of specifying, a plurality of differences between the first distance and the second distance are specified corresponding to the processes of capturing the second images a plurality of times; The strawberry plant information acquisition system of claim 2, wherein in the calculation process, the value of the petiole length is calculated based on the identified multiple differences and the wind speed when the process of taking the second images multiple times is performed.

4. The calculation device The strawberry plant information acquisition system according to any one of claims 1 to 3, wherein two or more petioles are identified from at least one of the first image and the second image, and the position where the identified two or more petioles intersect is detected as the position of the center of the plant; or two or more petioles are identified from at least one of the first image and the second image, and straight lines indicating the longitudinal direction of the central leaflet of each of the three leaflets connected to the two or more identified petioles are identified, and the intersection of the identified multiple straight lines is detected as the position of the center of the plant; or the position of the center of the plant is detected from at least one of the first image and the second image by deep learning.

Images