Strawberry plant information acquisition system

Abstract

To accurately calculate a petiole length or the leaf area of each strawberry plant.SOLUTION: A control device 30 controls a camera 24 in a first state of not blowing from a blower 26 to a plant or blowing from the blower 26 at a first speed, and captures a first image of a plant from above, and controls the camera 24 in a second state of blowing from the blower 26 at a second speed, and captures a second image of the plant from above. At this point, the second speed is a speed at which a movement amount of leaves becomes a predetermined speed (speed at which leaves barely move) when the blowing speed from the blower 26 is made higher. Further, an information processor 90 obtains a difference (MP) between distances from the plant center to a petiole tip of a predetermined leaf in the first image and the second image. Then, the information processor 90 obtains a petiole length L by using petiole angles θ, δ in first and second states and MP.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present invention relates to a strawberry plant information acquisition system. [Background technology]

[0002] Image measurement is known as an effective method for measuring the biometrics of plants, as it is non-destructive, allows for wide-area measurement, and is labor-saving. When the plant being measured is a strawberry, the measurement targets include petiole length and leaf area. These measurement targets are important factors in growth diagnosis and yield prediction, so it is desirable to be able to measure them accurately.

[0003] Conventionally, there is known a technique for determining the growth state of a plant (stem thickness) by capturing an image of the plant using a photographing unit while a fan is blowing air onto the plant and processing the image (see, for example, Patent Document 1). There is also known a technique for monitoring the growth state of a plant (such as harvest time) based on the time-series transition of images (see, for example, Patent Document 2). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-29940 [Patent Document 2] Japanese Patent Application Publication No. 2019-37225 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the above-mentioned petiole length and leaf area cannot be measured even using the above-mentioned Patent Documents 1 and 2. For example, since the petiole is located below the leaf blade (leaflet) and is difficult to see from the outside, it is difficult to measure the petiole length using an image. Furthermore, since leaf blades often overlap each other, it is also difficult to measure the leaf area with high accuracy.

[0006] Therefore, an object of the present invention is to provide a strawberry plant information acquisition system that can accurately calculate strawberry plant information. [Means for solving the problem]

[0007] The strawberry plant information acquisition system of the present invention includes a blowing device that blows wind onto strawberry plants from above, a photographing device that takes images of the plants from above, a control device that controls the photographing device to take a first image of the plants from above in a first state where no wind is blown onto the plants from the blowing device or where wind is blown onto the plants from the blowing device at a first speed, and controls the photographing device to take a second image of the plants from above in a second state where wind is blown onto the plants from the blowing device at a second speed, and a control device that determines the petiole length of a predetermined leaf of the strawberry plant from the images taken by the photographing device. The second speed is a speed at which the movement amount of the specified leaf becomes less than a predetermined amount when the speed of the wind blown from the blowing device is increased, and the calculation device determines the difference between a first distance from the center of the plant to the petiole tip of the specified leaf in the first image and a second distance from the center of the plant to the petiole tip of the specified leaf in the second image, and calculates the value of the petiole length of the specified leaf based on the value of the petiole angle of the specified leaf in each of the first state and the second state, and the difference between the first distance and the second distance. [Effects of the Invention]

[0008] The strawberry plant information acquisition system of the present invention has the effect of being able to accurately calculate strawberry plant information. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram schematically illustrating a configuration of an information acquisition system according to a first embodiment. [Figure 2] FIG. 2 is a diagram illustrating a hardware configuration of a control device and an information processing device. [Figure 3]FIG. 2 is a functional block diagram of a control device and an information processing device according to the first embodiment. [Figure 4] FIG. 1 is a diagram showing an experimental setup. [Figure 5] Figure 5(a) is an image of a strawberry plant taken by a camera in the experimental apparatus of Figure 4 when the blower was not operating and the wind speed was 0 m, and Figure 5(b) is an image of a strawberry plant taken by a camera when wind was blown from the blower at a specified wind speed. [Figure 6] FIG. 6(a) is a diagram schematically showing the state of the petiole and leaf blade in the first and second states, and FIG. 6(b) is a diagram for explaining the position of the tip of the petiole. [Figure 7] Figures 7(a) to 7(e) are graphs showing the results of actual measurements of the horizontal projection movement Mp of the petiole of each of the first to fifth leaves of a strawberry plant against wind speed v, obtained using the experimental apparatus of Figure 4. [Figure 8] 8(a) to 8(e) are diagrams showing examples in which the actual measurement values ​​of FIGS. 7(a) to 7(e) are approximated. [Figure 9] FIG. 1 is a schematic diagram (part 1) for explaining an approximate formula. [Figure 10] FIG. 2 is a schematic diagram (part 2) for explaining an approximation formula. [Figure 11] 11(a) to 11(c) are diagrams for explaining a method for detecting the center of a stock. [Figure 12] 5 is a flowchart showing processing by the control device according to the first embodiment. [Figure 13] 4 is a flowchart showing processing performed by the information processing device according to the first embodiment. [Figure 14] 14(a) to 14(c) are diagrams for explaining the processing of the pre-processing unit. [Figure 15] 15(a) to 15(e) are diagrams for explaining an example of determining the petiole length L and the leaf area A using approximate formulas. [Figure 16] FIG. 10 is a functional block diagram of a control device and an information processing device according to a second embodiment. [Figure 17]10 is a flowchart showing the processing of a control device according to a second embodiment. [Figure 18] 10 is a flowchart showing processing performed by an information processing device according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] First Embodiment The strawberry plant information acquisition system according to the first embodiment will be described in detail below.

[0011] FIG. 1 shows a schematic configuration of an information acquisition system 100 according to a first embodiment. In this embodiment, the information acquisition system 100 is a system that calculates and records the petiole length and leaf area of ​​each strawberry plant cultivated in an elevated cultivation bed 12 as shown in FIG. 1. The distance (height) from the ground to the upper surface of the cultivation bed 12 is H1 (H1 is, for example, 1000 mm). The strawberry plants do not have to be cultivated in the cultivation bed 12, but may be planted in the ground in a ridge, for example.

[0012] The information acquisition system 100 includes a monitoring device 10 and an information processing device 90 as a calculation device. The monitoring device 10 moves near a cultivation bed 12 and acquires images of strawberry plants photographed from above. The information processing device 90 is communicably connected to the monitoring device 10 and calculates the petiole length and leaf area of ​​each strawberry plant planted in the cultivation bed 12 based on the images and position information acquired from the monitoring device 10. The information processing device 90 and the monitoring device 10 may be connected via a wired LAN (Local Area Network) or the like, or wirelessly via Wi-Fi or the like. The information processing device 90 may also be mounted on the monitoring device 10. In this embodiment, the extension direction of the cultivation bed 12 is defined as the X-axis direction, the direction perpendicular to the X-axis direction in a horizontal plane is defined as the Y-axis direction, and the vertical direction is defined as the Z-axis direction.

[0013] 1, the monitoring device 10 is installed on a rail 14 laid along the X-axis direction below (on the ground) the cultivation bed 12. The monitoring device 10 is movable along the rail 14 in the X-axis direction.

[0014] The monitoring device 10 includes a housing 20, wheels 22, a camera 24 as a photographing device, a blower 26 as a spraying device, a motor 28, a position detection device 29, and a control device 30. The housing 20 has a substantially rectangular frame shape. A ceiling panel 20a of the housing 20 extends above the cultivation beds 12 and the strawberry plants, and the lower surface (-Z surface) of the ceiling panel 20a and the strawberry plants are vertically opposed to each other.

[0015] The wheels 22 are driven to rotate on the rails 14 by a motor 28. The rotation of the motor 28 is controlled by a control device 30. That is, in this embodiment, a configuration including the wheels 22 that run on the rails 14 and the motor 28 is adopted as a moving mechanism for moving the monitoring device 10. However, without being limited to this, the moving mechanism may also be configured to include a running unit such as a wheel or crawler that runs on the ground, and a drive unit (such as a motor) that drives the running unit.

[0016] The camera 24 and the blower 26 are mounted on the underside of the ceiling panel 20a of the housing 20. The camera 24 is positioned at a distance (height) H2 (e.g., 1000 mm) from the cultivation bed 12. The camera 24 photographs the strawberry plants from above. The blower 26 has the function of blowing air downward onto the strawberry plants. The speed (wind speed) of the air blown from the blower 26 is changeable and controlled by the control device 30. The control device 30 photographs the plants from above using the camera 24 with or without the blower 26 blowing air onto the plants. The images captured by the camera 24 are transmitted to the control device 30, which then transmits the photographed images to the information processing device 90. The control device 30 also transmits information on the speed (wind speed) of the air blown onto the plants from the blower 26 during photography to the information processing device 90.

[0017] The position detection device 29 may be, for example, a position detection device having an RFID reader capable of communicating with multiple RFID (radio frequency identifier) ​​tags installed near the cultivation bed 12, a position detection device having a camera capable of photographing markers installed near the cultivation bed 12, a position detection device that detects position from the amount of rotation of the wheels 22 or the motor 28, or an RTK-GNSS (Real Time Kinematic-Global Navigation Satellite System). The position detection device 29 detects the XY position of the camera 24. From the XY position of the camera 24, the XY position of each point in the captured image (capture range) can be identified. In addition, the detection result of the position detection device 29 is transmitted to the control device 30.

[0018] The control device 30 controls the motor 28 based on the detection result of the position detection device 29 to adjust the position of the monitoring device 10 (the shooting location of the camera 24). The control device 30 also transmits to the information processing device 90 the image captured by the camera 24, its position information (XY position), and information on the speed of the wind blown from the blower 26 when the image was captured. FIG. 2 shows the hardware configuration of the control device 30. As shown in FIG. 2, the control device 30 includes a CPU (Central Processing Unit) 190, a ROM (Read Only Memory) 192, a RAM (Random Access Memory) 194, a storage unit (such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive)) 196, a network interface 197, and a portable storage medium drive 199. These components of the control device 30 are connected to a bus 198. In the control device 30, the CPU 190 executes a program stored in the ROM 192 or the storage unit 196, or a program read by the portable storage medium drive 199 from the portable storage medium 191, thereby realizing the functions of the units shown in Fig. 3. Note that the functions of the units shown in Fig. 3 may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array), for example.

[0019] The information processing device 90 uses the location information transmitted from the control device 30 to identify the plant photographed in the image. The information processing device 90 also calculates the petiole length and leaf area of ​​each plant using the image and wind speed information. The information processing device 90 has the same hardware configuration as the control device 30 (see FIG. 2). In the information processing device 90, the CPU 190 executes a program to realize the functions of each unit shown in FIG. 3.

[0020] (Calculation formula for petiole length and leaf area) Here, we will explain the mathematical formulas used by the information processing device 90 (calculation unit 56 in FIG. 3, which will be described later) when calculating the petiole length and leaf area. Note that in this embodiment, the formulas (approximation formulas) described later are used, but this is not limiting and other formulas can also be used.

[0021] FIG. 4 shows a schematic diagram of an experimental apparatus 200 used to derive the formula. As shown in FIG. 4, a cultivation pot 202 containing a strawberry plant is installed in the experimental apparatus 200. The cultivation pot 202 is installed so that its upper end is at a height H0 (e.g., 500 mm) above the ground. The experimental apparatus 200 also includes a hollow cylinder 204 with a height H2 (e.g., 1000 mm) that covers the strawberry plant. The diameter of the hollow cylinder 204 is, for example, 500 mm. In the experimental apparatus shown in FIG. 4, the height H0 of the cultivation pot 202 is set equal to the diameter of the hollow cylinder 204 so that air passes through the hollow cylinder 204 (to achieve rectification). Furthermore, the experimental apparatus 200 includes a camera 24 and a blower 26 installed at the upper end of the hollow cylinder 204. The camera 24 and the blower 26 in FIG. 4 are the same as the camera 24 and the blower 26 in FIG. 1.

[0022] FIG. 5(a) is an image of a strawberry plant taken by camera 24 when blower 26 is not operating and the wind speed is 0 m. FIG. 5(b) is an image of a strawberry plant taken by camera 24 when wind is blown from blower 26 at a predetermined wind speed. As shown in FIG. 5(b), when wind is blown from above, the leaf blades are subjected to a force (wind load), and as can be seen by comparing with FIG. 5(a), the leaf petioles fall outward around the base (center of the plant (crown)). The leaf blades also fall outward, and the entire leaf moves outward.

[0023] In Fig. 6(a), the state of the entire leaf in a state where a wind speed of 0 m or a weak wind (wind with a first speed, which will be described later) is blowing against the leaf blade (first state) as in Fig. 5(a) is shown by a thick solid line. Also, in Fig. 6(a), the state of the entire leaf in a state where a medium or strong wind (wind with a second speed, which will be described later) is blowing against the leaf blade (second state) as in Fig. 5(b) is shown by a thick dashed line. In Fig. 6(a), the horizontal difference M between the position of the tip of the petiole in the first state and the position of the tip of the petiole in the second state is P is the horizontal projection movement of the petiole M p The position of the tip of the petiole shown in Figure 6(a) is actually the base (see symbol S) of the petiole (the thin petiole connecting the petiole and the leaf blade) shown in Figure 6(b).

[0024] 7(a) to 7(e) show the horizontal projection movement M of the petiole of each of the first to fifth leaves of a strawberry plant relative to the wind speed v, obtained using the experimental device 200. p The results of actual measurements are shown.

[0025] (About the approximation formula) This approximation formula is based on the actual measurements in Figures 7(a) to 7(e) and is calculated using the wind speed v and horizontal projection displacement M P This is an example in which the relationship between is approximated as shown by the solid lines in FIGS. 8(a) to 8(e).

[0026] 9 and 10 are schematic diagrams for explaining the approximation formula. In FIGS. 9 and 10, the state of the entire leaf when exposed to wind at a wind speed of 0 m or a first speed (weak wind) is shown by a thick solid line, and the state of the entire leaf when exposed to wind at a second speed (moderate wind or strong wind) is shown by a thick dashed line. The second speed (second wind speed) is the speed at which the leaf movement amount becomes less than a predetermined amount when the speed of the wind blowing against the leaf blade is increased (i.e., the speed at which the leaf hardly moves). In FIG. 9, the petiole angle (the angle between the direction in which the petiole extends and the horizontal plane) in the first state is θ (rad), and the petiole angle in the second state is δ (rad). θ can be said to be the value of the petiole angle when the leaf is attached. The actual movement amount of the petiole tip between each state is M (mm), and the horizontally projected movement amount is M (mm). P (mm).

[0027] Based on the knowledge that the wind load on the leaf blade is proportional to the square of the wind speed (v) and the pressure-receiving area (A), we decided to use the following formula (1), where l is the proportionality coefficient.

[0028]

number

[0029] In addition, this approximation formula expresses that the angle δ returns from the state shown by the dashed line due to bending (flexion) of the petiole, as shown in Figure 10.

[0030] Specifically, the following formula (2) was used.

[0031]

number

[0032] where p is the advance angle coefficient, and (v 4 / p) expresses the fact that the angle returns to its original position due to the bending of the petiole.

[0033] Furthermore, in this approximation formula, the formula for converting the length of the petiole inclination into a projection value was defined as follows: (3)

[0034]

number

[0035] In this embodiment, the petiole length L and the leaf area A are determined by using the above approximate formulas (formulas (1) to (3)).

[0036] (Functions of the control device 30 and the information processing device 90) Returning to FIG. 3, the functions of the control device 30 and the information processing device 90 will be described.

[0037] (Functions of the control device 30) As shown in FIG. 3, the control device 30 has the functions of a position control unit 40, a wind speed control unit 41, an imaging control unit 42, and an image acquisition and transmission unit 44.

[0038] The position control unit 40 controls the motor 28 based on the detection result of the position detection device 29, and adjusts the position of the monitoring device 10 (the shooting location of the camera 24). For example, the position control unit 40 moves the monitoring device 10 so that each plant falls sequentially within the shooting range of the camera 24.

[0039] When one stalk is within the photographing range of the camera 24, the wind speed control unit 41 controls the blower 26 to blow air at a predetermined speed onto the stalk.

[0040] The photography control unit 42 photographs the plants from above using the camera 24. The photography control unit 42 photographs the plants when they are not being exposed to the wind from the blower 26 and when they are being exposed to the wind from the blower 26. When the photography control unit 42 has completed photographing the plants, it notifies the position control unit 40 to that effect. Upon receiving this notification, the position control unit 40 adjusts the position of the monitoring device 10 so that the next plant falls within the photography range of the camera 24.

[0041] The image acquisition / transmission unit 44 acquires the image captured by the camera 24 and transmits the acquired image to the information processing device 90 together with the position information (XY position) at the time of image capture obtained from the position control unit 40 and the wind speed information at the time of image capture obtained from the wind speed control unit 41.

[0042] (Functions of information processing device 90) As shown in FIG. 3, the information processing device 90 includes an image acquisition unit 52, an image analysis unit 54, a calculation unit 56, a data management unit 58, an output unit 60, and a pre-processing unit 62.

[0043] The image acquisition unit 52 acquires images of stocks taken by the camera 24, information on the location where the images were taken, and information on the wind speed at the time the images were taken.

[0044] The image analysis unit 54 analyzes the image acquired by the image acquisition unit 52 and passes the analysis results to the calculation unit 56 .

[0045] The calculation unit 56 calculates the petiole length and leaf area of ​​the strawberry based on the analysis results of the image analysis unit 54, predetermined coefficients, and information input by the user.

[0046] The data management unit 58 acquires the calculation results (petiole length and leaf area) from the calculation unit 56 and manages them in association with the location information of the plant.

[0047] The output unit 60 outputs the information managed by the data management unit 58 .

[0048] The pre-processing unit 62 executes a process of determining a wind speed (second wind speed) to be set when calculating the petiole length and leaf area of ​​strawberries, using the image acquired by the image acquisition unit 52 and information on the wind speed at the time of image capture. A method for determining the second wind speed will be described below. As described above, the second wind speed is a speed at which the amount of leaf movement becomes equal to or less than a predetermined value when the speed of the wind blowing against the leaf blade is increased (i.e., a speed at which the leaf hardly moves at all).

[0049] (Second method of determining wind speed) The pre-processing unit 62 notifies the control device 30 that pre-processing will be performed. When the control device 30 receives this notification, the position control unit 40 moves the monitoring device 10 to a predetermined position (for example, a position where one of the plants grown in the cultivation bed 12 can be photographed). The wind speed control unit 41 and the photography control unit 42 photograph the plant from above multiple times using the camera 24 while changing the speed of the wind blowing on the specified plant. For example, the wind speed control unit 41 gradually increases the wind speed from a wind speed of 0 m / s or a weak wind, and the photography control unit 42 photographs the plant from above at predetermined time intervals. The image acquisition / transmission unit 44 then transmits the multiple images obtained by the photography and information on the wind speed at which each image was taken to the pre-processing unit 62 via the image acquisition unit 52.

[0050] The pre-processing unit 62 uses the multiple images acquired and information on the wind speed when each image was taken to determine the wind speed (second wind speed) to be set when calculating the petiole length and leaf area of ​​strawberries as follows.

[0051] (1) The pre-processing unit 62 first detects the center of each plant from multiple images. For example, the pre-processing unit 62 performs deep learning using a large number of images showing the plant's shape and its center as training data, and can use the resulting training model to detect the plant's center in each image. The pre-processing unit 62 can also detect the plant's center using methods other than deep learning. For example, as shown in FIG. 11(a), the image analysis unit 54 can detect multiple petioles from an image and detect the intersection of their extensions as the plant's center. This method allows for accurate detection of the plant's center even when the plant's center is hidden by leaves from other plants, as shown in FIG. 11(b). Furthermore, as shown in FIG. 11(c), strawberries typically have three leaflets connected to one petiole. Therefore, the pre-processing unit 62 can, for example, perform a process of drawing a straight line (see the dashed line in FIG. 11(c)) multiple times in the longitudinal direction of the middle leaflet among the three leaflets, and determine the intersection of these straight lines as the plant's center. In addition, the pre-processing unit 62 may prioritize detecting the center of the plant using deep learning, and if the center of the plant cannot be detected using deep learning, it may detect the center of the plant using the method of Figure 11(a), Figure 11(b), or Figure 11(c).

[0052] (2) Next, the pre-processing unit 62 detects petioles from each image. Here, the pre-processing unit 62 detects the leaf blades from each image in the order of the first leaf, second leaf, third leaf, etc., and detects the petioles of each leaf. Detecting the petioles of each leaf in this manner is for determining the second wind speed of each leaf. The reason for detecting the center of the plant in (1) and the petioles of each leaf in (2) is that even if the leaves and petioles are crowded together, detecting the petioles of each leaf and the center of the plant makes it possible to calculate the distances Dn and Dm (described later). For example, the pre-processing unit 62 performs deep learning using a large amount of learning data that associates leaf images with the degree of unfolding to obtain a learning model. Then, the pre-processing unit 62 uses the obtained learning model to detect unfolded leaves of the plant from the image and designates the detected unfolded leaf as the first leaf. Alternatively, the pre-processing unit 62 may detect leaves from the image and designate the smallest detected leaf as the first leaf. The pre-processing unit 62 may also detect leaves from the image and determine the lightest leaf among the detected leaves as the first leaf. Furthermore, the pre-processing unit 62 may also detect leaves from the image and determine the leaf that is closest to the center of the plant among the detected leaves as the first leaf. The pre-processing unit 62 may preferentially apply a method for identifying the first leaf using deep learning, and if the first leaf cannot be identified using that method, it may identify the first leaf using one of the other methods described above. The pre-processing unit 62 may also identify the first leaf using multiple of the methods described above and combine the results of the multiple identifications to identify the first leaf. Furthermore, the pre-processing unit 62 may detect the angle of the line connecting the first leaf and the center of the plant relative to a reference line, and identify the leaf located approximately 144° from the center of the plant in the direction of the first leaf's growth as the second leaf, the leaf located approximately 144° from the center of the plant in the direction of the second leaf's growth (approximately 288° from the direction of the first leaf's growth) as the third leaf, and so on. The pre-processing unit 62 then detects the petioles connected to the identified first leaf, second leaf, etc. Note that deep learning can be used as a method for detecting petioles. However, this is not limiting, and for example, a petiole and one leaf blade can be detected and the connection point between them can be regarded as the petiole. Alternatively, the position where the longitudinal directions of three leaf blades intersect can be regarded as the petiole.

[0053] (3) Next, the pre-processing unit 62 measures the distance from the center of the plant to the petiole of each leaf in each image (see Dn and Dm in FIG. 6(a)).

[0054] (4) Next, the pre-processing unit 62 calculates the horizontal projection shift amount M P Specifically, the horizontal projection movement amount M P Calculate. M P =Dm-Dn …(4)

[0055] Note that Dn is the distance from the center of the plant to the petiole of each leaf when the wind speed is 0 m or weak, and Dm is the distance from the center of the plant to the petiole of each leaf when the wind speed is gradually increased from 0 m or weak.

[0056] (5) Next, the pre-processing unit 62 calculates the horizontal projection movement amount M for each leaf even when the wind speed is increased. P The wind speed at which the horizontal projection movement amount M P The wind speed at which the change in wind speed is equal to or less than a predetermined threshold is determined as the second wind speed. Note that the second wind speed determined as described above can be applied to plants whose planting date is the same as or close to that of the plants photographed in the above process.

[0057] (Regarding the processing of the control device 30) FIG. 12 shows a flowchart of the processing of the control device 30. It should be noted that the plants to be photographed in the following processing are plants whose planting date is the same as or close to that of the plants photographed when the pre-processing unit 62 determines the second wind speed. In addition, in this embodiment, as an example, processing for measuring the petiole length and leaf area of ​​a predetermined leaf (e.g., the fourth leaf) will be described. Therefore, in the processing of FIG. 12, the control device 30 uses the second wind speed of a predetermined leaf from the second wind speeds for each leaf determined by the pre-processing unit 62.

[0058] 12 starts, first in step S10, the position control unit 40 controls the motor 28 based on the detection result of the position detection device 29, and moves the monitoring device 10 to a position where the strawberry plants can be photographed using the camera 24. Note that the approximate position of each plant on the cultivation bed 12 is assumed to be known in advance.

[0059] Next, in step S12, the wind speed control unit 41 turns off the blower 26 or controls the wind speed from the blower 26 to be a first wind speed (weak wind) (i.e., so that the stumps are in a first state). Here, the first state is assumed to be a windless state. When step S12 is performed, if the blower 26 is in the OFF state, the wind speed control unit 41 does not control the blower 26 and maintains the state as it is.

[0060] Next, in step S14, the photography control unit 42 uses the camera 24 to photograph the strawberry plant from above.

[0061] In step S16, the image acquisition / transmission unit 44 acquires the image (first image) taken in step S14 from the camera 24, and transmits the first image, information on the position when the first image was taken, which is obtained from the position control unit 40, and information on the wind speed of the blower 26 when the first image was taken, which is obtained from the wind speed control unit 41, to the information processing device 90.

[0062] Next, in step S18, the wind speed control unit 41 controls the wind speed from the blower 26 to be the second wind speed (medium wind or strong wind) determined by the pre-processing unit 62 (i.e., so that the stalks are in the second state).

[0063] Next, in step S20, the photography control unit 42 uses the camera 24 to photograph the strawberry plant from above.

[0064] Next, in step S22, the image acquisition / transmission unit 44 acquires the image (second image) taken in step S20 from the camera 24, and transmits the second image, information on the position when the second image was taken, which is obtained from the position control unit 40, and information on the wind speed of the blower 26 when the second image was taken, which is obtained from the wind speed control unit 41, to the information processing device 90.

[0065] Thereafter, the process returns to step S10. Note that the wind speed control unit 41 may stop the operation of the blower 26 before returning to step S10. When the process returns to step S10, the position control unit 40 moves to photograph the next stalk. Then, in steps S12 to S22, the control device 30 performs processes such as photographing the first and second images of the next stalk.

[0066] 12, the monitoring device 10 is moved (S10), a first image is captured at a first wind speed (S12, S14), a second image is captured at a second wind speed (S18, S20), and the monitoring device 10 is moved again (S10). However, the present invention is not limited to this. For example, the monitoring device 10 may be moved with the blower 26 turned off or maintained at a first wind speed, and images of multiple plants (first images) may be captured. Thereafter, the monitoring device 10 may be moved with the blower 26 maintained at a second wind speed, and images of multiple plants (second images) may be captured. The image acquisition / transmission unit 44 may then associate the first and second images captured at the same location (i.e., the first and second images of the same plant) and transmit them to the information processing device 90 together with location information and wind speed information.

[0067] (Regarding the processing of the information processing device 90) Next, the processing of the information processing device 90 will be described with reference to the flowchart of FIG.

[0068] When the processing of FIG. 13 starts, first in step S30, the image acquisition unit 52 acquires the image, position information, and wind speed information transmitted from the image acquisition / transmission unit 44 of the control device 30.

[0069] Next, in step S32, the image analysis unit 54 detects the center of the plant from either the image in the first state or the image in the second state. The method for detecting the center of the plant in step S32 is the same as the method described in the processing of the pre-processing unit 62 (for example, the method using deep learning or the method of FIGS. 11(a) to 11(c)).

[0070] Next, in step S34, the image analysis unit 54 detects petioles from the image in the first state. The processing in step S34 is similar to the processing in the pre-processing unit 62.

[0071] Next, in step S36, the image analysis unit 54 measures the distance Dn (see FIG. 6(a)) from the center of the plant to the petiole using the first image. Note that the distance Dn is measured only for the specified leaves that require measurement.

[0072] Next, in step S38, the image analysis unit 54 detects petioles from the image in the second state. The process in step S38 is the same as that in step S34 described above.

[0073] Next, in step S40, the image analysis unit 54 measures the distance Dm (see FIG. 6(a)) from the center of the plant to the petiole using the second image. Note that the distance Dm is measured only for the specified leaves that require measurement.

[0074] Next, in step S42, the calculation unit 56 calculates the horizontal projection movement amount M P is calculated based on the above formula (4).

[0075] Next, in step S44, the calculation unit 56 uses the above-mentioned approximation formulas (the above formulas (1) to (3)) to calculate the petiole length L and leaf area A of a predetermined leaf. The details of the processing in step S44 will be described later.

[0076] Next, in step S46, the data management unit 58 manages the petiole length L and the leaf area A in association with the position of the plant.

[0077] Next, in step S48, the output unit 60 determines whether an output request has been input. If the determination in step S48 is negative, the process returns to step S30, but if the determination is affirmative, the process proceeds to step S50.

[0078] When the process proceeds to step S50, the output unit 60 outputs the data for which an output request has been made. After that, the process returns to step S30, and the above processing is repeated.

[0079] In the above description, an example has been described in which photographs are taken using a second wind speed corresponding to a specified leaf and the petiole length L and leaf area A of the specified leaf are calculated. However, this is not limiting, and the petiole length L and leaf area A of leaves at multiple leaf positions may be calculated. In this case, it is necessary to photograph each plant using a second wind speed corresponding to each of the multiple leaf positions. Furthermore, the petiole length L and leaf area A of each leaf are calculated using an image (second image) taken at the second wind speed corresponding to each leaf. In this case, the petiole length L and leaf area A are managed in association with each leaf position.

[0080] (Example) Next, the processing of the pre-processing unit 62 and the processing of step S44 in FIG. 13 will be described using an example.

[0081] 14(a) and 14(b) show information on wind speed when the multiple images acquired by the pre-processing unit 62 were taken, and the horizontal projection movement amount M obtained from the multiple images. P Also, Fig. 14(c) shows the wind speed and the horizontal projection displacement M P The relationship between these is expressed in a coordinate system (horizontal axis: wind speed, vertical axis: horizontal projection movement M P ) shown above. In this example, the horizontal projection of the fourth leaf of the strawberry plant is M P This shows the following.

[0082] From Figure 14(c), a wind speed of 4.503 m / s is identified as the wind speed at which the amount of leaf movement is below a predetermined level (the wind speed at which leaves hardly move), so the pre-processing unit 62 determines this wind speed of 4.503 m / s as the second wind speed.

[0083] 15(a) to 15(e) are diagrams illustrating an example of calculating the petiole length L and the leaf area A using the above formulas (1) to (3). Note that these examples are values ​​for the fourth leaf of a strawberry plant.

[0084] 15(a) shows an example of coefficients l and p (advance angle coefficients) that need to be calculated in advance when using approximate formula 1. l is a coefficient in formula (1), and p is a coefficient in formula (2).

[0085] 15(b) shows an example of the petiole angle δ under pressure and the petiole angle θ (see FIG. 10), which are values ​​input by the user. δ and θ may be measured in advance, or if the relationship between the leaf position and δ and θ is known in advance, δ and θ may be calculated using that relationship.

[0086] 15(c) shows an example of a wind speed (first wind speed) v1 in a first state and a wind speed (second wind speed) v2 in a second state. The difference between the second wind speed v2 and the first wind speed v1 is v in equations (1) and (2).

[0087] The calculation unit 56 calculates M P The petiole length L is calculated by substituting the values ​​in Figure 15(d) for δ and the values ​​in Figure 15(b) for θ. As a result, the petiole length L is calculated to be 170.8311 mm, as shown in Figure 15(e). Note that the true value of petiole length L in this case was 165 mm, so it can be seen that the above method can derive a value for petiole length L that is close to the true value.

[0088] The calculation unit 56 also calculates the petiole movement amount M (see FIG. 10) by substituting the petiole length L calculated as above, δ and θ in FIG. 15(b), and p and v = 4.503 m / s in FIG. 15(a) into the above equation (3). As a result, the value shown in FIG. 15(e) (M = 48.37061 mm) is obtained as the petiole movement amount M.

[0089] Furthermore, the calculation unit 56 calculates the leaf area A by substituting the petiole movement amount M calculated as above, v=4.503 m / s, and the value of l in FIG. 15(a) into the above formula (1). As a result, the leaf area A is 20388.81 mm, as shown in FIG. 15(e). 2 The true value of leaf area A in this case is 20240.78 mm 2 Therefore, it can be seen that the above method can derive a value for leaf area A that is close to the true value.

[0090] As described above in detail, according to the first embodiment, the control device 30 controls the camera 24 to take a first image of the plant from above in a first state where no wind is blown from the blower 26 onto the plant or where wind is blown from the blower 26 at a first speed, and controls the camera 24 to take a second image of the plant from above in a second state where wind is blown from the blower 26 at a second speed (FIG. 12). At this time, the second speed (wind speed) is set to a speed at which the amount of leaf movement becomes equal to or less than a predetermined amount (a speed at which the leaves hardly move) when the speed of the wind blown from the blower 26 is increased. Furthermore, the information processing device 90 calculates the difference (horizontal projection movement amount M P ) is calculated. Furthermore, the information processing device 90 calculates the petiole angle θ in the first state, the petiole angle δ in the second state, and the horizontal projection movement amount M P The petiole length L is calculated using equation (3) using the above. As a result, even if the petiole is located below the leaf blade (leaflet) and is difficult to see from the outside, the petiole length L can be calculated with high accuracy by using the first and second images (see Figure 15(e)).

[0091] In this embodiment, the information processing device 90 calculates the value of the leaf area A based on the value of the petiole length L, the difference v in the speed of the air blown onto the plant from the blower 26 in the first state and the second state, and the petiole angles δ and θ. This allows the leaf area to be calculated accurately even when the leaf blades are overlapping (see FIG. 15(e)).

[0092] The wind speed distribution of the blower 26 is strongest directly below the blower 26 and becomes weaker at positions farther away from the blower 26. Therefore, when the monitoring device 10 is moved while the blower 26 is maintained in a constant state, the plants transition from a state in which they are exposed to weak wind (first state) to a state in which they are exposed to strong wind (second state). Therefore, in the above embodiment, images may be taken when the plants are located away from directly below the blower 26 (first state) and when they are located close to directly below the blower 26 (second state), and the processing of FIG. 13 may be performed using each image. In this case, the monitoring device 10 may have a first camera disposed away from the blower 26 in the X-axis direction to photograph the plants in the first state, and a second camera disposed near the blower 26 to photograph the plants in the second state. Alternatively, a single camera may be used to photograph the stock in the first state and the stock in the second state at different angles, and the images may be corrected to appear as if they were photographed from the same angle before being used in the processing shown in FIG. 13 above.

[0093] Second Embodiment Next, a second embodiment will be described. Fig. 16 shows a functional block diagram of a control device 30 and an information processing device 90 according to the second embodiment. As can be seen by comparing Fig. 16 with Fig. 3 (first embodiment), the information processing device 90 of the second embodiment does not have a pre-processing unit 62, and is therefore characterized in that the processing of the control device 30 and the information processing device 90 differs from that of the first embodiment. Specifically, in the second embodiment, the second speed is not determined in advance, but is determined (specified) for each plant based on the photographing results of each plant.

[0094] (Regarding the processing of the control device 30) FIG. 17 is a flowchart showing the processing of the control device 30 according to the second embodiment.

[0095] As shown in Fig. 17, the control device 30 executes steps S10 to S16 in the same manner as in the first embodiment. That is, in steps S10 to S16, the stocks are photographed with a wind speed of 0 m or with wind at a first wind speed blowing on them. Next, in step S18', the wind speed control unit 41 controls the wind speed from the blower 26 to the next wind speed. Note that in this embodiment, it is assumed that the wind speed values ​​to be set in stages in step S18' are predetermined in advance (for example, wind speeds v1 to v7 in Fig. 14(a)). Therefore, each time step S18' is executed, the wind speed control unit 41 increases the wind speed in stages, such as wind speed v1, v2, ...

[0096] After step S18', steps S20 and S22 are executed in the same manner as in the first embodiment. Then, in the next step S24, it is determined whether or not imaging has been completed at all wind speeds (for example, wind speeds v1 to v7). If the determination in step S24 is negative, the process returns to step S18', but if the determination is positive, the process returns to step S10.

[0097] In this way, in the second embodiment, when photographing stocks, images of stocks exposed to winds at a plurality of wind velocities (for example, v0 to v7) are acquired.

[0098] (Regarding the processing of the information processing device 90) 18 is a flowchart showing the processing of the information processing device 90 according to the second embodiment. The second embodiment is characterized in that the processing of step S43 is executed between steps S42 and S43 of the processing of the first embodiment (FIG. 13).

[0099] When the process of FIG. 18 starts, steps S30 to S42 are executed in the same manner as in the first embodiment. However, as the image of the second state, a plurality of images (for example, seven images) obtained at a plurality of wind speeds (for example, v1 to v7) are obtained. Therefore, in step S42, the horizontal projection movement amount M P A plurality of (for example, seven) second wind speeds are obtained. The method for identifying this second wind speed is the same as the method for determining the second wind speed by the pre-processing unit 62 in the first embodiment. That is, in step S30 of FIG. 18, data such as that shown in FIG. 14(a) is obtained, and in step S42, data such as that shown in FIG. 14(b) is obtained, and therefore in step S43, these data are used to identify the second wind speed (see FIG. 14(c)).

[0100] Thereafter, the second wind speed identified in step S43 and the horizontal projection movement amount M corresponding to the second wind speed are calculated. P The processes from step S43 onwards are carried out in the same manner as in the first embodiment using the above.

[0101] As a result, similar to the first embodiment, even when the petiole is located below the leaf blade (leaflet) and is difficult to see from the outside, the petiole length L can be calculated with high accuracy. Furthermore, even when the leaf blades are overlapping, the leaf area A can be calculated with high accuracy.

[0102] Furthermore, in the second embodiment, the second speed is specified for each plant, so that the petiole length L and leaf area A of each plant can be calculated with high accuracy.

[0103] In the second embodiment, the horizontal projection movement amount M P Calculate the change in horizontal projection movement M P The wind speed when the change in the wind speed is equal to or smaller than a predetermined value may be identified as the second wind speed. In this case, the determination in step S24 may be made positive when the second wind speed is identified.

[0104] In the above embodiment, the case where the wind speed value to be set in stages in step S18' is determined in advance has been described, but this is not limited to this, and the wind speed value to be set in stages may also be determined based on the second wind speed determined in a past process (for example, the previous process).

[0105] The process of the second embodiment (processing in FIGS. 17 and 18) may be performed on the first plant, and the process of the first embodiment (processing in FIGS. 12 and 13) may be performed on the second and subsequent plants. That is, the second wind speed identified in the process of the first plant may be used in the process of the second and subsequent plants.

[0106] In the first and second embodiments, the monitoring device 10 moves on the ground, but this is not limiting. For example, an unmanned aerial vehicle (drone or multicopter) equipped with a camera 24, a blower 26, a position detection device 29, and a control device 30 may be used as the monitoring device. In this case, the strength of the wind blowing on the plants can be adjusted by adjusting the height of the unmanned aerial vehicle and the relative positions of the unmanned aerial vehicle and the observation target. In this case, the blower 26 may be omitted. Note that adjusting the height of the unmanned aerial vehicle and the relative positions of the unmanned aerial vehicle and the observation target changes the relative positions of the camera 24 and the observation target, which changes the shooting distance and shooting angle. However, the above processing can be performed taking into account the relative positions of the unmanned aerial vehicle and the observation target.

[0107] The above-described embodiment is a preferred example of the present invention, but the present invention is not limited to this and can be modified in various ways without departing from the spirit of the present invention. [Explanation of symbols]

[0108] 24 Camera (photographic device) 26 Blower (spraying device) 30 Control device 90 Information processing device (calculating device) 100 Information Acquisition System

Claims

1. A blowing device that blows air onto strawberry plants from above, a photographing device for photographing an image of the stock from above; a control device that controls the photographing device to photograph a first image of the plant from above in a first state where no wind is blown from the blowing device to the plant or where wind is blown from the blowing device at a first speed, and that controls the photographing device to photograph a second image of the plant from above in a second state where wind is blown from the blowing device at a second speed; a calculation device that calculates the value of the petiole length of a predetermined leaf of the strawberry plant from the image captured by the photographing device; Equipped with the second speed is a speed at which the movement amount of the predetermined leaf becomes equal to or less than a predetermined amount when the speed of the wind blown from the blowing device is increased, The calculation device determining a difference between a first distance from the center of the plant to the tip of the petiole of the specified leaf in the first image and a second distance from the center of the plant to the tip of the petiole of the specified leaf in the second image; calculating a value of the petiole length of the specified leaf based on values ​​of the petiole angle of the specified leaf in each of the first state and the second state and a difference between the first distance and the second distance; A strawberry plant information acquisition system characterized by:

2. the control device captures the second image a plurality of times while varying the second speed; The strawberry plant information acquisition system described in claim 1, characterized in that the calculation device determines the second speed and second image to be used when calculating the petiole length value from among the multiple second speeds and multiple second images based on the multiple second images.

3. The strawberry plant information acquisition system described in claim 1, characterized in that the values ​​of the petiole angle of the specified leaf in each of the first state and the second state are pre-input values.

4. The calculation device In the calculation process, a value of the leaf area of ​​the specified leaf is calculated based on the value of the petiole length of the specified leaf, the difference in speed of the wind blown from the blowing device to the plant in the first state and the second state, and the value of the petiole angle of the specified leaf in each of the first state and the second state.

4. The strawberry plant information acquisition system according to claim 1, wherein the strawberry plant information acquisition system is a system for acquiring information on strawberry plants.

Images