Spraying device, spraying program, spraying method, and jetting control module

Abstract

An objective of the present invention is to provide a spraying device, a spraying program, a spraying method, and a jetting control module that enable the accurate performance of pollination work. The spraying device includes: a traveling vehicle body; an imaging device for acquiring image information of an agricultural crop; a jetting device capable of jetting liquid or powder on the agricultural crop from one or a plurality of jetting nozzles; and a control device. The imaging device includes a first imaging device provided in front of the jetting device in a traveling direction of the traveling vehicle body. The control device controls the jetting of the liquid or the powder on the agricultural crop by the jetting device on the basis of the image information acquired by the first imaging device.

Description

Spraying device, spraying program, spraying method, and spray control module

[0001] The present invention relates to a spraying device, a spraying program, a spraying method, and an injection control module.

[0002] As a smart agriculture system, the use of parameters obtained by image processing for operating agricultural equipment is being considered. For example, Patent Literature 1 discloses a system in which predetermined parameters of a predetermined location on a target crop are estimated based on a trained model and an image of the target crop, and an operating mechanism performs a predetermined operation, such as pollinating that location from a predetermined direction, based on the estimated predetermined parameters.

[0003] Patent No. 7090953

[0004] Pollination, which involves applying pollen one by one to the many flowers in a field, is one of the most labor-intensive agricultural tasks. Furthermore, the time available for pollination after each flower blooms is limited, typically 48 hours. Pollination must be timed appropriately within this limited flowering period, taking into account the plant's ever-changing condition. Delays in pollination can lead to reduced yields. Pollinating flowers that bloom randomly in various locations in a field becomes particularly difficult the larger the field. Therefore, pollination requires a concentrated workforce to be secured in advance for a specific period, but securing sufficient personnel is not easy because the timing and number of workers vary from year to year.

[0005] Therefore, it is conceivable to have a pollen-spraying device perform the pollination work instead of a human, but it is difficult to have a self-propelled device accurately spray pollen at any desired location on the crops.

[0006] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a spraying device, a spraying program, a spraying method, and an injection control module that enable accurate pollination work within a limited pollination period.

[0007] One aspect of the present invention is a spraying device that comprises a traveling vehicle body, an imaging device that acquires image information of agricultural crops, an injection device that can spray liquid or powder onto the crops from one or more injection nozzles, and a control device, wherein the imaging device has a first imaging device that is provided forward in the traveling direction of the traveling vehicle body than the injection device, and the control device controls the spraying of the liquid or powder onto the crops by the injection device based on the image information acquired by the first imaging device.

[0008] According to the present invention, it is possible to provide a spraying device, a spraying program, a spraying method, and an injection control module that enable pollination work to be performed with high accuracy within a limited pollination period.

[0009] 1 is a conceptual diagram of a smart agriculture system. FIG. 1 is a schematic side view of the configuration of a spraying device of this embodiment. FIG. 2 is a schematic top view of the configuration of a spraying device of this embodiment. FIG. 3 is a schematic rear view of the configuration of a spraying device of this embodiment. FIG. 4 is an example of image information of agricultural crops acquired in advance by an imaging device. FIG. 5 is an example of image information of agricultural crops acquired in advance by an imaging device. FIG. 6 is an example of image information of agricultural crops acquired in advance by an imaging device. FIG. 7 is a schematic diagram of the hardware configuration and functional configuration of a control device. FIG. 8 is a schematic diagram showing the data structure of traveling data. FIG. 9 is a flowchart of spray control of the spraying device of this embodiment. FIG. 10 is a sketch diagram showing a series of states from when a flower begins to bloom, until it reaches full bloom and then falls.

[0010] Hereinafter, an embodiment of the present invention (hereinafter referred to as "the present embodiment") will be described in detail with reference to the drawings as necessary. However, the present invention is not limited to this embodiment, and various modifications are possible without departing from the spirit of the present invention. In the drawings, the same elements are given the same reference numerals, and redundant explanations will be omitted. Furthermore, positional relationships such as up, down, left, and right are based on the positional relationships shown in the drawings unless otherwise specified. Furthermore, the dimensional ratios of the drawings are not limited to those shown in the drawings.

[0011] 1. System FIG. 1 is a schematic diagram showing a smart agriculture system 1 according to this embodiment. As shown in FIG. 1, in one example of the smart agriculture system 1 according to this embodiment, a user terminal 100, a server 200, a measurement device 300, and a spraying device 400 are connected via a network N. The measurement device 300 acquires information about agricultural crops in a farm field 500, which is then stored by an agricultural support device (hereinafter referred to as the "server 200"). The server 200 may then formulate plans for each agricultural task, such as a pollination work plan, and transmit the plans to the user terminal 100 used by a farm worker, for example. Furthermore, the server 200 may instruct the spraying device 400 to perform various tasks, such as pollination, based on the pollination work plan, and the spraying device 400 may then execute the plans.

[0012] In this embodiment, a "field" is a cultivation area where agricultural products are grown, and is not limited to an outdoor cultivation area, but may also be a facility with a roof, such as a greenhouse or agricultural product factory.

[0013] The user terminal 100 is a tablet terminal or smartphone carried by a worker who performs agricultural work in the field 500 or a manager who manages the field. The user terminal 100 issues work instructions and provides requested information to the spraying device 400, the measuring device 300, and the server 200 via the network N. Similarly, the user terminal 100 may also acquire information about the field, crops, and environment from the server 200, the measuring device 300, or the spraying device 400.

[0014] The server 200 functions as a management device for comprehensively managing the entire farm field. Specifically, it issues work instructions and provides requested information to the user terminal 100, the measurement device 300, or the spraying device 400 via the communication I / F 220 and the network N. Similarly, the server 200 also acquires information about the farm field, crops, and environment from the user terminal 100, the measurement device 300, or the spraying device 400. Furthermore, the server 200 not only controls and calculates the entire system, but may also function as an instruction unit that executes instructions to the spraying device 400, etc., a processing unit that performs various processes on the acquired information, and a prediction unit that predicts the crop yield and workload.

[0015] The measuring device 300 is a device that acquires information about environmental parameters such as the temperature and humidity of the field and about the crops, and may be a device equipped with a sensor for observing the condition of the field and the crops, or a device that measures the crops 520 in the field 500 and acquires image information and other information about the crops 520.

[0016] The measuring device 300 is not particularly limited, and may be, for example, various fixed sensors 310 installed at any position in the field 500, a drone equipped with various sensors and flying within the field 500, an unmanned aerial vehicle 320 that propels itself within the field, a smartphone equipped with various sensors, a handheld computing device, a wearable terminal, or other terminal operated by a person (not shown). Here, the sensor is not particularly limited, and may further include at least one of an image sensor, a component sensor, and an environmental sensor.

[0017] The measuring device 300 may be linked to the spraying device 400. In addition, although the spraying device 400 and the measuring device 300 are shown as separate devices in Fig. 1, the spraying device 400 may have the functions of the measuring device 300, and the spraying device 400 and the measuring device 300 may be an integrated device.

[0018] By adopting a system constructed in this manner, producers can achieve more productive farm management. Specifically, they can have the spraying device 400 carry out pollination work based on an appropriate pollination work plan.

[0019] Each component of the spraying device 400 of this embodiment will be described in detail below.

[0020] 2A shows a schematic side view of the configuration of the spraying device 400 of this embodiment. As shown in Fig. 2A, the spraying device 400 of this embodiment includes a traveling vehicle body 410, an imaging device 450 that acquires image information of the crops, a spraying device 420 that can spray liquid or powder onto the crops from one or more spray nozzles, and a control device 430.

[0021] The spraying device 400 sprays a liquid or powder onto the crops 520 while the traveling vehicle body 410 is traveling. For example, if the sprayed substance is liquid pollen or powder pollen, pollination work can be performed, and if the sprayed substance is a pesticide or the like, pesticide spraying work can be performed. Note that liquid pollen is pollen dispersed in a liquid such as water. Furthermore, powder pollen may be pollen itself, or may be pollen with a bulking agent added.

[0022] In this embodiment, "agricultural crops" refers to plants that are cultivated and harvested in agriculture. When simply referring to "agricultural crops" in this embodiment, it simply means plants that are cultivated in agriculture, without distinguishing between the trunks, branches, flowers, buds, leaves, stems, shoots, fruits, etc. of the plants. Therefore, in this embodiment, "agricultural crops" does not refer only to the fruits, etc. that are ultimately obtained.

[0023] The traveling vehicle body 410 is not particularly limited as long as it is a vehicle body that can travel within a field. For example, it may be a self-propelled spraying device that recognizes the environment using various sensors and control algorithms and can travel within a field autonomously, or it may be an operable spraying device that can travel within a field by being ridden by a person or remotely operated.

[0024] The traveling vehicle body 410 is not particularly limited, but may include, for example, a vehicle having a traveling mechanism 411 such as wheels or caterpillars that can travel, and a power unit (not shown) that drives the traveling mechanism 411. The traveling vehicle body 410 may also have various sensors such as a GPS, a speed sensor, an acceleration sensor, a distance sensor, and a geomagnetic sensor so that it can determine its own position and avoid obstacles when traveling on its own.

[0025] Furthermore, the traveling vehicle body 410 may be capable of autonomously traveling within a field, or may be capable of self-traveling within a field according to a traveling route instruction received from the user terminal 100 or the server 200. Alternatively, the traveling vehicle body 410 may be manually operated by a user according to an instruction from the user terminal 100. Furthermore, control related to the self-traveling of the traveling vehicle body 410, such as traveling speed, traveling direction, and traveling route, may be performed by a control device 430, which will be described later.

[0026] Furthermore, the various sensors of the traveling vehicle body 410 may cooperate with the control device 430 (described later) to realize SLAM (Simultaneous Localization and Mapping). Specifically, the various sensors may sense the surrounding environment while the traveling vehicle body 410 is traveling, thereby creating a two-dimensional or three-dimensional environmental map, and simultaneously estimating the vehicle's own position on the environmental map.

[0027] 1.1.2 Sprayer The sprayer 420 is not particularly limited as long as it has a configuration capable of spraying a liquid or powder onto the crops 520. Specifically, the sprayer 420 may have one or more spray nozzles 421 and a nozzle drive mechanism 424 that controls the extension and retraction of the one or more spray nozzles 421, and may also have a tank 425 containing the liquid or powder as needed. In this way, the liquid or powder sucked up from the tank 425 may be sprayed from the spray nozzle 421 brought close to the crops 520 by the nozzle drive mechanism 424. The control device 430, described below, may be responsible for controlling the spray timing and amount, and the extension and retraction of the spray nozzle 421 while the traveling vehicle body 410 is traveling under its own power.

[0028] 2B shows a schematic top view of the configuration of spraying device 400 of this embodiment, and Fig. 2C shows a schematic rear view of the configuration of spraying device 400 of this embodiment. Spray nozzle 421 is not particularly limited as long as it is a nozzle that sprays liquid or powder onto crop 520 in response to instructions from control device 430, which will be described later, or the like.

[0029] The spray nozzle 421 is not particularly limited as long as it is a known spray nozzle, but examples include a one-fluid nozzle that mainly sprays one liquid, a two-fluid nozzle that sprays two fluids such as a gas and a liquid, and a nozzle that sprays powder together with a gas. Specifically, a one-fluid nozzle may spray liquid pollen, or a two-fluid nozzle may spray a mixture of air and liquid pollen.

[0030] 2B shows an example of an arrangement of the injection nozzles 421. As an example, the injection nozzles 421 may be arranged in a single row in the width direction of the traveling vehicle body, or may be arranged in multiple rows as shown in FIG.

[0031] 2B , the spray ranges S21, S22 of adjacent spray nozzles 421 may partially overlap. A time lag occurs between the time a spray nozzle 421 sprays and the time it next sprays, such as when it needs to refill the spray material. In other words, pollen, etc., is not sprayed onto the crops 520 in the area through which the traveling vehicle 410 passes between the time a spray is made and the time a spray is made. If there is a flower in an area where pollen, etc., is not sprayed, it means that pollination has not occurred on that flower. By partially overlapping the spray ranges S21, S22 of adjacent spray nozzles 421, even if one spray nozzle 421 is unable to immediately spray pollen, etc., due to the time lag described above, the adjacent spray nozzle 421 can still spray pollen, etc., thereby avoiding a decrease in pollination efficiency.

[0032] 2B, some spray nozzles 421 may be positioned further back in the running direction than other spray nozzles 421. By providing spray nozzles 421 at different positions in the running direction in this way, as in the above, even if one spray nozzle 421 cannot immediately spray pollen or the like due to the time lag described above, the adjacent spray nozzle 421 can still spray pollen or the like, thereby avoiding a decrease in pollination efficiency.

[0033] The nozzle driving mechanism 424 is a mechanism that changes the length of the nozzle and the like using the control device 430 described below, and is not particularly limited as long as it is a known mechanism that can extend or retract the length of the injection nozzle 421. For example, the nozzle driving mechanism 424 may be made extendable by having a multi-layer sleeve structure. The nozzle driving mechanism 424 may also be capable of adjusting the inclination of the injection nozzle 421 and the direction of the nozzle opening. This allows the liquid or powder to be effectively sprayed in directions other than the extension direction of the nozzle. The nozzle driving mechanism 424 may also control the extension and retraction of one or more injection nozzles 421 independently, or may control the extension and retraction of the injection nozzles 421 in conjunction with each other.

[0034] For example, if the running vehicle body 410 sways due to external disturbances such as uneven ground or wind, the inclination of the running vehicle body 410 can be detected using a tilt sensor or the like, and the inclination of the spray nozzle 421 and the direction of the nozzle opening can be adjusted to spray at the desired location.

[0035] 1.1.3 Imaging Device The spraying device 400 of this embodiment has an imaging device 450 that acquires image information of the crop 520. As shown in FIG. 2A , the imaging device 450 may have a first imaging device 451 provided ahead of the spraying device 420 in the traveling direction of the traveling vehicle body 410, and may also have a second imaging device 452 provided behind the spraying device 420 in the traveling direction of the traveling vehicle body 410.

[0036] As traveling vehicle body 410 moves in the traveling direction, first imaging device 451 passes directly below crop 520 before spray nozzle 421, and second imaging device 452 passes directly below crop 520 after spray nozzle 421. Therefore, first imaging device 451 acquires image information of crop 520 in advance, control device 430 calculates the spray timing based on the image information, and spray nozzle 421 can spray the liquid or powder at the calculated appropriate timing. Furthermore, second imaging device 452 acquires image information of crop 520 onto which the liquid or powder has been sprayed, and control device 430 can confirm, based on the image information, whether the sprayed liquid or powder has adhered to crop 520 and the extent of adhesion.

[0037] The imaging device 450 is not particularly limited as long as it is capable of capturing still images or videos. Furthermore, a plurality of first imaging devices 451 and second imaging devices 452 may be arranged in the width direction of the traveling vehicle body. As shown in FIG. 2B , when a plurality of imaging devices 450 are provided, image information acquired by the plurality of imaging devices 450 may be combined into a single image. In this case, the fields of view S11 and S12 of the plurality of imaging devices 450 may partially overlap.

[0038] 2D shows an example of image information of crop 520 acquired by first imaging device 451 by combining fields of view S11 and S12. Areas S21 and S22 in FIG. 2D correspond to the spray range of the spray nozzle. In this way, control device 430 may identify the spray range based on the image information, calculate the spray timing based on the identified spray range, and spray nozzle 421 may spray liquid or powder at the calculated appropriate timing.

[0039] It is preferable that spraying device 400 monitors the condition of crops 520, which changes daily, and performs pollination work within a limited pollination period. From this perspective, for example, spraying device 400 may run within the field two or more times a day, and imaging device 450 may acquire image information of crops 520 at different times.

[0040] Then, based on such multiple pieces of image information at different points in time, the control device 4312 described below may identify newly bloomed flowers and spray pollen.

[0041] Furthermore, the control device 4312, which will be described later, may identify buds that will newly bloom based on multiple pieces of image information at different times. As a result, the control unit 4312 may determine the travel route of the spraying device 400 and perform self-propelled control at the timing when the flowers will bloom.

[0042] 1.1.4. Distance Measurement Sensor The spraying device 400 of this embodiment may further include a distance measurement sensor 460 facing the direction of the crops 520. The distance measurement sensor 460 is not particularly limited, but examples thereof include an image sensor, an ultrasonic sensor, an infrared sensor, and a laser sensor.

[0043] 2A and 2B, distance measurement sensor 460 may be provided further forward in the traveling direction of traveling vehicle body 410 than spray device 420. In this way, when traveling vehicle body 410 moves in the traveling direction, distance measurement sensor 460 passes directly below crop 520 before spray nozzle 421. Therefore, distance measurement sensor 460 can obtain information about the distance to crop 520 in advance, and spray nozzle 421 may be extended or retracted by nozzle drive mechanism 424 or the spray speed may be adjusted according to the distance information.

[0044] 1.1.5. Blower The spraying device 400 of this embodiment may have a blower 470. As shown in FIGS. 2A and 2B , the blower 470 may be provided further forward in the traveling direction than the spraying device 420, or further rearward in the traveling direction than the spraying device 420. Since the liquid or powder is sprayed while the traveling vehicle body 410 is traveling, the sprayed liquid or powder is located rearward in the traveling direction. Therefore, by providing the blower 470 further rearward in the traveling direction than the spraying device 420, the liquid or powder can be more efficiently dispersed by blowing air. Note that the number of blowers 470 may be one or more.

[0045] 1.1.6. Control Device The control device 430 performs various controls of the spraying device 400 of this embodiment, and may, for example, control the self-propelled movement of the traveling vehicle body 410 and the injection device 420. The control device 430 may, for example, be a general-purpose computer such as a desktop, laptop, or tablet mounted on the spraying device 400, or a programmable logic controller (PLC). In the following description, each process is performed by the control device 430 mounted on the spraying device 400. However, the processes performed by the control device 430 may alternatively be performed by another computer, such as the server 200, connected to the spraying device 400 via the network N.

[0046] Hereinafter, the hardware configuration and functional configuration of the control device 430 will be described with reference to FIG. 3A, and then each control will be described in detail in association with the functional configuration of the control device 430.

[0047] As shown in FIG. 3A, the control device 430 includes, for example, a processor 431, a communication interface 432, an input / output interface 433, a memory 434, a storage 435, and one or more communication buses 436 for interconnecting these components.

[0048] The processor 431 executes processes, functions, or methods implemented by codes or instructions included in a program stored in the storage 435. The processor 431 may include, for example and without limitation, one or more central processing units (CPUs), MPUs, GPUs, etc., and may implement the processes, functions, or methods disclosed in each embodiment by a logic circuit (hardware) formed in an integrated circuit or the like, or a dedicated circuit.

[0049] As shown in FIG. 3A, the processor 431 of this embodiment may be configured to function as a transceiver unit 4311 and a controller 4312.

[0050] The communication interface 432 transmits and receives various data to and from other devices via the network N. The communication may be performed either wired or wirelessly, and any communication protocol may be used as long as mutual communication is possible. For example, the communication interface 432 is implemented as hardware such as a network adapter, various types of communication software, or a combination of these.

[0051] The network N may be, by way of example and not limitation, an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), a portion of the Internet, a portion of the public switched telephone network (PSTN), a cellular network, integrated service digital networks (ISDNs), wireless LANs, long term evolution (LTE), code division multiple access (CDMA), Bluetooth, satellite communications, or the like, or any combination thereof. A network may include one or more networks.

[0052] The input / output interface 433 includes an input device for inputting various operations to the control device 430, and an output device for outputting processing results processed by the control device 430. For example, the input / output interface 433 includes information input devices such as a keyboard, a mouse, and a touch panel, and information output devices such as a display. Note that the control device 430 may receive a predetermined input or execute a predetermined output by connecting an external input / output interface 433.

[0053] The memory 434 temporarily stores programs loaded from the storage 435 and provides a working area for the processor 431. The memory 434 also temporarily stores various data generated while the processor 431 is executing the programs. The memory 434 may be, for example, a high-speed random access memory such as a DRAM, an SRAM, a DDR RAM, or other random access solid-state storage device, or a combination of these.

[0054] The storage 435 stores programs, each functional unit, and various data. The storage 435 may be, for example, one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or nonvolatile memories such as other nonvolatile solid-state storage devices, or a combination thereof. Another example of the storage 435 may be one or more storage devices installed remotely from the processor 431.

[0055] The control device 430 may record the travel of the spraying device 400 and various data acquired during travel as travel data 4351. As the spraying device 400 travels within the field, it can accumulate information such as acquired image information, detected flower pollination status, and pollen spray conditions in association with location information. Therefore, the information recorded in the travel data 4351 is map information such as image information within the field, flower pollination status, and spray conditions.

[0056] Furthermore, the spraying device 400 patrols the field and can acquire various image information, the pollination status of flowers, etc. at the same point at different times. Therefore, the information recorded in the travel data 4351 is data showing changes over time at specific points in the field, or map information recording changes over time.

[0057] 3B shows an example of the data structure of the driving data 4351. In the driving data 4351, a "driving ID" that uniquely identifies the autonomous driving data, a "driving log," "image information" acquired during driving, and a "nozzle control log" may be recorded in association with each other. In particular, the "image information" and "nozzle control log" may be recorded in association with the driving route of the driving log.

[0058] The "driving log" may store records of driving such as the driving location, driving route, driving speed, and time spent driving autonomously, as well as information regarding driving schedules or spraying locations received from the server 200, etc.

[0059] Furthermore, the "image information" may store image information of crop 520 captured by first imaging device 451 before the liquid or powder is sprayed, and / or image information of crop 520 captured by second imaging device 452 after the liquid or powder is sprayed. Here, image information of crop 520 before the liquid or powder is sprayed may be used as information for timing the spraying. Furthermore, image information of crop 520 after the liquid or powder is sprayed may be used as the result of the spraying work.

[0060] The "nozzle control log" may include control information about the extension / retraction position of the injection nozzle 421 by the nozzle drive mechanism 424, as well as control information about the injection timing and injection amount. This information may be recorded in association with the time when the extension / retraction or injection was performed and the travel route.

[0061] The transmitter / receiver 4311 may function as a transmitter that transmits various types of information to other devices such as the user terminal 100, the server 200, and the measurement device 300 via the communication interface 432 and the network N, or as a receiver that receives various types of information from other devices such as the user terminal 100, the server 200, and the measurement device 300.

[0062] For example, the transmitting / receiving unit 4311 may receive information on the spraying location and spraying method from the user terminal 100 or the server 200. The transmitting / receiving unit 4311 may also transmit various data such as acquired image information and information on the progress of the spraying work to the user terminal 100 or the server 200. The transmitting / receiving unit 4311 may refer to the traveling data 4351 and transmit this various information to the user terminal 100 or the server 200.

[0063] 1.1.6.2. Control Unit The control unit 4312 of the control device 430 performs various controls of the spraying device 400 of this embodiment, and may perform, for example, spray control of the spray device 420 and self-propelled control of the traveling vehicle body 410.

[0064] The control unit 4312 may also determine whether a blooming flower is suitable for pollination based on image information acquired by the imaging device 450. FIG. 5 is a sketch diagram illustrating a series of states from when a flower begins to bloom until it reaches full bloom and then falls. As shown in FIG. 5, it takes a certain period of time for a flower to reach full bloom after it begins to bloom. Although it depends on the crop, it is not necessarily suitable to perform pollination immediately after blooming; from the standpoint of fertilization ability, it is preferable to perform pollination taking into account the degree of blooming.

[0065] The relationship between the degree of flowering and fertility can be determined appropriately for each crop and is not particularly limited in this regard. However, for the sake of explanation, let us consider a crop that is suitable for pollination when in full bloom. In this case, the control unit 4312 may further determine whether a flower is suitable for pollination based on the image information and the degree of flowering of the flower. In this case, the control unit 4312 may refer to a data table, such as that shown in FIG. 5, that associates image data indicating the degree of flowering with information regarding the suitability of pollination. Specifically, the control unit 4312 may identify a flower from the image information and evaluate the degree of flowering of the flower by identifying image data similar to the identified flower image from image data of flowers with different degrees of flowering in the data table. The control unit 4312 may then refer to the data table to identify information regarding the suitability of pollination corresponding to the evaluated degree of flowering of the flower, and further determine whether the flower is suitable for pollination based on the information.

[0066] As a result, for example, the control unit 4312 can determine that flowers that are just before or just after full bloom, such as D to E shown in Fig. 5, are suitable for pollination, and can also determine that flowers that are not yet in full bloom, such as B to C, and flowers that are in full bloom, such as F to G, are not suitable for pollination. This allows for efficient pollination of flowers that are suitable for pollination, and also prevents wasteful pollination of flowers that are not suitable for pollination, thereby reducing the waste of pollen.

[0067] Furthermore, the control unit 4312 may predict whether the flower is in a state suitable for pollination, i.e., its fertility, based on information related to the image information.

[0068] Specifically, as shown in FIG. 5 , during the flowering process, the petals gradually open from a bud. Therefore, for example, the characteristics change from a green circular bud to a green and white rectangular half-bloomed state, and then to a white full-bloomed state. For an object recognized as a bud or half-bloomed flower, the control unit 4312 may further determine the pollination suitability based on the degree of blooming, as determined by the shape and color area of ​​the object. More specifically, the pollination suitability may be predicted by quantifying the color area of ​​the petals, such as white, and modeling the change, or the pollination suitability may be predicted from changes in a parameter indicating the shape of the flower.

[0069] Additionally, as time passes after flowering, the color of the petals changes as the flower wilts, for example, from white petals to yellow. The control unit 4312 may determine the suitability for pollination based on such colors or color changes that can be determined from the first image information. Furthermore, when making this determination, environmental data such as accumulated temperature and accumulated illuminance may be taken into consideration. This tends to further improve the accuracy of predicting the suitability for pollination.

[0070] The control unit 4312 may further determine whether the flower shown in the image is a male flower or a female flower from the image information. By distinguishing between male and female flowers through such a determination, it is possible to distinguish between unpigmented male flowers and unpigmented female flowers. This allows for more accurate identification of unpigmented female flowers and for pollination work to be performed. Furthermore, by preventing pollen from being sprayed onto male flowers, it is possible to reduce wasteful work and pollen.

[0071] Specifically, the control unit 4312 may determine the characteristic amount of the female flower, such as the presence or absence of a pistil, based on the image information.

[0072] Furthermore, the control unit 4312 may refer to the detection results recorded in the driving data 4351 and, for flowers after they have sprayed pollen, may consider the pollination process to be complete and exclude them from the targets for detecting the pollination status.

[0073] 1.1.6.2.1 Spray Control For example, the control unit 4312 controls the spraying of liquid or powder by the spray device 420 onto the crops 520 based on the image information acquired by the first imaging device 451. The control of the spraying may include control of the spray timing, spray duration, spray angle, or spray direction. In controlling the spray timing, the control device 430 may control the spray timing of the liquid or powder by the spray device 420 onto the crops based on the image information acquired by the first imaging device 451 and information related to the traveling speed of the traveling vehicle body 410.

[0074] The injection control will be described with reference to the flowchart of injection control of the injection nozzle 421 of the spraying device 400 of this embodiment shown in FIG.

[0075] As shown in FIG. 4, in step 021, the transmitter / receiver 4311 receives information on the spraying location and spraying method from the server 200 and the user terminal 100, and starts self-propelled based on the information.

[0076] Then, in step 022, while the spraying device 400 is traveling in the planned spraying area, the first image capturing device 451 captures image information of the crops 520 before spraying. Although not particularly limited, the spraying device 400 may patrol the field once or multiple times a day and record the image information in association with the location information in the travel data 4351. This allows image information from different points in time to be accumulated for the same location.

[0077] The control unit 4312 may also further identify the buds. For example, if the field is large, it is expected that the spraying device 400 will visit once a day or once every two days. In such a case, by further identifying the buds and predicting the flowering time from the image information, it is possible to estimate the location in the field where the flowers will bloom. This allows the spraying device 400 to quickly go to the location of the flowers that have bloomed and perform pollination work.

[0078] The control unit 4312 may predict the flowering time from image information of the bud stage before flowering, and specify the travel route based on the flowering time and position information within the field.

[0079] In step 023, the control unit 4312 of the control device 430 may control the injection timing, injection duration, injection angle, or injection direction based on the image information. Specifically, the control unit 4312 identifies the position of a target such as a flower from the image information by image processing, and identifies injection nozzles 421 that can inject into ranges S21 and S22 of the image information, as shown in Fig. 2D. For example, in Fig. 2D, when the traveling vehicle body 410 moves in the traveling direction, nozzles X and Y are identified as injection nozzles 421 that can inject into ranges S21 and S22 of the image information.

[0080] The distance between the position of the flower and spray nozzle 421 is then calculated, and based on that distance and the traveling speed of traveling vehicle body 410, the timing of how many seconds it will take for the target flower to be directly above spray nozzle 421 is calculated. For example, Figure 2D shows an example in which, when traveling vehicle body 410 moves in the traveling direction, nozzle X sprays after 5 seconds and nozzle Y sprays after 8 seconds, thereby depositing pollen on flowers in ranges S21 and S22. In this way, control unit 4312 of control device 430 can identify the timing for spraying pollen onto each of target positions S21 and S22.

[0081] Furthermore, in this case, the traveling direction may be taken into consideration in addition to the traveling speed. For example, when traveling on a road surface that is inclined to the left or right, it is expected that the spraying device 400 will gradually move toward the inclined side and will not be able to travel straight. In this way, when the spraying device 400 travels in a direction that deviates from the intended direction due to the road surface condition, the spray nozzle 421 to be used and the spray timing can be corrected by further considering the traveling direction.

[0082] Similarly, when the control unit 4312 identifies the position of a target such as a flower by image processing from image information and identifies a jet nozzle 421 that can spray at the position of the flower, it may identify a jet direction so that the jet nozzle 421 is directed toward the position of the flower, or it may identify a jet angle, which is the angle at which the liquid or gas sprayed from the jet nozzle 421 spreads, so that the target position is included in the jet range. Then, it may control the jet nozzle 421 according to the identified jet direction and jet angle.

[0083] Furthermore, the control unit 4312 of the control device 430 may control the timing and / or duration of spraying of the liquid or powder onto the crops by the spray device 420 so that, in the image information such as that shown in FIG. 2E , more flowers are included in the range of one spray of the spray nozzle. Specifically, the number of flowers included in the ranges S211 and S212 of one spray of the spray nozzle may be compared to identify the range S211 that includes more flowers, and the spray timing may be adjusted so that spray can be performed on the range S211. Alternatively, the duration of one spray of the spray nozzle may be adjusted to enlarge the range S213 of spray so that more flowers are included in the range of one spray of the spray nozzle.

[0084] 2E, the control unit 4312 of the control device 430 may control the spray width and / or spray amount of the spray nozzle so that more flowers are included in the spray range of the spray nozzle in one go. Specifically, the spray range S214 of the spray nozzle may be increased so that more flowers are included in the spray range of the spray nozzle in one go by increasing the spray range. Furthermore, since increasing the spray range (spray width) in this way reduces the pollen density per spray range, both the spray width and the spray amount may be controlled as necessary.

[0085] Furthermore, depending on the type of spray nozzle 421, a preparation time may be required between each spray, i.e., there may be a limit to continuous spraying. In such cases, the control unit 4312 of the control device 430 may further control the timing or duration of spraying of the liquid or powder onto the crops by the spray device 420, taking the preparation time into account, so as to include as many flowers as possible within the spray range of the spray nozzle 421 multiple times. Image information with the same flower arrangement is shown on the left and right sides of Figure 2F. In this case, the left image shows spray ranges S215 and S216 obtained when a second spray is performed immediately after a certain preparation period following the first spray, while the right image shows spray ranges S217 and S218 obtained when a second spray is performed after a certain additional preparation period following the first spray. In this case, the spray ranges S217 and S218, which have a longer time gap, include a larger number of flowers.

[0086] Furthermore, the control unit 4312 of the control device 430 may calculate a score based on any criteria for each flower included in a single spray range of the spray nozzle in image information such as that shown in FIG. 2G, and control at least one of the spray width, spray amount, timing of spraying the liquid or powder onto the crops by the spray device 420, and spray duration based on the score. For example, based on the image information, the control unit 4312 may assign a score to each flower included in a predetermined range as shown in FIG. 5 to evaluate the degree of flowering, and may calculate an average score for the flowers included in the predetermined range or calculate an overall score. Then, the control unit 4312 may control the spray width and / or spray amount of the spray nozzle to define a predetermined range such that the scores are equal to or greater than a predetermined threshold value as the spray range.

[0087] When assigning a score to evaluate the degree of flowering of each flower, for example, among those shown in A to F in FIG. 5, high scores may be assigned to flowers corresponding to D, E, etc., which have a high probability of pollination, and low scores may be assigned to other flowers. FIG. 2G shows an example of image information displaying the average scores and overall scores of flowers included in arbitrary ranges S221, S222, and S223. In this way, the control unit 4312 may set an arbitrary range, assign scores to each flower included in that range, and calculate the average value, etc. Then, the control unit 4312 may set the range in which the score meets a predetermined standard as the spray range, and control the spray width and / or spray amount of the spray nozzle.

[0088] As described above, by controlling the spray range to include multiple flowers and spraying them all at once, the number of sprays can be reduced compared to spraying each individual flower. This reduces the time required for spraying and eliminates the need to stop the traveling vehicle 410 or slow down its speed for each spray, allowing the vehicle to travel more widely within the field within the limited pollination period.

[0089] Furthermore, when controlling the injection timing, the traveling speed of the traveling vehicle body 410 may be controlled taking into account the time required to process image information and calculate the injection timing. For example, as shown in FIG. 2D , the distance L in the traveling direction between the imaging device 450 and the injection nozzle 421 is defined as the distance, and the reference time T is defined as a fixed time allocated for processing from image information acquisition to injection timing calculation. The control device may control the traveling speed S of the traveling vehicle body so that it does not exceed the ratio (distance L / reference time T). If the traveling speed S of the traveling vehicle body exceeds the ratio (distance L / reference time T), this means that the calculation time for image processing, etc. is long, and by the time the injection timing is instructed, the traveling vehicle will have passed the location of the crop 520 that was already captured in the image information. Therefore, even if the injection is performed, the flowers will no longer be there. Therefore, it is preferable to preset the reference time T estimated to be required for image processing, etc., and allow the traveling vehicle to self-propel so that the traveling speed S does not exceed the ratio (distance L / reference time T).

[0090] To give a specific example of the reference time T, it is assumed that it normally takes 0.10T to acquire an image, 0.30T to process the image, and 0.10T from issuing a spray command to spraying. However, if there are many flowers in the image to be processed, the time required to detect them and identify the spray range will be longer, and the time required for each of the above processes may vary depending on the situation, for example, 0.6T may be required for image processing. In order to buffer such fluctuations in processing time, the reference time T is set including a buffer time. Image acquisition 0.10T Image processing (flower detection, spray range identification) 0.30T From spray command to spraying 0.10T Buffer Remaining time

[0091] In this way, by setting a certain reference time T allocated to the processing from acquiring image information to calculating the injection timing, and controlling the traveling speed S based on that reference time T, it is possible to achieve injection at the appropriate position on the crops 520.

[0092] Control unit 4312 of control device 430 may estimate the spray timing based on traveling speed S and distance L. For example, liquid or powder may be sprayed onto crops 520 after L / S time has elapsed since image information was acquired.

[0093] Furthermore, the control unit 4312 of the control device 430 may estimate the injection timing based on the estimated movement distance L' in the traveling direction of the traveling vehicle body and the distance L. For example, when the traveling vehicle body 410 moves forward by the distance L from the acquisition of the image information, the injection nozzle 421 comes to the position of the imaging device 450 that acquired the image information. In other words, by injecting at a timing when the estimated movement distance L' matches the distance L, it is possible to inject pollen into the targeted area.

[0094] The estimated travel distance L' may be calculated based on the travel speed set by the control unit 4312, or may be estimated based on the travel distance calculated from the acceleration data of the traveling vehicle body 410, speed data (instantaneous value of speed), tire diameter, and rotation speed. Here, the "set travel speed" refers to, for example, "10 km / h" when the control unit 4312 drives the traveling mechanism 411 at a set value of 10 km / h. Apart from this, the actual travel speed may differ from the set travel speed due to reasons such as uneven ground. Therefore, the actual travel distance may be calculated based on the acceleration data, speed data (instantaneous value of speed), tire diameter, and rotation speed.

[0095] Next, in step 024, pollen is sprayed from the spray nozzle at the calculated spray timing. At this time, control unit 4312 of control device 430 may control blower 470 according to the distance between spray nozzle 421 and crop 520, and the difference between the range over which pollen should be sprayed specified by control unit 4312 and the range over which spray nozzle 421 can be mechanically sprayed. Specifically, the wind speed of blower 470 may be controlled so that the sprayed pollen reaches crop 520, or so that the sprayed pollen reaches a range over which pollen should be sprayed specified by control unit 4312 that is wider than the range over which spray nozzle 421 can mechanically spray.

[0096] Furthermore, the control device 430 may control at least one of the spray angle, spray intensity, or spray timing of the spray nozzle 421 depending on the external environment such as wind direction, the distance between the spray nozzle 421 and the crop 520, or the range into which the pollen identified by the control device 430 should be sprayed.

[0097] Then, in step 025, the second image capturing device 452 captures image information of the crop 520 after spraying. At this time, the control unit 4312 of the control device 430 may determine whether pollen is attached to the target spot based on the acquired image information and record this information as a pollination operation record. Note that the second image capturing device 452 used here may be a regular camera or an image capturing device for observing IR, fluorescence, etc.

[0098] Finally, in step 026, the control unit 4312 refers to the driving data 4351, and if driving to the location where spraying is scheduled has not been completed, returns to step 022 and repeats the processing, and if driving to the location where spraying is scheduled has been completed, completes the processing.

[0099] The control unit 4312 can perform known control to allow the traveling vehicle body 410 to travel independently within the field 500. Specifically, the control unit 4312 can grasp the absolute or relative position within the field 500, detect obstacles to travel, and control the travel of the traveling vehicle body 410 based on this information.

[0100] Separately, the control unit 4312 may calibrate the traveling speed based on the image information of the first imaging device 451 and the second imaging device 452. Specifically, by comparing the imaging information of the crops 520 and the cultivation structure 530 acquired by the first imaging device 451 and the time of acquisition thereof with the imaging information of the crops 520 and the cultivation structure 530 acquired by the second imaging device 452 and the time of acquisition thereof, the control unit 4312 can calculate the time required for the second imaging device 452 to acquire imaging information similar to the imaging information acquired by the first imaging device 451.

[0101] 2A , since the distance between the first imaging device 451 and the second imaging device 452 in the spraying device 400 is fixed, the actual running speed can be calculated from the calculated time and the distance between the first imaging device 451 and the second imaging device 452. The control unit 4312 may then adjust the drive control of the running mechanism 411 by comparing the actual running speed calculated from the imaging information with the running speed that is assumed to be achieved by driving the running mechanism 411.

[0102] Furthermore, instead of or in addition to controlling the drive of the traveling mechanism 411, the control unit 4312 may use the actual traveling speed calculated from the image capturing information as the speed when calculating the injection timing in step S023. This can further improve the accuracy of the injection timing.

[0103] Similarly, the control unit 4312 can evaluate whether the actual traveling direction is straight or curved by evaluating the rotation of the imaging information acquired by the second imaging device 452 relative to the imaging information acquired by the first imaging device 451. Then, the control unit 4312 may adjust the drive control of the traveling mechanism 411 by comparing the actual traveling direction with the assumed traveling direction.

[0104] 2. Spray Control Module The spray control module of this embodiment is a spray control module attached to traveling vehicle body 410 of spraying device 400 which includes spray device 420 capable of spraying liquid or powder onto crops 520 from one or more spray nozzles, and the spray control module includes imaging device 450 which acquires image information of crops 520, and control device 430, imaging device 450 has a first imaging device 451 which is provided further forward in the traveling direction of traveling vehicle body 410 than spray device 420, and control device 430 which controls the spraying of liquid or powder onto crops 520 by spray device 420 based on the image information acquired by first imaging device 451.

[0105] Since the elements constituting the injection control module of this embodiment have been described above, detailed description thereof will be omitted here.

[0106] 3. Spraying method The spraying method of the present embodiment includes traveling vehicle body 410, imaging device 450 that acquires image information of crop 520, spraying device 420 that can spray liquid or powder onto crop 520 from one or more spray nozzles, and control device 430, and spraying device 420 is provided behind imaging device 450 in the traveling direction of traveling vehicle body 410. Control device 430 of spraying device 400 executes a step of controlling the spraying of liquid or powder by spraying device 420 onto crop 520 based on the image information.

[0107] Note that the specific aspects of the method of this embodiment have been described above in the control process, so a detailed description thereof will be omitted here.

[0108] 4. Spraying Program The spraying program of this embodiment includes traveling vehicle body 410, imaging device 450 that acquires image information of crop 520, spraying device 420 that can spray liquid or powder onto crop 520 from one or more spray nozzles, and control device 430, and spraying device 420 causes control device 430 in spraying device 400, which is provided behind imaging device 450 in the traveling direction of traveling vehicle body 410, to execute a step of controlling the spraying of liquid or powder by spraying device 420 onto crop 520 based on the image information.

[0109] The program may be recorded on a readable recording medium. Note that the specific aspects of the processing executed by the program of this embodiment have been described in the control processing section above, and therefore will not be described in detail here.

[0110] The present invention has industrial applicability as a component technology that can be used in smart agriculture systems.

[0111] 100...user terminal, 200...server, 300...measuring device, 310...fixed sensor, 320...unmanned aerial vehicle, 400...spraying device, 410...traveling vehicle body, 411...traveling mechanism, 420...spraying device, 421...spraying nozzle, 424...nozzle driving mechanism, 425...tank, 430...control device, 431...processor, 432...communication interface, 433...input / output interface, 434...memory, 435...storage, 436...communication bus, 450...imaging device, 451...first imaging device, 452...second imaging device, 460...distance measuring sensor, 470...blower, 500...field, 520...crop, 530...cultivation structure, 4311...transmitting / receiving unit, 4312...control unit, 4351...traveling data

Claims

1. A spraying device comprising a traveling vehicle body, an imaging device that acquires image information of agricultural crops, a spraying device capable of spraying liquid or powder onto the agricultural crops from one or more spray nozzles, and a control device, wherein the imaging device has a first imaging device that is provided forward in the traveling direction of the traveling vehicle body relative to the spraying device, and the control device controls the spraying of the liquid or powder onto the agricultural crops by the spraying device based on the image information acquired by the first imaging device.

2. The spraying device of claim 1, wherein the control device controls the timing, duration, angle or direction of spraying of the liquid or powder by the spraying device onto the crops based on the image information acquired by the first imaging device.

3. The spraying device according to claim 1, wherein the control device controls the timing of spraying the liquid or powder by the spraying device onto the crops based on image information acquired by the first imaging device and information relating to the traveling speed of the traveling vehicle body.

4. The spraying device of claim 1, wherein the control device further controls at least one of the spray width, spray amount, spray timing of the liquid or powder against the crops by the spray device, and spray duration of the spray nozzle so as to cover as many flowers as possible within the range of a single spray of the spray nozzle.

5. The spraying device of claim 1, wherein the control device further calculates a score based on an arbitrary criterion for each flower that falls within the range of one spray of the spray nozzle, and controls at least one of the spray width and spray amount of the spray nozzle, the timing at which the liquid or powder is sprayed by the spray device against the crops, and the duration of spray based on the score.

6. The spraying device as described in claim 1, wherein, when the spray nozzle requires a preparation time between one spray and the next spray, the control device further controls the spray timing or duration of spraying of the liquid or powder by the spray device onto the crops, taking into account the preparation time, so that as many flowers as possible are included in the range of multiple sprays of the spray nozzle.

7. A spraying device as described in claim 1, wherein the distance in the traveling direction between the imaging device and the injection nozzle is distance L, and a fixed time allocated to processing from acquisition of the image information to calculation of the injection timing is reference time T, and the control device controls the traveling speed S of the traveling vehicle body so that it does not exceed the ratio (distance L / reference time T).

8. The spraying device according to claim 7, wherein the control device estimates the injection timing based on the traveling speed S and the distance L.

9. The spraying device according to claim 7, wherein the control device estimates the injection timing based on an estimated movement distance L' of the traveling vehicle body in the traveling direction and the distance L.

10. The spraying device according to claim 1, wherein the spraying device has a nozzle driving mechanism that independently extends and retracts one or more of the spray nozzles.

11. The spraying device according to claim 1, further comprising a blower provided rearward of the spray device in the traveling direction of the traveling vehicle body.

12. A spraying program comprising a spraying device comprising a traveling vehicle body, an imaging device for acquiring image information of agricultural crops, an injection device capable of spraying liquid or powder onto the agricultural crops from one or more injection nozzles, and a control device, the imaging device having a first imaging device arranged forward in the traveling direction of the traveling vehicle body relative to the injection device, the control device executing a step of controlling the spraying of the liquid or powder onto the agricultural crops by the injection device based on the image information acquired by the first imaging device.

13. A spraying method comprising: a traveling vehicle body; an imaging device for acquiring image information of agricultural crops; an injection device capable of spraying liquid or powder onto the agricultural crops from one or more injection nozzles; and a control device, wherein the imaging device has a first imaging device provided forward of the injection device in the traveling direction of the traveling vehicle body, the control device executing a step of controlling the spraying of the liquid or powder by the injection device onto the agricultural crops based on the image information acquired by the first imaging device.

14. A spray control module that is attached to the running body of a spraying device equipped with an injection device capable of spraying liquid or powder onto agricultural crops from one or more injection nozzles, the spray control module comprising: an imaging device that acquires image information of the agricultural crops; and a control device, the imaging device having a first imaging device that is provided forward in the traveling direction of the running body relative to the injection device, and the control device controls the spraying of the liquid or powder onto the agricultural crops by the injection device based on the image information acquired by the first imaging device.

Images