Spray control device, spray control program, spray control method, and spray device

Abstract

The purpose of the present invention is to provide a spray control device, a spray control program, and a spray control method that enable pollination work to be performed precisely. A spray control device comprises: a first acquisition unit that acquires first image information of a crop; an identification unit that identifies a flower in the first image information; and a detection unit that detects the pollination state of the flower on the basis of the first image information.

Description

Spray control device, spray control program, spray control method, and spray device

[0001] The present invention relates to a spray control device, a spray control program, a spray control method, and a spray device.

[0002] As a smart agriculture system, the use of parameters obtained by image processing for operating agricultural equipment is being considered. For example, Patent Literature 1 discloses a system in which predetermined parameters of a predetermined location on a target crop are estimated based on a trained model and an image of the target crop, and an operating mechanism performs a predetermined operation, such as pollinating that location from a predetermined direction, based on the estimated predetermined parameters.

[0003] Patent No. 7090953

[0004] Pollination, which involves applying pollen one by one to the many flowers in a field, is one of the most labor-intensive agricultural tasks. Furthermore, the time available for pollination after each flower blooms is limited, typically 48 hours. Pollination must be timed appropriately within this limited flowering period, taking into account the plant's ever-changing condition. Delays in pollination can lead to reduced yields. Pollinating flowers that bloom randomly in various locations in a field becomes particularly difficult the larger the field. Therefore, pollination requires a concentrated workforce to be secured in advance for a specific period, but securing sufficient personnel is not easy because the timing and number of workers vary from year to year.

[0005] Therefore, it is conceivable to have a pollen-spraying device perform the pollination work instead of a human, but it is difficult to have a self-propelled device accurately spray pollen at any desired location on the crops.

[0006] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a spraying control device, a spraying control program, a spraying control method, and a spraying device that enable pollination work to be performed accurately within a limited pollination period.

[0007] A spraying control device according to one aspect of the present invention has a first acquisition unit that acquires first image information of agricultural crops, an identification unit that identifies flowers in the first image information, and a detection unit that detects the pollination status of the flowers based on the first image information.

[0008] According to the present invention, it is possible to provide a spray control device, a spray control program, a spray control method, and a spray device that enable pollination work to be performed with high accuracy within a limited pollination period.

[0009] 1 is a conceptual diagram of a smart agriculture system. FIG. 2 is a schematic side view of the configuration of the spray control device of this embodiment. FIG. 3 is a schematic top view of the configuration of the spray control device of this embodiment. FIG. 4 is a schematic rear view of the configuration of the spray control device of this embodiment. FIG. 5 is an example of image information of agricultural crops acquired in advance by an imaging device. FIG. 6 is an example of image information of agricultural crops acquired in advance by an imaging device. FIG. 7 is a schematic diagram of the hardware configuration and functional configuration of a control device. FIG. 8 is a schematic diagram showing the data structure of traveling data. FIG. 9 is a flowchart of spray control of the spray control device of this embodiment. FIG. 10 is a sketch diagram showing a series of states from when a flower begins to bloom, until it reaches full bloom and then falls.

[0010] Hereinafter, an embodiment of the present invention (hereinafter referred to as "the present embodiment") will be described in detail with reference to the drawings as necessary. However, the present invention is not limited to this embodiment, and various modifications are possible without departing from the spirit of the present invention. In the drawings, the same elements are given the same reference numerals, and redundant explanations will be omitted. Furthermore, positional relationships such as up, down, left, and right are based on the positional relationships shown in the drawings unless otherwise specified. Furthermore, the dimensional ratios of the drawings are not limited to those shown in the drawings.

[0011] 1. System FIG. 1 is a schematic diagram showing a smart agriculture system 1 according to this embodiment. As shown in FIG. 1, in one example of the smart agriculture system 1 according to this embodiment, a user terminal 100, a server 200, a measurement device 300, and a spraying device 400 are connected via a network N. The measurement device 300 acquires information about agricultural crops in a farm field 500, which is then stored by an agricultural support device (hereinafter referred to as the "server 200"). The server 200 may then formulate plans for each agricultural task, such as a pollination work plan, and transmit the plans to the user terminal 100 used by a farm worker, for example. Furthermore, the server 200 may instruct the spraying device 400 to perform various tasks, such as pollination, based on the pollination work plan, and the spraying device 400 may then execute the plans.

[0012] In this embodiment, a "field" is a cultivation area where agricultural products are grown, and is not limited to an outdoor cultivation area, but may also be a facility with a roof, such as a greenhouse or agricultural product factory.

[0013] The user terminal 100 is a tablet terminal or smartphone carried by a worker who performs agricultural work in the field 500 or a manager who manages the field. The user terminal 100 issues work instructions and provides requested information to the spraying device 400, the measuring device 300, and the server 200 via the network N. Similarly, the user terminal 100 may also acquire information about the field, crops, and environment from the server 200, the measuring device 300, or the spraying device 400.

[0014] The server 200 functions as a management device for comprehensively managing the entire farm field. Specifically, it issues work instructions and provides requested information to the user terminal 100, the measurement device 300, or the spraying device 400 via the communication I / F 220 and the network N. Similarly, the server 200 also acquires information about the farm field, crops, and environment from the user terminal 100, the measurement device 300, or the spraying device 400. Furthermore, the server 200 not only controls and calculates the entire system, but may also function as an instruction unit that executes instructions to the spraying device 400, etc., a processing unit that performs various processes on the acquired information, and a prediction unit that predicts the crop yield and workload.

[0015] The measuring device 300 is a device that acquires information about environmental parameters such as the temperature and humidity of the field and about the crops, and may be a device equipped with a sensor for observing the condition of the field and the crops, or a device that measures the crops 520 in the field 500 and acquires image information and other information about the crops 520.

[0016] The measuring device 300 is not particularly limited, and may be, for example, various fixed sensors 310 installed at any position in the field 500, a drone equipped with various sensors and flying within the field 500, an unmanned aerial vehicle 320 that propels itself within the field, a smartphone equipped with various sensors, a handheld computing device, a wearable terminal, or other terminal operated by a person (not shown). Here, the sensor is not particularly limited, and may further include at least one of an image sensor, a component sensor, and an environmental sensor.

[0017] The measuring device 300 may be linked to the spraying device 400. In addition, although the spraying device 400 and the measuring device 300 are shown as separate devices in Fig. 1, the spraying device 400 may have the functions of the measuring device 300, and the spraying device 400 and the measuring device 300 may be an integrated device.

[0018] By adopting a system constructed in this manner, producers can achieve more productive farm management. Specifically, they can have the spraying device 400 carry out pollination work based on an appropriate pollination work plan.

[0019] The spraying control device 430 of this embodiment acquires image information of the crop 520 from the imaging device 450, estimates the timing of spraying pollen based on the image information, and controls the spraying device 420. The spraying control device 430 may be mounted on the spraying device 400 that has the spraying device 420 mounted on the traveling vehicle body 410, or may not be mounted on the spraying device 400 and may remotely operate the spraying device 400.

[0020] Before describing the spraying control device 430 of this embodiment in detail, a general configuration of the spraying device 400 will be described. Fig. 2A shows a general side view of the configuration of the spraying device 400 of this embodiment. As shown in Fig. 2A, the spraying device 400 is not particularly limited, but includes, for example, a traveling vehicle body 410, an imaging device 450 that acquires image information of the crops 520, an injection device 420 that can inject liquid or powder onto the crops, and a spraying control device 430.

[0021] The spraying device 400 sprays a liquid or powder onto the crops 520 using the spraying device 420 while the traveling vehicle body 410 is traveling. For example, if the sprayed material is liquid pollen or powder pollen, pollination work can be carried out, and if the sprayed material is a pesticide or the like, pesticide spraying work can be carried out. Note that liquid pollen is pollen dispersed in a liquid such as water. Furthermore, powder pollen may be pollen itself, or pollen with a bulking agent added.

[0022] In this embodiment, "agricultural crops" refers to plants that are cultivated and harvested in agriculture. When simply referring to "agricultural crops" in this embodiment, it simply means plants that are cultivated in agriculture, without distinguishing between the trunks, branches, flowers, buds, leaves, stems, shoots, fruits, etc. of the plants. Therefore, in this embodiment, "agricultural crops" does not refer only to the fruits, etc. that are ultimately obtained.

[0023] The spraying device 400 is not particularly limited as long as it is a vehicle that can travel within a field, and may be, for example, a self-propelled spraying device that recognizes the environment using various sensors and control algorithms and can travel autonomously within a field, or an operable spraying device that can travel within a field by being ridden by a person or remotely operated.

[0024] The traveling vehicle body 410 is not particularly limited, but may include, for example, a vehicle having a traveling mechanism 411 such as wheels or caterpillars that can travel, and a power unit (not shown) that drives the traveling mechanism 411. The traveling vehicle body 410 may also have various sensors such as a GPS, a speed sensor, an acceleration sensor, a distance sensor, and a geomagnetic sensor so that it can determine its own position and avoid obstacles when traveling on its own.

[0025] The traveling vehicle body 410 may be capable of autonomously traveling within the field, or may be capable of self-traveling within the field according to traveling route instructions received from the user terminal 100 or the server 200. Alternatively, the traveling vehicle body 410 may be manually operated by the user according to instructions from the user terminal 100. Furthermore, control of the traveling speed, traveling direction, traveling route, and other aspects of the traveling vehicle body 410's self-travel may be performed by the spraying control device 430, which will be described later.

[0026] Furthermore, the various sensors of the traveling vehicle body 410 may cooperate with the traveling control unit 4318 (described later) to realize SLAM (Simultaneous Localization and Mapping). Specifically, the various sensors may sense the surrounding environment while the traveling vehicle body 410 is traveling, thereby creating a two-dimensional or three-dimensional environmental map, while simultaneously estimating the traveling vehicle body's position on the environmental map.

[0027] There are no particular limitations on spray device 420, as long as it has a configuration capable of spraying liquid or powder onto crops 520. Specifically, spray device 420 may have one or more spray nozzles 421 and a nozzle drive mechanism 424 that independently controls the extension and retraction of one or more spray nozzles 421, and may also have a tank 425 containing liquid or powder as needed. In this way, liquid or powder sucked up from tank 425 may be sprayed from spray nozzles 421 brought close to crops 520 by nozzle drive mechanism 424. Control of the spray timing and amount sprayed, and the extension and retraction of spray nozzles 421 while traveling vehicle body 410 is self-propelled may be performed by spray control device 430, which will be described later.

[0028] 2B shows a schematic top view of the configuration of spraying device 400 of this embodiment, and Fig. 2C shows a schematic rear view of the configuration of spraying device 400 of this embodiment. Spray nozzle 421 is not particularly limited as long as it is a nozzle that sprays liquid or powder onto crop 520 in response to instructions from spraying control device 430, which will be described later, or the like.

[0029] The spray nozzle 421 is not particularly limited as long as it is a known spray nozzle, but examples include a one-fluid nozzle that mainly sprays one liquid, a two-fluid nozzle that sprays two fluids such as a gas and a liquid, and a nozzle that sprays powder together with a gas. Specifically, a one-fluid nozzle may spray liquid pollen, or a two-fluid nozzle may spray a mixture of air and liquid pollen.

[0030] 2B shows an example of an arrangement of the injection nozzles 421. As an example, the injection nozzles 421 may be arranged in a single row in the width direction of the traveling vehicle body, or may be arranged in multiple rows as shown in FIG.

[0031] 2B , the spray ranges S21, S22 of adjacent spray nozzles 421 may partially overlap. A time lag occurs between the time a spray nozzle 421 sprays and the time it next sprays, such as when it needs to refill the spray material. In other words, pollen, etc., is not sprayed onto the crops 520 in the area through which the traveling vehicle 410 passes between the time a spray is made and the time a spray is made. If there is a flower in an area where pollen, etc., is not sprayed, it means that pollination has not occurred on that flower. By partially overlapping the spray ranges S21, S22 of adjacent spray nozzles 421, even if one spray nozzle 421 is unable to immediately spray pollen, etc., due to the time lag described above, the adjacent spray nozzle 421 can still spray pollen, etc., thereby avoiding a decrease in pollination efficiency.

[0032] 2B, some spray nozzles 421 may be positioned further back in the running direction than other spray nozzles 421. By providing spray nozzles 421 at different positions in the running direction in this way, as in the above, even if one spray nozzle 421 cannot immediately spray pollen or the like due to the time lag described above, the adjacent spray nozzle 421 can still spray pollen or the like, thereby avoiding a decrease in pollination efficiency.

[0033] The nozzle drive mechanism 424 is a mechanism that changes the length of the nozzle using the spray control device 430 described below, and is not particularly limited as long as it is a known mechanism that can extend or retract the length of the spray nozzle 421. For example, the nozzle drive mechanism 424 may be extendable by having a multi-layer sleeve structure. The nozzle drive mechanism 424 may also be capable of adjusting the inclination of the spray nozzle 421 and the direction of the nozzle opening. This allows the liquid or powder to be effectively sprayed in directions other than the extension direction of the nozzle.

[0034] Spraying device 400 of this embodiment has imaging device 450 that acquires image information of crop 520. Furthermore, as shown in Fig. 2A , imaging device 450 may have a first imaging device 451 provided further forward in the traveling direction of traveling vehicle body 410 than spraying device 420, and may also have a second imaging device 452 provided further backward in the traveling direction of traveling vehicle body 410 than spraying device 420.

[0035] When traveling vehicle body 410 moves in the traveling direction, first imaging device 451 passes directly below crop 520 before spray nozzle 421, and second imaging device 452 passes directly below crop 520 after spray nozzle 421. Therefore, first imaging device 451 acquires image information of crop 520 in advance, spraying control device 430 calculates spray timing based on the image information, and spray nozzle 421 can spray liquid or powder at the calculated appropriate timing. Furthermore, second imaging device 452 acquires image information of crop 520 onto which liquid or powder has been sprayed, and based on the image information, spraying control device 430 can confirm whether the sprayed liquid or powder has adhered to crop 520 and the extent of adhesion.

[0036] The imaging device 450 is not particularly limited as long as it is capable of capturing still images or videos. Furthermore, a plurality of first imaging devices 451 and second imaging devices 452 may be arranged in the width direction of the traveling vehicle body. As shown in FIG. 2B , when a plurality of imaging devices 450 are provided, image information acquired by the plurality of imaging devices 450 may be combined into a single image. In this case, the fields of view S11, S12 of the plurality of imaging devices 450 may partially overlap.

[0037] 2D shows an example of image information of crop 520 acquired by first imaging device 451 by combining fields of view S11 and S12. Areas S21 and S22 in FIG. 2D correspond to the spray range of the spray nozzle. In this way, spray control device 430 may identify the spray range based on the image information, calculate the spray timing based on that, and spray nozzle 421 may spray the liquid or powder at the calculated appropriate timing.

[0038] Spraying device 400 of the present embodiment may further include distance measurement sensor 460 facing the direction of crops 520. The distance measurement sensor 460 is not particularly limited, but examples thereof include an image sensor, an ultrasonic sensor, an infrared sensor, and a laser sensor.

[0039] 2A and 2B, distance measurement sensor 460 may be provided further forward in the traveling direction of traveling vehicle body 410 than spray device 420. In this way, when traveling vehicle body 410 moves in the traveling direction, distance measurement sensor 460 passes directly below crop 520 before spray nozzle 421. Therefore, distance measurement sensor 460 can obtain information about the distance to crop 520 in advance, and spray nozzle 421 may be extended or retracted by nozzle drive mechanism 424 or the spray speed may be adjusted according to the distance information.

[0040] The spraying device 400 of this embodiment may have a blower 470. As shown in FIGS. 2A and 2B , the blower 470 may be provided further forward in the traveling direction than the spraying device 420, or further rearward in the traveling direction than the spraying device 420. Since the liquid or powder is sprayed while the traveling vehicle body 410 is traveling, the sprayed liquid or powder is located rearward in the traveling direction. Therefore, by providing the blower 470 further rearward in the traveling direction than the spraying device 420, the liquid or powder can be more efficiently dispersed by blowing air. Note that the number of blowers 470 may be one or more.

[0041] Hereinafter, each component of the spraying device 400 of this embodiment, which is mounted on the running vehicle body 410 and has the spraying device 420, will be described in detail.

[0042] 1.1 Spray Control Device The spray control device 430 performs various controls of the spray device 400 of this embodiment, and may, for example, control the spraying of the spray device 420 as well as the self-propelled control of the traveling vehicle body 410. The spray control device 430 may, for example, be an information processing device mounted on the spray device 400, or may be an information processing device that is not mounted on the spray device 400 and remotely controls the spray device 400. Examples of the information processing device that constitutes the spray control device 430 include a general-purpose computer such as a desktop, laptop, or tablet, or a PLC (Programmable Logic Controller).

[0043] In the following description, each process is executed by the spraying control device 430 mounted on the spraying device 400, but the various processes described below may be executed by the spraying control device 430 connected to the spraying device 400 via a network N, and the spraying control device 430 may remotely control the spraying device 400. In this case, for example, the server 200 including the functions and configuration of the spraying control device 430 may remotely control the spraying device 400.

[0044] Hereinafter, the hardware configuration and functional configuration of the spray control device 430 will be described with reference to FIG. 3A, and then each control will be described in detail in association with the functional configuration of the spray control device 430.

[0045] As shown in FIG. 3A, the spray control device 430 includes, for example, a processor 431, a communication interface 432, an input / output interface 433, a memory 434, a storage 435, and one or more communication buses 436 for interconnecting these components.

[0046] The processor 431 executes processes, functions, or methods implemented by codes or instructions included in a program stored in the storage 435. The processor 431 may include, for example and without limitation, one or more central processing units (CPUs), MPUs, GPUs, etc., and may implement the processes, functions, or methods disclosed in each embodiment by a logic circuit (hardware) formed in an integrated circuit or the like, or a dedicated circuit.

[0047] As shown in FIG. 3A, the processor 431 of this embodiment may be configured to function as a transceiver unit 4311, a first acquisition unit 4312, an identification unit 4313, an estimation unit 4315, an injection control unit 4316, a second acquisition unit 4317, a driving control unit 4318, and a nozzle control unit 4319.

[0048] The communication interface 432 transmits and receives various data to and from other devices via the network N. The communication may be performed either wired or wirelessly, and any communication protocol may be used as long as mutual communication is possible. For example, the communication interface 432 is implemented as hardware such as a network adapter, various types of communication software, or a combination of these.

[0049] The network N may be, by way of example and not limitation, an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), a portion of the Internet, a portion of the public switched telephone network (PSTN), a cellular network, integrated service digital networks (ISDNs), wireless LANs, long term evolution (LTE), code division multiple access (CDMA), Bluetooth, satellite communications, or the like, or any combination thereof. A network may include one or more networks.

[0050] The input / output interface 433 includes an input device for inputting various operations to the spray control device 430, and an output device for outputting processing results processed by the spray control device 430. For example, the input / output interface 433 may include, in addition to the above-described imaging device 450, distance measurement sensor 460, etc., information input devices such as a keyboard, mouse, and touch panel, and information output devices such as the above-described traveling mechanism 411, spray device 420, and display. Note that the spray control device 430 may accept predetermined inputs and execute predetermined outputs by connecting an external input / output interface 433.

[0051] The memory 434 temporarily stores programs loaded from the storage 435 and provides a working area for the processor 431. The memory 434 also temporarily stores various data generated while the processor 431 is executing the programs. The memory 434 may be, for example, a high-speed random access memory such as a DRAM, an SRAM, a DDR RAM, or other random access solid-state storage device, or a combination of these.

[0052] The storage 435 stores programs, each functional unit, and various data. The storage 435 may be, for example, one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or nonvolatile memories such as other nonvolatile solid-state storage devices, or a combination thereof. Another example of the storage 435 may be one or more storage devices installed remotely from the processor 431.

[0053] The spraying control device 430 may record the travel of the spraying device 400 and various data acquired during travel as travel data 4351. As the spraying device 400 travels within the field, it can accumulate information such as acquired image information, detected flower pollination status, and pollen spray conditions in association with location information. Therefore, the information recorded in the travel data 4351 is map information such as image information within the field, flower pollination status, and spray conditions.

[0054] Furthermore, the spraying device 400 patrols the field and can acquire various image information, the pollination status of flowers, etc. at the same point at different times. Therefore, the information recorded in the travel data 4351 is data showing changes over time at specific points in the field, or map information recording changes over time.

[0055] 3B shows an example of the data structure of the driving data 4351. In the driving data 4351, a "driving ID" that uniquely identifies the autonomous driving data, a "driving log," "image information" acquired during driving, and a "nozzle control log" may be recorded in association with each other. In particular, the "image information" and "nozzle control log" may be recorded in association with the driving route of the driving log.

[0056] The "driving log" may store records of driving such as the driving location, driving route, driving speed, and time spent driving autonomously, as well as information regarding driving schedules or spraying locations received from the server 200, etc.

[0057] Furthermore, the "image information" may store image information of crop 520 captured by first imaging device 451 before the liquid or powder is sprayed, and / or image information of crop 520 captured by second imaging device 452 after the liquid or powder is sprayed. Here, image information of crop 520 before the liquid or powder is sprayed may be used as information for timing the spraying. Furthermore, image information of crop 520 after the liquid or powder is sprayed may be used as the result of the spraying work.

[0058] The "nozzle control log" may include control information about the extension / retraction position of the injection nozzle 421 by the nozzle drive mechanism 424, as well as control information about the injection timing and injection amount. This information may be recorded in association with the time when the extension / retraction or injection was performed and the travel route.

[0059] 1.1.1. Transmitter / Receiver The transmitter / receiver 4311 may function as a transmitter that transmits various types of information to other devices such as the user terminal 100, the server 200, and the measurement device 300 via the communication interface 432 and the network N, or as a receiver that receives various types of information from other devices such as the user terminal 100, the server 200, and the measurement device 300.

[0060] For example, the transmitting / receiving unit 4311 may receive information on the spraying location and spraying method from the user terminal 100 or the server 200. The transmitting / receiving unit 4311 may also transmit various data such as acquired image information and information on the progress of the spraying work to the user terminal 100 or the server 200. The transmitting / receiving unit 4311 may refer to the traveling data 4351 and transmit this various information to the user terminal 100 or the server 200.

[0061] First acquisition unit 4312 acquires first image information of crop 520. Specifically, as shown in FIG. 2A , first acquisition unit 4312 may acquire image information of crop 520 from first imaging device 451 that is provided further forward in the traveling direction of traveling vehicle body 410 than injection device 420.

[0062] For example, suppose that the spraying device 400 receives a command from the server 200 to perform pollination work within a specific area in a farm field. When the spraying device 400 arrives at the specified area, the traveling vehicle body 410 travels within the area and captures an image of the crops using the first imaging device 451 provided ahead in the traveling direction. The first acquisition unit 4312 acquires the first image information captured by the first imaging device 451 in this manner.

[0063] At this time, the first imaging device 451 may record the first image information together with location information ( FIG. 3B ). The location information of the point where the first image information is acquired can be acquired by a sensor such as a GPS provided in the traveling vehicle body 410, or by various other sensors used to grasp the location of the traveling vehicle body 410 when the traveling vehicle body 410 is moving under its own power.

[0064] It is preferable that spraying device 400 monitors the condition of crop 520, which changes daily, and performs pollination work within a limited pollination period. From this perspective, for example, spraying device 400 may run within the field two or more times a day, and first acquisition unit 4312 may acquire first image information of crop 520 at different times.

[0065] Then, based on the plurality of pieces of first image information at different times, the identification unit 4313 (described later) may identify newly bloomed flowers. This allows the detection unit 4314 to detect the pollination state of the newly bloomed flowers, and the spray control device 4316 to more efficiently perform pollination processing.

[0066] Furthermore, the identification unit 4313, which will be described later, may identify a bud that will newly bloom into a flower based on a plurality of pieces of first image information at different times. In this way, the travel control unit 4318 may determine the travel route of the spraying device 400 by anticipating the timing when the flower will bloom.

[0067] 1.1.3 Identification Unit The identification unit 4313 identifies flowers in the first image information acquired by the first acquisition unit 4312. At this time, the identification unit 4313 may identify flowers to be pollinated. Specifically, the position of the flowers in the first image information may be identified by performing image processing on the first image information. The image processing method is not particularly limited as long as it is a known image recognition method that can identify the target object (flower) from the image information, and a model that has learned the feature amounts of the flower may be used.

[0068] 1.1.4. Detection Unit The detection unit 4314 detects the pollination state of the flower based on the first image information. Examples of the pollination state include, but are not limited to, "before pollination" and "pollination completed." The detection method is also not limited to, but may include, for example, detecting flowers with pollen attached using the color of the pollen as an identification means in the first image information. For example, if the pollen itself has a color such as yellow, the color may be used as an identification means. Alternatively, if liquid pollen or powder pollen is used as described above, a colorant may be added to the liquid pollen or powder pollen, and the color of the colorant may be used as an identification means. If the first imaging device 451 includes an ultraviolet camera or an infrared camera, the colorant may be a colorant that can be detected by such a camera.

[0069] Furthermore, instead of detecting the color of the pollen or coloring material, the state of the flower may be used to detect whether it is unpollinated or pollinated. For example, after pollination, petal shedding and the start of fruit formation can be confirmed, and the state of the flower may be detected based on such appearance factors.

[0070] Note that the result of detecting the pollination state of the flower is the result of detecting the pollination state before the pollen spraying process, and is different from the processing result acquired by the second acquisition unit, which is confirmation of whether pollen has adhered as a result of the pollen spraying process that will be performed soon.

[0071] The detection unit 4314 may further determine whether or not a blooming flower is suitable for pollination based on image information. Figure 5 shows a sketch illustrating a series of states from when a flower begins to bloom until it reaches full bloom and then falls. As shown in Figure 5, it takes a certain period of time for a flower to reach full bloom after it begins to bloom. Although it depends on the crop, it is not necessarily suitable to perform pollination immediately after blooming; from the standpoint of fertilization ability, it is preferable to perform pollination taking into account the degree of blooming.

[0072] The relationship between the degree of flowering and fertility can be determined appropriately for each crop and is not particularly limited in this regard. However, for the sake of explanation, let us consider a crop that is suitable for pollination when in full bloom. In this case, the detection unit 4314 may further determine whether a flower is suitable for pollination based on the image information and the degree of flowering of the flower. In this case, the detection unit 4314 may refer to a data table, such as that shown in FIG. 5, that associates image data indicating the degree of flowering with information regarding the suitability of pollination. Specifically, the detection unit 4314 may identify a flower from the image information and evaluate the degree of flowering of the flower by identifying image data similar to the identified flower image from image data of flowers with different degrees of flowering in the data table. The detection unit 4314 may then refer to the data table to identify information regarding the suitability of pollination corresponding to the evaluated degree of flowering of the flower, and further determine whether the flower is suitable for pollination based on the information.

[0073] As a result, for example, the detection unit 4314 can determine that flowers that are just before or just after full bloom, such as D to E shown in Fig. 5, are suitable for pollination, and can determine that flowers that are not yet in full bloom, such as B to C shown in Fig. 5, and flowers that are in full bloom, such as F to G, are not suitable for pollination. Therefore, it is possible to efficiently pollinate flowers that are suitable for pollination, and it is possible to eliminate the waste of pollination work on flowers that are not suitable for pollination, thereby reducing the waste of pollen.

[0074] Furthermore, the detection unit 4314 may predict whether the flower is in a state suitable for pollination, i.e., its fertility, based on information related to the image information.

[0075] Specifically, as shown in FIG. 5 , during the flowering process, the petals gradually open from a bud. Therefore, for example, the characteristics change from a green circular bud to a green and white rectangular half-bloomed state, and then to a white full-bloomed state. For an object recognized as a bud or half-bloomed flower, the detection unit 4314 may further determine the pollination suitability based on the degree of blooming, as determined by the shape and color area of ​​the object. More specifically, the pollination suitability may be predicted by quantifying the area of ​​the color of the petals, such as white, and modeling the change, or the pollination suitability may be predicted from changes in parameters indicating the shape of the flower.

[0076] Additionally, as time passes after flowering, the color of the petals changes as the flower wilts, for example, from white petals to yellow. The detection unit 4314 may determine the suitability for pollination based on such colors or color changes that can be determined from the first image information. Furthermore, when making this determination, environmental data such as accumulated temperature and accumulated illuminance may be taken into consideration. This tends to further improve the accuracy of predicting the suitability for pollination.

[0077] The detection unit 4314 may further determine whether the flower shown in the image is a male flower or a female flower from the image information. By distinguishing between male and female flowers through such a determination, it is possible to distinguish between unpigmented male flowers and unpigmented female flowers. This allows unpigmented female flowers to be more accurately identified and pollination work to be performed. Furthermore, by preventing pollen from being sprayed onto male flowers, it is possible to reduce wasteful work and pollen.

[0078] Specifically, the detection unit 4314 may determine the characteristic amount of the female flower, such as the presence or absence of a pistil, based on the image information.

[0079] Furthermore, the detection unit 4314 may refer to the detection results recorded in the driving data 4351 and, for flowers after they have sprayed pollen, may consider the pollination process to be complete and exclude them from the targets for detecting the pollination status.

[0080] The estimation unit 4315 estimates the timing of spraying pollen onto the flower identified by the identification unit 4313 as the target for pollination treatment, based on the first image information acquired by the first acquisition unit 4312. At this time, the estimation unit 4315 may estimate the timing of spraying pollen onto the identified flower based on the first image information and information related to the traveling direction and traveling speed of the traveling vehicle body 410 acquired from the traveling control unit 4318, which will be described later.

[0081] The estimated injection timing is not particularly limited, but may include, for example, information regarding the time, such as how many seconds after the acquisition of the first image information, after which the injection should be performed. In addition, if there are multiple injection nozzles, information regarding the nozzle that will inject pollen may also be included.

[0082] Furthermore, because the ground of a farm field is undulating and not flat, the travel distance of the traveling vehicle body 410 (hereinafter also referred to as the "set travel distance") derived from the traveling direction and traveling speed of the traveling vehicle body 410 controlled by the traveling control unit 4318 may differ from the actual travel distance of the traveling vehicle body 410. Considering the size and spray range of the flowers that spray pollen, the impact of such errors on the spray timing may not be negligible. Therefore, the estimation unit 4315 may estimate the spray timing based on the estimated travel distance L' in the traveling direction of the traveling vehicle body acquired from the traveling control unit 4318. Here, the estimated travel distance L' may be estimated based on the acceleration data of the traveling vehicle body 410, as well as speed data (instantaneous value of speed), a travel distance calculated from the tire diameter and rotation speed, and the like.

[0083] The spray control unit 4316 may cause the spray device 420 to spray pollen onto the flower in accordance with the spray timing estimated by the estimation unit 4315. Furthermore, if the difference between the set travel distance and the estimated travel distance becomes large between the time the estimation unit 4315 estimates the spray timing and the time the spray device 420 starts spraying, the spray timing may be corrected in accordance with the difference between the set travel distance and the estimated travel distance.

[0084] 1.1.7 Second Acquisition Unit The second acquisition unit 4317 may acquire processing results of the flowers onto which pollen has been sprayed. Specifically, the second acquisition unit 4317 may acquire second image information of the crop 520 from the second imaging device 452, which is provided further rearward in the traveling direction of the traveling vehicle body 410 than the spraying device 420, and may detect whether the flowers shown in the image have pollen attached based on the second image information, and output the result. The "processing result" here refers to the result of confirming whether pollen has attached as a result of the immediately preceding pollen spraying process, and is different from the result of detection of the pollination state before the pollen spraying process, detected by the detection unit 4314.

[0085] In addition, since the second image information contains only flowers with pollen or flowers without pollen, in this embodiment, the detection result of flowers with pollen is also the detection result of flowers without pollen.

[0086] The method for detecting whether or not pollen is attached is not particularly limited, but for example, flowers with pollen attached may be detected using the color of the pollen in the second image information as an identification means, or flowers with pollen attached may be detected by comparing the second image information with the first image information and observing the differences.

[0087] The second acquisition unit 4317 may record the detection results thus acquired together with location information in the travel data ( FIG. 3B ). The location information of the location where the detection results were acquired can be acquired by sensors such as a GPS installed in the travel vehicle 410, or by various other sensors used to determine the travel vehicle 410's location when traveling independently. This allows the location information of flowers without pollen to be recorded. Because the flowering period of flowers varies depending on the crop, if there are newly bloomed crops, the spraying device 400 will pass the same location in the field multiple times. In this case, by using the detection results recorded together with the location information, additional pollination can be performed when the spraying device 400 travels through a location where flowers without pollen are located. This prevents flowers from being left unpollinated, enabling efficient pollination.

[0088] In this case, the information recorded may be detailed information indicating an absolute position that can uniquely identify the position of a pollen-free flower, or may be abstract information such as the presence or absence of pollen-free flowers at a point within a certain range. In the case of abstract information, information regarding the number of pollen-free flowers may be included in addition to the presence or absence of pollen-free flowers. Furthermore, the "certain range" is not particularly limited, and may be, for example, a range approximately the same as the field of view of the second image information.

[0089] 1.1.8 Travel Control Unit The travel control unit 4318 controls the travel of the spraying device 400. Specifically, the travel mechanism 411 may be controlled to control the travel direction based on the orientation of the wheels, or the travel speed based on the rotation speed of the axle of the drive unit. In addition to issuing control instructions to the travel mechanism 411, the travel control unit 4318 may obtain information regarding the actual travel direction and actual travel speed from acceleration data, speed data (instantaneous speed), or tire diameter and rotation speed using various sensors provided in the spraying device 400.

[0090] Furthermore, the travel control unit 4318 may control the travel of the spraying device 400 in accordance with the spray timing estimated by the estimation unit 4315. For example, if the spray timing estimated by the estimation unit 4315 is 5 seconds after the acquisition of the first image information, the travel of the spraying device 400 may be controlled so that the target flower is positioned directly above the spray nozzle 421 5 seconds later.

[0091] More specifically, depending on the road surface conditions, the difference between the set travel distance and the estimated travel distance may become large between the time when the estimation unit 4315 estimates the estimated injection timing to be 5 seconds after the acquisition of the first image information and the time when injection by the injection device 420 begins. In such a case, the travel control unit 4318 controls the travel of the spraying device 400 so that the estimated travel distance is approximately the same as the set travel distance. In other words, if the estimated travel distance is longer than the set travel distance, the travel speed of the spraying device 400 is controlled to be slower, and if the estimated travel distance is shorter than the set travel distance, the travel speed of the spraying device 400 is controlled to be faster. In this way, the travel control unit 4318 can control the travel of the spraying device 400 in accordance with the injection timing estimated by the estimation unit 4315.

[0092] The travel control unit 4318 may determine a travel route based on the detection result acquired by the second acquisition unit 4317. Specifically, the travel route may include locations where flowers with no pollen are located. This prevents flowers from being left unpollinated, enabling efficient pollination.

[0093] When the distance in the traveling direction between the first image capturing device 451 and the spraying device 420 is defined as distance L, and a certain time allocated to the processing from acquisition of image information to calculation of the injection timing is defined as reference time T, the traveling control unit 4318 may control the traveling speed S of the traveling vehicle body so that it does not exceed the ratio (distance L / reference time T). This makes it possible to avoid a situation where the spraying device 400 passes under flowers when spraying pollen.

[0094] 1.1.9 Nozzle Control Unit When the spray device 420 has one or more spray nozzles 421 and a nozzle drive mechanism 424 that independently extends and retracts the spray nozzles 421, the nozzle control unit 4319 may control the extension and retraction of the spray nozzles 421 using the nozzle drive mechanism 424. This allows the spray nozzles 421 to be positioned closer to the flowers that are the target of pollen spraying, thereby improving pollination efficiency.

[0095] 1.2. Operation Processing 1.2.1. Spray Control Spray control by the spray control device 430 will be described with reference to the flowchart of spray control of the spray nozzle 421 of the spray device 400 of this embodiment shown in FIG.

[0096] As shown in FIG. 4, in step 01, the transmitter / receiver 4311 receives information on the spraying location and spraying method from the user terminal 100 and the server 200, and the spraying device 400 starts self-propelled based on this information.

[0097] Then, in step 02, while traveling within the planned spraying location, first imaging device 451 captures image information of crops 520 before spraying, and first acquisition unit 4312 acquires first image information of crops 520. Although not particularly limited, spraying device 400 may patrol the field once or multiple times a day and record the first image information in association with location information in travel data 4351. This allows first image information from different points in time to be accumulated for the same location.

[0098] In step 03, the identification unit 4313 identifies flowers to be pollinated in the first image information acquired by the first acquisition unit 4312. At this time, the detection unit 4314 may detect the pollination state of the flowers based on the first image information, and identify flowers that have not been pollinated as flowers to be pollinated.

[0099] The identification unit 4313 may also identify buds. For example, if the field is large, it is expected that the spraying device 400 will visit once a day or once every two days. In such a case, by further identifying buds and predicting the flowering time from the first image information, it is possible to estimate the location in the field where flowers will bloom. This allows the spraying device 400 to quickly go to the location of the flower that has bloomed and perform pollination work.

[0100] The identification unit 4313 may predict the flowering time from the first image information of the bud stage before flowering, and identify the travel route based on the flowering time and the position information within the field.

[0101] In step 04, the estimation unit 4315 estimates the timing of spraying pollen onto the flower identified by the identification unit 4313 as the target for pollination treatment, based on the first image information acquired by the first acquisition unit 4312. At this time, the estimation unit 4315 may calculate the spray timing based on the first image information and information regarding the traveling direction and traveling speed of the traveling vehicle body acquired from the traveling control unit 4318.

[0102] Specifically, as shown in Fig. 2D, the estimation unit 4315 identifies spray nozzles 421 that can spray into ranges S21 and S22 of the image information based on the position of the flower identified by the identification unit 4313 through image processing of the first image information. For example, in Fig. 2D, when the traveling vehicle body 410 moves in the traveling direction, nozzles X and Y are identified as spray nozzles 421 that can spray into ranges S21 and S22 of the image information.

[0103] The estimation unit 4315 then calculates the distance between the position of the flower and the spray nozzle 421, and calculates the time (spray timing) until the position of the target flower comes directly above the spray nozzle 421 based on that distance and the traveling direction and traveling speed of the traveling vehicle body 410. For example, Figure 2D shows an example in which, when the traveling vehicle body 410 moves in the traveling direction, nozzle X sprays after 5 seconds and nozzle Y sprays after 8 seconds, thereby depositing pollen on flowers in ranges S21 and S22. In this way, the estimation unit 4315 can identify the timing for spraying pollen onto each of the target positions S21 and S22.

[0104] In this case, the estimation unit 4315 may control the timing and / or duration of spraying of the liquid or powder onto the crops by the spray device 420 so that more flowers are included in the range of one spray of the spray nozzle in the first image information as shown in FIG. 2E. Specifically, the estimation unit 4315 may compare the numbers of flowers included in the ranges S211 and S212 of one spray of the spray nozzle to identify the spray range S211 that includes more flowers, and adjust the spray timing so that spray can be performed on the spray range S211. Alternatively, the duration of one spray of the spray nozzle may be adjusted to enlarge the spray range S213 so that more flowers are included in the range of one spray of the spray nozzle.

[0105] 2E, the estimation unit 4315 may control the spray width and / or spray amount of the spray nozzle so that more flowers are included in the spray range of the spray nozzle in one go. Specifically, the spray range S214 of the spray nozzle may be increased so that more flowers are included in the spray range of the spray nozzle in one go by increasing the spray range. Furthermore, since increasing the spray range (spray width) in this way reduces the pollen density per spray range, both the spray width and the spray amount may be controlled as necessary.

[0106] Furthermore, depending on the type of spray nozzle 421, a preparation time may be required between each spray, i.e., there may be a limit to continuous spraying. Here, preparation time refers to the time during which spraying is not possible. In such cases, the estimation unit 4315 may further control the timing or duration of spraying of the liquid or powder onto the crops by the spray device 420, taking the preparation time into account, so as to include as many flowers as possible within the range of multiple sprays by the spray nozzle 421. Image information with the same flower arrangement is shown on the left and right sides of Figure 2F. Here, the left image shows spray areas S215 and S216 obtained when a second spray is performed immediately after a certain preparation period following the first spray, while the right image shows spray areas S217 and S218 obtained when a second spray is performed after a certain additional preparation period following the first spray. In this case, the spray areas S217 and S218, which have a longer time gap, include a larger number of flowers.

[0107] Furthermore, when controlling the injection timing, the travel control unit 4318 may control the travel speed of the traveling vehicle body 410, taking into account the time required to process image information and calculate the injection timing. For example, as shown in FIG. 2D , if the distance L is the distance in the travel direction between the imaging device 450 and the injection nozzle 421, and the reference time T is a fixed time allocated for processing from image information acquisition to injection timing calculation, the control device may control the travel speed S of the traveling vehicle body so that it does not exceed the ratio (distance L / reference time T). If the travel speed S of the traveling vehicle body exceeds the ratio (distance L / reference time T), it means that the calculation time for image processing, etc. is long, and by the time the injection timing is instructed, the traveling vehicle will have passed the location of the crop 520 that was already captured in the image information. Therefore, even if the injection is performed, the flowers will no longer be there. Therefore, it is preferable to preset the reference time T estimated to be required for image processing, etc., and self-propel the traveling vehicle so that the travel speed S does not exceed the ratio (distance L / reference time T).

[0108] To give a specific example of the reference time T, it is assumed that it normally takes 0.10T to acquire an image, 0.30T to process the image, and 0.10T from issuing a spray command to spraying. However, if there are many flowers in the image to be processed, the time required to detect them and identify the spray range will be longer, and the time required for each of the above processes may vary depending on the situation, for example, 0.6T may be required for image processing. In order to buffer such fluctuations in processing time, the reference time T is set including a buffer time. Image acquisition 0.10T Image processing (flower detection, spray range identification) 0.30T From spray command to spraying 0.10T Buffer Remaining time

[0109] In this way, by setting a certain reference time T allocated to the processing from acquiring image information to calculating the injection timing, and controlling the traveling speed S based on that reference time T, it is possible to achieve injection at the appropriate position on the crops 520.

[0110] The estimation unit 4315 may also estimate the injection timing based on the estimated movement distance L' in the traveling direction of the traveling vehicle body and the distance L. For example, when the traveling vehicle body 410 moves forward by the distance L from the acquisition of the image information, the injection nozzle 421 comes to the position of the imaging device 450 that acquired the image information. In other words, by injecting at a timing when the estimated movement distance L' matches the distance L, it is possible to inject pollen into the targeted area.

[0111] The estimated travel distance L' may be calculated based on the travel speed set by the travel control unit 4318, or may be estimated based on the travel distance calculated from the acceleration data of the traveling vehicle body 41, speed data (instantaneous value of speed), tire diameter, and rotation speed. Here, the "set travel speed" refers to, for example, "10 km / h" when the travel control unit 4318 drives the traveling mechanism 411 at a set value of 10 km / h. Apart from this, the actual travel speed may differ from the set travel speed due to reasons such as uneven ground. Therefore, the actual travel distance may be calculated based on the acceleration data, speed data (instantaneous value of speed), tire diameter, and rotation speed.

[0112] Next, in step 04, the injection control unit 4316 injects pollen onto the flower from the injection device 420 at the calculated injection timing. At this time, when the injection device 420 has one or more injection nozzles 421 and a nozzle drive mechanism 424 that independently extends and retracts the injection nozzles 421, the nozzle control unit 4319 may control the extension and retraction of the injection nozzles 421 by the nozzle drive mechanism 424. Specifically, an objective sensor (not shown) may be provided at the tip of the injection nozzle 421, and the objective sensor may control the extension and retraction of the injection nozzle 421 by the nozzle drive mechanism 424 so that the tip of the injection nozzle 421 is at a constant distance from the flower.

[0113] At this time, spraying control device 430 may control blower 470 according to the distance between spray nozzle 421 and crop 520 and the difference between the range over which pollen should be sprayed as determined by estimation unit 4315 and the range that can be sprayed mechanically by spray nozzle 421. Specifically, the wind speed of blower 470 may be controlled so that the sprayed pollen reaches crop 520, or so that the sprayed pollen reaches a range that is wider than the range that can be sprayed mechanically by spray nozzle 421 and that can be determined by estimation unit 4315 to spray pollen.

[0114] Then, in step 05, the second imaging device 452 captures image information of the crop 520 after spraying, and the second acquisition unit 4317 acquires the detection result of the pollen-bearing flowers. At this time, the second acquisition unit 4317 may determine whether pollen is attached to the target spot based on the acquired image information and record this information as a pollination operation record. The detection result may also include location information from which it was acquired. This allows the travel control unit 4318 to determine the next travel route based on the detection result. Note that the second imaging device 452 used here may be a regular camera or an imaging device intended for observation of IR, fluorescence, etc.

[0115] Finally, in step 06, the driving control unit 4318 refers to the driving data 4351, and if driving to the location where spraying is scheduled has not been completed, it returns to step 02 and repeats the process, and if driving to the location where spraying is scheduled has been completed, it completes the process.

[0116] 1.2.2 Self-propelled Control The travel control unit 4318 can perform known control to allow the traveling vehicle body 410 to travel self-propelled within the field 500. Specifically, the travel control unit 4318 can grasp the absolute or relative position within the field 500, detect obstacles to travel, and control the travel of the traveling vehicle body 410 based on this information.

[0117] Separately, the travel control unit 4318 may calibrate the travel speed based on the image information of the first imaging device 451 and the second imaging device 452. Specifically, by comparing the imaging information of the crops 520 and the cultivation structure 530 acquired by the first imaging device 451 and the time of acquisition thereof with the imaging information of the crops 520 and the cultivation structure 530 acquired by the second imaging device 452 and the time of acquisition thereof, the travel control unit 4318 can calculate the time required for the second imaging device 452 to acquire imaging information similar to the imaging information acquired by the first imaging device 451.

[0118] 2A , since the distance between the first imaging device 451 and the second imaging device 452 in the spraying device 400 is fixed, the actual running speed can be calculated from the calculated time and the distance between the first imaging device 451 and the second imaging device 452. The running control unit 4318 may then adjust the drive control of the running mechanism 411 by comparing the actual running speed calculated from the imaging information with the running speed that is assumed to be achieved by driving the running mechanism 411.

[0119] Furthermore, instead of or in addition to controlling the drive of the traveling mechanism 411, the traveling control unit 4318 may use the actual traveling speed calculated from the image capturing information as the speed when calculating the injection timing in step S023. This can further improve the accuracy of the injection timing.

[0120] Similarly, the travel control unit 4318 can evaluate whether the actual traveling direction is straight or curved by evaluating the rotation of the imaging information acquired by the second imaging device 452 relative to the imaging information acquired by the first imaging device 451. Then, the travel control unit 4318 may adjust the drive control of the traveling mechanism 411 by comparing the actual traveling direction with the assumed traveling direction.

[0121] 2. Spray Control Method In the spray control method of the present embodiment, spray control device 430 of spray device 400, which is provided with spray device 420 on traveling vehicle body 410, executes the steps of acquiring first image information of crop 520, identifying flowers in the first image information, and detecting the pollination state of the identified flowers based on the first image information.

[0122] Note that the specific aspects of the method of this embodiment have been described above in the control process, so a detailed description thereof will be omitted here.

[0123] 3. Spraying Control Program The spraying control program of this embodiment causes spraying control device 430 of spraying device 400, which is equipped with spray device 420 on traveling vehicle body 410, to execute the steps of acquiring first image information of crop 520, identifying flowers in the first image information, and detecting the pollination state of the identified flowers based on the first image information.

[0124] The program may be recorded on a readable recording medium. Note that the specific aspects of the processing executed by the program of this embodiment have been described in the control processing section above, and therefore will not be described in detail here.

[0125] 4. Spraying Device Spraying device 400 of the present embodiment includes traveling vehicle body 410, spraying device 420 capable of spraying liquid or powder onto crops 520, and spraying control device 430. Spraying control device 430 has first acquisition unit 4312 that acquires first image information of the crops, identification unit 4313 that identifies flowers in the first image information, and detection unit 4314 that detects the pollination state of the flowers based on the first image information.

[0126] 5. Modification 1 The spraying control device 430 of Modification 1 may include a first acquisition unit 4312 that acquires first image information of agricultural crops, an identification unit 4313 that identifies flowers in the first image information, a travel control unit 4318 that controls the travel of the spraying device 420 that includes a traveling vehicle body 410, a spraying device 420 that can spray liquid or powder onto agricultural crops, and the spraying control device 430, and an estimation unit 4315 that estimates the spraying timing for spraying pollen onto the identified flowers based on the first image information and information related to the traveling direction and traveling speed of the traveling vehicle body 410 that is acquired from the travel control unit 4318.

[0127] The spraying device 400 of the first modified example may also include the spraying control device 430 described above.

[0128] The larger the field, the more difficult it becomes to pollinate flowers that bloom randomly in various locations within the field. Therefore, to enable pollination within a limited pollination period, it is preferable to optimize the spray timing and increase the traveling speed of the spraying device 400 within the field. From this perspective, in Modification 1, the spray timing is estimated based on the first image information and the traveling direction and traveling speed, and the spray device 420 is driven accordingly. Furthermore, when controlling the spray timing, the traveling control unit 4318 may control the traveling speed of the traveling vehicle body 410, taking into account the time required to process the image information and the time required to calculate the spray timing.

[0129] This allows the spraying device 400 to travel over a wider area within the field during the limited pollination period.

[0130] 6. Modification 2 The spraying control device 430 of Modification 2 may include a first acquisition unit 4312 that acquires first image information of agricultural crops, an identification unit 4313 that identifies flowers in the first image information, an injection control unit 4316 that causes an injection device to inject pollen onto the flowers based on the first image information, and a second acquisition unit 4317 that acquires processing results of the flowers onto which the pollen has been sprayed. In addition, the second acquisition unit 4317 may record the processing results in driving data 4351 ( FIG. 3B ).

[0131] The spraying device 400 of the second modification may also include the spraying control device 430 described above.

[0132] The time available for pollination after each flower blooms is limited, for example, 48 hours. Furthermore, pollination must be performed at the appropriate time within this limited flowering period, taking into account the daily changing condition of the plant. Delays in pollination can lead to reduced yields. In particular, the larger the field, the more difficult it becomes to pollinate flowers that bloom randomly in various locations. From this perspective, in the second modification, the second acquisition unit 4317 may record the processing results for flowers that have already sprayed pollen, and the identification unit 4313 may determine whether to spray pollen on flowers blooming in the same location.

[0133] For example, assume that the spraying device 400 passes by a certain point in a farm field several times. During the first pass, the first acquisition unit 4312 acquires the first image information, the identification unit 4313 identifies the flower, the spray control unit 4316 sprays pollen, and the second acquisition unit 4317 records the processing results. During the next pass, the first acquisition unit 4312 acquires the first image information, and the identification unit 4313 refers to the previous processing result and checks whether pollen spraying has finished. If pollen spraying has finished, the identification unit 4313 may decide not to spray pollen.

[0134] At this time, it is also assumed that new flowers that could not be confirmed during the first pass will be in bloom in the second first image information. In this case, the identification unit 4313 may refer to the previous processing result, and if there are new flowers, identify the flowers that should be sprayed with pollen, and the spray control unit 4316 may spray the pollen.

[0135] Furthermore, even when new flowers are blooming, it may be the case that a sufficient number of flowers have already finished spraying pollen at that point. In such a case, the specifying unit 4313 may refer to information on the number or density of flowers that have finished spraying pollen from the previous processing result, and if the number or density of flowers that have finished spraying pollen exceeds a predetermined value, the specifying unit 4313 may determine not to spray pollen.

[0136] The present invention has industrial applicability as a component technology that can be used in smart agriculture systems.

[0137] 100...user terminal, 200...server, 300...measuring device, 310...fixed sensor, 320...unmanned aerial vehicle, 400...spraying device, 410...traveling vehicle body, 411...traveling mechanism, 420...spraying device, 421...spraying nozzle, 424...nozzle driving mechanism, 425...tank, 430...spraying control device, 431...processor, 432...communication interface, 433...input / output interface, 434...memory, 435...storage, 43 6...Communication bus, 450...Imaging device, 451...First imaging device, 452...Second imaging device, 460...Distance measurement sensor, 470...Blower, 500...Field, 520...Crop, 530...Cultivation structure, 4311...Transmitting / receiving unit, 4312...First acquisition unit, 4313...Identification unit, 4314...Detection unit, 4315...Estimation unit, 4316...Injection control unit, 4317...Second acquisition unit, 4318...Travel control unit, 4319...Nozzle control unit, 4351...Travel data

Claims

1. A spraying control device having a first acquisition unit that acquires first image information of an agricultural crop, an identification unit that identifies a flower in the first image information, and a detection unit that detects the pollination state of the flower based on the first image information.

2. A spraying control device as described in claim 1, further comprising an estimation unit that estimates the timing of spraying pollen onto the identified flower based on the first image information.

3. The spray control device according to claim 2, further comprising an injection control unit that causes an injection device to inject the pollen onto the flower in accordance with the injection timing.

4. A spraying control device as described in claim 1, comprising: an injection control unit that causes an injection device to spray pollen onto the flower based on the first image information; and a second acquisition unit that acquires the processing results of the flower onto which the pollen has been sprayed.

5. A spraying control device as described in claim 1, comprising: a driving control unit that controls the driving of a spraying device comprising a traveling vehicle body, an injection device capable of spraying liquid or powder onto agricultural crops, and a spraying control device; and an estimation unit that estimates the injection timing for spraying pollen onto the identified flower based on the first image information and information regarding the traveling direction and traveling speed of the traveling vehicle body obtained from the driving control unit.

6. A spraying control device as described in claim 1, further comprising a travel control unit that controls the travel of a spraying device comprising a traveling vehicle body, a spraying device capable of spraying liquid or powder onto agricultural crops, and a spraying control device in accordance with spray timing.

7. The spray control device according to claim 1, wherein the identification unit identifies flowers that have not been pollinated as flowers to be pollinated based on the first image information.

8. A spraying control device as described in claim 1, further comprising a nozzle control unit that controls the extension and retraction of the spray nozzle by a nozzle driving mechanism, the nozzle driving mechanism extending and retracting one or more spray nozzles that spray pollen onto the flowers.

9. The spraying control device according to claim 4, wherein the detection result includes position information, and the travel control unit determines a travel route based on the detection result.

10. A spray control device as described in claim 1, wherein the distance in the traveling direction between the imaging device that captures the first image information and the injection device is distance L, and a certain amount of time allocated to processing from the acquisition of the first image information to the calculation of the injection timing is reference time T, the traveling control unit controls the traveling speed S of the traveling vehicle body so that it does not exceed the ratio (distance L / reference time T).

11. The spray control device according to claim 1, wherein the estimation unit estimates the injection timing based on an estimated travel distance L' in the traveling direction of the traveling vehicle body obtained from a travel control unit.

12. A spraying control program that causes a spraying control device of a spraying device equipped with an injection device on a traveling vehicle body to execute the steps of acquiring first image information of an agricultural crop, identifying flowers in the first image information, and detecting the pollination state of the flowers based on the first image information.

13. A spraying control method, comprising: a spraying control device of a spraying device equipped with an injection device on a traveling vehicle body, performing the steps of: acquiring first image information of an agricultural crop; identifying flowers in the first image information; and detecting the pollination state of the flowers based on the first image information.

14. A spraying device comprising a traveling vehicle body, an injection device capable of spraying liquid or powder onto agricultural crops, and a spraying control device, the spraying control device having a first acquisition unit that acquires first image information of the agricultural crops, an identification unit that identifies flowers in the first image information, and a detection unit that detects the pollination state of the flowers based on the first image information.

Images