Agricultural support systems, agricultural support devices, agricultural support methods, and agricultural support programs

Abstract

To provide a technology that enables various agricultural tasks, such as pollination, to be carried out more appropriately. [Solution] An agricultural support system comprising an operating mechanism that comes into contact with a plant and performs a predetermined operation, and a control unit, wherein the control unit includes a model generation unit that generates a three-dimensional model of the plant by combining predetermined generation conditions, a learning unit that learns a first trained model that is generated by learning using predetermined parameters in the three-dimensional model and an image of the plant as learning data, an image acquisition unit that acquires an image of the plant to be determined, a model acquisition unit that acquires the first trained model, a parameter estimation unit that estimates predetermined parameters of a predetermined location of the plant to be determined based on the first trained model and the image of the plant to be determined, and an operation instruction unit that causes the operating mechanism to perform a predetermined operation based on the estimated predetermined parameters.

Description

Technical Field

[0001] The present disclosure relates to an agricultural support system, an agricultural support device, an agricultural support method, and an agricultural support program.

Background Art

[0002] In agriculture, technologies for automatically pollinating fruits and the like are known.

[0003] Patent Document 1 describes a technology for a drone control device that controls a drone equipped with an attachment for attaching pollen or a fruit setting agent to a pistil.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the technology described in Patent Document 1, a technology for determining a flowering state based on image data of flowers acquired from a drone and attaching pollen and a fruit setting agent is described. However, in the technology of Patent Document 1, there is a possibility that pollen and a fruit setting agent cannot be appropriately attached when the flower is facing in the rotational direction. Therefore, there is a need for a technology that can more appropriately perform various operations related to agriculture, such as pollination.

Means for Solving the Problems

[0006] According to one embodiment, an agricultural support system is provided, comprising an operating mechanism that makes contact with a plant and performs a predetermined operation, and a control unit, wherein the control unit includes a model generation unit that generates a three-dimensional model of a plant by a combination of predetermined generation conditions, a learning unit that learns a first trained model generated by learning predetermined parameters in the three-dimensional model and an image of the plant as training data, an image acquisition unit that acquires an image of the plant to be judged, a model acquisition unit that acquires the first trained model, a parameter estimation unit that estimates predetermined parameters of a predetermined part of the plant to be judged based on the first trained model and the image of the plant to be judged, and an operation instruction unit that causes the operating mechanism to perform a predetermined operation based on the estimated predetermined parameters. [Effects of the Invention]

[0007] According to this disclosure, various agricultural tasks, such as pollination, can be carried out more appropriately. [Brief explanation of the drawing]

[0008] [Figure 1] Block diagram showing the overall configuration of System 1. [Figure 2] This diagram shows the functional configuration of terminal device 10. [Figure 3] This diagram shows the functional configuration of server 20. [Figure 4] This figure shows the functional configuration of the agricultural support device 30. [Figure 5] This figure shows an example of the external appearance (perspective view) of the agricultural support device 30 according to the first embodiment. [Figure 6] This figure shows an example of the external appearance (side view) of the agricultural support device 30 according to the first embodiment. [Figure 7] This diagram shows the detailed configuration of the main body 502 of the agricultural support device 30. [Figure 8] This flowchart shows a series of processes in which the agricultural support device 30 performs predetermined operations based on acquired plant images and a trained model. [Figure 9]The diagram shows how the operating mechanism 302 of the agricultural support device 30 performs a predetermined operation on a predetermined part of a plant based on predetermined parameters. [Figure 10] This shows an example screen displaying a notification to the user managing the field when the operating mechanism 302 of the agricultural support device 30 detects a tendency for plant disease. [Modes for carrying out the invention]

[0009] Embodiments of the present disclosure will be described below with reference to the drawings. In the following description, identical parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed descriptions of them will not be repeated.

[0010] <First Embodiment> <Overview> In the following embodiment, we describe a technique for acquiring images of plants, estimating predetermined parameters of the plant in three-dimensional space based on a trained model, and operating an operating mechanism provided at the tip of the arm of an agricultural support device based on those parameters.

[0011] As an example of comparison with the embodiment, the following configuration will be described. As a comparative example, we consider a technology that acquires images of plants, identifies the position of the pistil of a flower, and performs automatic pollination using a device. For example, a trained model that associates training images of flowers with the flowering state is used to determine the flowering state in the captured image of a flower, and the attachment is driven to perform pollination according to the orientation of the flower determined to be in the flowering state. However, in the comparative example, the orientation of the flower is merely an orientation in a plane (2D) based on the captured image, and if the flower is facing in the depth direction, it may not be possible to properly determine the flowering state and perform pollination, which may result in omissions in the pollination process and a reduction in the harvest yield.

[0012] Therefore, in the system 1 described in this embodiment, an agricultural support system includes an operation mechanism that contacts a plant and performs a predetermined operation, and a control unit. The control unit includes a model generation unit that generates a three-dimensional model of a plant according to a combination of predetermined generation conditions, a learning unit that learns a first learned model generated by learning using a predetermined parameter in the three-dimensional model and an image of the plant as learning data, an image acquisition unit that acquires an image of a plant to be determined, a model acquisition unit that acquires the first learned model, a parameter estimation unit that estimates a predetermined parameter of a predetermined location of the plant to be determined based on the first learned model and the image of the plant to be determined, and an operation instruction unit that causes the operation mechanism to perform a predetermined operation based on the estimated predetermined parameter.

[0013] As described above, the system 1 provides a technology for more appropriately performing various operations related to agriculture, such as pollination.

[0014] The system 1 can be used, for example, in a plant factory, greenhouse, or vinyl house in a scenario where automatic pollination, harvesting, leaf scraping, etc. are performed. Thereby, various operations related to agriculture, such as pollination, can be more appropriately performed.

[0015] Hereinafter, with reference to FIGS. 1 to 4, the configuration of the system 1 according to the first embodiment of the present disclosure will be described. The system 1 is mainly a system for systematically producing plants, such as a plant factory, greenhouse, or vinyl house, and includes the following devices. · A device for automatically performing a pollination process on a plant flower · A device for automatically performing a harvesting process on a plant fruit · A device for automatically performing leaf scraping, pruning, and thinning on plant leaves and stems · A device for automatically spraying fertilizer on plants Specifically, the above devices are devices that automatically perform a pollination process on self-pollinating flowers where pollen adheres to the stigma of the same flower, and devices that automatically perform a pollination process on cross-pollinating flowers where pollen adheres to the stigma of other flowers.

[0016] <1 System Configuration Diagram> Figure 1 shows the overall configuration of System 1 in the first embodiment.

[0017] As shown in Figure 1, System 1 includes a terminal device 10 (although only terminal device 10 is shown in Figure 1, it may consist of multiple terminal devices (10A, 10B, 10C, etc.)), a server 20, and an agricultural support device 30. The terminal device 10, the server 20, and the agricultural support device 30 communicate with each other via a network 80. The network 80 is composed of a wired or wireless network.

[0018] Terminal device 10 is a device operated by each user. Terminal device 10 is implemented by a stationary PC (Personal Computer), a laptop PC, etc. Alternatively, terminal device 10 may be a mobile terminal such as a tablet or smartphone compatible with a mobile communication system. As shown in Figure 1, terminal device 10 includes a communication interface (IF) 12, an input device 13, an output device 14, a memory 15, a storage unit 16, and a processor 19. Server 20 includes a communication interface 22, an input / output interface 23, a memory 25, storage 26, and a processor 29. Agricultural support device 30 includes a communication interface 32, an input / output interface 33, a memory 35, a storage unit 36, and a processor 39.

[0019] The terminal device 10 is connected to the server 20 and the agricultural support device 30 via the network 80, enabling communication between them. The terminal device 10 connects to the network 80 by communicating with communication equipment such as a wireless base station 81 that supports various communication standards (5G, LTE (Long Term Evolution), etc.) and a wireless LAN router 82 that supports wireless LAN (Local Area Network) standards such as IEEE (Institute of Electrical and Electronics Engineers) 802.11.

[0020] The communication interface 12 is an interface for inputting and outputting signals so that the terminal device 10 can communicate with external devices. The input device 13 is an input device (e.g., a touch panel, touchpad, mouse or other pointing device, keyboard, etc.) for receiving input operations from the user. The output device 14 is an output device (display, speaker, etc.) for presenting information to the user. The memory 15 is for temporarily storing programs and data processed by programs, etc., and is a volatile memory such as DRAM (Dynamic Random Access Memory). The storage unit 16 is a storage device for saving data, such as flash memory or an HDD (Hard Disk Drive). The processor 19 is hardware for executing the instruction set written in the program, and is composed of an arithmetic unit, registers, peripheral circuits, etc.

[0021] Server 20 manages information about the trained models acquired by the agricultural support device. Details about the trained models will be described later. In certain situations, server 20 may manage various information such as that of users who manage fields. Specifically, for example, server 20 may manage the following information as information about users who manage fields. • Types of vegetables and fruits the user is cultivating in their field • Area of ​​fields managed by the user • Types of equipment and machinery owned by the user • Slope angle of the fields managed by the user • Soil properties of the fields managed by the user • Types of fertilizers used by the user in the field • Areas where user-managed fields are located • Climate information (average temperature, rainfall, sunshine, etc.) for the region where the user's managed fields are located.

[0022] Communication IF22 is an interface for inputting and outputting signals so that the server 20 can communicate with external devices. Input / Output IF23 functions as an interface to an input device for receiving input operations from the user and an output device for presenting information to the user. Memory 25 is for temporarily storing programs and data processed by programs, etc., and is a volatile memory such as DRAM (Dynamic Random Access Memory). Storage 26 is a storage device for saving data, such as flash memory or HDD (Hard Disk Drive). Processor 29 is hardware for executing the instruction set written in the program, and is composed of an arithmetic unit, registers, peripheral circuits, etc.

[0023] In this embodiment, each device (terminal device, server, etc.) can also be considered as an information processing device. That is, the collection of each device can be considered as a single "information processing device," and System 1 may be formed as a collection of multiple devices. The way in which the multiple functions required to realize System 1 according to this embodiment are distributed to one or more hardware can be appropriately determined in view of the processing capacity of each hardware and / or the specifications required for System 1.

[0024] The agricultural support device 30 is a device that performs predetermined tasks in the field based on instructions from the user or pre-set conditions. Specifically, the predetermined tasks include, for example, the following tasks performed by the agricultural support device 30 in the field managed by the user. • Pollination (the process of attaching pollen to the pistil) • Harvesting work • Leaf pruning (removing overlapping or excess leaves) • Fertilizer application work • Thinning out diseased fruits, leaves, etc.

[0025] The communication IF32 is an interface for inputting and outputting signals so that the agricultural support device 30 can communicate with external devices. The input / output IF33 functions as an interface to an input device for receiving input operations from the user and an output device for presenting information to the user. The memory 35 is for temporarily storing programs and data processed by programs, etc., and is a volatile memory such as DRAM (Dynamic Random Access Memory). The storage unit 36 ​​is a storage device for saving data, such as flash memory or an HDD (Hard Disk Drive). The processor 39 is hardware for executing the instruction set written in the program, and is composed of an arithmetic unit, registers, peripheral circuits, etc.

[0026] <1.1 Configuration of terminal device 10> Figure 2 is a block diagram showing the functional configuration of the terminal device 10 that constitutes System 1 of Embodiment 1. As shown in Figure 2, the terminal device 10 includes a plurality of antennas (antenna 111, antenna 112), wireless communication units corresponding to each antenna (first wireless communication unit 121, second wireless communication unit 122), an operation reception unit 130 (including a keyboard 1301 and a mouse 1302), an audio processing unit 140, a microphone 141, a speaker 142, a display 150, a position information sensor 160, a storage unit 170, and a control unit 180. The terminal device 10 also has functions and configurations not specifically shown in Figure 2 (for example, a battery for maintaining power, a power supply circuit for controlling the supply of power from the battery to each circuit, etc.). As shown in Figure 2, each block included in the terminal device 10 is electrically connected by a bus or the like.

[0027] Antenna 111 radiates signals emitted by terminal device 10 as radio waves. Antenna 111 also receives radio waves from space and provides the received signals to first wireless communication unit 121.

[0028] Antenna 112 radiates signals emitted by terminal device 10 as radio waves. Antenna 112 also receives radio waves from space and provides the received signals to second wireless communication unit 122.

[0029] The first wireless communication unit 121 performs modulation and demodulation processing, etc., for the terminal device 10 to transmit and receive signals via the antenna 111 in order to communicate with other wireless devices. The second wireless communication unit 122 performs modulation and demodulation processing, etc., for the terminal device 10 to transmit and receive signals via the antenna 112 in order to communicate with other wireless devices. The first wireless communication unit 121 and the second wireless communication unit 122 are a communication module that includes a tuner, an RSSI (Received Signal Strength Indicator) calculation circuit, a CRC (Cyclic Redundancy Check) calculation circuit, a high-frequency circuit, etc. The first wireless communication unit 121 and the second wireless communication unit 122 perform modulation and demodulation, and frequency conversion of the wireless signals transmitted and received by the terminal device 10, and provide the received signal to the control unit 180.

[0030] The operation reception unit 130 has a mechanism for receiving user input operations. Specifically, the operation reception unit 130 includes a keyboard 1301 and a mouse 1302. The operation reception unit 130 may also be configured as a touchscreen that detects the user's contact position with the touch panel, for example, by using a capacitive touch panel.

[0031] The keyboard 1301 accepts user input operations from the terminal device 10. The keyboard 1301 is a device for character input and outputs the input character information as an input signal to the control unit 180.

[0032] The mouse 1302 accepts user input operations from the terminal device 10. The mouse 1302 is a pointing device for selecting objects displayed on the display 150, and outputs the selected position information on the screen and information indicating that a button is pressed as input signals to the control unit 180.

[0033] The audio processing unit 140 modulates and demodulates the audio signal. The audio processing unit 140 modulates the signal received from the microphone 141 and provides the modulated signal to the control unit 180. The audio processing unit 140 also provides the audio signal to the speaker 142. The audio processing unit 140 is implemented, for example, by an audio processing processor. The microphone 141 receives an audio input and provides the audio signal corresponding to that audio input to the audio processing unit 140. The speaker 142 converts the audio signal received from the audio processing unit 140 into sound and outputs the sound to the outside of the terminal device 10.

[0034] The display 150 displays data such as images, videos, and text in accordance with the control of the control unit 180. The display 150 is implemented by, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display.

[0035] The location information sensor 160 is a sensor that detects the position of the terminal device 10, and is, for example, a GPS (Global Positioning System) module. A GPS module is a receiving device used in a satellite positioning system. In a satellite positioning system, signals are received from at least three or four satellites, and the current position of the terminal device 10 equipped with the GPS module is detected based on the received signals. For example, if the system 1 makes the user's location information accessible, information regarding the location of a field managed by the user may be identified based on the user's location information, and the climate conditions in that field may be identified. Alternatively, if the system 1 has information about a user who manages a field, the system 1 may identify information about the user who manages the field in that area based on the location information. Furthermore, the location information sensor 160 may be a transmitting and receiving device based on a communication standard used in short-range communication systems between information devices. Specifically, the location information sensor 160 uses the 2.4GHz band, such as a Bluetooth® module, to receive beacon signals from other information devices equipped with a Bluetooth® module.

[0036] The storage unit 170 is composed of, for example, flash memory and stores data and programs used by the terminal device 10. In a certain scenario, the storage unit 170 may store the following information as information relating to the user who manages the field. • Types of vegetables and fruits the user is cultivating in their field • Area of ​​fields managed by the user • Types of equipment and machinery owned by the user • Slope angle of the fields managed by the user • Soil properties of the fields managed by the user • Types of fertilizers used by the user in the field • Areas where user-managed fields are located • Climate information (average temperature, rainfall, sunshine, etc.) for the region where the user's managed fields are located.

[0037] The control unit 180 controls the operation of the terminal device 10 by reading the program stored in the memory unit 170 and executing the instructions contained in the program. The control unit 180 is, for example, an application processor. By operating according to the program, the control unit 180 performs the functions of an input operation receiving unit 1801, a transmitting / receiving unit 1802, a data processing unit 1803, and a notification control unit 1804.

[0038] The input operation reception unit 1801 processes user input operations to an input device such as a keyboard 1301. When the input operation reception unit 1801 receives user input operations via an input device such as a touch-sensitive device (not shown), it determines the type of operation, including those listed below, based on the coordinate information of where the user's finger or other object has touched the touch-sensitive device. • Is the user performing a flick gesture? • Is the user's operation a tap operation? • Is the user's operation a drag (swipe) operation?

[0039] The transmitting / receiving unit 1802 performs processing to enable the terminal device 10 to send and receive data with an external device such as a server 20 in accordance with a communication protocol.

[0040] The data processing unit 1803 performs calculations on the data received as input by the terminal device 10 according to the program and outputs the calculation results to memory or the like.

[0041] The notification control unit 1804 performs processing to present information to the user. The notification control unit 1804 performs processing such as displaying the display image on the display 150 and outputting sound to the speaker 142.

[0042] <1.2 Functional Configuration of Server 20> Figure 3 shows the functional configuration of server 20. As shown in Figure 3, server 20 functions as a communication unit 201, a storage unit 202, and a control unit 203.

[0043] The communications unit 201 performs processing to enable the server 20 to communicate with external devices.

[0044] The memory unit 202 stores data and programs used by the server 20. The memory unit 202 also stores the trained model database 2021, etc. In certain situations, the memory unit 202 may also store various information about the user who manages the field.

[0045] The trained model database 2021 is a database for managing information on trained models that associate predetermined parameters at predetermined locations in a 3D model of a plant with images of the plant. Here, the associations are not limited to images of plants; they may also include videos of plants, etc.

[0046] The receiving control unit 2031 controls the process by which the server 20 receives signals from an external device according to a communication protocol.

[0047] The transmission control unit 2032 controls the process by which the server 20 transmits signals to external devices according to a communication protocol.

[0048] The model generation unit 2033 controls the process of generating a 3D model of a plant based on predetermined generation conditions. Specifically, for example, the model generation unit 2033 uses the combination of elements listed below as the generation conditions for the 3D model and generates a 3D model of a plant. • Plant leaves, stems, flowers (including pistils, stamens, etc.), and fruits • The size of each part of a plant: leaves, stems, flowers (including pistils, stamens, etc.), and fruits. The number of leaves, stems, flowers (including pistils, stamens, etc.), and fruits of a plant. • The colors of the leaves, stems, flowers (including pistils, stamens, etc.), and fruits of plants. • Background information • Lighting information • The orientation of each part of a plant: leaves, stems, flowers (including pistils, stamens, etc.), and fruits. • The position of each part of a plant: leaves, stems, flowers (including pistils, stamens, etc.), and fruits. Here, any existing 3D rendering software may be used to create the 3D model. Server 20 may store information about the 3D model generated by the combination of the above elements in storage unit 202.

[0049] The learning unit 2034 controls the process of learning a trained model that associates the generated 3D model with predetermined parameters of the plant, and storing it in the trained model database 2021 of the storage unit 202 of the server 20.

[0050] Here, we provide an example of how to create a pre-trained model in this disclosure. For example, the learning unit 2034 acquires information about the 3D model generated by the model generation unit 2033.

[0051] Next, the learning unit 2034 receives images (or videos) of plants to be associated with a 3D model from the user managing the field, and associates various parameters at each part of the plant (position, orientation, and positional relationship of the pistil) with various parameters at each part of the 3D model. For example, the learning unit 2034 receives an image of a plant in which the pistil is facing a predetermined direction. Next, the learning unit 2034 receives a 3D model from the user managing the field in which the pistil is facing the same direction as the received plant image, and associates various parameters in that model.

[0052] When server 20 receives an image of a plant as input, it identifies parameters (e.g., orientation, positional relationship, etc.) at a predetermined part of the plant (e.g., flower, pistil, leaf, etc.) based on a trained model, and outputs these parameters as parameters for operating the agricultural support device. That is, because the pistil is facing a predetermined direction (e.g., diagonally backward in depth), it outputs information for pollinating the pistil from the front. At this time, the output information may be a normal vector, and the agricultural support device may operate the operation mechanism described later based on the output normal vector information. This allows users managing fields to perform tasks automatically and accurately under various conditions.

[0053] In certain situations, the server 20 may store information on multiple trained models in its memory unit, not just the trained model exemplified above (the first trained model). Specifically, for example, the server 20 may store the following trained models. • A pre-trained model (second pre-trained model) that has been trained on at least one of the following: ventilation or sunlight in the field, and fruit yield at harvest time. • A pre-trained model (third pre-trained model) that associates fruit images with fruit weight, fruit sugar content, and fruit quality. • A pre-trained model (fourth pre-trained model) that associates plant images with images of plant diseases. • A pre-trained model (5th pre-trained model) that associates plant images with the nutritional status of plants. Here, we will illustrate the methods for generating each pre-trained model. In certain situations, the server 20 may store various pre-trained models for each plant. For example, it may store the aforementioned pre-trained models for each plant cultivated in the field, such as strawberries, tomatoes, melons, and cucumbers. In this case, the server 20 may determine which plant to acquire after identifying which plant it is from the acquired image. This allows users managing fields to instantly change the acquired pre-trained model and perform various operations appropriately, even when cultivating multiple types of plants.

[0054] An example of how to generate a second pre-trained model is given. Server 20 receives information from the user managing the field regarding the ventilation in the field (which may be approximated by the density of leaves, etc., on the plants) or the amount of sunlight in the field (the amount of sunlight at various points on the plants, based on the density of leaves, etc., on the plants). Next, Server 20 receives information from the user managing the field regarding the yield of fruits, etc., on the plants at harvest time and associates it with the ventilation and sunlight information. When Server 20 receives a plant image as input, it outputs a yield prediction based on the second pre-trained model. In a given situation, the agricultural support device may estimate the optimal positional relationship of leaves, etc., for achieving the best yield based on the outputted yield prediction, and perform leaf removal or other similar actions based on the estimated results. This allows users managing the fields to automatically adjust settings based on information such as leaf ventilation to achieve optimal yields.

[0055] The third example illustrates how to generate a pre-trained model. Server 20 may receive fruit images, fruit weight, fruit sugar content, and fruit quality from a user managing the field, and associate these with each other. When Server 20 receives fruit images, weight, and sugar content as input, it outputs the fruit quality. This allows users managing the fields to quantitatively evaluate the quality of the fruit.

[0056] The fourth example of generating a trained model is given below. Server 20 receives information from the user managing the field about a 3D plant model created based on images of plants affected by disease. At this time, the user managing the field may create a plant disease model with randomly set conditions as exemplified below. • Location of disease occurrence • Amount of disease occurrence (area on leaves, flowers, etc.) • Location of disease occurrence on a single plant Next, the server 20 receives images of plants that show signs of disease and associates them with 3D models. Specifically, if a predetermined disease (such as anthracnose, phylloxera, powdery mildew, etc., but not limited to these) is found on a part of a leaf, the server outputs information about the area where the disease trend was observed. In certain situations, the agricultural support device may perform actions such as removing or thinning plants in areas where the disease trend is observed, based on the information about those areas. This allows users managing fields to quickly detect disease trends, even in large fields, and prevent disease-related damage early on.

[0057] The fifth example illustrates the method for generating a pre-trained model. Server 20 receives information on 3D plant models created from images of plants under various nutritional conditions (nutrient deficiency, nutrient excess, etc.) from the user managing the field. Next, Server 20 receives images of plants that show the nutritional condition trends described above and associates them with these 3D models. That is, if some plants in the field show nutrient deficiency, Server 20 outputs information on the areas where this trend was observed. In certain situations, the agricultural support device may adjust the amount and proportion of fertilizer to be applied based on the information about the areas where a tendency toward nutrient deficiency was observed. This allows users managing the fields to understand imbalances in fertilizer application and harvest uniform fruits and other produce.

[0058] The learning unit 2034 controls the process of storing the generated trained model in the trained model database 2021 of the server 20's storage unit 202. Specifically, for example, the learning unit 2034 may transmit the trained model to the agricultural support device 30 in response to receiving an operation instruction from a user managing a field, or it may transmit a trained model corresponding to a plant in response to receiving an image of a plant from the agricultural support device 30.

[0059] <1.3 Functional configuration of agricultural support device 30> Figure 3 shows the functional configuration of the agricultural support device 30. As shown in Figure 3, the agricultural support device 30 includes a communication unit 301, an operating mechanism 302, a storage unit 303, a control unit 304, and a shooting mechanism 350.

[0060] The communication unit 201 allows the agricultural support device 30 to communicate with external devices.

[0061] The operating mechanism 302 allows the agricultural support device 30 to perform predetermined operations on plants. These predetermined operations include, for example, the following: • Pollination (the process of attaching pollen to the pistil) • Harvesting work • Leaf pruning (removing overlapping or excess leaves) • Fertilizer application work • Thinning out diseased fruits, leaves, etc.

[0062] The memory unit 303 stores data and programs used by the agricultural support device 30. In certain situations, the memory unit 303 may also store various trained models and various information about the user who manages the field.

[0063] The control unit 304 controls the operation of the agricultural support device 30 by reading the program stored in the memory unit 303 and executing the instructions contained in the program. The control unit 304 is, for example, an application processor. By operating according to the program, the control unit 304 performs the functions shown in the image acquisition unit 3031, the model acquisition unit 3032, the parameter estimation unit 3033, and the operation instruction unit 3034.

[0064] The image acquisition unit 3031 controls the operation of acquiring images or videos captured by the shooting mechanism 350, which will be described later. At this time, the format of the images or videos acquired by the image acquisition unit 3031 is not limited. For example, the image acquisition unit 3031 acquires images or videos in the following formats: ·JPEG (Joint Photographic Experts Group) format ·PNG (Portable Network Graphics) format ·GIF (Graphics Interchange Format) format • TIFF (Tagged Image File Format) format • Bitmap format ·MP4 (Moving Picture Experts Group) format ·MOV (QuickTime file format) format ·AVI (Audio Video still Images) format ·VOB (Video Object file) format

[0065] The model acquisition unit 3032 controls the process of acquiring various pre-trained models from the server 20. Specifically, for example, the model acquisition unit 3032 acquires various pre-trained models in response to the image acquisition unit 3031 acquiring images of plants. At this time, the model acquisition unit 3032 may store the acquired pre-trained models in the memory unit 303 and omit the process of acquiring pre-trained models when performing various operations on the same plant. Alternatively, the model acquisition unit 3032 may temporarily store the acquired pre-trained models in non-volatile memory and acquire the corresponding pre-trained model each time an operation is performed. This allows users to perform various operations based on pre-trained models, even in situations where it is difficult to ensure sufficient memory capacity in agricultural support devices.

[0066] The parameter estimation unit 3033 controls the process of estimating predetermined parameters at predetermined locations on a plant based on the plant image acquired by the image acquisition unit 3031 and the trained model acquired by the model acquisition unit. These predetermined locations specifically include, for example, the following: • Flowers and petals of plants • Plant pistil • Stamens of plants • Plant leaves • Fruits of plants • Plant stems Furthermore, the specified parameters include, for example, the following: • The orientation of the specified part of the plant as described above • The positional relationship of the specified parts of the plants mentioned above In this case, the parameter estimation unit 3033 may calculate the orientation parameter not as an upward, downward, etc., but as a normal vector in three-dimensional space. The normal vector in three-dimensional space may be, for example, a representation of various orientations as vectors in a 360° celestial sphere model. This allows users managing the fields to accurately estimate the direction of plant flowers, leaves, etc., even if they are facing in any direction, and to perform appropriate operations such as pollination, harvesting, and leaf pruning.

[0067] The operation instruction unit 3034 controls the process of causing the operating mechanism 302 of the agricultural support device 30 to perform a predetermined operation. The predetermined operation includes, for example, the following: • Pollination (the process of attaching pollen to the pistil) • Harvesting work • Leaf pruning (removing overlapping or excess leaves) • Fertilizer application work • Thinning out diseased fruits, leaves, etc.

[0068] The imaging mechanism 350 acquires images of plants in the field. At this time, the imaging mechanism 350 may also acquire video of the plants. The imaging mechanism 350 is not limited and may be any existing camera. The imaging mechanism 350 may be a camera that uses silver halide film or a digital camera.

[0069] In certain situations, the agricultural support device 30 may have functions other than those described above, depending on its application. For example, the agricultural support device may have functions including the following (none of which are shown): • Water spraying section for spraying water • Fertilizer spreading unit for spreading fertilizer • A unit for spraying pesticides. Other functions are not limited to those mentioned above and may include any functions necessary for agricultural activities.

[0070] <1.3 Configuration of the agricultural support device 30>

[0071] Figures 5 and 6 show examples of the external appearance of the agricultural support device 30 according to the first embodiment. Figure 5 is a perspective view of the agricultural support device 30, and Figure 6 is a side view of the agricultural support device 30.

[0072] As shown in Figures 5 and 6, the agricultural support device 30 includes a tray mounting rack 501, a harvest tray 5011, a main body 502, an information processing unit 503, and a travel unit 504. In this case, the main body 502 may be installed on rails that can move up and down.

[0073] The tray mounting rack 501 and the harvest tray 5011 are mechanisms for storing fruits and other items harvested by the agricultural support device 30. Specifically, for example, the agricultural support device 30 places the harvest tray 5011 for storing harvested fruits on the tray mounting rack 501. The agricultural support device 30 stores the harvested fruits (for example, strawberries) in the harvest tray 5011. The tray mounting rack 501 is configured in a multi-tiered manner, and when the storage capacity of the harvest tray 5011 reaches a specified value, the main body 502 moves up and down to store fruits in the harvest tray 5011 that has available capacity. This allows users managing the fields to harvest fruit efficiently.

[0074] In a given scenario, the operation of placing the harvest tray 5011 on the tray rack 501 may be performed by a user managing the field, or it may be performed automatically. Specifically, the agricultural support device 30 may be equipped with a tray placement arm (not shown) on its main body 502 that can move forward, backward, left, and right, and the harvest tray 5011 may be placed in a predetermined position on the tray rack 501 by grasping and moving the harvest tray 5011 with the tray placement arm. This allows users managing the fields to reduce the effort required to manually install the tray racks of the agricultural support device 30, even if the number of tiers on the racks increases.

[0075] Furthermore, in certain situations, the tray mounting rack 501 and harvest tray 5011 may be used not for harvesting fruit, but for storing plant seedlings. In that case, the main unit 502 may sequentially remove the seedlings from the harvest tray 5011 and perform the operation of planting them in the field.

[0076] The main body 502 is a mechanism for performing various tasks in the field (pollination, harvesting, leaf removal, etc.). Further details will be described later.

[0077] The information processing unit 503 is a mechanism for the agricultural support device 30 to communicate with an external terminal or to issue operation instructions to the main unit 502, etc., based on a program pre-stored in the information processing unit 503. The information processing unit 503 may be a computer equipped with a processor or a microcomputer.

[0078] The running section 504 is a mechanism for moving the agricultural support device 30 within the field. Specifically, for example, the running section 504 may be wheels installed on the underside of the agricultural support device 30 for running on the ground, or it may be wheels that can move on rails. Alternatively, the running section 504 may move the agricultural support device 30 using the mechanism shown below. • A mechanism equipped with a magnetic sensor that detects magnets laid on the field surface to drive the agricultural support device 30. • A mechanism equipped with a line sensor that detects lines drawn on the field and causes the agricultural support device 30 to move. In addition, the running unit 504 may have any mechanism used for autonomous driving of automobiles and the like.

[0079] Figure 7 shows a detailed configuration of the main body 502 of the agricultural support device 30.

[0080] As shown in Figure 7, the agricultural support device 30 includes a main body 502, an arm 701 positioned on the main body 502, an operating mechanism 302 provided at the tip of the arm, a drive mechanism 751, and an attachment 752.

[0081] The operating mechanism 302 may be mounted at the tip of the arm 701. The imaging mechanism 350 (not shown) may be mounted at a location on the arm 701 other than the operating mechanism 302 (i.e., the tip of the arm 701). In a given scenario, the arm 701 may include a pollination imaging unit (not shown) that captures the moment of pollination, and a yield prediction unit (not shown) that predicts the yield based on the captured image of the moment of pollination. In addition, the agricultural support device 30 may present the predicted yield to the user. This allows the user managing the field to determine whether pollination was carried out appropriately based on the information of the moment of pollination and the yield prediction. Furthermore, in certain situations, if the yield prediction results show that the yield is lower than past yield predictions, the agricultural support device 30 may perform the pollination operation again. This allows users managing the fields to harvest fruits and other produce stably without pollination shortages.

[0082] As shown in Figure 7, the operating mechanism 302 is provided at the tip of the movable arm 701 and performs predetermined operations by contacting the flowers, fruits, leaves, etc., of a plant. The operating mechanism 302 may have different mechanisms at its tip depending on the type of operation. For example, when performing plant pollination, the tip that comes into contact with the plant's flower may have a fibrous member. Also, when performing tasks such as fruit harvesting or leaf pruning, the tip may have an arm for pruning and harvesting the fruit. The operating mechanism 302 performs predetermined operations on the plant using the various mechanisms it has at its tip. The operation when performing pollination is described below.

[0083] For example, the operating mechanism 302 includes a drive mechanism 751 and an attachment 752. The operating mechanism 302 drives the drive mechanism 751 to a predetermined location on a plant to be pollinated (e.g., the pistil of a flower) and brings the attachment 752 into contact with it, according to the control of the control unit 304. Based on the image acquired from the imaging mechanism 350 and a trained model, the control unit 304 outputs the position of the pistil in the image and the normal vector of the pistil. The operating mechanism 302 drives the drive mechanism 751 in three dimensions according to the normal vector information output by the control unit 304, controlling the attachment 752 to contact the pistil perpendicularly (i.e., directly in front of the pistil), thereby evenly and uniformly attaching pollen to the pistil. The operating mechanism 302 can also be moved vertically and horizontally while in contact with the stamens of the flower.

[0084] <2 operations> The following describes a series of processes in which the agricultural support device 30, which constitutes System 1, performs predetermined operations based on acquired plant images and a trained model.

[0085] Figure 8 is a flowchart showing a series of processes in which the agricultural support device 30 performs predetermined operations based on acquired plant images and a trained model.

[0086] In step S811, the control unit 180 of the terminal device 10 receives operation input from the user managing the field. Specifically, for example, the control unit 180 receives operation input from the user managing the field for a predetermined operation in the field (pollination work, fruit harvesting work, etc.). At this time, the control unit 180 may transmit the operation content entered by the user to the agricultural support device 30, or it may transmit information about the operation that has been set in advance by the user managing the field to the agricultural support device 30.

[0087] In step S801, the control unit 304 of the agricultural support device 30 acquires an image of the plant to be judged. Specifically, for example, the agricultural support device 30 moves in front of the target plant using rails installed on the underside of the device, and the shooting mechanism 350 takes an image of the plant. The control unit 304 may also transmit the acquired image to the server 20. The shooting method may, for example, involve the shooting mechanism 350 automatically identifying flowers, focusing on those areas and taking an image, and also detecting the distance and angle to the photographed flowers using a sensor such as an infrared sensor or a depth sensor, and further calculating the distance and angle from the operating mechanism 302 to the flowers.

[0088] In step S851, the control unit 203 of the server 20 transmits the first trained model to the agricultural support device 30. Specifically, for example, the control unit 203 transmits the first trained model, which was trained by the method described above, to the agricultural support device 30 in accordance with instructions from the agricultural support device 30. At this time, the trained model to be transmitted is not limited to the first trained model. The control unit 203 may determine the trained model to transmit based on the plant images acquired by the agricultural support device 30. Alternatively, information on the trained model to be transmitted may be received from the user who manages the field.

[0089] In step S802, the control unit 304 of the agricultural support device 30 acquires a first trained model from the server. Specifically, the model acquisition unit 3032, which constitutes the control unit 304, acquires various trained models in response to the image acquisition unit 3031 acquiring images of plants. At this time, the model acquisition unit 3032 may store the acquired trained models in the storage unit 303 and omit the process of acquiring trained models when performing various operations on the same plant. Alternatively, the model acquisition unit 3032 may temporarily store the acquired trained models in non-volatile memory and acquire the corresponding trained model each time an operation is performed.

[0090] In step S803, the control unit 304 of the agricultural support device 30 estimates predetermined parameters based on the first trained model and the plant image. Specifically, the parameter estimation unit 3033, which constitutes the control unit 304, controls the process of estimating predetermined parameters at predetermined locations on the plant based on the plant image acquired by the image acquisition unit 3031 and the trained model acquired by the model acquisition unit. The predetermined locations specifically include, for example, the following: • Flowers and petals of plants • Plant pistil • Stamen of a crafted grain • Plant leaves • Fruits of plants • Plant stems Furthermore, the specified parameters include, for example, the following: • The orientation of the specified part of the plant as described above • The positional relationship of the specified parts of the plants mentioned above In this case, the parameter estimation unit 3033 may calculate the orientation parameter not as an upward, downward, etc., but as a normal vector in three-dimensional space. The normal vector in three-dimensional space may be, for example, a representation of various orientations as vectors in a 360° celestial sphere model.

[0091] In step S804, the control unit 304 of the agricultural support device 30 causes a predetermined operation to be performed based on the estimated predetermined parameters. Specifically, the operation instruction unit 3034, which constitutes the control unit 304, controls the process of causing the operating mechanism 302 of the agricultural support device 30 to perform a predetermined operation. The predetermined operation includes, for example, the following: • Pollination (the process of attaching pollen to the pistil) • Harvesting work • Leaf pruning (removing overlapping or excess leaves) • Thinning of fruit • Fertilizer application work • Thinning out diseased fruits, leaves, etc.

[0092] Through the above process, users managing the fields can appropriately perform various treatments (pollination, harvesting, leaf removal, etc.) regardless of the orientation or positional relationship of each part of the plant (flowers, stamens, leaves, stems, fruits, etc.). As a result, it becomes possible to cultivate higher quality fruits more consistently.

[0093] In certain situations, server 20 may maintain records of the quality of fruit when self-pollinating plants are cross-pollinated. Based on these results, server 20 may inform the user managing the field that cross-pollination is expected to improve fruit quality even for self-pollinating plants. This allows users managing the fields to properly control operations such as pollination, including quality improvement.

[0094] <3 Example of Operation> Figures 9 to 10 illustrate examples of the operation of a series of processes in which the system 1 disclosed in the present invention operates the operating mechanisms of agricultural support devices, etc., based on input for predetermined operations in the field received from a user who manages the field.

[0095] Figure 9 shows an operation diagram of the operating mechanism 302 of the agricultural support device 30, which performs a predetermined operation on a predetermined part of a plant based on predetermined parameters.

[0096] In Figure 9, the agricultural support device 30 drives the arm 701 based on predetermined parameters at a predetermined location on the plant, which are identified based on the plant image and a trained model. Figure 9 illustrates an example of operation when performing pollination on the pistil of a flower. As shown in Figure 9, the agricultural support device 30 moves the arm 701 and the drive mechanism 751 of the operating mechanism 302 in a predetermined direction based on the identified parameters, driving the attachment 752 of the operating mechanism 302, which is provided at the tip of the arm 701, to be perpendicular (directly in front) of the pistil of the flower. The predetermined direction is, for example, output by the trained model and determined by a combination of vectors shown below as a normal vector. • Roll (rotation in the x-axis direction) • Pitch (rotation in the y-axis direction) • Yaw (rotation in the z-axis direction) This allows the system to output the orientation in 3D space based on a trained model, even from 2D image information of plants. The agricultural support device 30 combines the movement of the arm 701 and the movement of the drive mechanism 751 to orient the attachment 752 in any direction and bring it into contact with plants. Therefore, a user managing a field can perform operations such as pollination appropriately, even if, for example, the pistil of a flower is facing in the depth direction from the perspective of the agricultural support device 30.

[0097] The operations described above are not limited to pollination. Similar mechanisms may be used for operations that occur in the field (such as leaf removal, pruning, and fruit thinning).

[0098] Figure 10 shows an example screen displaying a notification to the user managing the field when the operating mechanism 302 of the agricultural support device 30 detects a tendency for plant disease.

[0099] In Figure 9, the display 150 of the terminal device 10 owned by the user managing the field shows the captured image 1001, the disease alert 1002, and the poor growth alert 1003. The captured image 1001 is an image of a plant taken by the shooting mechanism 350 provided in the agricultural support device 30.

[0100] The disease alert 1002 indicates an alert displayed for plants that show signs of disease in the captured plant images. Specifically, the control unit 304 of the agricultural support device 30 estimates whether the trend observed in the plant is a disease based on the acquired plant image and the aforementioned fourth trained model. If the estimation results in the plant being diseased, the disease alert 1002 is presented to the user managing the field. In a certain scenario, if the plant is diseased, the agricultural support device 30 may present the disease alert 1002 to the user managing the field and also cause the operation mechanism 302 to perform an operation to remove the diseased parts. This allows users managing fields to detect diseases early and prevent losses due to disease.

[0101] The poor growth alert 1003 indicates an alert displayed for plants that show signs of poor growth in the captured plant images. Specifically, the control unit 304 of the agricultural support device 30 estimates whether the trend observed in the plant is due to poor growth, based on the acquired plant images and the aforementioned fifth trained model. If the estimation results in the plant being found to be poorly growing, the poor growth alert 1003 is presented to the user managing the field. In a certain scenario, if the plant is found to be poorly growing, the agricultural support device 30 may present the poor growth alert 1003 to the user managing the field and also cause the fertilizer adjustment unit to adjust the fertilizer in the area of ​​the field where the poorly growing plants are being cultivated. This allows field managers to detect poor growth caused by uneven fertilizer application and other factors early on, enabling them to harvest uniform fruit.

[0102] In certain situations, the agricultural support device may also notify the user managing the field of the same result based on the assessment of leaf overlap.

[0103] In the embodiments described herein, the agricultural support system is illustrated as one in which a server 20 and an agricultural support device 30 communicate information, but the embodiments are not limited thereto. For example, a series of processes may be executed without communication with the server 20 based on a trained model held in the storage unit of the agricultural support device 30. Furthermore, the control unit of the agricultural support device 30 may have a function to generate a 3D model and train the trained model.

[0104] <4 Variations> Modifications of this embodiment will now be described. That is, the following embodiments may be adopted. (1) An information processing device, which may have this program pre-installed, or may install it afterward, or may store such a program on an external non-temporary storage medium, or may run on cloud computing. (2) A method in which a computer is made to function as an information processing device, and the program may be pre-installed on the information processing device or installed thereafter, or such program may be stored on an external non-temporary storage medium or run on cloud computing.

[0105] <Note> The details described in each of the above embodiments are noted below.

[0106] (Note 1) Agricultural support system 1 comprising an operating mechanism 302 that makes contact with a plant and performs a predetermined operation, and a control unit 304, wherein the control unit 304 includes a model generation unit 2033 that generates a three-dimensional model of a plant by a combination of predetermined generation conditions, a learning unit 2034 that learns a first trained model generated by learning predetermined parameters in the three-dimensional model and an image of the plant as training data, an image acquisition unit 3041 that acquires an image of the plant to be judged, a model acquisition unit 3042 that acquires the first trained model, a parameter estimation unit 3043 that estimates predetermined parameters of a predetermined part of the plant to be judged based on the first trained model and the image of the plant to be judged, and an operation instruction unit 3044 that causes the operating mechanism 302 to perform a predetermined operation based on the estimated predetermined parameters.

[0107] (Note 2) The agricultural support system 1 as described in Appendix 1, wherein the predetermined parameter is at least one selected from the group consisting of the orientation of a predetermined part of a plant in three-dimensional space and the positional relationship of a predetermined part of a plant.

[0108] (Note 3) The agricultural support system 1 according to Appendix 1 or 2, wherein the predetermined generation conditions are at least one selected from the group consisting of the size of the 3D model, the number of petals in the 3D model, the color of the 3D model, the shape of the 3D model, the background of the space in which the 3D model is generated, the position of the lighting in the space in which the 3D model is generated, and the orientation of the 3D model.

[0109] (Note 4) The agricultural support system 1 described in any of the appendices 1 to 3, wherein the designated location of the plant to be judged is at least one selected from the group consisting of the location of the flower pistil, the location of the flower stamen, the location of the fruit, and the location of the leaf.

[0110] (Note 5) The agricultural support system 1 according to any one of the appendices 1 to 4, wherein the prescribed operation is at least one selected from the group consisting of flower pollination, fruit harvesting, leaf removal, and fruit thinning.

[0111] (Note 6) An agricultural support system 1 as described in any of Appendix 1 to 5, including a shooting mechanism 350 for capturing images of plants.

[0112] (Note 7) The agricultural support system 1 according to any one of the appendices 1 to 6, wherein the model acquisition unit 3032 acquires a second trained model that has learned at least one of the ventilation or sunlight in the field and the fruit yield at harvest time.

[0113] (Note 8) The agricultural support system 1 as described in Appendix 7, wherein the control unit 304 includes a positional relationship estimation unit that estimates the optimal positional relationship of any of the groups consisting of leaves, flowers, and fruits based on the yield prediction output by the second trained model, and the operation instruction unit 3044 causes the operation mechanism 302 to perform a predetermined operation based on the optimal positional relationship of any of the groups consisting of leaves, flowers, and fruits estimated by the positional relationship estimation unit.

[0114] (Note 9) The control unit 304 includes a pollination imaging unit that causes the imaging mechanism 350 to capture images of the moment of pollination, a yield prediction unit that predicts the yield based on the captured images of the moment of pollination, and an output unit that presents the predicted yield to the user, as described in any of appendices 6 to 8.

[0115] (Note 10) The agricultural support system 1 according to any one of the appendices 1 to 9 further includes a weight sensor for measuring the weight of the fruit and a sugar content sensor for measuring the sugar content of the fruit, the model acquisition unit 3032 acquires a third trained model that associates an image of the fruit with the weight of the fruit, the sugar content of the fruit, and the quality of the fruit, and the output unit identifies and outputs the quality of the fruit based on the image of the fruit, the weight, the sugar content, and the third trained model.

[0116] (Note 11) The agricultural support system 1 described in any of the appendices 1 to 10, wherein the model acquisition unit 3032 acquires a fourth trained model that associates plant images with plant disease images, the output unit, upon receiving a plant image as input based on the fourth trained model, outputs whether or not the plant is diseased, and the operation instruction unit 3044, if it is output that the plant is diseased, instructs the operation mechanism 302 to perform an operation to remove the diseased parts.

[0117] (Note 12) The agricultural support system 1 as described in any of the appendices 1 to 11, wherein the control unit 304 includes a fertilizer adjustment unit, the model acquisition unit 3032 acquires a fifth trained model that associates plant images with the plant's nutritional status, the output unit, upon receiving plant images as input based on the fifth trained model, determines and outputs whether the plant's nutritional status is excessive or insufficient, and the fertilizer adjustment unit adjusts the amount of fertilizer to be applied to the field based on the plant's nutritional status.

[0118] (Note 13) An agricultural support device 30 comprising an operating mechanism 302 that makes contact with a plant and performs a predetermined operation, and a control unit 304, wherein the control unit 304 includes a model generation unit 2033 that generates a three-dimensional model of a plant by a combination of predetermined generation conditions, a learning unit 2034 that learns a first trained model generated by learning predetermined parameters in the three-dimensional model and an image of the plant as training data, an image acquisition unit 3041 that acquires an image of the plant to be judged, a model acquisition unit 3042 that acquires the first trained model, a parameter estimation unit 3043 that estimates predetermined parameters of a predetermined part of the plant to be judged based on the first trained model and the image of the plant to be judged, and an operation instruction unit 3044 that causes the operating mechanism 302 to perform a predetermined operation based on the estimated predetermined parameters.

[0119] (Note 14) An agricultural support method using an operating mechanism 302 that makes contact with a plant and performs a predetermined operation, and a control unit 304, wherein the control unit 304 executes a model generation step of generating a three-dimensional model of a plant by a predetermined combination of generation conditions, a learning step of learning a first trained model generated by learning predetermined parameters in the three-dimensional model and an image of the plant as training data, an image acquisition step (S801) of acquiring an image of the plant to be judged, a model acquisition step (S802) of acquiring the first trained model, a parameter estimation step (S803) of estimating predetermined parameters of a predetermined part of the plant to be judged based on the first trained model and the image of the plant to be judged, and an operation instruction step (S804) of causing the operating mechanism 302 to perform a predetermined operation based on the estimated predetermined parameters.

[0120] (Note 15) A program for causing a computer 20 to perform agricultural support using an operating mechanism 302 that makes contact with a plant and performs predetermined operations, and a control unit 304, wherein the program causes the control unit 304 to execute: a model generation step of generating a 3D model of a plant by a combination of predetermined generation conditions; a learning step of learning a first trained model generated by learning predetermined parameters in the 3D model and an image of the plant as training data; an image acquisition step (S801) of acquiring an image of the plant to be judged; a model acquisition step (S802) of acquiring the first trained model; a parameter estimation step (S803) of estimating predetermined parameters of a predetermined part of the plant to be judged based on the first trained model and the image of the plant to be judged; and an operation instruction step (S804) of causing the operating mechanism 302 to perform predetermined operations based on the estimated predetermined parameters. [Explanation of symbols]

[0121] 1 System, 10 Terminal device, 12 Communication interface, 13 Input device, 14 Output device, 15 Memory, 16 Storage unit, 19 Processor, 20 Server, 22 Communication interface, 23 Input / Output interface, 25 Memory, 26 Storage, 29 Processor, 30 Agricultural support device, 32 Communication interface, 33 Input / Output interface, 35 Memory, 36 Storage unit, 39 Processor, 80 Network, 170 Storage unit, 180 Control unit, 1801 Input operation reception unit, 1802 Transmit / receive unit, 1803 Data processing unit, 1804 Notification control unit, 130 Operation reception unit, 1301 Keyboard, 1302 Mouse, 140 Voice processing unit, 141 Microphone, 142 Speaker, 150 Display, 160 Location information sensor, 202 Storage unit, 2021 Trained model database, 203 Control unit, 2031 Receive control unit, 2032 Transmit control unit, 2033 Model generation unit, 3034 Learning unit, 302 Operation mechanism, 303 Storage unit, 304 Control unit, 3041 Image acquisition unit, 3042 Model acquisition unit, 3043 Parameter estimation unit, 3044 Operation instruction unit, 350 Imaging mechanism

Claims

1. An operating mechanism that makes contact with a plant to perform a predetermined operation, An agricultural support system comprising a control unit, The control unit, A learning unit that trains a first trained model generated by training it using predetermined parameters related to plants and images of plants as training data, An image acquisition unit that acquires images of the plant to be judged, The model acquisition unit acquires the aforementioned first trained model, A parameter estimation unit estimates predetermined parameters of a predetermined location on a plant to be determined based on the first trained model and an image of the plant to be determined, The operating mechanism includes an operation instruction unit that causes the operating mechanism to perform a predetermined operation from a predetermined direction at a predetermined location on the plant to be judged, based on the predetermined parameters estimated above, The predetermined parameter includes a vector component indicating direction. Agricultural support system.

2. The predetermined parameter is a normal vector indicating a predetermined direction at a predetermined location of the plant. The operation instruction unit causes the operation mechanism to perform the predetermined operation on the predetermined location of the plant to be judged, from the direction of the normal vector. The agricultural support system according to claim 1.

3. The predetermined location of the plant subject to determination is the pistil of the plant. The aforementioned predetermined operation is the pollination operation of flowers. The predetermined parameter is a vector indicating the direction in which the pistil is facing. The operation instruction unit causes the operation mechanism to perform the pollination operation from the front of the pistil based on the vector. The agricultural support system according to claim 1.

4. The agricultural support system according to claim 1, wherein the predetermined parameter is at least one selected from the group consisting of the orientation of the predetermined part of the plant and the positional relationship of the predetermined part of the plant.

5. The agricultural support system according to any one of claims 1 to 4, wherein the predetermined location of the plant to be determined is at least one selected from the group consisting of the location of the flower, the location of the petals, the location of the pistil, the location of the stamen, the location of the leaf, the location of the fruit, and the location of the stem.

6. The agricultural support system according to any one of claims 1 to 4, wherein the predetermined operation is at least one selected from the group consisting of flower pollination, fruit harvesting, leaf removal, and fruit thinning.

7. An operating mechanism that makes contact with a plant to perform a predetermined operation, An agricultural support device comprising a control unit and, The control unit, A learning unit that trains a first trained model generated by training it using predetermined parameters related to plants and images of plants as training data, An image acquisition unit that acquires images of the plant to be judged, The model acquisition unit acquires the aforementioned first trained model, A parameter estimation unit that estimates predetermined parameters at predetermined locations of a plant to be determined based on the first trained model and an image of the plant to be determined, The operating mechanism includes an operation instruction unit that causes the operating mechanism to perform a predetermined operation from a predetermined direction at a predetermined location on the plant to be judged, based on the predetermined parameters estimated above, The predetermined parameter includes a vector component indicating direction. Agricultural support equipment.

8. The predetermined parameter is a normal vector indicating a predetermined direction at a predetermined location of the plant. The operation instruction unit causes the operation mechanism to perform the predetermined operation on the predetermined location of the plant to be judged, from the direction of the normal vector. The agricultural support device according to claim 7.

9. The predetermined location of the plant subject to determination is the pistil of the plant. The aforementioned predetermined operation is the pollination operation of flowers. The predetermined parameter is a vector indicating the direction in which the pistil is facing. The operation instruction unit causes the operation mechanism to perform the pollination operation from the front of the pistil based on the vector. The agricultural support device according to claim 7.

10. An operating mechanism that makes contact with a plant to perform a predetermined operation, A method for supporting agriculture using a control unit and The aforementioned agricultural support method includes the control unit, A learning step involves training a first pre-trained model, which is generated by training it using predetermined parameters related to plants and images of plants as training data. An image acquisition step to obtain an image of the plant to be judged, The first step of acquiring a pre-trained model, A parameter estimation step in which predetermined parameters are estimated for predetermined locations of the plant to be determined based on the first trained model and an image of the plant to be determined, Based on the predetermined parameters estimated above, the operation instruction step is performed to cause the operation mechanism to perform a predetermined operation from a predetermined direction at a predetermined location on the plant to be judged, The predetermined parameter includes a vector component indicating direction. method.

11. The predetermined parameter is a normal vector indicating a predetermined direction at a predetermined location of the plant. The operation instruction step involves causing the operation mechanism to perform the predetermined operation on the predetermined location of the plant to be judged, from the direction of the normal vector. The method according to claim 10.

12. The predetermined location of the plant subject to determination is the pistil of the plant. The aforementioned predetermined operation is the pollination operation of flowers. The predetermined parameter is a vector indicating the direction in which the pistil is facing. The operation instruction step causes the operation mechanism to perform the pollination operation from the front of the pistil based on the vector. The method according to claim 10.

13. An operating mechanism that makes contact with a plant to perform a predetermined operation, A program for causing a computer to perform agricultural support using a control unit, wherein the program is configured to allow the control unit to perform agricultural support using the control unit. A learning step involves training a first pre-trained model, which is generated by training it using predetermined parameters related to plants and images of plants as training data. An image acquisition step to obtain an image of the plant to be judged, The first step of acquiring a pre-trained model, A parameter estimation step in which predetermined parameters are estimated for predetermined locations of the plant to be determined based on the first trained model and an image of the plant to be determined, Based on the predetermined parameters estimated above, the operation mechanism is instructed to perform an operation instruction step that causes it to perform a predetermined operation from a predetermined direction at a predetermined location on the plant to be judged, The predetermined parameter includes a vector component indicating direction. program.

14. The predetermined parameter is a normal vector indicating a predetermined direction at a predetermined location of the plant. The operation instruction step involves causing the operation mechanism to perform the predetermined operation on the predetermined location of the plant to be judged, from the direction of the normal vector. The program according to claim 13.

15. The predetermined location of the plant subject to determination is the pistil of the plant. The aforementioned predetermined operation is the pollination operation of flowers. The predetermined parameter is a vector indicating the direction in which the pistil is facing. The operation instruction step causes the operation mechanism to perform the pollination operation from the front of the pistil based on the vector. The program according to claim 13.

Images