Method and device for multi-machine collaborative farming

Abstract

A method for coordinating machines to perform a task includes establishing multiple communication channels among a plurality of agricultural machines located in a geographic area, receiving data regarding the capabilities and possibly locations of each of the plurality of machines, and determining a collaboration plan for completing the task using the plurality of machines, at least a portion of which is transmitted to each of the plurality of machines. The collaboration plan may be based on the locations and capabilities of each of the plurality of machines. The collaboration plan may include determining a plurality of operations for each of the plurality of machines.

Description

[Technical field]

[0001] <CROSS REFERENCE TO RELATED APPLICATIONS> This application claims the benefit of U.S. Provisional Application No. 62 / 902,452, filed September 19, 2019, and U.S. Patent Application No. 17 / 015109, filed September 9, 2020, the disclosures of both of which are incorporated herein by reference in their entireties.

[0002] The present disclosure relates generally to multiple machines engaged in the same task, and more specifically to multi-machine collaborative agriculture. [Background technology]

[0003] Farms, construction sites, and other locations typically require one or more tasks to be performed, such as harvesting crops, treating an area with a solid or liquid, smoothing a surface, etc. Each of these tasks requires a specific amount of time to complete.

[0004] These tasks are often completed using machines. A single machine can complete the task quickly compared to manual methods. Multiple machines can complete the task faster than a single machine can perform the task.

[0005] Using multiple machines to complete a task, on the other hand, can result in wasted time and / or resources. Using multiple machines can result in overloading as more than one machine is working on a particular area. Also, using multiple machines can cause a task to take longer than expected due to overlapping machine efforts. Summary of the Invention

[0006] A method of machine coordination to perform a task includes establishing a number of communication channels between a number of machines located in a geographic area, receiving data regarding the capabilities of each of the number of machines and possibly the location of each of the number of machines, and determining a collaboration plan for completing the task using the number of machines. At least a portion of the collaboration plan is transmitted to each of the number of machines. The collaboration plan may be based on the location and the capabilities of each of the number of machines. The collaboration plan may include determining a number of operations for each of the number of machines. The operations may include paths, speeds, and machine implement operations. The collaboration plan may include start and end times for operations by machines and / or time periods for which the machines will perform operations. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 illustrates a single machine performing a task. [Diagram 2] FIG. 2 illustrates multiple machines working together to complete a particular task. [Diagram 3] FIG. 3 is a diagram illustrating how multiple machines working in a field can cause overlapping of work. [Figure 4] FIG. 4 is a flow chart of an agricultural method according to one embodiment. [Diagram 5] FIG. 5 illustrates a device communicating with other devices within signal range. [Figure 6] FIG. 6 illustrates a set of connections between devices that form a Multi-Machine Collaborative Farming ("MMCF") network. [Figure 7] FIG. 7 illustrates a device transmitting information to another device over a multi-hop network. [Figure 8] FIG. 8 illustrates a network that has been repaired following device unavailability. [Figure 9]FIG. 9 illustrates separate networks that may be connected using adjacent or cloud-based bridge nodes. [Figure 10] FIG. 10 illustrates MMCF software that communicates by abstraction. [Figure 11] FIG. 11 is a diagram showing the types of information stored in an MMCF device. [Figure 12] FIG. 12 is a high-level schematic diagram showing a computer for implementing the various machines and devices described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] FIG 1 illustrates an agricultural machine 102 performing a task in a field 100. In this example, the task is the application of liquid fertilizer. The machine 102 travels along a path 104 from one end of the field 100 to the opposite end in a first direction indicated by arrow 106, then turns around and travels in the opposite direction indicated by arrow 108. The machine 102 continues traversing through the path 104 as indicated by arrows 110, 112. The machine illustrated in FIG 1 has traveled a portion of the path 104, with the path that has been traveled being shown in solid lines and the portion of the path that has not been traveled being shown in dashed lines.

[0009] A task may include one or more activities such as applying a liquid or solid substance (e.g., fertilizer or pesticide), harvesting, planting, etc. As shown in Figure 1, since only a single machine 102 is performing a task, the task takes a certain amount of time to complete. The time it takes a single machine to complete a task may depend on, for example, the size of the field being worked on, the speed of the machine, the time it takes the machine to turn around, the specific task being performed, etc.

[0010] Multi-machine collaborative farming ("MMCF") is a method in which multiple machines work together to complete a particular task faster than a single machine can complete it. While many of the embodiments described herein relate to agriculture, the methods and techniques described are also applicable to construction and / or surface modification tasks. FIG. 2 shows machines 200, 202, 204, and 206 working together to perform a task in a field 100. The task performed in FIG. 2 may be, for example, spreading fertilizer. In FIG. 2, each of the illustrated machines 200, 202, 204, and 206 travels in the same direction as indicated by arrows 201, 203, 205, and 207. By using multiple machines as shown in FIG. 2, fertilizer can be spread on the field 100 in a much shorter time than it would take a single machine to spread fertilizer on the field 100. By running the field in a coordinated manner, the multiple machines can complete the task for the majority of the field in a single pass. However, the use of multiple machines requires coordination between the machines. For example, as shown in Figure 2, machines 200, 202, 204, and 206 travel in the same direction in a coordinated manner with space between them, front to back, left to right, and right to left. The speed and direction of each of the machines 200, 202, 204, and 206 are controlled to maintain the spacing. Coordination is required to complete tasks efficiently and prevent overlapping and / or undesirable activities, such as overspreading of seed or product.

[0011] FIG. 3 shows liquid application machines 300 and 302 operating in a field 100. Machine 300 is shown moving in a direction indicated by arrow 301 and applying liquid fertilizer to area 304. Machine 302 is shown moving in a direction indicated by arrow 303 and applying liquid fertilizer to area 306. The paths of machines 300 and 302 overlap in area 308. Since both machines 300 and 302 apply liquid fertilizer to area 308, twice as much liquid fertilizer would be applied to area 308 if the operations of machines 300 and 302 were not coordinated. To prevent such undesirable results, in one embodiment, a method of multi-machine collaborative farming is used to coordinate the operations of multiple machines performing tasks.

[0012] Coordination between machines requires communication between the machines. To allow for mobility of the machines, the communication between the machines is wireless. In one embodiment, communication between the machines is facilitated by each machine communicating through its device with devices of other machines within range of its device's wireless transceiver. A collaboration plan for multiple machines to complete a task (e.g., farming, construction, surface modification, etc.) can be determined by a device located on one of the multiple machines. The collaboration plan can also be determined by one or two devices associated with the multiple machines (e.g., distributed computing). Alternatively, the collaboration plan can be determined by a computing device located remotely from each of the multiple machines. For example, the collaboration plan can be determined by a computer capable of wireless communication with one or more of the multiple machines. FIG. 4 shows a flow chart of a method 400 of MMCF. In step 402, multiple communication channels are established between multiple machines located in a geographic area. Machines such as agricultural and construction machines are generally far apart and have difficulty communicating and cannot transmit through obstacles.

[0013] FIG. 5 illustrates a device 502 associated with an agricultural machine and including a transceiver. The device 502 has a signal range 520 that is the maximum distance over which the device 502 can communicate with one of the other devices via one of multiple communication channels between the device 502 and the other device. In one embodiment, the signal range of the device 502 is fixed (e.g., the device 502 can communicate within a range of approximately 3 kilometers). In FIG. 5, the device 502 has established communication channels with each of the devices 504, 506, 508, 510, and 512 that are within the signal range 520 of the device 502. The devices 514, 516, and 518 are located outside the signal range 520 and the device 502 cannot establish a communication channel with each of these devices. In one embodiment, the device 502 can also communicate with a cloud 500, which in one embodiment is comprised of one or more computers that are remote from the device 502. In one embodiment, the area in which all of the devices that can communicate with each other are located is referred to as a geographic area. In one embodiment, the range of a particular device can be expanded by using multiple other devices located remotely from the particular device.

[0014] FIG. 6 illustrates an MMCF network, also referred to as a graph. The graph includes the set of all connections between devices. FIG. 6 illustrates a device 502 having a signal range 520 and a device 516 having a signal range 530. Devices 506, 508, and 510 are located within the signal range of both devices 502 and 516, allowing them to communicate even though devices 502 and 516 are outside of each other's signal range. Devices 502 and 516 are not within each other's range (e.g., 3 kilometers) and therefore cannot communicate with each other directly. However, devices 502 and 516 can communicate by sending information to each other through one or more of devices 506, 508, and 510 that are located within the range of both devices 502 and 516. When a communication channel between multiple devices determines that two devices cannot communicate directly due to distance or physical obstruction, the two devices can communicate through a third device.

[0015] Returning to FIG. 4, at step 404, after the establishment of the communication channel, data regarding each of the plurality of machines in the area is received by each device in the MMCF network. That is, data is transmitted from each device and received by each device in the MMCF network. The data includes the functional capabilities of the machines, such as the tasks the machines can perform (e.g., harvesting, liquid application, solid application, seed distribution / planting, etc.). The data may also include information regarding the physical aspects of each machine and / or details such as implement width, machine width, machine speed, machine limitations (e.g., ground slope, etc.), machine location, application range, task parameters, etc. The data may also include information regarding the quantity, quality, and / or type of physical tangible materials available to each of the plurality of machines. For example, the data may be regarding fuel, grain, pesticides, etc. associated with the machine. The data is transmitted from each device and received by each device in the MMCF network.

[0016] FIG. 7 illustrates how devices are connected to a source device. A source device is a device that is currently transmitting to other devices. Thus, each device in the MMCF network can be a source device. In one embodiment, a source device is connected to a source device through the fewest number of other devices. The number of devices used to connect to a source device is known as the hop count. As shown in FIG. 7, the hop count from source device 502 to each of devices 504, 506, 508, 510, and 512 is 1. The hop count from source device 502 to devices 514 and 516 is 2. This is because for data from source device 502 to be received at device 514, the data must pass through one of devices 506 or 508, and for data from source device 502 to be received at device 516, the data must pass through one of devices 508 or 510. The hop count from source device 502 to device 518 is 3. This is because, for data to be received by device 518, the data must pass through one of devices 506, 508, or 510 and then through one of devices 514 or 516. Note that each device in the MMCF network can be a source device and can send data to other devices. In order to maintain the network, the MMCF network must take into account devices that stop communicating or become unavailable.

[0017] FIG. 8 illustrates how the MMCF network repairs itself when a device stops communicating. Device 508 was in communication with other devices such as devices 502, 504, 506, 510, 512, 514, 516, and 518, but device 508, marked with an X in FIG. 8, has stopped communicating with the other devices. Devices 502, 504, and 512 can no longer communicate with device 508 or with other devices through device 508. Devices 502, 504, and 512 can communicate with devices 514, 516, and 518 through devices 506 and 510 instead of device 508. When a device starts communicating or becomes operational again, it is added to the existing MMCF network. Similarly, if there are no devices to connect the separate networks, multiple separate MMCF networks are formed.

[0018] 9 illustrates two separate MMCF networks that are not currently connected to each other. Devices 502, 504, and 512 form a first network that is not currently communicating with devices 514, 516, and 518, which form a second network. When any of devices 514, 516, or 518 initiates or resumes communication with another device, the first and second separate networks become connected. Similarly, if the devices of the first and second separate networks are too far away from any of devices 514, 516, or 518 to communicate, the networks become connected when at least two of the devices come within range of each other. The range over which devices can communicate is based in part on the type of network over which the devices communicate.

[0019] There is a limit to the bandwidth for data transmission between devices. In one embodiment, both subscription-based and request-receiving types are used for device-to-device communication to reduce bandwidth requirements. This is important to maximize the limited expected bandwidth value. For example, consider devices A, B, and C. Device A generates data streams for resources X, Y, and Z. The communication channel between devices A, B, and C saturates at a data rate R. The data rates of the streams for resources X, Y, and Z are represented by Rx, Ry, and Rz respectively, and Rx + Ry + Rz > R. In this case, device A cannot perform data communication for all resources. In the subscription-based type, devices B and C can register the required resources with device A. If B and C only subscribe to X, A only needs to send Rx < R automatically. This data reaches B and C with minimal delay. If either B or C needs the current state of Y but does not require continuous updates regarding the state of Y, B or C can make a one-time request using the request-receiving type. Although there is an inherent delay because B or C does not receive new information about Y until a change occurs, this is appropriate for resources with low priority or importance and can keep the bandwidth available for the higher-priority resource X.

[0020] Consider the above case assuming Rx + Ry < R. Further, assume that C is not within the communication range of A and communication must be carried out via device B which is within the communication ranges of A and C. When C requests resource Y, since C is not within the communication range of A, it cannot request a subscription from A. Instead, it requests a subscription from B. In this case, B must request a subscription for Y from A.

[0021] FIG. 10 illustrates how devices in a multi-machine collaborative agricultural network communicate over different networks. In one embodiment, MMCF software 1002 is executed by a device associated with the agricultural machine. The MMCF software 1002 is network interface agnostic and communicates through one or more of Ethernet 1006, WiFi 1008, or 4G / LTE 1010 located in a network 1012 by a kernel network abstraction 1004. The kernel network abstraction 1004 may also communicate using other types of network protocols. For example, in one embodiment, if a connection between devices cannot be established through normal communication channels, a cloud-based bridge node may be used for communication. A bridge node is software that acts like a device, allowing devices that cannot connect due to their distance from each other to connect through the bridge node.

[0022] Returning to FIG. 4, at step 406, a task to be performed is identified. In one embodiment, a user identifies a task to be performed on one of the devices in the MMCF network. For example, the user identifies a particular field to be harvested using a desired one of the devices. In one embodiment, candidate tasks that can be completed are determined and presented to the one of the devices for selection by the user. In one embodiment, the candidate tasks that can be completed are determined based on machines currently available to perform each task.

[0023] In step 408, a collaboration plan for completing the identified task is determined based on the data for each of the plurality of machines and the information about the identified task received in step 404. For example, if a harvesting task is selected, a number of combine harvesters available to perform the harvesting is determined, and a collaboration plan for performing harvesting in the field is determined based on the number of combine harvesters available.

[0024] In one embodiment, the collaboration plan is determined based on the location of each of the multiple machines and the capabilities of each of the multiple machines. For example, the collaboration plan may include combine harvesters located near fields that need to be harvested, while excluding combine harvesters far from the fields. In one embodiment, the collaboration plan determines a number of operations for each of the multiple machines. The number of operations may include determining a speed for each machine, and an implement operation for each machine. In one embodiment, determining a path for each of the multiple machines may include determining a start time and a start location for each machine, and a duration for each machine to traverse the path.

[0025] In one embodiment, determining a collaborative plan to complete a task may include determining machine spacing. Front-to-back machine spacing may be based on the speed of the machine, the stopping ability of the machine, visibility from the cabin of the machine, etc. Side-to-side machine spacing may be based on the implement geometry and the task being performed. For example, for harvesting machines, the portions of the field where the machines harvest may overlap, but for machines that apply pesticides or fertilizers, the work portions should generally not overlap to prevent overapplication.

[0026] In one embodiment, the collaboration plan for multiple storable machines (e.g., grain bins and harvest bins) can be based on the collaboration plan for multiple harvesters. For example, a combine harvester can determine a collaboration plan for multiple combine harvesters harvesting a field. The collaboration plan for the combine harvester can then be used to determine a collaboration plan for multiple storable machines receiving the harvest harvested by the combine harvester. The collaboration plan for the storable machines generated can use information about the physical locations and spacing of all machines, current capacity data of the combine harvester and the storable machines, and instruct the storable machines to follow the combine harvester (in an auto-steering manner) so that the harvest can be unloaded in a more or less optimal manner. This embodiment can be used to minimize the downtime of the combine harvester.

[0027] In step 410, the one or more devices that have determined the collaboration plan transmit the collaboration plan to each of the multiple devices over each of the multiple communication channels. In one embodiment, the multiple machines are machines that perform the identified tasks. The collaboration plan may also be received by machines that do not perform tasks, allowing operators of those machines to know which tasks are being performed nearby. Each device in the MMCF network can store various information.

[0028] In one embodiment, the collaborative plan transmitted to each device of the multiple machines is presented to each operator of each machine so that the operator can steer the machine according to the collaborative plan. In another embodiment, the collaborative plan is automatically executed by each machine involved in the collaborative plan. In one embodiment, the collaborative plan is achieved through a combination of both automatic control of the machines and operator input. For example, the machines may be automatically controlled to execute the collaborative plan, and an operator may override the automatic machine operations if an unexpected situation occurs, such as an unexpected dangerous terrain or an object to avoid.

[0029] A device that can join the MMCF network is referred to as an MMCF device. FIG. 11 shows an MMCF device 1102 that stores a resource list 1101 and an MMCF device list 1103. Each resource in the resource list 1101 is a piece of information. For example, a resource may be vehicle location data, application area data, product data (e.g., for fertilizer or pesticide). In one embodiment, the MMCF device list 1103 includes an identification of a physical device that runs the MMCF software. The information stored in the MMCF device 1102 may be received from, sent to, or requested from a variety of locations, such as, for example, a cloud server 1104, a personal computer 1106, and / or a tractor 1108.

[0030] In one embodiment, each of the above elements can be implemented using a computer. FIG. 12 is a high-level block diagram of such a computer. The computer 1202 includes a processor 1204 that controls the overall operation of the computer 1202 by executing computer program instructions that define such operations. The computer program instructions may be stored in a storage device 1212 or other computer-readable medium (e.g., a magnetic disk, a CD-ROM, a USB drive, an SD card, etc.) and loaded into the memory 1210 when it is desired to execute the computer program instructions. Thus, the above-described methods can be defined by computer program instructions stored in the memory 1210 and / or the storage device 1212 and controlled by the processor 1204 that executes the computer program instructions. For example, the computer program instructions can be implemented as computer executable code programmed by a person skilled in the art to execute the algorithms defined in the above-described methods and functions. Thus, by executing the computer program instructions, the processor 1204 executes the algorithms defined in the above-described methods and functions. The computer 1202 also includes one or more network interfaces 1206 for communicating with other devices over a network. Computer 1202 also includes input / output devices 1208 (e.g., a display, keyboard, mouse, speakers, buttons, etc.) that allow a user to interact with computer 1202. Those skilled in the art will appreciate that an actual computer implementation may include other components as well, and FIG. 12 shows a high-level view of some of the components of such a computer for illustrative purposes.

[0031] The foregoing Detailed Description is to be understood in all respects as illustrative and not restrictive, and the scope of the inventive concepts disclosed herein should be determined not from the Detailed Description, but from the claims which are interpreted in their entirety as permitted by each patent law. It should be understood that the embodiments shown and described herein are merely illustrative of the principles of the inventive concepts, and that various modifications may be made by those skilled in the art without departing from the scope and spirit of the inventive concepts. Those skilled in the art could implement various other feature combinations without departing from the scope and spirit of the inventive concepts.

Claims

1. Establishing multiple communication channels between multiple devices; performing communication between the plurality of devices using a subscription type and a request reception type in response to a case where the data stream from the plurality of devices exceeds an available bandwidth; Each of the plurality of devices is associated with one of a plurality of agricultural machines disposed in a geographic area, each of the plurality of agricultural machines having functional capabilities, physical aspects, and physical tangible assets, and establishing the plurality of communication channels comprises: determining that two devices among the plurality of devices cannot communicate with each other using either a subscription type or a request reception type; establishing a communication channel between the two devices with a third device within range of each of the two devices; receiving, by at least one of the plurality of devices disposed on one of the plurality of agricultural machines, data regarding the functional capabilities, the physical aspects, and the quantity, quality, and type of the physical tangible materials of each of the plurality of agricultural machines; determining a collaborative plan for completing a task using the plurality of agricultural machines based on the received data; and transmitting at least a portion of the collaboration plan from at least one of a plurality of devices to each of the plurality of agricultural machines via the plurality of communication channels, the portion of the collaboration plan being for controlling execution of a portion of the collaboration plan involving the plurality of agricultural machines. method.

2. the received data includes data regarding a location of each of the plurality of agricultural machines; 2. The method according to claim 1 .

3. Determining the collaboration plan further comprises: determining a plurality of operations for each of the plurality of agricultural machines; 3. The method according to claim 2 .

4. Determining the plurality of operations includes: determining a path for each of the plurality of agricultural machines; determining a speed at which each of the plurality of agricultural machines travels along the path; determining an implement task for each of the plurality of agricultural machines; 4. The method according to claim 3 .

5. determining a path for each of the plurality of agricultural machines includes determining a start time and a start location for each of the plurality of agricultural machines to commence operation; 5. The method according to claim 4 .

6. determining a path for each of the plurality of agricultural machines includes determining a time period it will take for each of the plurality of agricultural machines to complete the path.

6. The method according to claim 5 .

7. A processor; a memory for storing computer program instructions; The computer program instructions, when executed by the processor, cause the processor to: Establishing multiple communication channels between multiple devices; performing communication between the plurality of devices using a subscription type and a request reception type in response to a case where the data stream from the plurality of devices exceeds an available bandwidth; Each of the plurality of devices is associated with one of a plurality of agricultural machines disposed in a geographic area, each of the plurality of agricultural machines having functional capabilities, physical aspects, and physical tangible assets, and establishing the plurality of communication channels comprises: determining that two devices among the plurality of devices cannot communicate with each other using either a subscription type or a request reception type; establishing a communication channel between the two devices with a third device within range of each of the two devices; receiving, by at least one of the plurality of devices disposed on one of the plurality of agricultural machines, data regarding the functional capabilities, the physical aspects, and the quantity, quality, and type of the physical tangible materials of each of the plurality of agricultural machines; determining a collaborative plan for completing a task using the plurality of agricultural machines based on the received data; and transmitting at least a portion of the collaboration plan from at least one of a plurality of devices to each of the plurality of agricultural machines via the plurality of communication channels, the portion of the collaboration plan being for controlling the plurality of agricultural machines to execute a portion of the collaboration plan involving the plurality of agricultural machines.

8. the received data includes data regarding a location of each of a plurality of agricultural machines; 8. Apparatus according to claim 7, characterized in that

9. determining the collaborative plan further includes determining a plurality of operations for each of the plurality of agricultural machines.

9. The device according to claim 8.

10. Determining the plurality of operations may further include determining a path for each of the plurality of agricultural machines; determining a speed at which each of the plurality of agricultural machines travels along the path; determining an implement task for each of the plurality of agricultural machines; 10. The device according to claim 9.

11. determining a path for each of the plurality of agricultural machines includes determining a start time and a start location for each of the plurality of agricultural machines to commence operation; 11. The device according to claim 10 .

12. determining a path for each of the plurality of agricultural machines includes determining a time period it will take for each of the plurality of agricultural machines to complete the path.

12. The device according to claim 11 .

13. A computer readable medium storing computer program instructions, comprising: The computer program instructions, when executed by a processor, cause the processor to: Establishing multiple communication channels between multiple devices; performing communication between the plurality of devices using a subscription type and a request reception type in response to a case where the data stream from the plurality of devices exceeds an available bandwidth; Each of the plurality of devices is associated with one of a plurality of agricultural machines disposed in a geographic area, each of the plurality of agricultural machines having functional capabilities, physical aspects, and physical tangible assets, and establishing the plurality of communication channels comprises: determining that two devices among the plurality of devices cannot communicate with each other using either a subscription type or a request reception type; establishing a communication channel between the two devices with a third device within range of each of the two devices; receiving, by at least one of the plurality of devices disposed on one of the plurality of agricultural machines, data regarding the functional capabilities, the physical aspects, and the quantity, quality, and type of the physical tangible materials of each of the plurality of agricultural machines; determining a collaborative plan for completing a task using the plurality of agricultural machines based on the received data; and transmitting at least a portion of the collaboration plan from at least one of a plurality of devices to each of the plurality of agricultural machines via the plurality of communication channels, the portion of the collaboration plan being for controlling execution of a portion of the collaboration plan involving the plurality of agricultural machines; A computer readable medium comprising:

14. the received data includes data regarding a location of each of a plurality of agricultural machines; 14. The computer readable medium of claim 13.

15. determining the collaborative plan further includes determining a plurality of operations for each of the plurality of agricultural machines.

15. The computer readable medium of claim 14.

Images