Cooperative scheduling method and device based on multi-heterogeneous agricultural machinery collaborative operation simulation scene
By transforming the collaborative operation scenario of multi-heterogeneous agricultural machinery into Markov decision-making process, and using the gym framework to build a multi-agent reinforcement learning environment, the optimality and strategy failure problems of collaborative operation scheduling of agricultural machinery are solved, and dynamic response and high efficiency are achieved Scheduling.
Patent Information
- Application Number
- CN202411101976.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-12
- Publication Date
- 2024-11-08
AI Technical Summary
The existing scheduling scheme for collaborative operations of multi-heterogeneous agricultural machinery is difficult to ensure optimality, and the static scheduling strategy based on the demand point pre-generation algorithm has the problem of policy failure under dynamic circumstances.
Transform the collaborative operation scenario of multi-heterogeneous agricultural machinery into Markov decision-making process, use the gym framework to build a multi-agent reinforcement learning environment, design farmland, harvester, and grain truck functions, and adopt preset static scheduling strategies and dynamic scheduling The strategy solves the Markov decision-making process and generates a collaborative scheduling scheme between the main task machine and the collaborative machine.
It realizes dynamic response to meet demand points under complex farmland operation conditions, improves the optimality and flexibility of scheduling, avoids the failure of static scheduling strategies, and improves the efficiency and economic benefits of coordinated agricultural machinery operations.
Smart Images

Figure CN118917208A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of agricultural machinery collaborative technology, and in particular to a collaborative scheduling method and device based on a multi-heterogeneous agricultural machinery collaborative operation simulation scenario. Background Art
[0002] Agricultural activities are an important part of social and economic activities, the foundation of my country's national economy, and an important guarantee for social stability and national development. With the continuous advancement of urbanization and aging, my country's cultivated land area and agricultural population are decreasing, while the cost of agricultural materials, labor and land rental is gradually increasing, which further leads to the gradual shift of rural land management from decentralized management to centralized management. Therefore, the production efficiency and management efficiency of land need to be further paid attention to. With the development of information science and technology and the continuous improvement of automation level, world agriculture is developing towards intelligence, precision and efficiency. With the continuous popularization of farm scale, on the one hand, the automation, intelligence and precision of agricultural machinery are constantly improving, and the cost is also constantly decreasing, making it possible for advanced agricultural machinery to operate in large-scale farmland; on the other hand, advanced planning and scheduling algorithms such as intelligent optimization algorithms and artificial intelligence algorithms have begun to be initially deployed and applied in the scheduling and decision-making tasks of agricultural machinery. Especially with the rise of artificial intelligence in recent years, deep neural networks have broad application prospects in many aspects such as automatic driving, environmental perception, and intelligent autonomous operation of agricultural machinery. Traditional agricultural machinery has also been further expanded to the scope of agricultural robots, with certain autonomous perception, automatic control, and intelligent decision-making capabilities.
[0003] However, there are still two major technical issues to be solved for large-scale autonomous smart farms. In terms of individual agricultural machinery, the level of autonomy, precision and intelligence of agricultural machinery is still low; in addition, many advanced agricultural machinery and equipment cannot be domestically produced. In terms of group operations, many rural areas still generally adopt extensive management models, and lack the use of advanced agricultural machinery and equipment in the four important agricultural operation links of tillage, planting, management and harvesting.
[0004] In various agricultural group operations including tilling, planting, management, and harvesting, multiple heterogeneous agricultural machines are often required to collaborate to complete a specific task. For example, the replenishment of fertilizers to fertilizer applicators during land fertilization, the replenishment of pesticides to sprayers during spraying, the replenishment of seedlings to transplanters during transplanting, the replenishment of seeds to seeders during sowing, and the timely transportation of grain to harvesters during harvesting all require other heterogeneous agricultural robots such as transfer vehicles and grain transport vehicles to replenish agricultural materials or transport crops to the operating agricultural machinery in a timely manner. The above-mentioned heterogeneous multi-agricultural machinery collaborative operation processes are highly similar, that is, the agricultural materials or crop reserves of the operating agricultural machinery are constantly decreasing or increasing throughout the process, and other types of heterogeneous agricultural machinery are required to replenish agricultural materials or transport crops in a timely manner, otherwise the agricultural machinery will not be able to continue agricultural operations when the agricultural materials are exhausted or the warehouse is full. Due to the high similarity of these tasks, taking the harvesting process of the harvester as an example, there are currently three main modes to complete the harvester's grain unloading task: one is that after the harvester's granary is full, it goes to the transfer vehicle or a fixed unloading point to unload the grain, and then returns to the nearest operation point to continue working after completing the unloading. However, this mode is inefficient, and the harvester will incur a lot of time and fuel costs when going back and forth to the granary; the second is that each harvester is equipped with a grain transport vehicle with a large storage capacity to follow the operation. While harvesting, the harvester transfers the grain to the grain transport vehicle through conveyor belts and other equipment. After the grain transport vehicle is full, it goes to a fixed point to unload the grain, and the harvester waits for the grain transport vehicle to return to continue working, but this mode expands the harvester's granary at a high cost. The grain trucks in this scheme are in a following state most of the time, and their own transportation and storage capabilities are not maximized; third, multiple harvesters are equipped with a small number of grain trucks. When the harvester's own granary is full or almost full, it sends a request signal to the nearest grain truck, and the grain truck goes to its location to assist in unloading grain. After unloading, the harvester continues to work, but this mode takes into account both efficiency and cost, and fully utilizes the transportation capacity of the grain truck. However, this method lacks a reasonable scheduling mode, and most of it relies on manual experience. When the harvester's granary is full, it often sends a grain unloading request to the nearest grain truck, which will also cause extra waiting time for the harvester, and lacks reasonable overall scheduling and planning.
[0005] like Figure 1As shown in the figure, most of the current research at home and abroad adopts a two-stage method to decouple the problem into two problems: the task allocation problem of the harvester and the response mode of the grain truck during the field collaborative operation of the harvester and the grain truck. The task allocation problem of the harvester can be transformed into the classic mTSP problem, which can be solved by a heuristic algorithm to obtain the operation rows or operation plots of each harvester that minimize the total operation time or operation distance. There are two main research methods for the response mode of grain trucks: one is dynamic response based on artificial experience, such as the grain truck scheduling algorithm based on preset values and nearest neighbor response will send a grain unloading request to the nearest harvester when the capacity of each harvester reaches the preset value (such as 80%), and the harvester will go to unload the grain; another example is the grain truck response algorithm based on travel estimation and capacity estimation. It will calculate the distance and corresponding driving time from the current grain truck to each harvester in real time, and estimate the harvester full warehouse time, and decide whether the grain truck will go to the harvester and which harvester to go to for unloading based on this time. The artificial experience strategies mentioned above are often easy to deploy and execute, but because they do not consider global information and the cooperative relationship between grain trucks, it is difficult to ensure optimality.
[0006] In addition to the dynamic response strategy based on manual experience, given the known operation path, driving speed, and storage capacity of the harvester, the time and location of the harvester when the storage is full can be pre-generated before the operation begins, and these locations can be defined as demand points. With the pre-generated demand points, the grain truck scheduling problem can be transformed into a vehicle routing problem with time windows and capacity constraints (Constrained Vehicle Routing Problem with Time Window, CVRPTW). When modeling the CVRPTW problem, the unsatisfied time constraints are penalized and the penalty is minimized to ensure that the time constraints are met as much as possible. However, in the scenario of harvester-grain truck collaborative operation, if a demand point is met earlier or later than the set time window, the demand time and location of the subsequent demand points generated by the harvester will change accordingly, and the pre-generated scheduling strategy will no longer apply. In addition, the length of the transit time will also affect the demand time of subsequent demand points. That is, the specific location and time window constraints of each demand point in the CVRPTW problem cannot be generated in advance strictly according to manually designed rules, but are highly dependent on the decisions made and the actual satisfaction time of each demand point. The problem is highly dynamic and coupled, and static scheduling strategies are not applicable. Summary of the invention
[0007] The present invention provides a collaborative scheduling method based on a simulation scenario of collaborative operation of multiple heterogeneous agricultural machinery, so as to solve the problems that the optimality of the existing scheduling scheme of collaborative operation of multiple heterogeneous agricultural machinery is difficult to ensure by using artificial dynamic strategies, and the static scheduling strategy based on the demand point pre-generation algorithm has the problem of strategy failure.
[0008] The first aspect of the present invention provides a collaborative scheduling method based on a multi-heterogeneous agricultural machinery collaborative operation simulation scenario, comprising the following steps: converting a target multi-heterogeneous agricultural machinery collaborative operation scenario into a Markov decision process; constructing the target multi-heterogeneous agricultural machinery collaborative operation scenario as a multi-heterogeneous agricultural machinery collaborative operation simulation scenario based on a gym framework; based on the multi-heterogeneous agricultural machinery collaborative operation simulation scenario, solving the Markov decision process using preset static scheduling strategies and dynamic scheduling strategies to generate a collaborative scheduling scheme for a main task machine and a collaborative machine.
[0009] Optionally, the Markov decision process includes a set of state variables, an action space and a state transfer function, wherein the set of state variables includes the state of farmland, the state of each main task machine and the state of each collaborative machine; the action space adopts the underlying navigation strategy function and / or the upper-level scheduling decision function to determine the action at the next moment; the state transfer function causes the current state of the set of state variables to undergo a state transfer according to the action at the next moment determined by the action space.
[0010] Optionally, based on the multi-heterogeneous agricultural machinery collaborative operation simulation scenario, the Markov decision process is solved by using a preset static scheduling strategy and a dynamic scheduling strategy to generate a collaborative scheduling scheme for the main task machine and the collaborative machine, including:
[0011] Based on the multi-heterogeneous agricultural machinery collaborative operation simulation scenario, a waiting operation path is assigned to each main task machine, and each main task machine is made to perform a harvesting task according to its corresponding waiting operation path;
[0012] Without introducing cooperative machines, the Markov decision process is simulated and deduced, and the env.step() function is called at the first time step to obtain the current state of the farmland and the state of each main task machine;
[0013] Determining whether each main task machine reaches a preset collaboration threshold according to the current farmland state and the state of each main task machine;
[0014] If the preset collaboration threshold is not reached, the simulation is continued until the preset collaboration threshold is reached, the current collaboration demand is generated, and the current demand generation time point, the current main task machine position and the current main task machine orientation are obtained and saved;
[0015] According to the current cooperation demand, the current state of each cooperation machine is traversed, and the cooperation machine that best performs the transfer task is selected;
[0016] Send the current demand generation time point, the current main task machine position and the current main task machine orientation to the cooperative machine that currently best performs the transfer task, and set and save the execution action of the cooperative machine that currently best performs the transfer task;
[0017] The env.step() function is called at each time step, and the judgment, traversal and setting processes are iteratively executed until each main task machine completes the job, and the collaborative scheduling plan of the main task machine and the cooperative machine is output.
[0018] Optionally, traversing the current state of each collaborative machine according to the current collaboration requirement and selecting the collaborative machine that best performs the transfer task at present includes:
[0019] Traversing the current state of each collaborative machine according to the current collaboration requirements;
[0020] If only one collaborative machine is in an idle state, the idle collaborative machine is used as the collaborative machine that best performs the transfer task at present;
[0021] If there are multiple collaborative machines in an idle state, the collaborative machine that is closest to the main task machine that is currently full or has reached a preset storage volume is selected as the collaborative machine that is currently best at performing the transfer task.
[0022] Optionally, it also includes:
[0023] Based on the multi-heterogeneous agricultural machinery collaborative operation simulation scenario, the visualization function is turned on. When the env.step() function is called at each time step, the env.render() function is called synchronously to visualize the working process of each main task machine and each collaborative machine in each time step. When outputting the collaborative scheduling plan of the main task machine and the collaborative machine, the visualized collaborative scheduling plan of the main task machine and the collaborative machine is output synchronously.
[0024] The second aspect of the present invention provides a collaborative scheduling device based on a multi-heterogeneous agricultural machinery collaborative operation simulation scenario, including: a conversion module, used to convert a target multi-heterogeneous agricultural machinery collaborative operation scenario into a Markov decision process; a construction module, used to build the target multi-heterogeneous agricultural machinery collaborative operation scenario into a multi-heterogeneous agricultural machinery collaborative operation simulation scenario based on a gym framework; a solution module, used to solve the Markov decision process based on the multi-heterogeneous agricultural machinery collaborative operation simulation scenario using preset static scheduling strategies and dynamic scheduling strategies to generate a collaborative scheduling plan for the main task machine and the collaborative machine.
[0025] A third aspect of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a collaborative scheduling method based on a multi-heterogeneous agricultural machinery collaborative operation simulation scenario as described in the above embodiment.
[0026] A fourth aspect of the present invention provides a computer program product, which, when executed by a processor, implements the collaborative scheduling method based on a multi-heterogeneous agricultural machinery collaborative operation simulation scenario.
[0027] A fifth aspect of the present invention provides a computer-readable storage medium, which stores a computer program. When the program is executed by a processor, it implements the above-mentioned collaborative scheduling method based on a multi-heterogeneous agricultural machinery collaborative operation simulation scenario.
[0028] The collaborative scheduling method and device based on the multi-heterogeneous agricultural machinery collaborative operation simulation scenario proposed in the embodiment of the present invention abandons the traditional CVRP modeling method, abstracts the multi-heterogeneous agricultural machinery collaborative operation into a Markov decision process, and uses the gym library to establish a multi-agent reinforcement learning environment. By designing and encapsulating class functions such as farmland, harvester, and grain truck, navigation points are pre-allocated to the farmland class to ensure farmland driving constraints. At the same time, in order to solve the problem of static scheduling strategy failure, the static scheduling strategy is transformed into an environment-based dynamic strategy. Strategies such as strategies based on artificial design, reinforcement learning, and multi-agent reinforcement learning can be designed, and the Markov decision process is solved using the strategy to obtain the optimal collaborative scheduling solution for the main task machine and the collaborative machine.
[0029] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0031] Figure 1 This is a schematic diagram of the field collaborative operation environment of harvesters and grain transport vehicles;
[0032] Figure 2 A flowchart of a collaborative scheduling method based on a multi-heterogeneous agricultural machinery collaborative operation simulation scenario provided by an embodiment of the present invention;
[0033] Figure 3Schematic diagram of the navigation strategy and scheduling strategy provided by the embodiment of the present invention, wherein (a) is the navigation strategy and (b) is the scheduling strategy;
[0034] Figure 4 A pseudo code schematic diagram of a demand point pre-generation algorithm provided in an embodiment of the present invention;
[0035] Figure 5 A schematic diagram of demand points obtained by the demand point pre-generation algorithm provided in an embodiment of the present invention;
[0036] Figure 6 A schematic diagram of a dynamic dispatching strategy for grain transport vehicles based on nearest neighbor responses of demand points provided by an embodiment of the present invention;
[0037] Figure 7 A schematic diagram showing a comparison between the harvester waiting time and the grain transport vehicle travel distance under different preset values provided in an embodiment of the present invention;
[0038] Figure 8 A schematic diagram of the structure of a collaborative scheduling device based on a multi-heterogeneous agricultural machinery collaborative operation simulation scenario provided by an embodiment of the present invention;
[0039] Fig. 9 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0040] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.
[0041] The following describes a collaborative scheduling method and device based on a multi-heterogeneous agricultural machinery collaborative operation simulation scenario according to an embodiment of the present invention with reference to the accompanying drawings.
[0042] Figure 2 A flowchart of a collaborative scheduling method based on a multi-heterogeneous agricultural machinery collaborative operation simulation scenario provided by an embodiment of the present invention.
[0043] like Figure 2 As shown, the collaborative scheduling method based on the multi-heterogeneous agricultural machinery collaborative operation simulation scenario includes the following steps:
[0044] In step S201, the target multi-heterogeneous agricultural machinery collaborative operation scenario is converted into a Markov decision process.
[0045] In some embodiments, the Markov decision process includes a set of state variables, an action space, and a state transition function, wherein:
[0046] The state variable set includes the farmland state, the state of each main task machine and the state of each collaborative machine;
[0047] The action space uses the underlying navigation strategy function and / or the upper scheduling decision function to determine the action at the next moment;
[0048] The state transfer function causes the current state of the state variable set to undergo state transfer according to the next action at the next moment determined by the action space.
[0049] During the actual execution process, the embodiment of the present invention first converts the collaborative operation process of the main task machine and the collaborative machine in the target multi-heterogeneous agricultural machinery collaborative operation scenario into a Markov decision process, wherein the Markov decision process includes a state variable set of the target multi-heterogeneous agricultural machinery, an action space of the target multi-heterogeneous agricultural machinery, and a state transfer function of the target multi-heterogeneous agricultural machinery.
[0050] Taking the task path allocation and response problem of the harvester-grain truck in the field collaborative operation process as an example, the highly dynamic agricultural machinery and the high uncertainty of the demand point bring difficulties to the problem modeling. The embodiment of the present invention uses the Markov property of the environment to transform the harvester-grain truck collaborative operation process into a Markov decision process (MRP), that is, a Markov decision process. A random process is said to have the Markov property if and only if the state at a certain moment depends only on the state at the previous moment, which is expressed by the formula P(s t+1 |s0,s1,…,s t )=P(s t+1 |s t ). A Markov process refers to a random process with Markov properties, also known as a Markov chain.
[0051] Furthermore, for the multi-harvester-grain truck collaborative operation environment, in order to ensure the Markov property, it is necessary to reasonably select state variables to fully describe the state of the environment at the current moment. The state variable set of the Markov decision process is described by the following equation:
[0052] s=[s farm ,s trans,1 ,…,s trans,n ,s harv,1 ,…,s harv,m ]
[0053] Among them, s farmThe farmland status includes the length and width of the farmland to be operated, the width of the operation row, the width of the field head, the location of the hangar, the current operation status of each operation row (not operating, operating, completed operation), etc. trans,i , i = 1, 2, ..., n is the state of the grain truck, which needs to include the current position, speed, operation stage (idle, transfer, unloading), grain storage capacity and other information of the grain truck, s harv,j ,j=1,2,…,m is the harvester status, which needs to include the current position, driving direction, current operation line, all operations and current granary capacity of all harvesters.
[0054] In step S202, the target multi-heterogeneous agricultural machinery collaborative operation scenario is constructed as a multi-heterogeneous agricultural machinery collaborative operation simulation scenario based on the gym framework.
[0055] During the actual execution process, the embodiment of the present invention builds the target multi-heterogeneous agricultural machinery collaborative operation scenario into a multi-heterogeneous agricultural machinery collaborative operation simulation scenario based on the gym framework, wherein the simulation scenario satisfies the gym style, and can call the step() function in the system, and reasonably design its observation() observation function and reward() reward function. It can also design and adopt appropriate reinforcement learning or multi-agent reinforcement learning methods, obtain the optimal strategy through training, and then control the collaborative operation of multi-heterogeneous agricultural machinery.
[0056] like Figure 3 As shown, for the action selection of the intelligent agent in the harvester-grain transporter collaborative operation simulation scenario, the embodiment of the present invention designs two action spaces, continuous action and discrete action, according to the control level of the agricultural machinery, specifically:
[0057] (1) The agent performs the bottom-level control of the agricultural machinery through the artificial strategy input in advance, that is, the bottom-level navigation strategy function: a = [v x ,v y ,θ], the agent outputs the speed and direction of the grain truck, where the speed is limited by the maximum speed of the grain truck. If the harvester-grain truck collaborative optimization is considered, the speed and direction of the harvester are determined according to the pre-set artificial strategy. The artificial strategy can also consider strict cross-ridge constraints and its own parameter constraints.
[0058] (2) The intelligent agent makes scheduling decisions for agricultural machinery by training a preset neural network algorithm, that is, the upper-level scheduling decision function including the separated scheduling strategy and navigation strategy: a = [stop, back_to_depot, move_to_harv i], the agent only makes scheduling decisions, that is, it only decides at which time to go to which harvester to unload grain or to go to the hangar to unload grain. After the agricultural machinery receives the scheduling instruction, the navigation strategy can be specified by artificial experience (that is, the bottom layer navigation strategy function is adopted), or it can be generated by PID controller, learning-based methods, etc., and decides at which time the grain truck goes to which harvester to unload grain or to go to the hangar to unload grain.
[0059] In step S203, based on the multi-heterogeneous agricultural machinery collaborative operation simulation scenario, the Markov decision process is solved using a preset static scheduling strategy and a dynamic scheduling strategy to generate a collaborative scheduling solution for the main task machine and the collaborative machine.
[0060] In some embodiments, based on the multi-heterogeneous agricultural machinery collaborative operation simulation scenario, the Markov decision process is solved using a preset static scheduling strategy and a dynamic scheduling strategy to generate a collaborative scheduling scheme for the main task machine and the collaborative machine, including:
[0061] Based on the multi-heterogeneous agricultural machinery collaborative operation simulation scenario, a waiting operation path is assigned to each main task machine, and each main task machine is made to perform a harvesting task according to its corresponding waiting operation path;
[0062] Without introducing cooperative machines, the Markov decision process is simulated and the env.step() function is called at the first time step to obtain the current state of the farmland and the state of each main task machine.
[0063] Determine whether each main task machine has reached the preset collaboration threshold based on the current farmland status and the status of each main task machine;
[0064] If the preset collaboration threshold is not reached, the simulation continues until the preset collaboration threshold is reached, the current collaboration demand is generated, and the time point of the current demand generation, the current main task machine position, and the current main task machine orientation are obtained and saved;
[0065] According to the current cooperation requirements, the current status of each cooperative machine is traversed, and the cooperative machine that best performs the transfer task is selected;
[0066] Send the current demand generation time point, the current main task machine position and the current main task machine orientation to the collaborative machine that currently best performs the transfer task, and set and save the execution action of the collaborative machine that currently best performs the transfer task;
[0067] The env.step() function is called at each time step, and the judgment, traversal, and setting processes are iteratively executed until each main task machine completes the job, and the collaborative scheduling plan for the main task machine and the collaborative machine is output.
[0068] In some embodiments, traversing the current state of each collaborative machine according to the current collaboration requirement and selecting the collaborative machine that best performs the transfer task at present includes:
[0069] Traversing the current state of each collaborative machine according to the current collaboration requirements;
[0070] If only one collaborative machine is in an idle state, the idle collaborative machine is used as the collaborative machine that best performs the transfer task at present;
[0071] If there are multiple collaborative machines in an idle state, the collaborative machine closest to the main task machine that is currently full or has reached a preset storage volume is selected as the collaborative machine that is currently the best to perform the transfer task.
[0072] In the actual implementation process, taking the collaborative operation environment of multiple harvesters and grain transporters as an example, we first combined the actual survey results of farms and agricultural machinery parameters in my country to construct the following scenario to be as close to the real farm environment as possible: the length of the farmland is 320 meters, the width is 100 meters, the width of the field is 10 meters, the width of the operation row is 3 meters, and the yield per unit area of the farmland is 0.8 kg / m 3 . The maximum grain storage capacity of the grain truck is 8000kg, and the maximum driving speed is 6m / s. Consider the unloading problem of three harvesters in the field operation scenario under this plot, in which the operation path of the harvester has been pre-generated according to manual experience or heuristic algorithm. The driving speed, grain bin volume, and operation row parameters of each harvester are shown below. Taking into account the edge of the farmland, the total number of operation rows is obtained by dividing the farmland width by the operation row width and rounding up. The operation rows in Table 1 below are represented by integers from 0 to 33, representing each operation row from east to west.
[0073] Table 1 Harvester parameters
[0074] Harvester name Operation speed Granary capacity Job row number Harvester 0 1.8m / s 1600kg [30,10,21,16,0,25,18,3,28,7,6,2] Harvester 1 1.6m / s 2000kg [31,27,9,15,29,12,20,32,23,5,13] Harvester 2 15m / s 1800kg [22 19 17 14 4 8 1 26 33 24 11]
[0075] Assume that all harvesters start from a hangar located at random positions at the four corners of the entire farmland. In order to ensure that the harvester completes the operation without crossing the ridge, the harvester traverses the assigned operation rows in sequence according to the pre-designed navigation strategy and finally returns to the hangar. Specifically, before the operation task starts, all navigation points of the entire farmland are pre-generated and saved according to the size of the farmland and the location of the hangar. After the harvester obtains a specific operation task, it allocates its operation navigation points to each agricultural machine according to the pre-generated farmland navigation points, so that it can complete all operation rows and return to the hangar.
[0076] First, the existing demand point pre-generation algorithm is used to simulate the collaborative operation simulation scenario of harvesters and grain trucks to generate the demand point location and time when each harvester is full or reaches a specified percentage of the warehouse in advance, and the problem is converted into a vehicle routing problem with hard time window constraints (Constrained Vehicle Problem with TimeWindow, CVRPTW). The specific process of the demand point pre-generation algorithm is as follows:
[0077] like Figure 4 As shown, the initial demand point list is empty, and each element in the list stores the demand time, demand location and current grain truck direction of the current demand point. The waiting operation paths of all harvesters are allocated according to the heuristic algorithm. Through the simulation platform built by this patent, simulation is performed without introducing grain trucks. The env.step() function is called at each time step to return the current farmland state and the state of each harvester s = [s farm ,…,s harv,m ]. Note that since the specific response and transfer time of the grain truck are not considered in the demand point pre-generation algorithm, only the demand position and time of each demand point are obtained through simulation, so the state does not need to include the state of the grain truck. After obtaining the farm state and the state of each harvester, determine whether the grain bin of each harvester is full or reaches the specified preset storage volume. If not, continue to the next simulation. For harvesters with transfer demand, add a tuple consisting of the current time, harvester position and orientation to the demand point list. Note that the purpose of adding the harvester orientation is to retain the operation information, because when scheduling the grain truck, it is necessary to judge the operation status of the current operation row based on this information, and catch up with the harvester from the part that has not been operated to perform the transfer task. After recording, clear the current grain bin volume of the harvester to continue the operation task. At the next time step, continue to call the env.step() function and repeat the above cycle until all harvesters complete the operation. If the visualization function is turned on, call the env.render() function at each time step to visualize the environment.
[0078] like Figure 5 As shown in Figure 2, based on the above farmland and agricultural machinery parameters, the constructed simulation environment is visualized, and the pre-generated demand locations and times of each demand point are shown in Table 2.
[0079] Table 2 Demand points obtained by the demand point pre-generation algorithm
[0080]
[0081]
[0082] Through analysis, it can be seen that since most smart farms tend to purchase agricultural machinery of the same brand and type for large-scale operations to facilitate unified management, the parameters of each agricultural machinery are close to each other. According to the data in this table, when the harvesters start from the same position without load and the speed, storage capacity and other parameters of each harvester are close, the corresponding demand time of each demand point generated is also relatively close. For example, the three demand points numbered 0, 4, and 7 in the table are the demand time of the first demand point generated by each harvester, and the time interval is about 50 seconds. In this case, if the number of harvesters is significantly larger than that of grain trucks, it cannot be guaranteed that the demand of each harvester can be responded to in a timely manner. Taking this scenario as an example, harvester 0 generates a grain unloading demand at 517.5 seconds. Assuming that this demand can be responded to and processed in time, harvester 2 generates a grain unloading demand at 551.7 seconds, 34.2 seconds later. There is a transfer cost of 382.05 meters between the two demand points. Assuming that the speed of the grain truck is 6m / s, the time required is more than 63s, which cannot meet the timely response. In the case of large-scale farms, there are more harvesters. If the number of harvesters is significantly greater than the number of grain trucks, and the distance between the harvesters is far, it is difficult to get a timely response and meet the generated demand points. When a demand point cannot be met due to the lack of a suitable grain truck response, all subsequent work plans of the harvester will be affected, that is, the location and demand time of the demand point pre-generated by calculation before the start of the operation will change, and the change is directly related to the response time of the grain unloading demand. Since the subsequent demand points and demand times have changed, this problem is no longer a static scheduling problem solved by the demand point pre-generation algorithm before the start of the operation, and the solution obtained is no longer applicable.
[0083] Therefore, based on the simulation platform built, the embodiment of the present invention adds a dynamic grain truck response strategy based on artificial strategy (i.e., based on the nearest neighbor algorithm of the demand point) to solve the problems that the static strategy cannot consider the early or delayed unloading of grain and the transportation time. The specific process based on the nearest neighbor algorithm of the demand point is as follows:
[0084] First, the waiting operation paths of all harvesters are allocated according to the heuristic algorithm, and the current farmland status and the status of each harvester are obtained. The simulation platform constructed by the embodiment of the present invention is used to perform simulation deduction without introducing a grain transport vehicle. The env.step() function is called at each time step to return the current farmland status and the status of each harvester s=[s farm ,…,s harv,m ,s harv,1 ,…,s harv,m], the state includes the farmland state and the state of each harvester and grain transport vehicle; after obtaining the farm state and the state of each harvester, determine whether the grain storage of each harvester reaches the specified preset storage volume. If not, proceed to the next step of simulation deduction. If reached, there is a harvester with transfer demand, obtain and save the current demand generation time point, the current position of the harvester with transfer demand and the current direction of the harvester with transfer demand, traverse all grain transport vehicles, and according to the grain transport vehicle state s harv,i Select a grain transport vehicle that currently has no transfer task. If there are multiple grain transport vehicles available, select the grain transport vehicle that is closest to the current harvester; send the current demand generation time point, the current harvester position with transfer demand, and the current harvester orientation with transfer demand to the current best performing grain transport vehicle, use the underlying navigation strategy function and / or the upper scheduling decision function in the action space to generate specific actions, save the specific actions, and set the corresponding actions for the selected grain transport vehicle, that is, a j =[stop,back_to_depot,move_to_harv i ], the actions of all harvesters form the current joint action: a joint =[a i ,…,a n ], so that the grain truck goes to the current harvester to unload grain according to this specific action; the joint action is used as the input of the environment, under this action, the state transfer function is used to transfer the environment to a new state, and in the next time step, the env.step() function is continued to be called, and the above cycle is repeated until all harvesters complete the work, and the coordinated scheduling plan of harvesters and grain trucks is output.
[0085] Furthermore, a visualization function is provided in the multi-heterogeneous agricultural machinery collaborative operation simulation scenario. If the visualization function is turned on when solving the Markov decision process, the embodiment of the present invention synchronously calls the env.render() function when calling the env.step() function at each time step to visualize the working process of each harvester and each grain truck in each time step, and synchronously outputs the visualized collaborative scheduling plan of the harvester and the grain truck when outputting the collaborative scheduling plan of the harvester and the grain truck.
[0086] During the transfer process, the grain truck goes to the designated harvester through the bottom navigation strategy and / or the upper scheduling decision function. After completing the transfer task, it returns to the field and waits. When the grain truck's granary is full, it returns to the hangar to unload the grain according to the bottom navigation strategy. Among them, in order to ensure that the grain truck does not crush the unused plots during driving, the bottom navigation strategy of the grain truck to the harvester can also adopt the catch-up strategy, adding the old navigation point of the designated harvester to its own navigation point until the two meet. The return strategy of the grain truck adopts returning along the edge to minimize the crushing of the ridges.
[0087] Furthermore, the embodiments of the present invention can also design a scheduling strategy based on harvester capacity estimation and grain truck travel estimation in the upper-level scheduling decision function, and estimate in real time the distance from the grain truck to each harvester and the harvester full-warehouse time. When the time it takes for the grain truck to arrive at the corresponding harvester is equal to the expected full-warehouse time of the harvester, the grain truck will be dispatched.
[0088] like Figure 6 As shown, a simulation is performed on a simulation platform built according to an embodiment of the present invention to obtain a visualization of the dynamic scheduling process. The small squares represent harvesters, and their colors are consistent with the corresponding rows to be operated. The larger circles represent the positions of the grain trucks. Figure 6 (a)-(f) are screenshots of the simulation system every 300 seconds.
[0089] like Figure 7 As shown, in the embodiment of the present invention, different decision points of greedy strategies can be set in the upper scheduling decision function, and the total waiting time of each harvester, the total transfer distance of the grain truck, and the total time to complete the task are shown in Table 3. It can be seen that setting a lower critical value will cause the grain truck to go to the harvester to unload grain in advance, the waiting time of the harvester and the total task completion time will be reduced, but the total scheduling distance of the harvester will increase, and there is no good cooperation between the two. In addition, the current grain truck dispatching strategy uses the Euclidean distance to measure the nearest grain truck, but the actual distance that the grain truck needs to travel is not the Euclidean distance, so a more intelligent grain truck dispatching strategy is needed to dispatch the grain truck. The grain truck of the embodiment of the present invention is dispatched based on the greedy strategy. The navigation strategy when going to the harvester adopts the same navigation point as the harvester to ensure that the unharvested ridges are not crushed. The navigation strategy also has room for further improvement. It can pass through the operation line that has completed the operation to reduce the travel distance.
[0090] Table 3 Comparison of results of different greedy strategies
[0091]
[0092] In summary, according to the collaborative scheduling method based on the multi-heterogeneous agricultural machinery collaborative operation simulation scenario proposed in the embodiment of the present invention, the traditional CVRP modeling method is abandoned, and the multi-heterogeneous agricultural machinery collaborative operation is abstracted into a Markov decision process, and a multi-agent reinforcement learning environment is established using the gym library. By designing and encapsulating class functions such as farmland, harvester, and grain truck, navigation points are pre-allocated to the farmland class to ensure farmland driving constraints; at the same time, in order to solve the problem of static scheduling strategy failure, the static scheduling strategy is transformed into an environment-based dynamic strategy. Strategies such as strategies based on artificial design, reinforcement learning, and multi-agent reinforcement learning can be designed, and the Markov decision process is solved using the strategy to obtain the optimal collaborative scheduling solution for the main task machine and the collaborative machine.
[0093] Next, a collaborative scheduling device based on a multi-heterogeneous agricultural machinery collaborative operation simulation scenario proposed according to an embodiment of the present invention will be described with reference to the accompanying drawings.
[0094] Figure 8 It is a block diagram of a collaborative scheduling device based on a multi-heterogeneous agricultural machinery collaborative operation simulation scenario according to an embodiment of the present invention.
[0095] like Figure 8 As shown, the collaborative scheduling device 80 based on the multi-heterogeneous agricultural machinery collaborative operation simulation scenario includes: a conversion module 801, a joining module 802 and a solution module 803.
[0096] Among them, the conversion module 801 is used to convert the target multi-heterogeneous agricultural machinery collaborative operation scenario into a Markov decision process. The construction module 802 is used to build the target multi-heterogeneous agricultural machinery collaborative operation scenario into a multi-heterogeneous agricultural machinery collaborative operation simulation scenario based on the gym framework. The solution module 803 is used to solve the Markov decision process based on the multi-heterogeneous agricultural machinery collaborative operation simulation scenario using preset static scheduling strategies and dynamic scheduling strategies to generate a collaborative scheduling plan for the main task machine and the collaborative machine.
[0097] It should be noted that the aforementioned explanation of the embodiment of the collaborative scheduling method based on the multi-heterogeneous agricultural machinery collaborative operation simulation scenario is also applicable to the collaborative scheduling device based on the multi-heterogeneous agricultural machinery collaborative operation simulation scenario of this embodiment, and will not be repeated here.
[0098] According to the collaborative scheduling device based on the multi-heterogeneous agricultural machinery collaborative operation simulation scenario proposed in the embodiment of the present invention, in view of the complex farmland operation conditions, many constraints, and the failure of the static scheduling strategy when the demand points are not met, the traditional CVRP modeling method is abandoned, and the collaborative operation of multi-heterogeneous agricultural machinery is abstracted as a Markov decision process. The gym library is used to establish a multi-agent reinforcement learning environment. By designing and encapsulating class functions such as farmland, harvester, and grain transporter, navigation points are pre-allocated to the farmland class to ensure farmland driving constraints. At the same time, in view of the failure problem of the static scheduling strategy, the static scheduling strategy is transformed into an environment-based dynamic strategy. Strategies such as strategies based on artificial design, reinforcement learning, and multi-agent reinforcement learning can be designed, and the Markov decision process is solved using the strategy to obtain the optimal collaborative scheduling solution for the main task machine and the collaborative machine.
[0099] Fig. 9 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device may include:
[0100] A memory 901 , a processor 902 , and a computer program stored in the memory 901 and executable on the processor 902 .
[0101] When the processor 902 executes the program, the collaborative scheduling method based on the multi-heterogeneous agricultural machinery collaborative operation simulation scenario provided in the above embodiment is implemented.
[0102] Furthermore, the electronic device further comprises:
[0103] The communication interface 903 is used for communication between the memory 901 and the processor 902 .
[0104] The memory 901 is used to store computer programs that can be executed on the processor 902 .
[0105] The memory 901 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.
[0106] If the memory 901, the processor 902 and the communication interface 903 are implemented independently, the communication interface 903, the memory 901 and the processor 902 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0107] Optionally, in a specific implementation, if the memory 901, the processor 902 and the communication interface 903 are integrated on a chip, the memory 901, the processor 902 and the communication interface 903 can communicate with each other through an internal interface.
[0108] The processor 902 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention.
[0109] An embodiment of the present invention also provides a computer program product, which, when executed by a processor, implements the above-mentioned collaborative scheduling method based on a multi-heterogeneous agricultural machinery collaborative operation simulation scenario.
[0110] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the collaborative scheduling method based on a multi-heterogeneous agricultural machinery collaborative operation simulation scenario as described above.
[0111] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0112] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0113] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.
[0114] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary and then storing it in a computer memory.
[0115] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0116] A person skilled in the art will appreciate that all or part of the steps in the above-mentioned embodiment can be completed by instructing the relevant hardware through a program.
[0117] The program can be stored in a computer-readable storage medium, and when the program is executed, it includes one or a combination of the steps of the method embodiment.
[0118] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
[0119] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.
Claims
1. A collaborative scheduling method based on a multi-heterogeneous agricultural machinery collaborative operation simulation scenario, characterized in that: The following steps are involved: The target multi-heterogeneous agricultural machinery collaborative operation scenario is transformed into a Markov decision process; Based on the gym framework, the target multi-heterogeneous agricultural machinery collaborative operation scenario is constructed as a multi-heterogeneous agricultural machinery collaborative operation simulation scenario; Based on the multi-heterogeneous agricultural machinery collaborative operation simulation scenario, the Markov decision process is solved using preset static scheduling strategies and dynamic scheduling strategies to generate a collaborative scheduling solution for the main task machine and the collaborative machine.
2. The collaborative scheduling method based on the multi-heterogeneous agricultural machinery collaborative operation simulation scenario according to claim 1 is characterized in that: The Markov decision process includes a set of state variables, an action space and a state transfer function, wherein: The state variable set includes the farmland state, the state of each main task machine and the state of each collaborative machine; The action space uses the bottom layer navigation strategy function and / or the upper layer scheduling decision function to determine the action at the next moment; The state transfer function causes the current state of the state variable set to undergo a state transfer according to the next moment action determined by the action space.
3. The collaborative scheduling method based on the multi-heterogeneous agricultural machinery collaborative operation simulation scenario according to claim 1 is characterized in that: Based on the multi-heterogeneous agricultural machinery collaborative operation simulation scenario, the Markov decision process is solved by using a preset static scheduling strategy and a dynamic scheduling strategy to generate a collaborative scheduling scheme for the main task machine and the collaborative machine, including: Based on the multi-heterogeneous agricultural machinery collaborative operation simulation scenario, a waiting operation path is assigned to each main task machine, and each main task machine is made to perform a harvesting task according to its corresponding waiting operation path; Without introducing cooperative machines, the Markov decision process is simulated and deduced, and the env.step() function is called at the first time step to obtain the current state of the farmland and the state of each main task machine; Determining whether each main task machine reaches a preset collaboration threshold according to the current farmland state and the state of each main task machine; If the preset collaboration threshold is not reached, the simulation is continued until the preset collaboration threshold is reached, the current collaboration demand is generated, and the time point of the current demand generation, the current main task machine position and the current main task machine orientation are obtained and saved; According to the current cooperation demand, the current state of each cooperation machine is traversed, and the cooperation machine that best performs the transfer task is selected; Send the current demand generation time point, the current main task machine position and the current main task machine orientation to the cooperative machine that currently best performs the transfer task, and set and save the execution action of the cooperative machine that currently best performs the transfer task; The env.step() function is called at each time step, and the judgment, traversal and setting processes are iteratively executed until each main task machine completes the job, and the collaborative scheduling plan of the main task machine and the cooperative machine is output.
4. The collaborative scheduling method based on the multi-heterogeneous agricultural machinery collaborative operation simulation scenario according to claim 3 is characterized in that: The step of traversing the current state of each collaborative machine according to the current collaborative demand and selecting the collaborative machine that best performs the transfer task at present includes: Traversing the current state of each collaborative machine according to the current collaboration requirements; If only one collaborative machine is in an idle state, the idle collaborative machine is used as the collaborative machine that best performs the transfer task at present; If there are multiple collaborative machines in an idle state, the collaborative machine closest to the main task machine that is currently full or has reached a preset storage volume is selected as the collaborative machine that is currently the best to perform the transfer task.
5. The collaborative scheduling method based on the multi-heterogeneous agricultural machinery collaborative operation simulation scenario according to claim 3 is characterized in that: Also includes: Based on the multi-heterogeneous agricultural machinery collaborative operation simulation scenario, the visualization function is turned on. When the env.step() function is called at each time step, the env.render() function is called synchronously to visualize the working process of each main task machine and each collaborative machine in each time step. When outputting the collaborative scheduling plan of the main task machine and the collaborative machine, the visualized collaborative scheduling plan of the main task machine and the collaborative machine is output synchronously.
6. A collaborative scheduling device based on a multi-heterogeneous agricultural machinery collaborative operation simulation scenario, characterized in that: include: A conversion module, used to convert the target multi-heterogeneous agricultural machinery collaborative operation scenario into a Markov decision process; A building module, used to build the target multi-heterogeneous agricultural machinery collaborative operation scenario into a multi-heterogeneous agricultural machinery collaborative operation simulation scenario based on the gym framework; A solution module is used to solve the Markov decision process based on the multi-heterogeneous agricultural machinery collaborative operation simulation scenario by using preset static scheduling strategies and dynamic scheduling strategies to generate a collaborative scheduling plan for the main task machine and the collaborative machine.
7. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the collaborative scheduling method based on a multi-heterogeneous agricultural machinery collaborative operation simulation scenario as described in any one of claims 1 to 5.
8. A computer program product, characterized in that When the computer program / instruction is executed by a processor, the collaborative scheduling method based on a multi-heterogeneous agricultural machinery collaborative operation simulation scenario as described in any one of claims 1 to 5 is implemented.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement a collaborative scheduling method based on a multi-heterogeneous agricultural machinery collaborative operation simulation scenario as described in any one of claims 1 to 5.
Citation Information
Cited By
Receiving and transporting cooperative scheduling method based on heuristic algorithm
CN119599228A
A pickup and delivery collaborative scheduling method based on heuristic algorithm
CN119599228B
Rice transplanter operation method, device and equipment based on Internet of Things and medium
CN121942395A
Equipment cluster operation simulation planning method based on multi-agent collaboration
CN122509035A
Method for simulating and planning of device cluster job based on multi-agent cooperation
CN122509035B
