Agricultural machinery scheduling method and device based on production planning joint optimization

Abstract

The invention provides an agricultural machinery scheduling method and device based on production planning joint optimization. The agricultural machinery scheduling method comprises the following steps: according to sowing time, plant protection time, harvesting time, crop water content and related operation cost factors such as oil consumption, manpower, seeds and pesticides, performing agricultural machinery scheduling; according to the method, a mathematical model which aims at full-period agricultural machinery sowing, plant protection and harvesting plans of corn crops and aims at maximizing enterprise profit is proposed, then the mathematical model is converted into a reinforcement learning model, the model is solved through a deep reinforcement learning method, and an agricultural machinery scheduling strategy is given. Specifically, the method comprises the following steps: constructing a full-period agricultural machinery resource scheduling model taking maximization of enterprise profit as a target based on farm actual data and external factor influence; and an approximately optimal agricultural machinery scheduling strategy is obtained based on the depth deterministic strategy gradient model.

Description

Technical Field

[0001] The present invention belongs to the technical field of agricultural information technology, and particularly relates to a method and a device for agricultural machinery scheduling based on joint optimization of production planning. Background Art

[0002] As a major agricultural country, China has 1.8 billion mu of arable land. Corn, as one of China's three major staple foods, is mainly distributed in the north and mainly produced in the field. According to statistics, the corn planting area in 2022 is about 600 million mu, and China's corn output accounts for more than 40% of the total grain output. However, agricultural production is a complex process. Global climate change has led to the high frequency of extreme weather, breaking the traditional agricultural production rules. Therefore, operating in accordance with traditional methods will inevitably reduce farm output and reduce corporate profits. In addition, affected by the global market economy, the cost input of labor, production materials, agricultural machinery, etc. has increased year by year, further compressing corporate profits. Therefore, how to propose the optimal agricultural production plan and dynamic scheduling plan of agricultural machinery resources based on actual data is the key to increasing output and reducing costs.

[0003] Improving agricultural production efficiency and promoting agricultural economic development by optimizing the allocation of agricultural machinery resources has always attracted widespread attention in the industry. A DDS decision system was developed based on regression analysis to improve harvesting income. However, the system only schedules the harvesting process and cannot support agricultural machinery scheduling processes such as sowing and plant protection. Linear and multivariate linear regression analysis was used to study the power, weight, working width, number of rows and price of the seeder, and parameter equations related to the seeder were established to evaluate and predict the performance and cost of the seeder, providing decision support for agricultural machinery selection and guidance. By considering parameters such as labor time, agricultural machinery type, and real-time harvesting operations, linear and nonlinear programming models were constructed to achieve agricultural machinery scheduling. However, the above model does not consider the impact of factors such as labor cost, maintenance cost, operating area, and sowing cycle on agricultural machinery selection, so it has limitations. By considering factors such as planting area, labor costs, agricultural machinery costs, and crop moisture content, a corn crop management model is established with the goal of maximizing farmers' profits, or a system model that integrates agricultural machinery costs and energy requirements during sowing, plant protection, and harvesting is established to determine the most economical agricultural machinery combination operation. In other words, most existing agricultural machinery scheduling problems are modeled through operations research as MIP problems, linear programming problems, dynamic programming problems, etc., and solved through branch and bound methods or heuristic algorithms.

[0004] Although the above studies have optimized and provided guidance on agricultural machinery operation scheduling methods based on actual operation scenarios, the assumptions about the scenarios in these studies are relatively simple and the factors considered are not comprehensive enough.

[0005] In addition, the optimization process given in the above research does not cover the entire cycle. At the same time, when solving the MINP, linear programming and other problems given, it usually takes a lot of time and multiple iterations to obtain the optimization solution, or it is necessary to use expensive commercial solvers to solve it. Summary of the invention

[0006] The purpose of the present invention is to solve the problem of agricultural machinery resource scheduling and production operation planning for the whole cycle of corn in my country. At present, there is no agricultural machinery resource scheduling model for the whole cycle, and the existing scheduling models are all based on operations research methods, which are either difficult to solve or the models are too simplified and do not consider various influencing factors. The present invention proposes a full-cycle agricultural machinery resource scheduling method based on deep reinforcement learning.

[0007] In view of the shortcomings of existing technologies, such as Figure 6 As shown, the present invention provides an agricultural machinery scheduling method based on production planning joint optimization, which includes:

[0008] Step 1: Construct an agricultural machinery configuration model N according to the total farm area, agricultural machinery width, number of agricultural machinery, operation rate, operation time, and operation efficiency in the harvesting operation;

[0009] Step 2: Based on the yield of standard sowing date, early sowing area, standard sowing area, and the yield influencing factors of early sowing and standard sowing, construct the actual yield model Y under the influence of sowing date ac ;

[0010] Step 3: Based on the fuel price, the number of days for sowing operations, the number of agricultural machinery used in sowing operations, and the fuel consumption of agricultural machinery, a fuel consumption cost model C for sowing agricultural machinery operations is constructed. s ;

[0011] Step 4: Construct a labor cost model C based on the operation area, sowing amount per mu, number of sowing machines, labor cost per unit time, and early sowing operation time. c ;

[0012] Step 5: Based on the relationship between weed growth time and yield, construct the plant protection operation and yield relationship model Y p ;

[0013] Step 6: Based on the fuel consumption, fuel price and plant protection time of agricultural machinery, construct the agricultural machinery input cost model C for the plant protection operation process p ;

[0014] Step 7: Based on the plant protection time, fertilizer weight per unit area, fertilizer price, and labor cost per unit time, construct the fertilizer and labor cost model C of the plant protection operation process x ;

[0015] Step 8: Based on the harvest yield and crop moisture content, construct the moisture content and yield relationship model Yre ;

[0016] Step 9: Construct the fuel consumption cost model C of harvesting agricultural machinery operation based on fuel price, harvesting operation days, number of agricultural machinery used in harvesting operation, and agricultural machinery fuel consumption. h ;

[0017] Step 10: Construct the harvest period labor cost model C based on the labor cost per unit time and the harvesting agricultural machinery operation time z ;

[0018] Step 11: Based on the agricultural machinery configuration model N and the actual yield model Y ac 、Fuel consumption cost model C s , Labor cost model C c , Plant protection operation and yield relationship model Y p , Agricultural machinery input cost model C in the process of plant protection operation p , Fertilizer and labor cost model for plant protection operation process C x , moisture content and yield relationship model Y re , Fuel consumption cost model of harvesting agricultural machinery operation C h , Harvest period labor cost model C z , build a mathematical model for maximizing profits throughout the entire cycle;

[0019] Step 12: Based on the upper limit of the daily operation time of agricultural machinery, the upper limit of the total number of agricultural machinery, the upper limit of the total farm area, and the range of moisture content of crops, the restriction constraints of the mathematical model for maximizing profit in the whole cycle are constructed. Under the restriction constraints, the agricultural machinery scheduling strategy when the output of the mathematical model for maximizing profit in the whole cycle is solved to maximize the profit is obtained; the sowing, harvesting and plant protection operations of the crop are performed using the agricultural machinery scheduling strategy.

[0020] The agricultural machinery scheduling method based on joint optimization of production planning, wherein the agricultural machinery configuration model N is:

[0021]

[0022] Among them, N represents the number of agricultural machinery, A is the total area of ​​the farm, w represents the width of the agricultural machinery, v represents the operation rate, t represents the operation time, and e represents the operation efficiency of the agricultural machinery;

[0023] The actual yield model Y under the influence of the sowing date ac for:

[0024]

[0025] Y ac represents the actual yield under the influence of sowing date, Y ha is the yield of the standard sowing period, A ea is the early sowing area, A stis the standard sowing area, is the yield influencing factor under early sowing time j, is the yield influencing factor under standard sowing time j;

[0026]

[0027]

[0028]

[0029] in, It represents the number of sowing operation hours of the agricultural machinery with horsepower segment i on the jth day in the early sowing stage. It represents the number of sowing operation hours of the agricultural machinery with horsepower segment i on the jth day during the normal sowing stage. To delay the sowing time;

[0030] Fuel consumption cost model of seeding agricultural machinery operation C s for:

[0031]

[0032] L represents the number of types of agricultural machinery horsepower segments, and J represents the number of operating days; is the operation time of the agricultural machinery with horsepower segment i on the jth day in the early sowing stage, is the number of agricultural machinery with horsepower segment i in the early sowing period operating on day j, is the normal sowing operation time, f i s represents the fuel consumption of the seed drill with horsepower segment i, θ d is the unit diesel price;

[0033] The labor cost model C c :

[0034]

[0035] Where A represents the total farm area, θ seed is the sowing amount per mu, is the number of early sowing machines, θ p is the daily labor cost, It is the time for early sowing. The number of agricultural machinery used in the normal sowing period, Normal sowing operation time;

[0036] The relationship model Y between plant protection operation and yield p :

[0037] Y p =Y ha A c xj

[0038]

[0039] where Y p is the yield after weed management, x j is the percentage of weed impact on yield on day j, where j is the jth day in the plant protection cycle;

[0040] The agricultural machinery input cost model C of the plant protection operation process p :

[0041]

[0042] Among them C p Represents the fuel cost of agricultural machinery in the plant protection process, It is expressed as the operation time of the agricultural machinery with horsepower segment i on the jth day, θ d is the diesel market price, represents the number of agricultural machinery operating in horsepower segment i on day j;

[0043] The fertilizer and labor cost model C of the plant protection operation process x :

[0044]

[0045] Among them C x represents the labor cost of the plant protection process, θ pe Indicates the amount of pesticide required per unit area, θ cf Indicates the amount of fertilizer required per unit area. is the operating time of the agricultural machinery with horsepower range i on the jth day during the plant protection operation, is the number of agricultural machinery inputs in the operation process of horsepower segment i on day j;

[0046] The water volume and yield relationship model Y re :

[0047]

[0048] Among them, Y re To obtain the actual yield, is the loss rate of agricultural machinery harvesting process, M q is the crop water content, θ ma is the harvest price corresponding to the moisture content; the relationship between crop moisture content and crop price is:

[0049]

[0050] where p e Indicates the crop price when the moisture content is less than or equal to 14%.

[0051] The fuel consumption cost model C of the harvesting agricultural machinery operation h :

[0052]

[0053] Among them C h represents the fuel cost of agricultural machinery during the harvesting process, represents the operation time of agricultural machinery with horsepower segment i in the harvesting process on day j, θ d is the diesel market price, It represents the number of harvesting agricultural machinery operating on day j in horsepower segment i;

[0054] The labor cost model C for the harvest period z :

[0055]

[0056] Among them C z represents the labor cost of the harvesting process, It is expressed as the operation time of the harvesting agricultural machinery with horsepower segment i in j days, θ p For each person's daily salary, It represents the number of harvesting agricultural machinery operating on day j in horsepower segment i;

[0057] The mathematical model for maximizing profits throughout the entire cycle is:

[0058]

[0059] The restrictions are:

[0060]

[0061]

[0062]

[0063]

[0064]

[0065]

[0066]

[0067]

[0068]

[0069]

[0070]

[0071]

[0072]

[0073]

[0074]

[0075]

[0076]

[0077] M s ≤M q ≤M (19)

[0078] The above constraints (2) to (4) respectively represent the operating time range of agricultural machinery with horsepower range i on the jth day; constraints (5) and (7) indicate that the number of agricultural machinery in the early sowing, standard sowing and late sowing periods is less than or equal to the number of sowing agricultural machinery already on the farm; constraint (8) indicates that the actual number of agricultural machinery operations during the plant protection process is less than or equal to the total number of plant protection machines already on the farm; constraint (9) indicates that the actual number of agricultural machinery operations during the harvest process is less than or equal to the total number of harvesters already on the farm; constraints (10) to (12) indicate that the number of agricultural machinery is limited to an integer, constraints (13) to (17) respectively indicate that the operating area during the harvest, plant protection and sowing process is less than or equal to the total area of ​​the farm, and constraint (18) indicates that the sum of the operating areas of early sowing, standard sowing and late sowing is less than or equal to the total area of ​​the farm; constraint (19) indicates the range of moisture content of crops.

[0079] The agricultural machinery scheduling method based on joint optimization of production planning is described, wherein the mathematical model of maximizing profit in the whole cycle is solved based on reinforcement learning, and the area X required for operation and the number of days H for remaining operation are used as the state space S. Among them, f represents plant protection, h represents harvesting, and the number of various agricultural machinery, water content m and operation time are used as the action space A. Subscript t represents the tth day; for a given action a t , the reward R is calculated based on the mathematical model of maximizing profit in the whole cycle t , calculate the first operation area according to the agricultural machinery configuration model N, reduce the corresponding remaining area in the state space S by the first operation area, and reduce the corresponding remaining days by 1;

[0080] The specific process of solving the mathematical model of maximizing profit in the whole cycle based on reinforcement learning includes:

[0081] Step 121, use the initial parameter θμ Initialize the training policy network, which is used to output the action a in a given state t ;Copy the parameters of the policy network to the target policy network, the initial parameters of the target policy network are θ' μ =θ μ ; Using the initial parameters θ Q Initialize the training value network, which is trained according to the input action a t The state st gives the current state value estimate Q; the parameters of the training value network are copied to the target Q value network, and the initial parameters of the target Q value network are θ' Q =θ Q ; Create an experience replay pool for training neural networks;

[0082] Step 122: Randomly initialize state s0 from the state space S; use the training strategy network to select action a t =μ(s t |θ μ )+N t , where N t For noise; action a t Enter the mathematical model of maximizing profit in the whole cycle and get the reward R t and the next state s t+1 , the quaternion (s t ,a t ,R t ,s t+1 ) is stored in the experience replay pool; the current state is updated to s t+1 ;

[0083] Step 123: Randomly sample experience (s) from the experience replay pool i ,a i ,R i ,s i+1 );

[0084] Step 124: Calculate the target Q value using the target policy network and the target Q value network:

[0085] y i =r i +γQ′(s t+1 ,μ′(s t+1 |θ′ μ )|θ′ Q )

[0086] Where γ is the discount factor;

[0087] Update the parameters θ of the training value network by minimizing the loss function L Q :

[0088]

[0089] Step 125: Calculate the policy gradient using the current training value network To use the gradient ascent method to update the parameters θ of the training strategy network μ ;

[0090]

[0091] Step 126: Copy the parameters of the training strategy network and the training value network to the target strategy network and the target Q value network respectively:

[0092] θ′ Q =τθ Q +(1-τ)θ′ Q

[0093] θ′ μ =τθ μ +(1-τ)θ′ μ

[0094] Step 127, loop iteratively execute steps 122 to 126 until the reward tends to stabilize or converge over time, or reaches a predetermined number of training steps, then the strategy is considered to converge, and the current training strategy network π is saved. online As a candidate network. Execute step 121 again to try different initial parameters to obtain multiple candidate networks, obtain the operation strategy of each candidate network according to the full-cycle maximization profit mathematical model, action space A and state space S, and select the operation strategy that maximizes the profit output by the full-cycle maximization profit mathematical model as the final operation strategy.

[0095] The agricultural machinery scheduling method based on joint optimization of production planning, wherein the crop is corn.

[0096] like Figure 7 As shown, the present invention also proposes an agricultural machinery scheduling device B based on joint optimization of production planning, which includes:

[0097] Module 1: Construct the agricultural machinery configuration model N according to the total farm area, agricultural machinery width, number of agricultural machinery, operation rate, operation time and operation efficiency in the harvesting operation;

[0098] Module 2: Based on the yield of standard sowing period, early sowing area, standard sowing area, and the yield influencing factors of early sowing and standard sowing, the actual yield model Y under the influence of sowing period is constructed. ac ;

[0099] Module 3: Construct the fuel consumption cost model C of the sowing agricultural machinery operation based on the fuel price, the number of sowing operation days, the number of agricultural machinery used in the sowing operation, and the fuel consumption of agricultural machinerys ;

[0100] Module 4: Construct the labor cost model C based on the operation area, sowing amount per mu, number of sowing machines, labor cost per unit time, and early sowing operation time. c ;

[0101] Module 5: Construct a plant protection operation and yield relationship model Y based on the relationship between weed growth time and yield p ;

[0102] Module 6: Construct the agricultural machinery input cost model C in the process of plant protection operation based on the fuel consumption, fuel price and plant protection time of agricultural machinery for plant protection p ;

[0103] Module 7: Construct the fertilizer and labor cost model of the plant protection operation process based on the plant protection time, fertilizer weight per unit area, fertilizer price, and labor cost per unit time. x ;

[0104] Module 8: Construct the relationship model between moisture content and yield based on harvest yield and crop moisture content re ;

[0105] Module 9: Construct the fuel consumption cost model C of harvesting agricultural machinery operation based on fuel price, harvesting operation days, number of agricultural machinery used in harvesting operation, and agricultural machinery fuel consumption h ;

[0106] Module 10: Construct the labor cost model C for the harvest period based on the labor cost per unit time and the operation time of the harvesting agricultural machinery z ;

[0107] Module 11: Based on the agricultural machinery configuration model N and the actual yield model Y ac 、Fuel consumption cost model C s , Labor cost model C c , Plant protection operation and yield relationship model Y p , Agricultural machinery input cost model C in the process of plant protection operation p , Fertilizer and labor cost model for plant protection operation process C x , moisture content and yield relationship model Y re , Fuel consumption cost model of harvesting agricultural machinery operation C h , Harvest period labor cost model C z , build a mathematical model for maximizing profits throughout the entire cycle;

[0108] Module 12: Based on the upper limit of the daily operation time of agricultural machinery, the upper limit of the total number of agricultural machinery, the upper limit of the total farm area, and the range of moisture content of crops, the constraints of the mathematical model for maximizing profits throughout the entire cycle are constructed. Under the constraints, the agricultural machinery scheduling strategy is solved when the output of the mathematical model for maximizing profits throughout the entire cycle is maximized; the sowing, harvesting and plant protection operations of the crop are performed using the agricultural machinery scheduling strategy.

[0109] The agricultural machinery scheduling device based on production planning joint optimization, wherein the agricultural machinery configuration model N is:

[0110]

[0111] Among them, N represents the number of agricultural machinery, A is the total area of ​​the farm, w represents the width of the agricultural machinery, v represents the operation rate, t represents the operation time, and e represents the operation efficiency of the agricultural machinery;

[0112] The actual yield model Y under the influence of the sowing date ac for:

[0113]

[0114] Y ac represents the actual yield under the influence of sowing date, Y ha is the yield of the standard sowing period, A ea is the early sowing area, A st is the standard sowing area, is the yield influencing factor under early sowing time j, is the yield influencing factor under standard sowing time j;

[0115]

[0116]

[0117]

[0118] in, It represents the number of sowing operation hours of the agricultural machinery with horsepower segment i on the jth day in the early sowing stage. It represents the number of sowing operation hours of the agricultural machinery with horsepower segment i on the jth day during the normal sowing stage. To delay the sowing time;

[0119] Fuel consumption cost model of seeding agricultural machinery operation C s for:

[0120]

[0121] L represents the number of types of agricultural machinery horsepower segments, and J represents the number of operating days; is the operation time of the agricultural machinery with horsepower segment i on the jth day in the early sowing stage, is the number of agricultural machinery with horsepower segment i in the early sowing period operating on day j, is the normal sowing operation time, f i s represents the fuel consumption of the seed drill with horsepower segment i, θ d is the unit diesel price;

[0122] The labor cost model C c :

[0123]

[0124] Where A represents the total farm area, θ seed is the sowing amount per mu, is the number of early sowing machines, θ p is the daily labor cost, It is the time for early sowing. The number of agricultural machinery used in the normal sowing period, Normal sowing operation time;

[0125] The relationship model Y between plant protection operation and yield p :

[0126] Y p =Y ha A c x j

[0127]

[0128] where Y p is the yield after weed management, x j is the percentage of weed impact on yield on day j, where j is the jth day in the plant protection cycle;

[0129] The agricultural machinery input cost model C of the plant protection operation process p :

[0130]

[0131] Among them C p Represents the fuel cost of agricultural machinery in the plant protection process, It is expressed as the operation time of the agricultural machinery with horsepower segment i on the jth day, θ d is the diesel market price, represents the number of agricultural machinery operating in horsepower segment i on day j;

[0132] The fertilizer and labor cost model C of the plant protection operation process x :

[0133]

[0134] Among them C x represents the labor cost of the plant protection process, θ pe Indicates the amount of pesticide required per unit area, θ cf Indicates the amount of fertilizer required per unit area. is the operating time of the agricultural machinery with horsepower range i on the jth day during the plant protection operation, is the number of agricultural machinery inputs in the operation process of horsepower segment i on day j;

[0135] The water volume and yield relationship model Y re :

[0136]

[0137] Among them, Y re To obtain the actual yield, is the loss rate of agricultural machinery harvesting process, M q is the crop water content, θ ma is the harvest price corresponding to the moisture content; the relationship between crop moisture content and crop price is:

[0138]

[0139] where p e Indicates the crop price when the moisture content is less than or equal to 14%.

[0140] The fuel consumption cost model C of the harvesting agricultural machinery operation h :

[0141]

[0142] Among them C h represents the fuel cost of agricultural machinery during the harvesting process, represents the operation time of agricultural machinery with horsepower segment i in the harvesting process on day j, θ d is the diesel market price, It represents the number of harvesting agricultural machinery operating on day j in horsepower segment i;

[0143] The labor cost model C for the harvest period z :

[0144]

[0145] Among them C z represents the labor cost of the harvesting process, It is expressed as the operation time of the harvesting agricultural machinery with horsepower segment i in j days, θ p For each person's daily salary, It represents the number of harvesting agricultural machinery operating on day j in horsepower segment i;

[0146] The mathematical model for maximizing profits throughout the entire cycle is:

[0147]

[0148] The restrictions are:

[0149]

[0150]

[0151]

[0152]

[0153]

[0154]

[0155]

[0156]

[0157]

[0158]

[0159]

[0160]

[0161]

[0162]

[0163]

[0164]

[0165]

[0166] M s ≤M q ≤M (19)

[0167] The above constraints (2) to (4) respectively represent the operating time range of agricultural machinery with horsepower range i on the jth day; constraints (5) and (7) indicate that the number of agricultural machinery in the early sowing, standard sowing and late sowing periods is less than or equal to the number of sowing agricultural machinery already on the farm; constraint (8) indicates that the actual number of agricultural machinery operations during the plant protection process is less than or equal to the total number of plant protection machines already on the farm; constraint (9) indicates that the actual number of agricultural machinery operations during the harvest process is less than or equal to the total number of harvesters already on the farm; constraints (10) to (12) indicate that the number of agricultural machinery is limited to an integer, constraints (13) to (17) respectively indicate that the operating area during the harvest, plant protection and sowing process is less than or equal to the total area of ​​the farm, and constraint (18) indicates that the sum of the operating areas of early sowing, standard sowing and late sowing is less than or equal to the total area of ​​the farm; constraint (19) indicates the range of moisture content of crops.

[0168] The agricultural machinery scheduling device based on production planning joint optimization solves the full-cycle profit maximization mathematical model based on reinforcement learning, and takes the required operation area X and the remaining operation days H as the state space S. Among them, f represents plant protection, h represents harvesting, and the number of various agricultural machinery, water content m and operation time are used as the action space A. Subscript t represents the tth day; for a given action a t , the reward R is calculated based on the mathematical model of maximizing profit in the whole cycle t , calculate the first operation area according to the agricultural machinery configuration model N, reduce the corresponding remaining area in the state space S by the first operation area, and reduce the corresponding remaining days by 1;

[0169] The specific process of solving the mathematical model of maximizing profit in the whole cycle based on reinforcement learning includes:

[0170] Module 121, using initial parameters θ μ Initialize the training policy network, which is used to output the action a in a given state t ;Copy the parameters of the policy network to the target policy network, the initial parameters of the target policy network are θ' μ =θ μ ; Using the initial parameters θ Q Initialize the training value network, which is trained according to the input action a t and status t Given the current state value estimate Q; copy the parameters of the training value network to the target Q value network, the initial parameters of the target Q value network are θ' Q =θ Q ; Create an experience replay pool for training neural networks;

[0171] Module 122: Randomly initialize state s0 from state space S; use the trained policy network to select action a t =μ(s t |θ μ )+N t , where N t For noise; action a t Enter the mathematical model of maximizing profit in the whole cycle and get the reward R t and the next state s t+1 , the quaternion (s t ,a t ,R t ,s t+1 ) is stored in the experience replay pool; the current state is updated to s t+1 ;

[0172] Module 123, randomly sample experience (s) from the experience replay pool i ,a i ,R i ,s i+1 );

[0173] Module 124, using the target policy network and the target Q value network to calculate the target Q value:

[0174] y i =r i +γQ′(s t+1 ,μ′(s t+1 |θ′ μ )|θ′ Q )

[0175] Where γ is the discount factor;

[0176] Update the parameters θ of the training value network by minimizing the loss function L Q :

[0177]

[0178] Module 125: Calculate the policy gradient using the current training value network To use the gradient ascent method to update the parameters θ of the training strategy network μ ;

[0179]

[0180] Module 126, copying the parameters of the training strategy network and the training value network to the target strategy network and the target Q value network respectively:

[0181] θ′ Q =τθ Q +(1-τ)θ′ Q

[0182] θ′ μ =τθ μ +(1-τ)θ′ μ

[0183] Module 127, loop iteratively executes modules 122 to 126 until the reward tends to stabilize, converge over time, or reaches a predetermined number of training modules, then the strategy is considered to converge, and the current training strategy network π is saved online As a candidate network. Module 121 is executed again to try different initial parameters to obtain multiple candidate networks, and the operation strategy of each candidate network is obtained according to the full-cycle maximization profit mathematical model, the action space A and the state space S, and the operation strategy that makes the full-cycle maximization profit mathematical model output the maximum profit is selected as the final operation strategy.

[0184] The agricultural machinery scheduling device based on production planning joint optimization, wherein the crop is corn.

[0185] The present invention also proposes an electronic device, which includes the agricultural machinery scheduling device based on production planning joint optimization. The electronic device may be connected to an information display device, and the information display device is used to display the final operation strategy based on display parameters and attributes set by the user or through an artificial intelligence model.

[0186] The present invention also proposes a storage medium for storing a computer program for executing the agricultural machinery scheduling method based on joint optimization of production planning.

[0187] It can be seen from the above scheme that the advantages of the present invention are: compared with the existing method, the proposed method reduces the running time by about 60% in terms of time complexity; and improves the total profit by 11%. BRIEF DESCRIPTION OF THE DRAWINGS

[0188] Figure 1 This is a schematic diagram of the DDPG algorithm;

[0189] Figure 2 This is the effect diagram of ddpg vs gurobi;

[0190] Figure 3 This is a comparison chart between DDPG and branch and bound method;

[0191] Figure 4 This is a time comparison chart of deep reinforcement learning and branch-and-bound method under different area conditions;

[0192] Figure 5 This is the relationship diagram between the total number of agricultural machinery and profits;

[0193] Figure 6 is a flow chart of the method of the present invention;

[0194] Figure 7 It is a module diagram of the device of the present invention;

[0195] Figure 8 This is a schematic diagram of the structure of a first electronic device of the present invention;

[0196] Fig. 9 This is a schematic diagram of the application environment structure of the first electronic device of the present invention;

[0197] Fig.10 It is a schematic diagram of the structure of a second electronic device of the present invention.

[0198] Reference numerals:

[0199] A-First electronic device;

[0200] B-Agricultural machinery scheduling device based on joint optimization of production planning;

[0201] C-data acquisition equipment;

[0202] D-information display device;

[0203] 1000 - second electronic device;

[0204] Ⅰ-computational unit;

[0205] II-ROM;

[0206] III-RAM;

[0207] IV-bus;

[0208] V-interface;

[0209] VI - input unit;

[0210] VII-output unit;

[0211] VIII- Storage medium;

[0212] Ⅸ-Communication unit. DETAILED DESCRIPTION

[0213] It should be noted that the processor described in the present invention is the control center of the electronic device, which can be a processor or a general term for multiple processing elements. For example, it can be one or more central processing units (CPUs), or application specific integrated circuits (ASICs), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (FPGAs).

[0214] Optionally, the processor can perform various functions of the electronic device by running or executing a software program stored in the memory, and calling data stored in the memory.

[0215] In a specific implementation, as an embodiment, the processor may include one or more CPUs. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions). Electronic devices may include: servers, desktop computers, laptops, smart phones, tablet computers, embedded computers, etc., wherein the embedded computers include vehicles and robots, etc.

[0216] The memory is used to store the software program for executing the solution of the present invention, and the execution is controlled by the processor. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0217] It should be noted that the structure of the electronic device shown in the drawings of the present invention does not constitute a limitation thereto, and the actual knowledge structure recognition device may include more or fewer components than shown in the drawings, or a combination of certain components, or a different arrangement of components.

[0218] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0219] It should also be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0220] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0221] It should also be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0222] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0223] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0224] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0225] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0226] The present invention proposes a mathematical model for the full-cycle agricultural machinery sowing, plant protection, and harvesting plan for corn crops based on factors such as sowing time, plant protection time, harvesting time, crop moisture content, and operation-related costs such as fuel consumption, labor, seeds, and pesticides, with the goal of maximizing corporate profits. The mathematical model is then converted into a reinforcement learning model, which is solved using a deep reinforcement learning method to provide an agricultural machinery scheduling strategy. Specifically, the present invention constructs a full-cycle agricultural machinery resource scheduling model based on actual farm data and the influence of external factors with the goal of maximizing corporate profits; and obtains a near-optimal agricultural machinery scheduling strategy based on a deep deterministic policy gradient model.

[0227] In order to make the above features and effects of the present invention more clearly and understandably described, embodiments are given below and described in detail with reference to the accompanying drawings. This specification discloses one or more embodiments that include the features of the present invention. The disclosed embodiments are only for illustration. The scope of protection of the present invention is not limited to the disclosed embodiments, and the present invention is defined by the attached claims.

[0228] 1 Modeling and problem description of the full-cycle agricultural machinery scheduling system

[0229] 1.1 Agricultural Machinery Scale Model

[0230] For operation scenarios such as sowing, plant protection, and harvesting, it is necessary to configure the corresponding number of agricultural machinery based on factors such as farm area, agricultural machinery speed, and width. In addition, in order to make the configuration more reasonable and accurate, agricultural machinery is divided into different horsepower segments, and the horsepower segment is represented by i. The specific agricultural machinery configuration model is as follows:

[0231]

[0232] Among them, N represents the number of agricultural machinery, A is the operating area, w represents the width of the agricultural machinery, v represents the operating rate, t represents the operating time, e represents the operating efficiency of agricultural machinery, and 0.36 is the conversion factor from square meters per second to acres per hour.

[0233] 1.2 Model of relationship between sown area and yield

[0234] Different sowing cycles will have a greater impact on yields, and appropriate early or late sowing can increase yields. For example, the traditional sowing period in Dahewan, Inner Mongolia, is concentrated in April, but due to the influence of global climate, the annual temperature has increased compared with previous years, and the accumulated temperature has increased, so it is suitable for early sowing. The relationship between the actual sowing area and yield is as follows:

[0235] Here Y ac represents the actual yield under the influence of sowing date, Y ha is the yield of the standard sowing period, A ea is the early sowing area, A stis the standard sowing area, is the yield influencing factor under early sowing time j, is the yield influencing factor under the standard sowing time j. The relationship between sowing time and yield influencing factor is as follows:

[0236]

[0237]

[0238]

[0239] in, It represents the number of sowing operation hours of the agricultural machinery with horsepower segment i on the jth day in the early sowing stage. It represents the number of sowing operation hours of the agricultural machinery with horsepower segment i on the jth day during the normal sowing stage. To delay the sowing time. When the operation is completed within the specified time, the yield can be increased by 20% based on the standard yield. When the sowing time is delayed, the yield will be reduced by 20%.

[0240] 1.3 Agricultural machinery input costs during sowing operations

[0241] Here we mainly use no-till seeders, and the input cost of agricultural machinery operation is mainly fuel consumption cost. The specific fuel consumption cost model is as follows:

[0242]

[0243] L represents the number of types of agricultural machinery horsepower segments, and J represents the number of operating days; is the operation time of the agricultural machinery with horsepower segment i on the jth day in the early sowing stage, is the number of agricultural machinery with horsepower segment i in the early sowing period operating on day j, is the normal sowing operation time, f i s represents the fuel consumption of the seed drill with horsepower segment i, θ d is the unit diesel price.

[0244] 1.4 Seed and labor costs

[0245] In addition to fuel costs, there are also seed and labor costs during the sowing process, of which the seed and labor costs are expressed as follows:

[0246]

[0247] Where A represents the operating area, θ seed is the sowing rate per mu (RMB / hectare, θ seed The seed cost is calculated by the operating area, based on the amount of seeds sown per acre and the unit price of seeds). is the number of early sowing machines, θ p is the daily labor cost, It is the time for early sowing. The number of agricultural machinery used in the normal sowing period, This is the normal sowing operation time.

[0248] 1.5 Plant protection operation and yield relationship model

[0249] Adequate fertility and effective weed management can increase corn yields. Since the amount of land fertilizer applied is related to the content of soil elements in the area and has nothing to do with the operation time, the early or delayed fertilization operation has no effect on the yield. However, weeds gradually grow taller over time, thus competing with crops for soil components, thus affecting crop yields. Therefore, in the process of plant protection, the focus is on building a weed management model. Here, the weed growth height is simply mapped to the growth time, so the relationship between weed growth time and yield is modeled as follows:

[0250] Y p =Y ha A c x j

[0251]

[0252] where Y p is the actual yield after weed management, x j It is the percentage of weed impact on yield on the jth day, where j is the jth day in the plant protection cycle. If May to June is the plant protection cycle, the jth day is the jth day after entering May.

[0253] 1.6 Agricultural machinery input costs during plant protection operations

[0254] The mowing here is mainly suspended, based on the weight of the machine, the width is matched with different horsepower agricultural machinery, and the fuel consumption model of plant protection agricultural machinery is as follows:

[0255]

[0256] Among them C p Represents the fuel cost of agricultural machinery in the plant protection process, It is expressed as the operation time of the agricultural machinery with horsepower segment i on the jth day, θ d is the diesel market price, It represents the number of agricultural machinery operating in horsepower segment i on day j.

[0257] 1.7 Fertilizer, pesticide and labor costs

[0258] In addition to fuel costs, the plant protection process also includes labor, fertilizer and pesticide costs.

[0259]

[0260] where θ pe Indicates how many kilograms of pesticides are needed per mu of land, θ cf How many kilograms of fertilizer are needed per acre of land? is the operating time of the agricultural machinery with horsepower range i on the jth day during the plant protection operation, is the number of agricultural machinery inputs in horsepower segment i during the operation on the jth day.

[0261] 1.8 Relationship model between moisture content, harvest loss rate and yield

[0262] During the harvesting stage, crop moisture content and the loss rate of agricultural machinery harvesting directly affect the yield level. Crop moisture content is mainly affected by the external temperature, and the crop moisture content gradually decreases over time. Therefore, the earlier the harvest time, the higher the moisture content and the higher the yield, and vice versa. The relationship between harvest time, moisture content and yield is as follows:

[0263]

[0264] Among them, Y re To obtain the actual yield, is the loss rate of agricultural machinery harvesting process, M q is the crop water content, θ ma is the harvest price corresponding to the moisture content. The relationship between crop moisture content and crop price is as follows:

[0265]

[0266] where p e Indicates the crop price when the moisture content is less than or equal to 14%.

[0267] 1.9 Relationship between harvesting time and agricultural machinery input cost

[0268]

[0269] Among them C h represents the fuel cost of agricultural machinery during the harvesting process, represents the operation time of agricultural machinery with horsepower segment i in the harvesting process on day j, θ d is the diesel market price, It represents the number of harvesting agricultural machinery operating in horsepower segment i on day j.

[0270] 1.10 Labor costs during the harvest period

[0271] The cost during the harvest period mainly comes from the labor cost incurred during the operation of manual agricultural machinery. The specific harvest cost model is as follows:

[0272]

[0273] Among them C h represents the labor cost of the harvesting process, It is expressed as the operation time of the harvesting agricultural machinery with horsepower segment i in j days, θ p For each person's daily salary, It represents the number of harvesting agricultural machinery operating in horsepower segment i on day j.

[0274] 1.11 Mathematical model for maximizing profits throughout the entire cycle

[0275] By integrating the income and input of the previous steps, we can get the mathematical model for calculating corporate profits:

[0276]

[0277]

[0278]

[0279]

[0280]

[0281]

[0282]

[0283]

[0284]

[0285]

[0286]

[0287]

[0288]

[0289]

[0290]

[0291]

[0292]

[0293]

[0294] M s ≤M q≤M (19)

[0295] Get the number of hours of x-level agricultural machinery operations arranged on the xth day. Among them, constraints (2) to (4) respectively represent the operating time range of the seed drill (agricultural machinery) with horsepower segment i on the jth day; constraints (5) and (7) indicate that the number of agricultural machinery in the early sowing, standard sowing and late sowing periods cannot exceed the number of sowing agricultural machinery on the farm; constraint (8) indicates that the actual number of agricultural machinery operations during the plant protection process does not exceed the total number of plant protection machines on the farm; constraint (9) indicates that the actual number of agricultural machinery operations during the harvesting process does not exceed the total number of harvesters on the farm; constraints (10) to (12) indicate that the number of agricultural machinery is limited to an integer, constraints (13) to (17) respectively indicate that the operating area during the harvesting, plant protection and sowing process cannot exceed the total area of ​​the farm, and constraint (18) indicates that the sum of the operating areas of early sowing, standard sowing and late sowing cannot exceed the total area of ​​the farm. Constraint (19) indicates the range of crop moisture content.

[0296] Since the above model is a mixed integer nonlinear programming problem composed of continuous variables and discrete variables, it has high complexity. Therefore, using traditional non-convex optimization solution methods or algorithms based on continuous action space to solve the mixed integer nonlinear programming problem will result in very low efficiency.

[0297] 2 Based on deep reinforcement learning DDPG algorithm

[0298] The present invention uses a neural network to design a low-complexity algorithm to obtain an approximate optimal solution. Reinforcement learning (RL) is an important branch of machine learning. Unlike supervised learning and unsupervised learning, reinforcement learning finds the optimal action strategy by continuously interacting with the agent and the environment. An RL framework usually includes the following main elements: state space, which is used to represent the current state of the agent. Through the state space, the intelligent agent can perceive and understand the information in the environment; action space, which is used to represent the set of actions that the agent can take. The agent selects one or more actions to interact with the environment according to its own state. During the interaction, the intelligent agent will adjust its strategy according to the actions taken and the rewards obtained; reward, which is used to represent the reward given by the environment. As a standard for evaluating the quality of the agent's actions, it will be assigned according to the actions of the intelligent agent and the feedback of the environment. The reward can be used to adjust the agent's strategy to better adapt to the environment; state transition, which describes the new state obtained by the intelligent agent after performing an action in the current state. State transition is usually determined by the environment, and the agent can only learn and infer the state transition law of the environment through observed information. Deep reinforcement learning (DRL) combines deep neural networks and reinforcement learning. Unlike traditional reinforcement learning, deep reinforcement learning models use deep neural networks (DNNs) to represent action-value functions, which allows them to learn from high-dimensional perceptual inputs. Therefore, in real-world applications, the learning and decision-making capabilities of intelligent agents have been significantly improved.

[0299] 2.1 Problem Transformation Based on Deep Reinforcement Learning

[0300] The present invention proposes a method based on deep reinforcement learning to find A, n, and t that optimize the result of formula (1). According to 1.1, since A, n, and t satisfy the relationship A=ntWVe0.36, it is only necessary to find the optimal n and t.

[0301] In order to convert the original mathematical optimization problem into the form of reinforcement learning, the area X required for operation and the number of days for remaining operation H are used as the state space S. Among them, f represents plant protection, h represents harvesting, and the number of various agricultural machinery, water content m and operation time are used as the action space A. The reason for including moisture content is to determine the maximum profit for harvesting corn at what moisture content. The subscript t represents time (days) and also represents the number of steps in the reinforcement learning process. For example, t = 1 represents the number of agricultural machinery working strategies on the first day. For a given action a t , the reward R is calculated according to formula (1)t , calculate the area of ​​the operation according to the formula in 1.1, reduce the corresponding remaining area in the state space S by the operation area, and the corresponding remaining days - 1. In this way, based on the mathematical model, an environment (env) that can be used for reinforcement learning is constructed.

[0302] 2.2 Solution based on DDPG algorithm

[0303] Since the action space contains continuous values, the present invention selects the Deep Deterministic Policy Gradient (DDPG) algorithm to model and solve the original problem.

[0304] The present invention creates an experience playback pool to store (s t ,a t ,R t ,s t+1 ) is used to train neural networks.

[0305] The overall architecture of the algorithm is shown in the figure Figure 1 As shown, the algorithm consists of a training strategy network π online , a target policy network π target , a training value function network Q online , a target value function network Q target The strategy network π is composed according to the input state s t Output action a t , the value network Q is based on the input action a t and status t Give the current state value estimate Q. The neural network is iteratively trained by fitting the output of the training value function network with the output of the target value function network.

[0306] Step 1. Initialization

[0307] Policy Network (Actor Network): Use parameter θ μ Initialize the policy network and output the action for a given state.

[0308] Target Actor Network: Copy the parameters of the policy network to the target policy network, with the initial parameters being θ' μ =θ μ

[0309] Train the Critic Network: use the parameter θ Q Initialize the Q-value network (train the value network) to evaluate the Q-value for a given state-action pair.

[0310] Target Critic Network: Copy the parameters of the Q-value network to the target Q-value network, with the initial parameters being θ' Q =θ Q .

[0311] Replay Buffer: Initialize an experience replay buffer to store past experiences (state, action, reward, next state).

[0312] Step 2. Interaction and experience storage

[0313] State initialization: Randomly initialize state s0 from the environment.

[0314] Action selection: Use the current training policy network to select action a t =μ(s t |θ μ )+N t , where N t is the exploration noise, in this case Gaussian noise, which ensures that the policy has enough exploration.

[0315] Execute actions and store experience: Execute action a_t in the environment (Formula 1), obtain reward r_t and next state s_{t+1}, and store the four-tuple (s t ,a t ,r t ,s t+1 ) is stored in the experience replay pool.

[0316] Status update: Update the current status to s t+1 .

[0317] Step 3. Sampling from the experience replay pool

[0318] Mini-batch sampling: Randomly sample a mini-batch of experiences (s i ,a i ,r i ,s i+1 ). Batch sampling of past data has the following two advantages: First, when doing reinforcement learning, the most time-consuming part is interacting with the environment, while training the network is faster. Using a replay buffer can reduce the number of interactions with the environment. Second, when training the network, the more diverse the data in a batch, the better. If the data in the batch training are all of the same nature, the training results are likely to be poor.

[0319] The mini-batch sampling results are fed into four networks π online , π target , Q online , Qtarget Composition. Figure 1 The data transfer process can be seen in .

[0320] Step 4. Update of Critic Network

[0321] Calculate the target Q value: Calculate the target Q value using the target policy network and the target Q value network:

[0322] y i =r i +γQ′(s t+1 ,μ′(s t+1 |θ′ μ )|θ′ Q )

[0323] where γ is the discount factor.

[0324] Update the Critic network: Update the Critic network parameters by minimizing the following loss function:

[0325]

[0326] Update θ using gradient descent Q .

[0327] Step 5. Update the Actor Network

[0328] Calculate policy gradient: Use the current Critic network to calculate the policy gradient:

[0329]

[0330] Update the Actor network: Use gradient ascent to update the policy network parameters θ μ

[0331] Step 6. Update the target network

[0332] Soft update: Use soft update to copy the parameters of the policy network and Q value network to the corresponding target network:

[0333] θ′ Q =τθ Q +(1-τ)θ′ Q

[0334] θ′ μ =τθ μ +(1-τ)θ′ μ

[0335] Step 7. Repeat steps 2 to 6.

[0336] Repeat the above steps at each time step until the reward tends to stabilize and converge over time, or reaches a predetermined number of training steps, then the strategy is considered to converge, and the current training strategy network π is saved. online As a candidate network. Execute step 1 again to try different random seeds to obtain multiple candidate networks. According to the full-cycle profit maximization mathematical model, action space A and state space S, obtain the operation strategy of each candidate network, and select the operation strategy that maximizes the profit output by the full-cycle profit maximization mathematical model as the final operation strategy.

[0337] 2.2 Result analysis based on DDPG algorithm

[0338] Figure 2 The comparison of the maximum profit obtained by solving the mathematical model using the deep reinforcement learning algorithm and the branch and bound method under different total areas (unit: hectares) is described. It can be seen from the figure that the deep reinforcement learning algorithm can approximately obtain the optimal solution.

[0339] Figure 3 The efficiency comparison between the deep reinforcement learning algorithm training process and the iterative solution using the branch and bound method is described. In the experiment, the maximum operating area is set to 2000 hectares, and the upper limit of each type of agricultural machinery is 10. All are implemented using python on a computer with i5-11400 CPU and 16G memory. It can be seen from the figure that compared with the branch and bound method, the deep reinforcement learning training model can fit faster and find the approximate optimal solution.

[0340] Figure 4 The time comparison between deep reinforcement learning algorithm training and branch-and-bound iterative solution under different farm area conditions is described. Figure 4 It can be seen that the training time of the deep reinforcement learning algorithm is not much different under different farm areas, while the calculation time of the branch and bound method tends to increase with the increase of parameters.

[0341] Figure 5 The figure describes the estimated profits of the deep reinforcement learning algorithm under different total number of agricultural machinery under the condition of a total farm area of ​​800 hectares. As can be seen from the figure, compared with the algorithm that does not consider the early sowing process (nsddpg), the total profit of the proposed algorithm is always higher than that of nsddpg as the number of agricultural machinery increases. However, the increasing trend gradually slows down, and finally the profit is maximized due to the limitation of the total operating area.

[0342] The following is a system embodiment corresponding to the above method embodiment. This embodiment can be implemented in conjunction with the above embodiment. The relevant technical details mentioned in the above embodiment are still valid in this embodiment. In order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the above embodiment.

[0343] like Figure 7 As shown, the present invention also proposes an agricultural machinery scheduling device B based on joint optimization of production planning, which includes:

[0344] Module 1: Construct the agricultural machinery configuration model N according to the total farm area, agricultural machinery width, number of agricultural machinery, operation rate, operation time and operation efficiency in the harvesting operation;

[0345] Module 2: Based on the yield of standard sowing period, early sowing area, standard sowing area, and the yield influencing factors of early sowing and standard sowing, the actual yield model Y under the influence of sowing period is constructed. ac ;

[0346] Module 3: Construct the fuel consumption cost model C of the sowing agricultural machinery operation based on the fuel price, the number of sowing operation days, the number of agricultural machinery used in the sowing operation, and the fuel consumption of agricultural machinery s ;

[0347] Module 4: Construct the labor cost model C based on the operation area, sowing amount per mu, number of sowing machines, labor cost per unit time, and early sowing operation time. c ;

[0348] Module 5: Construct a plant protection operation and yield relationship model Y based on the relationship between weed growth time and yield p ;

[0349] Module 6: Construct the agricultural machinery input cost model C in the process of plant protection operation based on the fuel consumption, fuel price and plant protection time of agricultural machinery for plant protection p ;

[0350] Module 7: Construct the fertilizer and labor cost model of the plant protection operation process based on the plant protection time, fertilizer weight per unit area, fertilizer price, and labor cost per unit time. x ;

[0351] Module 8: Construct the relationship model between moisture content and yield based on harvest yield and crop moisture content re ;

[0352] Module 9: Construct the fuel consumption cost model C of harvesting agricultural machinery operation based on fuel price, harvesting operation days, number of agricultural machinery used in harvesting operation, and agricultural machinery fuel consumption h ;

[0353] Module 10: Construct the labor cost model C for the harvest period based on the labor cost per unit time and the operation time of the harvesting agricultural machinery z ;

[0354] Module 11: Based on the agricultural machinery configuration model N and the actual yield model Y ac 、Fuel consumption cost model C s, Labor cost model C c , Plant protection operation and yield relationship model Y p , Agricultural machinery input cost model C in the process of plant protection operation p , Fertilizer and labor cost model for plant protection operation process C x , moisture content and yield relationship model Y re , Fuel consumption cost model of harvesting agricultural machinery operation C h , Harvest period labor cost model C z , build a mathematical model for maximizing profits throughout the entire cycle;

[0355] Module 12: Based on the upper limit of the daily operation time of agricultural machinery, the upper limit of the total number of agricultural machinery, the upper limit of the total farm area, and the range of moisture content of crops, the constraints of the mathematical model for maximizing profits throughout the entire cycle are constructed. Under the constraints, the agricultural machinery scheduling strategy is solved when the output of the mathematical model for maximizing profits throughout the entire cycle is maximized; the sowing, harvesting and plant protection operations of the crop are performed using the agricultural machinery scheduling strategy.

[0356] The agricultural machinery scheduling device based on production planning joint optimization, wherein the agricultural machinery configuration model N is:

[0357]

[0358] Among them, N represents the number of agricultural machinery, A is the total area of ​​the farm, w represents the width of the agricultural machinery, v represents the operation rate, t represents the operation time, and e represents the operation efficiency of the agricultural machinery;

[0359] The actual yield model Y under the influence of the sowing date ac for:

[0360]

[0361] Y ac represents the actual yield under the influence of sowing date, Y ha is the yield of the standard sowing period, A ea is the early sowing area, A st is the standard sowing area, is the yield influencing factor under early sowing time j, is the yield influencing factor under standard sowing time j;

[0362]

[0363]

[0364]

[0365] in, It represents the number of sowing operation hours of the agricultural machinery with horsepower segment i on the jth day in the early sowing stage. It represents the number of sowing operation hours of the agricultural machinery with horsepower segment i on the jth day during the normal sowing stage. To delay the sowing time;

[0366] Fuel consumption cost model of seeding agricultural machinery operation C s for:

[0367]

[0368] L represents the number of types of agricultural machinery horsepower segments, and J represents the number of operating days; is the operation time of the agricultural machinery with horsepower segment i on the jth day in the early sowing stage, is the number of agricultural machinery with horsepower segment i in the early sowing period operating on day j, is the normal sowing operation time, f i s represents the fuel consumption of the seed drill with horsepower segment i, θ d is the unit diesel price;

[0369] The labor cost model C c :

[0370]

[0371] Where A represents the total farm area, θ seed is the sowing amount per mu, is the number of early sowing machines, θ p is the daily labor cost, It is the time for early sowing. The number of agricultural machinery used in the normal sowing period, Normal sowing operation time;

[0372] The relationship model Y between plant protection operation and yield p :

[0373] Y p =Y ha A c x j

[0374]

[0375] where Y p is the yield after weed management, x j is the percentage of weed impact on yield on day j, where j is the jth day in the plant protection cycle;

[0376] The agricultural machinery input cost model C of the plant protection operation process p :

[0377]

[0378] Among them C p Represents the fuel cost of agricultural machinery in the plant protection process, It is expressed as the operation time of the agricultural machinery with horsepower segment i on the jth day, θ d is the diesel market price, represents the number of agricultural machinery operating in horsepower segment i on day j;

[0379] The fertilizer and labor cost model C of the plant protection operation process x :

[0380]

[0381] Among them C x represents the labor cost of the plant protection process, θ pe Indicates the amount of pesticide required per unit area, θ cf Indicates the amount of fertilizer required per unit area. is the operating time of the agricultural machinery with horsepower range i on the jth day during the plant protection operation, is the number of agricultural machinery inputs in the operation process of horsepower segment i on day j;

[0382] The water volume and yield relationship model Y re :

[0383]

[0384] Among them, Y re To obtain the actual yield, is the loss rate of agricultural machinery harvesting process, M q is the crop water content, θ ma is the harvest price corresponding to the moisture content; the relationship between crop moisture content and crop price is:

[0385]

[0386] where p e Indicates the crop price when the moisture content is less than or equal to 14%.

[0387] The fuel consumption cost model C of the harvesting agricultural machinery operation h :

[0388]

[0389] Among them C h represents the fuel cost of agricultural machinery during the harvesting process, represents the operation time of agricultural machinery with horsepower segment i in the harvesting process on day j, θ d is the diesel market price, It represents the number of harvesting agricultural machinery operating on day j in horsepower segment i;

[0390] The labor cost model C for the harvest periodz :

[0391]

[0392] Among them C z represents the labor cost of the harvesting process, It is expressed as the operation time of the harvesting agricultural machinery with horsepower segment i in j days, θ p For each person's daily salary, It represents the number of harvesting agricultural machinery operating on day j in horsepower segment i;

[0393] The mathematical model for maximizing profits throughout the entire cycle is:

[0394]

[0395] The restrictions are:

[0396]

[0397]

[0398]

[0399]

[0400]

[0401]

[0402]

[0403]

[0404]

[0405]

[0406]

[0407]

[0408]

[0409]

[0410]

[0411]

[0412]

[0413] Ms ≤M q ≤M (19)

[0414] The above constraints (2) to (4) respectively represent the operating time range of agricultural machinery with horsepower range i on the jth day; constraints (5) and (7) indicate that the number of agricultural machinery in the early sowing, standard sowing and late sowing periods is less than or equal to the number of sowing agricultural machinery already on the farm; constraint (8) indicates that the actual number of agricultural machinery operations during the plant protection process is less than or equal to the total number of plant protection machines already on the farm; constraint (9) indicates that the actual number of agricultural machinery operations during the harvest process is less than or equal to the total number of harvesters already on the farm; constraints (10) to (12) indicate that the number of agricultural machinery is limited to an integer, constraints (13) to (17) respectively indicate that the operating area during the harvest, plant protection and sowing process is less than or equal to the total area of ​​the farm, and constraint (18) indicates that the sum of the operating areas of early sowing, standard sowing and late sowing is less than or equal to the total area of ​​the farm; constraint (19) indicates the range of moisture content of crops.

[0415] The agricultural machinery scheduling device based on production planning joint optimization solves the full-cycle profit maximization mathematical model based on reinforcement learning, and takes the required operation area X and the remaining operation days H as the state space S. Among them, f represents plant protection, h represents harvesting, and the number of various agricultural machinery, water content m and operation time are used as the action space A. Subscript t represents the tth day; for a given action a t , the reward R is calculated based on the mathematical model of maximizing profit in the whole cycle t , calculate the first operation area according to the agricultural machinery configuration model N, reduce the corresponding remaining area in the state space S by the first operation area, and reduce the corresponding remaining days by 1;

[0416] The specific process of solving the mathematical model of maximizing profit in the whole cycle based on reinforcement learning includes:

[0417] Module 121, using initial parameters θ μ Initialize the training policy network, which is used to output the action a in a given state t ;Copy the parameters of the policy network to the target policy network, the initial parameters of the target policy network are θ' μ =θ μ ; Using the initial parameters θ Q Initialize the training value network, which is trained according to the input action a t The state st gives the current state value estimate Q; the parameters of the training value network are copied to the target Q value network, and the initial parameters of the target Q value network are θ' Q =θ Q; Create an experience replay pool for training neural networks;

[0418] Module 122: Randomly initialize state s0 from state space S; use the trained policy network to select action a t =μ(s t |θ μ )+N t , where N t For noise; action a t Enter the mathematical model of maximizing profit in the whole cycle and get the reward R t and the next state s t+1 , the quaternion (s t ,a t ,R t ,s t+1 ) is stored in the experience replay pool; the current state is updated to s t+1 ;

[0419] Module 123, randomly sample experience (s) from the experience replay pool i ,a i ,R i ,s i+1 );

[0420] Module 124, using the target policy network and the target Q value network to calculate the target Q value:

[0421] y i =r i +γQ′(s t+1 ,μ′(s t+1 |θ′ μ )|θ′ Q )

[0422] Where γ is the discount factor;

[0423] Update the parameters θ of the training value network by minimizing the loss function L Q :

[0424]

[0425] Module 125: Calculate the policy gradient using the current training value network To use the gradient ascent method to update the parameters θ of the training strategy network μ ;

[0426]

[0427] Module 126, copying the parameters of the training strategy network and the training value network to the target strategy network and the target Q value network respectively:

[0428] θ′Q =τθ Q +(1-τ)θ′ Q

[0429] θ′ μ =τθ μ +(1-τ)θ′ μ

[0430] Module 127, loop iteratively executes modules 122 to 126 until the reward tends to stabilize, converge over time, or reaches a predetermined number of training modules, then the strategy is considered to converge, and the current training strategy network π is saved online As a candidate network. Module 121 is executed again to try different initial parameters to obtain multiple candidate networks, and the operation strategy of each candidate network is obtained according to the full-cycle maximization profit mathematical model, the action space A and the state space S, and the operation strategy that makes the full-cycle maximization profit mathematical model output the maximum profit is selected as the final operation strategy.

[0431] The agricultural machinery scheduling device based on production planning joint optimization, wherein the crop is corn.

[0432] like Figure 8 As shown, the present invention further proposes a first electronic device A in another embodiment, which includes the agricultural machinery scheduling device based on joint optimization of production planning.

[0433] like Fig. 9 As shown, the first electronic device A can also be connected to the data acquisition device C and the information display device D through a wired or wireless information transmission scheme. The data acquisition device C is used to collect farm area, agricultural machinery speed, width, fuel consumption, hourly labor cost, factors affecting yield and unit price per kilogram of crops. The information display device D is used to display the operation strategy obtained by the analysis of the present invention.

[0434] The information display device D can sort and process the data output by the first electronic device A based on the information display mechanism to improve the readability of the data output by the first electronic device A. The information display mechanism can be manually preset, for example, the data output by the first electronic device A is visually displayed, which can be based on the display parameters and / or attributes set by the user. The display parameters can be, for example, the display data range, and the display attributes can be, for example, the display font, color, whether to scroll and play, etc. The user is presented with the key information specified by the user, and the user can understand the information more promptly without having to access the secondary page or scroll the page, saving the user's operation. Or the information display mechanism can be an artificial intelligence AI display model, which can learn the user's key information based on the user's previous usage habits, such as viewing time, number of clicks, number of edits, etc., and then automatically present rich and necessary key information to the user.

[0435] The present invention also proposes a storage medium VIII in another embodiment for storing a computer program for executing the agricultural machinery scheduling method based on production planning joint optimization. It should be understood that the storage medium in the embodiment of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0436] Fig.10 A schematic block diagram of a second electronic device 1000 that can be used to implement an embodiment of the present invention is shown. The second electronic device 1000 electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The second electronic device 1000 can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples, and are not intended to limit the implementation of the present invention described and / or required herein. The second electronic device 1000 may be the same or different from the first electronic device A.

[0437] The second electronic device 1000 includes a computing unit I, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory II (ROM) or a computer program loaded from a storage medium VIII into a random access memory (RAM) III. In RAM III, various programs and data required for the operation of the device 1000 can also be stored. The computing unit I, ROM II, and RAM III are connected to each other via a bus IV. An input / output (I / O) interface V is also connected to the bus IV.

[0438] A plurality of components in the second electronic device 1000 are connected to the I / O interface V, including: an input unit VI, such as a keyboard, a mouse, etc.; an output unit VII, such as various types of displays, speakers, etc.; a storage medium VIII, such as a disk, an optical disk, etc.; and a communication unit IX, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit IX allows the second electronic device 1000 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0439] The computing unit I may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of computing unit I include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit I performs the various methods and processes described above, such as method steps S1-S3. For example, in some embodiments, the method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage medium VIII. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1000 via ROM II and / or a communication unit IX. When the computer program is loaded into RAM III and executed by the computing unit I, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit I may be configured to execute the method in any other appropriate manner (e.g., by means of firmware).

[0440] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and the implementation modes, and they can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.

Claims

1. A method for agricultural machinery scheduling based on joint optimization of production planning, characterized in that: include: Step 1: Construct an agricultural machinery configuration model N according to the total farm area, agricultural machinery width, number of agricultural machinery, operation rate, operation time, and operation efficiency in the harvesting operation; Step 2: Based on the yield of standard sowing date, early sowing area, standard sowing area, and the yield influencing factors of early sowing and standard sowing, construct the actual yield model Y under the influence of sowing date ac ; Step 3: Based on the fuel price, the number of days for sowing operations, the number of agricultural machinery used in sowing operations, and the fuel consumption of agricultural machinery, a fuel consumption cost model C for sowing agricultural machinery operations is constructed. s ; Step 4: Construct a labor cost model C based on the operation area, sowing amount per mu, number of sowing machines, labor cost per unit time, and early sowing operation time. c ; Step 5: Based on the relationship between weed growth time and yield, construct the plant protection operation and yield relationship model Y p ; Step 6: Based on the fuel consumption, fuel price and plant protection time of agricultural machinery, construct the agricultural machinery input cost model C for the plant protection operation process p ; Step 7: Based on the plant protection time, fertilizer weight per unit area, fertilizer price, and labor cost per unit time, construct the fertilizer and labor cost model C of the plant protection operation process x ; Step 8: Based on the harvest yield and crop moisture content, construct the moisture content and yield relationship model Y re ; Step 9: Construct the fuel consumption cost model C of harvesting agricultural machinery operation based on fuel price, harvesting operation days, number of agricultural machinery used in harvesting operation, and agricultural machinery fuel consumption. h ; Step 10: Construct the labor cost model C for the harvest period based on the labor cost per unit time and the operation time of the harvesting agricultural machinery z ; Step 11: Based on the agricultural machinery configuration model N and the actual yield model Y ac 、Fuel consumption cost model C s , Labor cost model C c , Plant protection operation and yield relationship model Y p , Agricultural machinery input cost model C in the process of plant protection operation p , Fertilizer and labor cost model for plant protection operation process C x , moisture content and yield relationship model Y re , Fuel consumption cost model of harvesting agricultural machinery operation C h , Harvest period labor cost model C z , build a mathematical model for maximizing profits throughout the entire cycle; Step 12: Based on the upper limit of the daily operation time of agricultural machinery, the upper limit of the total number of agricultural machinery, the upper limit of the total farm area, and the range of moisture content of crops, the restriction constraints of the mathematical model for maximizing profit in the whole cycle are constructed. Under the restriction constraints, the agricultural machinery scheduling strategy when the output of the mathematical model for maximizing profit in the whole cycle is solved to maximize the profit is obtained; the sowing, harvesting and plant protection operations of the crop are performed using the agricultural machinery scheduling strategy.

2. The agricultural machinery scheduling method based on production planning joint optimization according to claim 1 is characterized in that: The agricultural machinery configuration model N is: Among them, N represents the number of agricultural machinery, A is the total area of ​​the farm, w represents the width of the agricultural machinery, v represents the operation rate, t represents the operation time, and e represents the operation efficiency of the agricultural machinery; The actual yield model Y under the influence of the sowing date ac for: Y ac represents the actual yield under the influence of sowing date, Y ha is the yield of the standard sowing period, A ea is the early sowing area, A st is the standard sowing area, is the yield influencing factor under early sowing time j, is the yield influencing factor under standard sowing time j; in, It represents the number of sowing operation hours of the agricultural machinery with horsepower segment i on the jth day in the early sowing stage. It represents the number of sowing operation hours of the agricultural machinery with horsepower segment i on the jth day during the normal sowing stage. To delay the sowing time; Fuel consumption cost model of seeding agricultural machinery operation C s for: L represents the number of types of agricultural machinery horsepower segments, and J represents the number of operating days; is the operation time of the agricultural machinery with horsepower segment i on the jth day in the early sowing stage, is the number of agricultural machinery with horsepower segment i in the early sowing period operating on day j, is the normal sowing operation time, f i s represents the fuel consumption of the seed drill with horsepower segment i, θ d is the unit diesel price; The labor cost model C c : Where A represents the total farm area, θ seed is the sowing amount per mu, is the number of early sowing machines, θ p is the daily labor cost, It is the time for early sowing. The number of agricultural machinery used in the normal sowing period, Normal sowing operation time; The relationship model Y between plant protection operation and yield p : AND p =And ha TO c x j where Y p is the yield after weed management, x j is the percentage of weed impact on yield on day j, where j is the jth day in the plant protection cycle; The agricultural machinery input cost model C of the plant protection operation process p : Among them C p Represents the fuel cost of agricultural machinery in the plant protection process, It is expressed as the operation time of the agricultural machinery with horsepower segment i on the jth day, θ d is the diesel market price, represents the number of agricultural machinery operating in horsepower segment i on day j; The fertilizer and labor cost model C of the plant protection operation process x : Among them C x represents the labor cost of the plant protection process, θ pe Indicates the amount of pesticide required per unit area, θ cf Indicates the amount of fertilizer required per unit area. is the operating time of the agricultural machinery with horsepower range i on the jth day during the plant protection operation, is the number of agricultural machinery inputs in the operation process of horsepower segment i on day j; The water volume and yield relationship model Y re : Among them, Y re To obtain the actual yield, is the loss rate of agricultural machinery harvesting process, M q is the crop water content, θ ma is the harvest price corresponding to the moisture content; the relationship between crop moisture content and crop price is: where p e Indicates the crop price when the moisture content is less than or equal to 14%. The fuel consumption cost model C of the harvesting agricultural machinery operation h : Among them C h represents the fuel cost of agricultural machinery during the harvesting process, represents the operation time of agricultural machinery with horsepower segment i in the harvesting process on day j, θ d is the diesel market price, It represents the number of harvesting agricultural machinery operating on day j in horsepower segment i; The labor cost model C for the harvest period z : Among them C z represents the labor cost of the harvesting process, It is expressed as the operation time of the harvesting agricultural machinery with horsepower segment i in j days, θ p For each person's daily salary, It represents the number of harvesting agricultural machinery operating on day j in horsepower segment i; The mathematical model for maximizing profits throughout the entire cycle is: The restrictions are: M s ≤M q ≤M (19) The above constraints (2) to (4) respectively represent the operating time range of agricultural machinery with horsepower range i on the jth day; constraints (5) and (7) indicate that the number of agricultural machinery in the early sowing, standard sowing and late sowing periods is less than or equal to the number of sowing agricultural machinery already on the farm; constraint (8) indicates that the actual number of agricultural machinery operations during the plant protection process is less than or equal to the total number of plant protection machines already on the farm; constraint (9) indicates that the actual number of agricultural machinery operations during the harvest process is less than or equal to the total number of harvesters already on the farm; constraints (10) to (12) indicate that the number of agricultural machinery is limited to an integer, constraints (13) to (17) respectively indicate that the operating area during the harvest, plant protection and sowing process is less than or equal to the total area of ​​the farm, and constraint (18) indicates that the sum of the operating areas of early sowing, standard sowing and late sowing is less than or equal to the total area of ​​the farm; constraint (19) indicates the range of moisture content of crops.

3. The agricultural machinery scheduling method based on production planning joint optimization according to claim 1, characterized in that: Based on reinforcement learning, the mathematical model of maximizing profit in the whole cycle is solved, and the area X required for operation and the number of days H for remaining operation are used as the state space S. Among them, f represents plant protection, h represents harvesting, and the number of various agricultural machinery, water content m and operation time are used as the action space A. Subscript t represents the tth day; for a given action a t , the reward R is calculated based on the mathematical model of maximizing profit in the whole cycle t , calculate the first operation area according to the agricultural machinery configuration model N, reduce the corresponding remaining area in the state space S by the first operation area, and reduce the corresponding remaining days by 1; The specific process of solving the mathematical model of maximizing profit in the whole cycle based on reinforcement learning includes: Step 121, use initial parameters θ μ Initialize the training policy network, which is used to output the action a in a given state t ;Copy the parameters of the policy network to the target policy network, the initial parameters of the target policy network are θ' μ =θ μ ; Using the initial parameters θ Q Initialize the training value network, which is trained according to the input action a t and status t Given the current state value estimate Q; copy the parameters of the training value network to the target Q value network, the initial parameters of the target Q value network are θ′ O =θ Q ; Create an experience replay pool for training neural networks; Step 122: Randomly initialize state s0 from the state space S; use the training strategy network to select action a t =μ(s t |θ μ )+N t , where N t For noise; action a t Enter the mathematical model of maximizing profit in the whole cycle and get the reward R t and the next state s t+1 , the quaternion (s t , a t , R t ,s t+1 ) is stored in the experience replay pool; Update the current state to s t+1 ; Step 123: Randomly sample experience (s) from the experience replay pool i , a i , R i ,s i+1 ); Step 124: Calculate the target Q value using the target policy network and the target Q value network: y i =r i +γQ′(s t+1 ,μ′(s t+1 |θ′ μ )|θ′ Q ) Where y is the discount factor; Update the parameters θ of the training value network by minimizing the loss function L Q : Step 125: Calculate the policy gradient using the current training value network Use the gradient ascent method to update the parameters θμ of the training strategy network; Step 126: Copy the parameters of the training strategy network and the training value network to the target strategy network and the target Q value network respectively: θ′ Q =tθ Q +(1-τ)θ′ Q θ′ μ =tθ μ +(1-τ)θ′ μ Step 127, loop iteratively execute steps 122 to 126 until the reward tends to stabilize or converge over time, or reaches a predetermined number of training steps, then the strategy is considered to converge, and the current training strategy network π is saved. online As a candidate network. Execute step 121 again to try different initial parameters to obtain multiple candidate networks, obtain the operation strategy of each candidate network according to the full-cycle maximization profit mathematical model, action space A and state space S, and select the operation strategy that maximizes the profit output by the full-cycle maximization profit mathematical model as the final operation strategy.

4. The agricultural machinery scheduling method based on production planning joint optimization according to claim 1, characterized in that: The crop is corn.

5. An agricultural machinery scheduling device based on joint optimization of production planning, characterized in that: include: Module 1: Construct the agricultural machinery configuration model N according to the total farm area, agricultural machinery width, number of agricultural machinery, operation rate, operation time and operation efficiency in the harvesting operation; Module 2: Based on the yield of standard sowing period, early sowing area, standard sowing area, and the yield influencing factors of early sowing and standard sowing, the actual yield model Y under the influence of sowing period is constructed. ac ; Module 3: Construct the fuel consumption cost model C of the sowing agricultural machinery operation based on the fuel price, the number of sowing operation days, the number of agricultural machinery used in the sowing operation, and the fuel consumption of agricultural machinery s ; Module 4: Construct the labor cost model C based on the operation area, sowing amount per mu, number of sowing machines, labor cost per unit time, and early sowing operation time. c ; Module 5: Construct a plant protection operation and yield relationship model Y based on the relationship between weed growth time and yield p ; Module 6: Construct the agricultural machinery input cost model C in the process of plant protection operation based on the fuel consumption, fuel price and plant protection time of agricultural machinery for plant protection p ; Module 7: Construct the fertilizer and labor cost model of the plant protection operation process based on the plant protection time, fertilizer weight per unit area, fertilizer price, and labor cost per unit time. x ; Module 8: Construct the relationship model between moisture content and yield based on harvest yield and crop moisture content re ; Module 9: Construct the fuel consumption cost model C of harvesting agricultural machinery operation based on fuel price, harvesting operation days, number of agricultural machinery used in harvesting operation, and agricultural machinery fuel consumption h ; Module 10: Construct the labor cost model C during the harvest period based on the labor cost per unit time and the operation time of the harvesting agricultural machinery z ; Module 11: Based on the agricultural machinery configuration model N and the actual yield model Y ac 、Fuel consumption cost model C s , Labor cost model C c , Plant protection operation and yield relationship model Y p , Agricultural machinery input cost model C in the process of plant protection operation p , Fertilizer and labor cost model for plant protection operation process C x , moisture content and yield relationship model Y re , Fuel consumption cost model of harvesting agricultural machinery operation C h , Harvest period labor cost model C z , build a mathematical model for maximizing profits throughout the entire cycle; Module 12: Based on the upper limit of the daily operation time of agricultural machinery, the upper limit of the total number of agricultural machinery, the upper limit of the total farm area, and the range of moisture content of crops, the constraints of the mathematical model for maximizing profits throughout the entire cycle are constructed. Under the constraints, the agricultural machinery scheduling strategy is solved when the output of the mathematical model for maximizing profits throughout the entire cycle is maximized; the sowing, harvesting and plant protection operations of the crop are performed using the agricultural machinery scheduling strategy.

6. The agricultural machinery scheduling device based on production planning joint optimization according to claim 5, characterized in that: The agricultural machinery configuration model N is: Among them, N represents the number of agricultural machinery, A is the total area of ​​the farm, w represents the width of the agricultural machinery, v represents the operation rate, t represents the operation time, and e represents the operation efficiency of the agricultural machinery; The actual yield model Y under the influence of the sowing date ac for: Y ac represents the actual yield under the influence of sowing date, Y ha is the yield of the standard sowing period, A ea is the early sowing area, A st is the standard sowing area, is the yield influencing factor under early sowing time j, is the yield influencing factor under standard sowing time j; in, It represents the number of sowing operation hours of the agricultural machinery with horsepower segment i on the jth day in the early sowing stage. It represents the number of sowing operation hours of the agricultural machinery with horsepower segment i on the jth day during the normal sowing stage. To delay the sowing time; Fuel consumption cost model of seeding agricultural machinery operation C s for: L represents the number of types of agricultural machinery horsepower segments, and J represents the number of operating days; is the operation time of the agricultural machinery with horsepower segment i on the jth day in the early sowing stage, is the number of agricultural machinery with horsepower segment i in the early sowing period operating on day j, is the normal sowing operation time, f i s represents the fuel consumption of the seed drill with horsepower segment i, θ d is the unit diesel price; The labor cost model C c : Where A represents the total farm area, θ seed is the sowing amount per mu, is the number of early sowing machines, θ p is the daily labor cost, It is the time for early sowing. The number of agricultural machinery used in the normal sowing period, Normal sowing operation time; The relationship model Y between plant protection operation and yield p : AND p =And ha TO c x j where Y p is the yield after weed management, x j is the percentage of weed impact on yield on day j, where j is the jth day in the plant protection cycle; The agricultural machinery input cost model C of the plant protection operation process p : Among them C p Represents the fuel cost of agricultural machinery in the plant protection process, It is expressed as the operation time of the agricultural machinery with horsepower segment i on the jth day, θ d is the diesel market price, represents the number of agricultural machinery operating in horsepower segment i on day j; The fertilizer and labor cost model C of the plant protection operation process x : Among them C x represents the labor cost of the plant protection process, θ pe Indicates the amount of pesticide required per unit area, θ cf Indicates the amount of fertilizer required per unit area. is the operating time of the agricultural machinery with horsepower range i on the jth day during the plant protection operation, is the number of agricultural machinery inputs in the operation process of horsepower segment i on day j; The water volume and yield relationship model Y re : Among them, Y re To obtain the actual yield, is the loss rate of agricultural machinery harvesting process, M q is the crop water content, θ ma is the harvest price corresponding to the moisture content; the relationship between crop moisture content and crop price is: where p e Indicates the crop price when the moisture content is less than or equal to 14%. The fuel consumption cost model C of the harvesting agricultural machinery operation h : Among them C h represents the fuel cost of agricultural machinery during the harvesting process, represents the operation time of agricultural machinery with horsepower segment i in the harvesting process on day j, θ d is the diesel market price, It represents the number of harvesting agricultural machinery operating on day j in horsepower segment i; The labor cost model C for the harvest period z : Among them C z represents the labor cost of the harvesting process, It is expressed as the operation time of the harvesting agricultural machinery with horsepower segment i in j days, θ p For each person's daily salary, It represents the number of harvesting agricultural machinery operating on day j in horsepower segment i; The mathematical model for maximizing profits throughout the entire cycle is: The restrictions are: M s ≤M q ≤M (19) The above constraints (2) to (4) respectively represent the operating time range of agricultural machinery with horsepower range i on the i-th day; constraints (5) and (7) indicate that the number of agricultural machinery in the early sowing, standard sowing and late sowing periods is less than or equal to the number of sowing agricultural machinery already on the farm; constraint (8) indicates that the actual number of agricultural machinery operations during the plant protection process is less than or equal to the total number of plant protection machines already on the farm; constraint (9) indicates that the actual number of agricultural machinery operations during the harvest process is less than or equal to the total number of harvesters already on the farm; constraints (10) to (12) indicate that the number of agricultural machinery is limited to an integer, constraints (13) to (17) respectively indicate that the operating area during the harvest, plant protection and sowing process is less than or equal to the total area of ​​the farm, and constraint (18) indicates that the sum of the operating areas of early sowing, standard sowing and late sowing is less than or equal to the total area of ​​the farm; constraint (19) indicates the range of moisture content of crops.

7. The agricultural machinery scheduling device based on production planning joint optimization according to claim 5, characterized in that: Based on reinforcement learning, the mathematical model of maximizing profit in the whole cycle is solved, and the area X required for operation and the number of days H for remaining operation are used as the state space S. Among them, f represents plant protection, h represents harvesting, and the number of various agricultural machinery, water content m and operation time are used as the action space A. Subscript t represents the tth day; for a given action a t , the reward R is calculated based on the mathematical model of maximizing profit in the whole cycle t , calculate the first operation area according to the agricultural machinery configuration model N, reduce the corresponding remaining area in the state space S by the first operation area, and reduce the corresponding remaining days by 1; The specific process of solving the mathematical model of maximizing profit in the whole cycle based on reinforcement learning includes: Module 121, using initial parameters θμ to initialize the training strategy network, which is used to output the action a in a given state t ;Copy the parameters of the policy network to the target policy network, and the initial parameters of the target policy network are θ′ μ =θ μ ; Using the initial parameters θ Q Initialize the training value network, which is trained according to the input action a t and status t Given the current state value estimate Q; copy the parameters of the training value network to the target Q value network, the initial parameters of the target Q value network are θ′ Q =θ Q ; Create an experience replay pool for training neural networks; Module 122: Randomly initialize state s0 from state space S; use the trained policy network to select action a t =μ(s t |θ μ )+N t , where N t For noise; action a t Enter the mathematical model of maximizing profit in the whole cycle and get the reward R t and the next state s t+1 , the quaternion (s t , a t , R t ,s t+1 ) is stored in the experience replay pool; Update the current state to s t+1 ; Module 123, randomly sample experience (s) from the experience replay pool i , a i , R i ,s i+1 ); Module 124, using the target policy network and the target Q value network to calculate the target Q value: y i =r i +γQ′(s t+1 ,μ′(s t+1 |θ′ μ )|θ′ Q ) Where y is the discount factor; Update the parameters θ of the training value network by minimizing the loss function L Q : Module 125: Calculate the policy gradient using the current training value network Use the gradient ascent method to update the parameters θμ of the training strategy network; Module 126, copying the parameters of the training strategy network and the training value network to the target strategy network and the target Q value network respectively: θ′ Q =tθ Q +(1-τ)θ′ Q θ′ μ =tθ μ +(1-τ)θ′ μ Module 127, loop iteratively executes modules 122 to 126 until the reward tends to stabilize, converge over time, or reaches a predetermined number of training modules, then the strategy is considered to converge, and the current training strategy network π is saved online As a candidate network. Module 121 is executed again to try different initial parameters to obtain multiple candidate networks, and the operation strategy of each candidate network is obtained according to the full-cycle maximization profit mathematical model, the action space A and the state space S, and the operation strategy that makes the full-cycle maximization profit mathematical model output the maximum profit is selected as the final operation strategy.

8. The agricultural machinery scheduling device based on production planning joint optimization as claimed in claim 5, characterized in that: The crop is corn.

9. An electronic device, characterized in that: It includes an agricultural machinery scheduling device based on joint optimization of production planning as described in claims 5-8, and the electronic device is connected to an information display device, which is used to display the final operation strategy with display parameters and attributes set by the user or through an artificial intelligence model.

10. A storage medium for storing a computer program for executing the agricultural machinery scheduling method based on production planning joint optimization as described in claims 1-4.

Images