Production task allocation method and device, model training method and device, equipment, medium and product

Abstract

The invention discloses a production task allocation method and device, a model training method and device, equipment, a medium and a product. The production task allocation method comprises the following steps: acquiring a first production object and a collaborative production object for producing the same batch of tobaccos; for at least one production device, according to a first production strategy prediction model corresponding to the first production object, a second production strategy prediction model corresponding to a current production workshop where the current production equipment is located and a third production strategy prediction model corresponding to the current production equipment, the current production object is obtained; determining a first tobacco production plan corresponding to the current production equipment in the future time range according to the first object production associated data of the first production object and the second object production associated data of the collaborative production object, so that the current production equipment performs tobacco production according to the first tobacco production plan. According to the technical scheme, the tobacco production plan is determined based on the production strategy prediction model of the object level, the workshop level and the equipment level in sequence.

Description

Technical Field

[0001] The present invention relates to the technical field of tobacco production, and in particular, to a production task allocation, model training method, device, equipment, medium and product. Background Art

[0002] With the development of modern supply chain systems, a tobacco production supply chain is usually formed by the collaborative production of multiple cooperative production entities. For the allocation and optimization method of the collaborative production tasks of cooperative production entities, a centralized optimization method is usually adopted in the prior art. The centralized optimization method usually relies on a centralized decision-making entity, which collects and processes the information of the entire tobacco supply chain and formulates a global optimization strategy.

[0003] However, in the actual tobacco supply chain environment, each participating entity is often distributed in different geographical locations and has its own production capacity, constraints and optimization goals. Therefore, it is difficult for the centralized optimization method to comprehensively consider the characteristics and requirements of all participating entities, resulting in limitations in the applicability and feasibility of the optimized production task allocation strategy. In the centralized framework, once a change occurs in a certain link of the supply chain, the entire optimization strategy needs to be re-formulated and adjusted. Such frequent re-optimization is difficult to respond in a timely manner to the real-time state of the supply chain, resulting in the lag of production task allocation. Summary of the Invention

[0004] The present invention provides a production task allocation, model training method, device, equipment, medium and product, so as to realize determining a tobacco production plan based on a production strategy prediction model at the object level, workshop level and equipment level in sequence, and applying distributed collaborative optimization at multiple levels when formulating the tobacco production plan to obtain the effect of an optimal tobacco production plan.

[0005] According to one aspect of the present invention, a production task allocation method is provided, and the method includes:

[0006] Obtain a first production entity and a collaborative production entity for producing the same batch of tobacco; wherein, at least one production workshop for producing the same batch of tobacco is included under the first production entity, and at least one production equipment for producing the same batch of tobacco is deployed in the production workshop;

[0007] For at least one of the production devices, based on the first production strategy prediction model corresponding to the first production object, the second production strategy prediction model corresponding to the current production workshop where the current production device is located, the third production strategy prediction model corresponding to the current production device, the first object production association data corresponding to the first production object in the first historical time range, and the second object production association data corresponding to the collaborative production object in the first historical time range, determine the first tobacco production plan corresponding to the current production device in the future time range, so that the current production device conducts tobacco production according to the first tobacco production plan;

[0008] Wherein, the first object production association data and the second object production association data are the model inputs of the first production strategy prediction model, the model output of the first production strategy prediction model is part of the model inputs of the second production strategy prediction model, and the model output of the second production strategy prediction model is part of the model inputs of the third production strategy model; the first object production association data is used to evaluate the tobacco production efficiency of the first production object in the first historical time range; the first tobacco production plan is used to represent the quantity of tobacco planned to be produced by the current production device in the future time range.

[0009] According to another aspect of the present invention, a model training method is provided, and the method includes:

[0010] Obtain a first sample production object and a collaborative sample production object for producing the same batch of tobacco; wherein, the first sample production object includes at least one sample production workshop for producing the same batch of tobacco, and at least one sample production device for producing the same batch of tobacco is deployed in the sample production workshop;

[0011] Obtain a plurality of training samples corresponding to the first sample production object; wherein, the training samples include the first sample object production association data corresponding to the first sample tobacco production object in the historical production cycle and the second sample object production association data corresponding to the collaborative sample production object in the historical production cycle;

[0012] Train a first reinforcement learning model corresponding to the first sample production object, a second reinforcement learning model corresponding to at least one of the sample production workshops, and a third reinforcement learning model corresponding to at least one of the sample production devices under each sample production workshop according to the plurality of training samples;

[0013] When the preset training end condition is reached, a first production strategy prediction model corresponding to the first sample production object, a second production strategy prediction model corresponding to the sample production workshop, and a third production strategy prediction model corresponding to the sample production equipment are obtained, so as to predict the tobacco production plan based on the first production strategy prediction model, the second production strategy prediction model, and the third production strategy prediction model.

[0014] According to another method of the present invention, a production task allocation device is provided, and the device includes:

[0015] An object acquisition module, configured to acquire a first production object and a collaborative production object for producing the same batch of tobacco; wherein, at least one production workshop for producing the same batch of tobacco is included under the first production object, and at least one production equipment for producing the same batch of tobacco is deployed in the production workshop;

[0016] A production plan determination module, configured to, for at least one of the production equipment, determine a first tobacco production plan corresponding to the current production equipment in a future time range according to the first production strategy prediction model corresponding to the first production object, the second production strategy prediction model corresponding to the current production workshop where the current production equipment is located, the third production strategy prediction model corresponding to the current production equipment, the first object production association data corresponding to the first production object in a first historical time range, and the second object production association data corresponding to the collaborative production object in a first historical time range, so that the current production equipment performs tobacco production according to the first tobacco production plan; wherein, the first object production association data and the second object production association data are model inputs of the first production strategy prediction model, the model output of the first production strategy prediction model is a partial model input of the second production strategy prediction model, and the model output of the second production strategy prediction model is a partial model input of the third production strategy model; the first object production association data is used to evaluate the tobacco production efficiency of the first production object within the first historical time range; the first tobacco production plan is used to represent the quantity of tobacco planned to be produced by the current production equipment within the future time range.

[0017] According to another aspect of the present invention, a model training device is provided, and the device includes:

[0018] A sample object acquisition module, configured to acquire a first sample production object and a collaborative sample production object for producing the same batch of tobacco; wherein, the first sample production object includes at least one sample production workshop for producing the same batch of tobacco, and at least one sample production equipment for producing the same batch of tobacco is deployed in the sample production workshop;

[0019] A training sample acquisition module, configured to acquire a plurality of training samples corresponding to the first sample production object; wherein, the training samples include first sample object production association data corresponding to the first sample tobacco production object in a historical production cycle and second sample object production association data corresponding to the collaborative sample production object in the historical production cycle;

[0020] A model training module, configured to train a first reinforcement learning model corresponding to the first sample production object, a second reinforcement learning model corresponding to at least one of the sample production workshops, and a third reinforcement learning model corresponding to at least one of the sample production devices under each of the sample production workshops according to the plurality of training samples;

[0021] A policy prediction model determination module, configured to obtain a first production policy prediction model corresponding to the first sample production object, a second production policy prediction model corresponding to the sample production workshop, and a third production policy prediction model corresponding to the sample production device when a preset training end condition is reached, so as to predict a tobacco production plan based on the first production policy prediction model, the second production policy prediction model, and the third production policy prediction model.

[0022] According to another aspect of the present invention, there is provided an electronic device, which includes:

[0023] At least one processor; and

[0024] A memory communicatively connected to the at least one processor; wherein,

[0025] The memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the at least one processor is enabled to execute the production task allocation and model training method according to any embodiment of the present invention.

[0026] According to another aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions for enabling a processor to execute the production task allocation and model training method according to any embodiment of the present invention when executed.

[0027] According to another aspect of the present invention, there is provided a computer program product including a computer program, and when the computer program is executed by a processor, it implements the production task allocation and model training method according to any embodiment of the present invention.

[0028] The technical solution of the embodiment of the present invention obtains a first production object and a collaborative production object for producing the same batch of tobacco; wherein, the first production object includes at least one production workshop for producing the same batch of tobacco, and at least one production device for producing the same batch of tobacco is deployed in the production workshop, which provides a basis for obtaining data for subsequent distribution of tobacco production tasks, and reflects the application scenario of multiple production objects collaborating to produce the same batch of tobacco; further, for at least one production device, according to the first production strategy prediction model corresponding to the first production object, the second production strategy prediction model corresponding to the current production workshop where the current production device is located, the third production strategy prediction model corresponding to the current production device, the first object production association data corresponding to the first production object, and the second object production association data corresponding to the collaborative production object, determine the first tobacco production plan corresponding to the current production device in the future time range, so that the current production device produces tobacco according to the first tobacco production plan, achieving the effect of determining the tobacco production plan based on the production strategy prediction models at the object level, workshop level, and device level in sequence, realizing the application of distributed collaborative optimization at multiple levels when formulating the tobacco production plan to obtain the optimal tobacco production plan, and being able to better handle the dynamically changing tobacco production supply chain environment based on the production strategy prediction model, with stronger adaptability and robustness.

[0029] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0031] Figure 1 is a flowchart of a production task allocation method provided according to an embodiment of the present invention;

[0032] Figure 2 is a flowchart of a production task allocation method provided according to an embodiment of the present invention;

[0033] Figure 3 is a flowchart of a model training method provided according to an embodiment of the present invention;

[0034] Figure 4 is a flowchart of a model training method provided according to an embodiment of the present invention;

[0035] Figure 5 It is a schematic structural diagram of a production task allocation device provided according to an embodiment of the present invention;

[0036] Figure 6 It is a schematic structural diagram of a model training device provided according to an embodiment of the present invention;

[0037] Figure 7 It is a schematic structural diagram of an electronic device for implementing the production task allocation and model training method according to an embodiment of the present invention. Specific embodiments

[0038] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

[0039] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those clearly listed steps or units, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0040] Figure 1 It is a flowchart of a production task allocation method provided by an embodiment of the present invention. This embodiment is applicable to the situation of predicting the tobacco production plan of any tobacco production object in the future time range. This method can be executed by a production task allocation device, which can be implemented in the form of hardware and / or software, and the production task allocation device can be configured in a terminal and / or a server. As Figure 1 shown, the method includes:

[0041] S110. Obtain a first production object and a collaborative production object for producing the same batch of tobacco; wherein, the first production object includes at least one production workshop for producing the same batch of tobacco, and at least one production device for producing the same batch of tobacco is deployed in the production workshop.

[0042] Among them, the first production object can be an object engaged in multiple tobacco-related links, including the tobacco production and processing link. The collaborative production object can be a tobacco production object that collaborates with the first production object to produce the same batch of tobacco. The number of collaborative production objects can be one or more. It can be understood that in the case where the demand for producing one or more batches of tobacco has been determined, in order to improve production efficiency, multiple tobacco production objects can be used to jointly produce the batch of tobacco. At this time, one of the multiple tobacco production objects participating in the collaborative production can be used as the first production object, and the other tobacco production objects among the multiple tobacco production objects except the first production object can be used as collaborative production objects. The same batch of tobacco produced by the first production object and the collaborative production objects can be one or more batches of tobacco, and the same batch of tobacco includes at least one category of tobacco. In this embodiment, the first production object includes at least one production workshop for producing the same batch of tobacco. The production workshop is a specific place in the first production object for the production and processing of tobacco products, and undertakes the task of converting raw materials such as tobacco leaves into finished tobacco products. In order to improve tobacco production efficiency, at least one production device for producing the same batch of tobacco is deployed in a production workshop. The production device refers to the machines and equipment used to complete a series of production processes from tobacco raw material processing to the final packaging of tobacco products during the tobacco production process.

[0043] In a specific implementation, in the case where the production demand for a batch of tobacco is determined, multiple tobacco production objects for producing the batch of tobacco can be determined. Furthermore, one of the multiple tobacco production objects can be used as the first production object, and the other tobacco production objects among the multiple tobacco production objects except the first production object can be used as collaborative production objects corresponding to the first production object.

[0044] S120. For at least one production device, according to the first production strategy prediction model corresponding to the first production object, the second production strategy prediction model corresponding to the current production workshop where the current production device is located, the third production strategy prediction model corresponding to the current production device, the first object production correlation data of the first production object in the first historical time range, and the second object production correlation data of the collaborative production object in the first historical time range, determine the first tobacco production plan corresponding to the current production device in the future time range, so that the current production device conducts tobacco production according to the first tobacco production plan.

[0045] Among them, the first production strategy prediction model can be used to provide tobacco production strategies for the first production object. In other words, the first production strategy prediction model can output the tobacco production strategy corresponding to the first production object based on the first object production correlation data corresponding to the first production object and the second object production correlation data corresponding to the collaborative production object. The first object production correlation data can be used to evaluate the tobacco production efficiency of the first production object within the first historical time range. The first object production correlation data can include multiple data that can evaluate the tobacco production efficiency of the first production object. Optionally, it includes tobacco production costs, delivery delay rates, and production capacity utilization rates, etc. It can be understood that the tobacco production cost refers to the manifestation of various resource consumptions incurred by the tobacco production object during a certain historical production cycle for producing tobacco products. The delivery delay rate refers to the ratio of the order data of the tobacco production object that fails to deliver products according to the agreed delivery time to the total order quantity during the historical production cycle. A lower delivery delay rate indicates that the production plan arrangement of the tobacco production object is reasonable, the supply chain management is smooth, and the products or services can be delivered on time. On the contrary, a higher delivery delay rate may affect the user experience and may also bring additional resource consumption to the tobacco production object. The production capacity utilization rate refers to the ratio of the actual output of the tobacco production object within a certain period to the potential maximum production capacity (i.e., the tobacco production plan), and it is an important indicator to measure the degree of utilization of the tobacco production object and resources. For example, if the tobacco production plan of a certain tobacco production object is to produce 1 million cases of cigarettes per year, and the actual production in a certain year is 800,000 cases, then the production capacity utilization rate of this tobacco production object in that year is (80÷100)×100% = 80%. A higher production capacity utilization rate usually means that the production equipment and human resources of the tobacco production object are fully utilized, and the production process is relatively smooth. A lower production capacity utilization rate may indicate that there are some problems in the production process of the tobacco production object, such as equipment failures, unreasonable production plans, supply chain interruptions, etc., resulting in the failure of resources to fully play their roles.

[0046] Similarly, the second object production correlation data can be used to evaluate the tobacco production efficiency of the collaborative production object within the first historical time range. Optionally, the second object production correlation data includes tobacco production costs, delivery delay rates, and production capacity utilization rates, etc. The first historical time range can be any time interval before the current moment. Optionally, the first historical time range can be the previous year, the previous six months, the previous three months, the previous month, or the previous week, etc.

[0047] Among them, the second production strategy prediction model can be used to provide tobacco production strategies for the production workshop. The third production strategy prediction model can be used to provide tobacco production strategies for the production equipment.

[0048] It should be noted that the first production strategy prediction model, the second production strategy prediction model, and the third production strategy prediction model can be reinforcement learning models trained based on the reinforcement learning algorithm. Moreover, the first object production correlation data and the second object production correlation data are the model inputs of the first production strategy prediction model, the model output of the first production strategy prediction model is part of the model inputs of the second production strategy prediction model, and the model output of the second production strategy prediction model is part of the model inputs of the third production strategy model. The model output of the first production strategy prediction model can be the tobacco production plan corresponding to the first production object. The model output of the second production strategy prediction model can be the tobacco production plan corresponding to the production workshop. The model output of the third production strategy prediction model can be the tobacco production plan corresponding to the production equipment, that is, the first tobacco production plan. Among them, the first tobacco production plan can be used to represent the quantity of tobacco planned to be produced by the current production equipment within the future time range. The future time range can be any time interval after the current moment. Optionally, the future time range can be the next week, the next month, the next three months, the next six months, or the next year, etc.

[0049] It should also be noted that the advantage of applying the second object production correlation data of the collaborative production object to determine the first tobacco production plan corresponding to each production equipment under the first production object is that it can achieve the communication and collaboration between the hierarchical production strategy prediction models corresponding to the first production object (including the first production strategy prediction model, the second production strategy prediction model, and the third production strategy prediction model) and the hierarchical production strategy prediction models corresponding to the collaborative production object, enabling the hierarchical production strategy prediction models corresponding to the first production object to determine the tobacco production plan based on the global information. Moreover, by applying the second object production correlation data of the collaborative production object, the production capacity of the entire collaborative system can be more accurately evaluated. Furthermore, it helps to reasonably allocate relevant resources according to the overall demand and supply situation, achieving the effect of improving the utilization efficiency of resources while reducing production costs, and improving the rationality and accuracy of the tobacco production plan.

[0050] In a specific implementation, when determining the first production object and the collaborative production object for producing the same batch of tobacco, the first object production correlation data corresponding to the first production object within the first historical time range and the second object production correlation data corresponding to the collaborative production object within the first historical time range can be obtained. Further, for at least one production device, the first production strategy prediction model corresponding to the first production object, the second production strategy prediction model corresponding to the current production workshop where the current production device is located, and the third production strategy prediction model corresponding to the current production device can be retrieved. Further, the first object production correlation data and the second object production correlation data can be input into the first production strategy prediction model, and the model output of the first production strategy prediction model can be obtained. Subsequently, this model output can be used as part of the model input for the second production strategy prediction model, and the other model inputs of the second production strategy prediction model and this model output can be input into the second production strategy prediction model, and the model output of the second production strategy prediction model can be obtained. Subsequently, this model output can be used as part of the model input for the third production strategy prediction model, and the other model inputs of the third production strategy prediction model and this model output can be input into the third production strategy prediction model, and the first tobacco production plan corresponding to the current production device within the future time range can be output. Furthermore, the current tobacco device can be made to carry out tobacco production according to its corresponding first tobacco production plan.

[0051] The technical solution of the embodiment of the present invention provides a basis for data acquisition for subsequent allocation of tobacco production tasks by obtaining the first production object and the collaborative production object for producing the same batch of tobacco; wherein, the first production object includes at least one production workshop for producing the same batch of tobacco, and at least one production device for producing the same batch of tobacco is deployed in the production workshop, and reflects the application scenario of multiple production objects collaborating to produce the same batch of tobacco; further, for at least one production device, according to the first production strategy prediction model corresponding to the first production object, the second production strategy prediction model corresponding to the current production workshop where the current production device is located, the third production strategy prediction model corresponding to the current production device, the first object production correlation data corresponding to the first production object within the first historical time range, and the second object production correlation data corresponding to the collaborative production object within the first historical time range, the first tobacco production plan corresponding to the current production device within the future time range is determined, so that the current production device carries out tobacco production according to the first tobacco production plan, achieving the effect of determining the tobacco production plan based on the production strategy prediction models at the object level, workshop level, and equipment level in sequence, realizing the application of distributed collaborative optimization at multiple levels when formulating the tobacco production plan to obtain the optimal tobacco production plan, and being able to better handle the dynamically changing tobacco production supply chain environment based on the production strategy prediction model, with stronger adaptability and robustness.

[0052] Figure 2 This is a flowchart of a production task allocation method provided by an embodiment of the present invention. Based on the foregoing embodiment, the determination process of the first tobacco production plan is further refined. For the specific implementation manner, reference may be made to the technical solution of this embodiment. Among them, the technical terms that are the same or similar to the above embodiments will not be described in detail here.

[0053] As Figure 2 shown, the method includes:

[0054] S210. Obtain a first production object and a collaborative production object for producing the same batch of tobacco; wherein, the first production object includes at least one production workshop for producing the same batch of tobacco, and at least one production device for producing the same batch of tobacco is deployed in the production workshop.

[0055] S220. For at least one production device, obtain first object production association data corresponding to the first production object in the first historical time range, and second object production association data corresponding to the collaborative production object in the first historical time range.

[0056] In specific implementation, when determining the first production object and the collaborative production object for producing the same batch of tobacco, the first object identifier of the first production object and the collaborative object identifier of the collaborative production object can be obtained. Further, the first object production association data corresponding to the first production object in the first historical time range can be retrieved from the pre-constructed database according to the first object identifier; and the second object production association data corresponding to the collaborative production object in the first historical time range can be retrieved from the pre-constructed database according to the collaborative object identifier.

[0057] S230. Process the first object production association data and the second object production association data according to the first production strategy prediction model to obtain a second tobacco production plan corresponding to the first production object in the future time range.

[0058] Among them, the second tobacco production plan can be used to represent the quantity of tobacco planned to be produced by the first production object in the future time range. The second tobacco production plan is the model output of the first production strategy prediction model. In this embodiment, the first object production association data and the second object production association data are the data included in the state space of the first production strategy prediction model; the second tobacco production plan is the data included in the action space of the first production strategy prediction model. The first production object can be regarded as an intelligent agent, and the first production strategy prediction model can be regarded as the environmental model where the intelligent agent is located. Through the interaction between the intelligent agent and the environmental model, the prediction of the second tobacco production plan is realized.

[0059] In a specific implementation, the first object production - related data corresponding to the first production object within the first historical time range and the second object production - related data corresponding to the collaborative production object within the first historical time range are obtained. Further, the first object production - related data and the second object production - related data can be input into the first production strategy prediction model. Then, based on the first production strategy prediction model, the first object production - related data and the second object production - related data are processed, and the second tobacco production plan corresponding to the first production object within the future time range is output.

[0060] S240. According to the second tobacco production plan and the second production strategy prediction model corresponding to the current production workshop where the current production equipment is located, determine the third tobacco production plan corresponding to the current production workshop within the future time range.

[0061] Among them, the current production workshop is the production workshop where the current production equipment is located among at least one production workshop. The third tobacco production plan can be used to represent the quantity of tobacco planned to be produced by the current production workshop within the future time range. The third tobacco production plan is the model output of the second production strategy prediction model.

[0062] In this embodiment, after obtaining the second tobacco production plan output by the first production strategy prediction model, the second tobacco production plan can be input into the second production strategy prediction model corresponding to the current production workshop. To make the finally obtained third tobacco production plan fit the production capacity of the current production workshop, the production - related data associated with the current production workshop can also be input into the second production strategy prediction model. Then, according to the second production strategy prediction model, the second tobacco production plan and the production - related data associated with the current production workshop are processed to obtain the third tobacco production plan corresponding to the current production workshop within the future time range.

[0063] Optionally, determining the third tobacco production plan corresponding to the current production workshop within the future time range according to the second tobacco production plan and the second production strategy prediction model corresponding to the current production workshop where the current production equipment is located includes: obtaining the first workshop production - related data corresponding to the current production workshop within the first historical time range, and the second workshop production - related data corresponding to other production workshops except the current production workshop among at least one production workshop; according to the second production strategy prediction model, processing the first workshop production - related data, the second workshop production - related data, and the second tobacco production plan to obtain the third tobacco production plan corresponding to the current production workshop within the future time range.

[0064] Among them, the production-related data of the first workshop is used to characterize the tobacco production efficiency of the current production workshop in the first historical time range. The production-related data of the first workshop may include multiple data that can evaluate the tobacco production efficiency of the current production workshop. Optionally, it includes tobacco production cost, delivery delay rate, production capacity utilization rate, etc. The production-related data of the second workshop can be used to characterize the tobacco production efficiency of other production workshops in the first historical time range. In this embodiment, the production-related data of the first workshop, the production-related data of the second workshop, and the second tobacco production plan are the data included in the state space of the second production strategy prediction model; the third tobacco production plan is the data included in the action space of the second production strategy prediction model. The current production workshop can be regarded as an agent, and the second production strategy prediction model can be regarded as the environmental model where the agent is located. Through the interaction between the agent and the environmental model, the prediction of the third tobacco production plan can be realized.

[0065] In a specific implementation, for at least one production device, when determining the current production workshop where the current production device is located, other production workshops except the current production workshop among the at least one production workshop included in the first production object can be determined. Further, the first workshop identifier of the current production workshop and the second workshop identifier of other production workshops can be determined, and the production-related data of the current production workshop corresponding to the first historical time range can be retrieved from the pre-constructed database according to the first workshop identifier, and the production-related data of other production workshops corresponding to the first historical time range can be retrieved from the database according to the second workshop identifier. Further, the production-related data of the first workshop, the production-related data of the second workshop, and the second tobacco production plan can be input into the second production strategy prediction model corresponding to the current production workshop. Furthermore, the production-related data of the first workshop, the production-related data of the second workshop, and the second tobacco production plan can be processed according to the second production strategy prediction model, and the third tobacco production plan corresponding to the current production workshop in the future time range can be obtained.

[0066] S250. According to the third tobacco production plan and the third production strategy prediction model corresponding to the current production device, determine the first tobacco production plan corresponding to the current production device in the future time range, so that the current production device conducts tobacco production according to the first tobacco production plan.

[0067] In this embodiment, after obtaining the third tobacco production plan output by the second production strategy prediction model, the third tobacco production plan can be input into the third production strategy prediction model corresponding to the current production equipment. In order to make the finally obtained first tobacco production plan conform to the production capacity of the current production equipment, the production correlation data associated with the current production equipment can also be input into the third production strategy prediction model. Furthermore, the third production strategy prediction model can process the third tobacco production plan and the production correlation data associated with the current production equipment to obtain the first tobacco production plan corresponding to the current production equipment in the future time range.

[0068] Optionally, determining the first tobacco production plan corresponding to the current production equipment in the future time range according to the third tobacco production plan and the third production strategy prediction model corresponding to the current production equipment includes: obtaining the first equipment production correlation data corresponding to the current production equipment in the first historical time range, and the second equipment production correlation data corresponding to other production equipment except the current production equipment in at least one production equipment; processing the first equipment production correlation data, the second equipment production correlation data and the third tobacco production plan according to the third production strategy prediction model to obtain the first tobacco production plan corresponding to the current production equipment in the future time range.

[0069] Among them, the first equipment production correlation data is used to characterize the tobacco production efficiency of the current production equipment in the first historical time range. The first equipment production correlation data may include multiple data that can evaluate the tobacco production efficiency of the current production equipment. Optionally, it includes tobacco production cost, delivery delay rate, production capacity utilization rate, etc. The second equipment production correlation data can be used to characterize the tobacco production efficiency of other production equipment in the first historical time range. In this embodiment, the first equipment production correlation data, the second equipment production correlation data and the third tobacco production plan are the data included in the state space of the third production strategy prediction model; the first tobacco production plan is the data included in the action space of the third production strategy prediction model. The current production equipment can be regarded as an agent, and the third production strategy prediction model can be regarded as the environmental model where the agent is located. Through the interaction between the agent and the environmental model, the prediction of the first tobacco production plan can be realized.

[0070] In a specific implementation, for at least one production device, when determining the current production workshop where the current production device is located, the first device identifier of the current production device and the second device identifiers of other production devices except the current production device among at least one production device subordinate to the current production workshop can be determined. Further, according to the first device identifier, the first device production association data corresponding to the current production device in the first historical time range is retrieved from the pre-constructed database, and according to the second device identifier, the second device production association data corresponding to other production devices in the first historical time range is retrieved from the database. Further, the first device production association data, the second device production association data, and the third tobacco production plan can be input into the third production strategy prediction model corresponding to the current production device. Furthermore, the first device production association data, the second device production association data, and the third tobacco production plan can be processed according to the third production strategy prediction model, and the first tobacco production plan corresponding to the current production device in the future time range can be obtained. Furthermore, the current device can be made to carry out tobacco production according to the first tobacco production plan in the future time range.

[0071] The technical solution of the embodiment of the present invention is to obtain the first object production association data corresponding to the first production object in the first historical time range, and the second object production association data corresponding to the collaborative production object in the first historical time range; further, process the first object production association data and the second object production association data according to the first production strategy prediction model to obtain the second tobacco production plan corresponding to the first production object in the future time range; further, determine the third tobacco production plan corresponding to the current production workshop in the future time range according to the second tobacco production plan and the second production strategy prediction model corresponding to the current production workshop where the current production device is located; further, determine the first tobacco production plan corresponding to the current production device in the future time range according to the third tobacco production plan and the third production strategy prediction model corresponding to the current production device, achieving the effect of formulating a globally optimized production plan based on the upper-layer production strategy prediction model to coordinate and optimize the model output of the lower-layer production strategy prediction model, and achieving the effect of dynamically optimizing the production task allocation of the tobacco supply chain formed by multi-level tobacco cooperative production objects.

[0072] Figure 3 It is a flowchart of a model training method provided by an embodiment of the present invention. This embodiment is applicable to any situation of training a policy network included in a reinforcement learning algorithm. This method can be executed by a model training device, which can be implemented in the form of hardware and / or software, and the model training device can be configured in a terminal and / or a server. As Figure 3 shown, the method includes:

[0073] S310. Obtain a first sample production object and a collaborative sample production object for producing the same batch of tobacco; wherein, the first sample production object includes at least one sample production workshop for producing the same batch of tobacco, and at least one sample production device for producing the same batch of tobacco is deployed in the sample production workshop.

[0074] Among them, the first sample production object can be an object engaged in multiple links related to tobacco, and the multiple links include the tobacco production and processing link. The first sample production object can be the first production object or other tobacco production objects. The collaborative sample production object can be a tobacco production object that collaborates with the first sample production object to produce the same batch of tobacco. The number of collaborative sample production objects can be one or more. The same batch of tobacco produced by the first sample production object and the collaborative sample production object can be one batch or multiple batches of tobacco, and the same batch of tobacco includes at least one category of tobacco. In this embodiment, the first sample production object includes at least one sample production workshop for producing the same batch of tobacco. At least one sample production device for producing the same batch of tobacco is deployed in the sample production workshop.

[0075] In a specific implementation, multiple sample tobacco production objects that need to jointly produce the same batch of tobacco can be obtained. Further, one of the other sample tobacco production objects among the multiple sample tobacco production objects can be used as the first sample production object, and the other sample tobacco production objects except the first sample production object among the multiple sample tobacco production objects can be used as the collaborative sample production objects corresponding to the first sample production object.

[0076] S320. Obtain multiple training samples corresponding to the first sample production object; wherein, the training samples include first sample object production association data corresponding to the first sample tobacco production object in the historical production cycle and second sample object production association data corresponding to the collaborative sample production object in the historical production cycle.

[0077] Among them, the historical production cycle can be any time interval before the current moment. Optionally, the historical production cycle can be the previous year, the previous six months, the previous three months, the previous month, or the previous week, etc. The historical production cycle and the first historical time range can be the same historical time range or different historical time ranges. The first sample object production association data can be used to evaluate the tobacco production efficiency of the first sample production object in the historical production cycle. The first sample object production association data can include multiple data items that can evaluate the tobacco production efficiency of the first sample production object. Optionally, it includes tobacco production cost, delivery delay rate, and production capacity utilization rate, etc. The second sample object production association data can be used to evaluate the tobacco production efficiency of the collaborative sample production object in the historical production cycle. The second sample object production association data can include multiple data items that can evaluate the tobacco production efficiency of the collaborative sample production object. Optionally, it includes tobacco production cost, delivery delay rate, and production capacity utilization rate, etc.

[0078] In a specific implementation, after determining the first sample production object and the collaborative sample production object for producing the same batch of tobacco, the first sample object identifier of the first sample production object and the second sample object identifier of the collaborative sample production object can be determined. Further, the first sample object production association data of the first sample production object in multiple historical production cycles can be retrieved from a pre-constructed database based on the first sample object identifier; and, the second sample object production association data of the collaborative sample production object in multiple historical production cycles can be retrieved from the pre-constructed database based on the second sample object identifier. Further, for multiple historical production cycles, a training sample can be constructed based on the first sample object production association data of the first sample production object in the current historical production cycle and the second sample object production association data of the collaborative sample production object in the current historical production cycle. Furthermore, multiple training samples corresponding to the first sample production object can be obtained.

[0079] S330. Train the first reinforcement learning model corresponding to the first sample production object, the second reinforcement learning model corresponding to at least one sample production workshop, and the third reinforcement learning model corresponding to at least one sample production device in each sample production workshop according to multiple training samples.

[0080] Among them, the first reinforcement learning model can be a neural network model with model parameters as initial values or default values. In this embodiment, the first sample production object can be regarded as an agent, and the first reinforcement learning model can be regarded as the environment where the agent is located. Further, the first reinforcement learning model can be iteratively trained through a reinforcement learning algorithm and a preset object reward function to obtain a first production strategy prediction model corresponding to the first sample production object. The object reward function can be constructed based on the object production constraint conditions and the object production objective function corresponding to the first sample production object.

[0081] Among them, the second reinforcement learning model can be a neural network model with initial or default model parameters. In this embodiment, the first sample production workshop can be used as the agent, and the second reinforcement learning model can be used as the environment where the agent is located. Furthermore, the second reinforcement learning model can be iteratively trained through a reinforcement learning algorithm and a preset workshop reward function to obtain a second production strategy prediction model corresponding to the sample production workshop. The workshop reward function can be constructed based on the workshop production constraint conditions and the workshop production objective function corresponding to the sample production workshop.

[0082] Among them, the third reinforcement learning model can be a neural network model with initial or default model parameters. In this embodiment, the first sample production equipment can be used as the agent, and the third reinforcement learning model can be used as the environment where the agent is located. Furthermore, the third reinforcement learning model can be iteratively trained through a reinforcement learning algorithm and a preset equipment reward function to obtain a third production strategy prediction model corresponding to the sample production equipment. The equipment reward function can be constructed based on the equipment production constraint conditions and the equipment production objective function corresponding to the sample production equipment.

[0083] Among them, the first reinforcement learning model, the second reinforcement learning model, and the third reinforcement learning model are all composed of a state-action value network and a policy network. And the reinforcement learning algorithm used to train the first reinforcement learning model, the second reinforcement learning model, and the third reinforcement learning model can be any reinforcement learning algorithm. Optionally, it includes Q-learning algorithm, Deep Q-Network (DQN) algorithm, policy gradient-based algorithm, model-based algorithm, etc. In this embodiment, the Q-learning algorithm can be used for model training.

[0084] In specific implementation, when obtaining multiple training samples corresponding to the first sample production object, the first reinforcement learning model corresponding to the first sample production object, the second reinforcement learning corresponding to at least one sample production workshop included in the first sample production object, and the third reinforcement learning model corresponding to at least one sample production equipment subordinate to each sample production workshop can be trained according to the multiple training samples. During the model training process, the model parameters in the first reinforcement learning model can be corrected through a reinforcement learning algorithm and an object reward function, and the model parameters in the second reinforcement learning model can be corrected through a reinforcement learning algorithm and a workshop reward function, and the model parameters in the third reinforcement learning model can be corrected through a reinforcement learning algorithm and an equipment reward function.

[0085] S340. When the preset training end condition is met, obtain the first production strategy prediction model corresponding to the first sample production object, the second production strategy prediction model corresponding to the sample production workshop, and the third production strategy prediction model corresponding to the sample production equipment, so as to predict the tobacco production plan based on the first production strategy prediction model, the second production strategy prediction model, and the third production strategy prediction model.

[0086] Among them, the preset training end condition can be the condition for the end of the iterative training of the model set in advance. Optionally, the preset training end condition includes that the training error is less than the preset error, the error change trend tends to be stable, or the current number of iterative training times reaches the preset number threshold, etc.

[0087] In specific implementation, during the model training process, when the preset training end condition is met, for example, when the number of iterative training times reaches the preset number threshold, the training can be stopped, and the first reinforcement learning model obtained at this time is used as the first production strategy prediction model corresponding to the first sample production object, the second reinforcement learning model obtained at this time is used as the second production strategy prediction model corresponding to the corresponding sample production workshop, and the third reinforcement learning model obtained at this time is used as the third production strategy prediction model corresponding to the corresponding sample production equipment. Furthermore, the tobacco production plan of any production object in the future time range can be predicted based on the first production strategy prediction model; and / or, the workshop tobacco production plan of at least one production workshop included under any production object in the future time range can be predicted based on the first production strategy prediction model and the second production strategy prediction model; and / or, the equipment tobacco production plan of at least one production equipment subordinate to at least one production workshop included under any production object in the future time range can be predicted based on the first production strategy prediction model, the second production strategy prediction model, and the third production strategy prediction model.

[0088] The technical solution of the embodiment of the present invention is to obtain a first sample production object and a collaborative sample production object that produce the same batch of tobacco; wherein the first sample production object includes at least one sample production workshop that produces the same batch of tobacco, and at least one sample production equipment for producing the same batch of tobacco is deployed in the sample production workshop; further, a plurality of training samples corresponding to the first sample production object are obtained; wherein the training samples include first sample object production-related data corresponding to the first sample tobacco production object in a historical production cycle and second sample object production-related data corresponding to the collaborative sample production object in a historical production cycle; further, according to the plurality of training samples, a first reinforcement learning model corresponding to the first sample production object, a second reinforcement learning model corresponding to at least one sample production workshop, and at least one sample production equipment under each sample production workshop are trained. A third reinforcement learning model corresponding to the equipment is used for training; further, when the preset training end conditions are met, a first production strategy prediction model corresponding to the first sample production object, a second production strategy prediction model corresponding to the sample production workshop, and a third production strategy prediction model corresponding to the sample production equipment are obtained, so as to predict the tobacco production plan based on the first production strategy prediction model, the second production strategy prediction model, and the third production strategy prediction model, thereby achieving the effect of obtaining production strategy prediction models at the object level, the workshop level, and the equipment level through training samples at the object level, and applying the reinforcement learning algorithm to the tobacco production plan allocation scenario, thereby achieving the effect of collaborative optimization between multi-level production strategy prediction models when predicting the tobacco production plan, so as to obtain the optimal tobacco production plan.

[0089] Figure 4 This is a flow chart of a model training method provided by an embodiment of the present invention. On the basis of the above embodiment, the training process of the reinforcement learning model is further refined. The specific implementation method can refer to the technical solution of this embodiment. Among them, the technical terms that are the same or similar to the above embodiment are not repeated here.

[0090] like Figure 4 As shown, the method includes:

[0091] S410, obtaining a first sample production object and a collaborative sample production object that produce the same batch of tobacco; wherein the first sample production object includes at least one sample production workshop that produces the same batch of tobacco, and at least one sample production device for producing the same batch of tobacco is deployed in the sample production workshop.

[0092] S420, obtaining multiple training samples corresponding to the first sample production object; wherein the training samples include first sample object production-related data corresponding to the first sample tobacco production object in the historical production cycle and second sample object production-related data corresponding to the collaborative sample production object in the historical production cycle.

[0093] S430. For multiple training samples, input the current training sample into the first reinforcement learning model to obtain the object tobacco production plan and the actual object value parameter corresponding to the first sample production object in the current historical production cycle, and process the object tobacco production plan and the first sample object production association data according to the object reward function corresponding to the first reinforcement learning model to determine the first object reward parameter corresponding to the current historical production cycle.

[0094] Among them, the object tobacco production plan can be the production decision action output after inputting the first sample object production association data and the second sample object production association data corresponding to the historical production cycle into the first reinforcement learning model. The actual object value parameter can be the value parameter obtained by evaluating the first sample object production association data, the second sample object production association data, and the object tobacco production plan in the current historical production cycle by the state-action value network. The object reward function can be a function used to measure the goodness or badness of the object tobacco production plan of the first sample production object under a specific state, which helps the first sample production object learn a strategy that can maximize the long-term cumulative reward. In this embodiment, the object reward function is constructed based on the object production constraint conditions and the object production objective function corresponding to the first sample production object. The object production constraint conditions can be used as the penalty term in the object reward function, and the object production objective function can be used as the reward term in the object reward function. Furthermore, minimizing the object production objective function can be transformed into maximizing the reward in reinforcement learning. The object production constraint conditions refer to various factors that restrict and constrain production activities during the production process. Optionally, the object constraint conditions include production capacity constraints, market demand satisfaction constraints, and non-negativity constraints on decision variables, etc. The object production objective function can be used to describe the goal that the first sample production object needs to achieve under the object production constraint conditions. Optionally, the object production objective function includes a production cost objective function, a delivery delay objective function, and a production capacity utilization objective function.

[0095] Exemplarily, the object production constraint conditions can be expressed based on the following formula:

[0096] The production capacity constraint is:

[0097]

[0098] where x ijt represents the quantity of product of tobacco category j produced by the first sample production object i in the historical production cycle t; j represents the tobacco product category; J represents the set of tobacco product categories; p ijt represents the quantity allocated to tobacco product j by the first sample production object i in the historical production cycle t; ε itrepresents the upper limit of the allowable production capacity constraint violation probability; i represents the first sample production object; I represents the set of sample tobacco production objects; t represents the historical production cycle; T represents the set of production cycles.

[0099] The market demand satisfaction constraint is:

[0100]

[0101] Among them, d jt represents the market demand quantity of tobacco category j in the historical production cycle t.

[0102] The non - negative constraint of decision variables is:

[0103]

[0104] The object production objective function can be expressed based on the following formula:

[0105] The production cost objective function is:

[0106]

[0107] Among them, f1 represents the production cost objective function; α t represents the weight coefficient of the production cost objective in the historical production cycle t; c ijt represents the unit production cost of the first sample production object i in the historical production cycle t.

[0108] The delivery delay objective function is:

[0109]

[0110] Among them, f2 represents the delivery delay objective function; α t represents the weight coefficient of the delivery delay objective in the historical production cycle t; δ ijt represents the unit production time of the first sample production object i in the historical production cycle t; τ jt represents the delivery deadline of tobacco category j in the historical production cycle t.

[0111] The production capacity utilization objective function is:

[0112]

[0113] Among them, f3 represents the production capacity utilization objective function; γ t represents the weight coefficient of the production capacity utilization objective in the historical production cycle t; ρ ijt represents the production capacity utilization rate of the first sample production object i in the historical production cycle t; p ijt represents the production capacity allocated to tobacco category j by the first sample production object i.

[0114] Exemplarily, the object reward function can be represented based on the following formula:

[0115]

[0116] where represents the expected value of the unit production cost of the first sample production object i in the historical production cycle t; ξ ijt is a random variable, representing the uncertainty of the unit production cost of the first sample production object i in producing tobacco category j in the historical production cycle t, and it is assumed to follow a normal distribution, that is represents the variance of the unit production cost of the first sample production object i in the historical production cycle t; ω1 represents the weight coefficient of the production capacity constraint penalty term; ω2 represents the weight coefficient of the market demand satisfaction constraint penalty term; ω3 represents the weight coefficient of the non - negativity constraint penalty term of the decision variable.

[0117] In a specific implementation, the object production constraint conditions and the object production objective function corresponding to the first sample production object can be pre - constructed. Further, an object reward function can be constructed based on the object production constraint conditions and the object production objective function. Further, in the case of obtaining multiple training samples corresponding to the first sample production object, for the multiple training samples, the current training sample can be input into the first reinforcement learning model, and based on the first reinforcement learning model, the first sample object production association data and the second sample object production association data are processed to obtain the object tobacco production plan and the actual object value parameters corresponding to the first sample production object in the current historical production cycle. Further, the first sample object production association data for the next historical production cycle can be determined according to the object tobacco production plan and the first sample object production association data. Furthermore, the first sample object production association data and the second sample object production association data for the next historical production cycle can be processed according to the object reward function to obtain the first object reward parameter corresponding to the current historical production cycle.

[0118] Among them, the first object reward parameter can be a numerical parameter used to characterize the degree of reward brought by the object tobacco production plan. The first object reward parameter is an immediate reward. In other words, the first object reward parameter can be understood as an evaluation or reward for the first sample object production association data and the second sample object production association data and their corresponding object tobacco production plans in the current historical production cycle.

[0119] S440. Process the first object reward parameter and the actual object value parameter according to a preset loss function, determine the first loss value corresponding to the current historical production cycle, and correct the model parameters in the first reinforcement learning model based on the first loss value.

[0120] Among them, the preset loss function can be pre-set and used to characterize the degree of difference between the actual output of the action state value network and the reward in the current historical production cycle. The preset loss function can be a temporal difference function including an object reward parameter, a hierarchical reward parameter, and a global reward parameter.

[0121] Optionally, processing the first object reward parameter and the actual object value parameter according to the preset loss function to determine the first loss value corresponding to the current historical production cycle includes: obtaining the second object reward parameter corresponding to the collaborative sample production object in the current historical production cycle; determining the object hierarchical reward parameter corresponding to the current historical production cycle according to the first object reward parameter and the second object reward parameter; obtaining the workshop hierarchical reward parameter corresponding to the sample production workshop in the current historical production cycle and the equipment hierarchical reward parameter corresponding to the sample production equipment in the current historical production cycle, and processing the object hierarchical reward parameter, the workshop hierarchical reward parameter, and the equipment hierarchical reward parameter through mean calculation or weighted average calculation to obtain the global reward parameter corresponding to the first sample production object in the current historical production cycle; processing the first object reward parameter, the actual object value parameter, the object hierarchical reward parameter, and the global reward parameter according to the preset loss function to determine the first loss value corresponding to the current historical production cycle.

[0122] Among them, the second object reward parameter can be used to characterize the immediate reward obtained by the collaborative sample production object in the current historical production cycle. The object hierarchical reward parameter can be used to characterize the average immediate reward of each agent in this level, and this level is the level where the sample tobacco production object is located, and each agent represents each sample tobacco production object producing the same batch of tobacco. The global reward parameter can be used to characterize the average hierarchical reward of the entire level composed of the sample tobacco production object - sample production workshop - sample production equipment.

[0123] In a specific implementation, the second object reward parameter of the collaborative sample production object in the current historical production cycle can be obtained. Further, the first object reward parameter and the second object reward parameter can be processed through mean calculation or weighted mean calculation to obtain the object-level reward parameter corresponding to the current historical production cycle. Further, the workshop-level reward parameter corresponding to the sample production workshop in the current historical production cycle and the equipment-level reward parameter corresponding to the sample production equipment in the current historical production cycle can be obtained, and the object-level reward parameter, the workshop-level reward parameter, and the equipment-level reward parameter can be processed through mean calculation or weighted average calculation to obtain the global reward parameter corresponding to the first sample production object in the current historical production cycle. Further, according to the first object reward parameter, the actual object value parameter, the object-level reward parameter, and the global reward parameter being input into a preset loss function to obtain a loss value, this loss value can be used as the first loss value corresponding to the current historical production cycle.

[0124] Exemplarily, the preset loss function can be represented based on the following formula:

[0125]

[0126] Among them, Q li (s li , a li ) represents the actual object value parameter corresponding to the current historical production cycle; η represents the learning rate; r li represents the first object reward parameter; μ1 represents the weight coefficient of the object-level reward parameter; R li represents the object-level reward parameter; μ2 represents the weight coefficient of the global reward parameter; λ represents the discount factor; represents the maximum Q value of all possible actions a l ′ i for the first sample production object in the next state s ′ li , that is, the training target of the first sample production object in the current historical production cycle.

[0127] S450. For at least one sample production workshop, according to the object tobacco production plan and the second reinforcement learning model corresponding to the current sample production workshop, determine the workshop tobacco production plan and the actual workshop value parameter corresponding to the current sample production workshop in the current historical production cycle, and according to the workshop reward function corresponding to the second reinforcement learning model and the workshop tobacco production plan, determine the first workshop reward parameter corresponding to the current historical production cycle.

[0128] Among them, the workshop tobacco production plan can be the production decision action determined by the second reinforcement learning model based on the workshop state information of the current sample production workshop in the current historical production cycle. The actual workshop value parameter can be the value parameter obtained by the state-action value network evaluating the workshop state information and the workshop tobacco production plan of the current sample production workshop in the current historical production cycle. In this embodiment, the workshop reward function is constructed based on the workshop production constraint conditions and the workshop production objective function corresponding to the sample production workshop. Optionally, the workshop production constraint conditions include production capacity constraint, market demand satisfaction constraint, and non-negativity constraint of decision variables, etc.; the workshop production objective function includes production cost objective function, delivery delay objective function, and production capacity utilization objective function.

[0129] Exemplarily, the workshop production constraint conditions can be expressed based on the following formula:

[0130] The production capacity constraint is:

[0131]

[0132] Among them, x wjt represents the quantity of products of tobacco category j produced by the sample production workshop w in the historical production cycle t; j represents the tobacco product category; J represents the set of tobacco product categories; p wjt represents the quantity allocated to tobacco product j by the sample production workshop w in the historical production cycle t; ε wt represents the upper limit of the allowable probability of violating the production capacity constraint; w represents the sample production workshop; W represents the set of sample production workshops; t represents the historical production cycle; T represents the set of production cycles.

[0133] The market demand satisfaction constraint is:

[0134]

[0135] Among them, d jt represents the market demand quantity of tobacco category j in the historical production cycle t.

[0136] The non-negativity constraint of decision variables is:

[0137]

[0138] The workshop production objective function can be expressed based on the following formula:

[0139] The production cost objective function is:

[0140]

[0141] Among them, f1 represents the production cost objective function; α t represents the weight coefficient of the production cost objective in the historical production cycle t; cwjt represents the unit production cost of the sample production workshop w in the historical production cycle t.

[0142] The delivery delay objective function is:

[0143]

[0144] where f2 represents the delivery delay objective function; α t represents the weight coefficient of the delivery delay objective in the historical production cycle t; δ wjt represents the unit production time of the sample production workshop w in the historical production cycle t; τ jt represents the delivery deadline of the tobacco category j in the historical production cycle t.

[0145] The capacity utilization objective function is:

[0146]

[0147] where f3 represents the capacity utilization objective function; γ t represents the weight coefficient of the capacity utilization objective in the historical production cycle t; ρ wjt represents the capacity utilization rate of the sample production workshop w in the historical production cycle t; p wjt represents the production capacity allocated by the sample production workshop w to the tobacco category j.

[0148] Exemplarily, the workshop reward function can be expressed based on the following formula:

[0149]

[0150] where, represents the expected value of the unit production cost of the sample production workshop w in the historical production cycle t; ξ wjt is a random variable, representing the uncertainty of the unit production cost of the sample production workshop w in producing the tobacco category j in the historical production cycle t, and it is assumed to follow a normal distribution, that is represents the variance of the unit production cost of the sample production workshop w in the historical production cycle t; ψ1 represents the weight coefficient of the production capacity constraint penalty term; ψ2 represents the weight coefficient of the market demand satisfaction constraint penalty term; ψ3 represents the weight coefficient of the decision variable non - negativity constraint penalty term.

[0151] In this embodiment, after obtaining the object tobacco production plan output by the first reinforcement learning model, the object tobacco production plan can be input into the second reinforcement learning model corresponding to the current sample production workshop. In order to make the finally obtained workshop tobacco production plan fit the production capacity of the current sample production workshop, the production correlation data associated with the current sample production workshop can also be input into the second reinforcement learning model. Furthermore, the second reinforcement learning model can process the object tobacco production plan and the production correlation data associated with the current sample production workshop to obtain the workshop tobacco production plan corresponding to the current sample production workshop in the current historical production cycle.

[0152] Optionally, according to the object tobacco production plan and the second reinforcement learning model corresponding to the current sample production workshop, determining the workshop tobacco production plan and the actual workshop value parameters corresponding to the current sample production workshop in the current historical production cycle includes: obtaining the first sample workshop production correlation data corresponding to the current sample production workshop in the current historical production cycle and the second sample workshop production correlation data corresponding to other sample production workshops except the current sample production workshop in the current historical production cycle; processing the object tobacco production plan, the first sample workshop production correlation data, and the second sample workshop production correlation data according to the second reinforcement learning model corresponding to the current sample production workshop to obtain the workshop tobacco production plan and the actual workshop value parameters corresponding to the current sample production workshop in the current historical production cycle.

[0153] Among them, the first sample workshop production correlation data can be used to evaluate the tobacco production efficiency of the current sample production workshop in the historical production cycle. The first sample workshop production correlation data may include multiple data that can evaluate the tobacco production efficiency of the current sample production workshop. Optionally, it includes tobacco production cost, delivery delay rate, and production capacity utilization rate, etc. The second sample workshop production correlation data can be used to evaluate the tobacco production efficiency of other sample production workshops in the historical production cycle. The second sample workshop production correlation data may include multiple data that can evaluate the tobacco production efficiency of other sample production workshops. Optionally, it includes tobacco production cost, delivery delay rate, and production capacity utilization rate, etc.

[0154] In a specific implementation, the workshop production constraint conditions and the workshop production objective function corresponding to the sample production workshop can be pre-constructed. Further, the workshop reward function can be constructed based on the workshop production constraint conditions and the workshop production objective function. Further, for multiple training samples, when obtaining the object tobacco production plan corresponding to the current training sample, for at least one sample production workshop included under the first sample production object, the first sample workshop production correlation data corresponding to the current sample production workshop in the current historical production cycle and the second sample workshop production correlation data corresponding to other sample production workshops except the current sample production workshop among at least one sample production workshop in the current historical production cycle can be obtained. Further, the first sample workshop production correlation data, the second sample workshop production correlation data, and the object tobacco production plan can be input into the second reinforcement learning model, and the input data can be processed based on the second reinforcement learning model to obtain the workshop tobacco production plan and the actual workshop value parameters corresponding to the current sample production workshop in the current historical production cycle. Further, the first sample workshop production correlation data for the next historical production cycle can be determined according to the workshop tobacco production plan and the first sample workshop production correlation data. Furthermore, the first sample workshop production correlation data and the second sample workshop production correlation data for the next historical production cycle can be processed according to the workshop reward function to obtain the first workshop reward parameter corresponding to the current historical production cycle.

[0155] Among them, the first workshop reward parameter can be a numerical parameter used to characterize the reward degree brought by the workshop tobacco production plan. The first workshop reward parameter is an immediate reward. In other words, the first workshop reward parameter can be understood as an evaluation or reward made for the first sample workshop production correlation data and the second sample workshop production correlation data and their corresponding workshop tobacco production plans in the current historical production cycle.

[0156] S460. Determine the second loss value corresponding to the current historical production cycle according to the preset loss function for the first workshop reward parameter and the actual workshop value parameter, and correct the model parameters in the second reinforcement learning model based on the second loss value.

[0157] Optionally, determine a second loss value corresponding to the current historical production cycle according to a preset loss function for the first workshop reward parameter and the actual workshop value parameter, including: obtaining a second workshop reward parameter corresponding to other sample production workshops in the current historical production cycle; determining a workshop-level reward parameter corresponding to the current historical production cycle according to the first workshop reward parameter and the second workshop reward parameter; obtaining an object-level reward parameter corresponding to the first sample production object in the current historical production cycle and an equipment-level reward parameter corresponding to the sample production equipment in the current historical production cycle, and processing the object-level reward parameter, the workshop-level reward parameter, and the equipment-level reward parameter through mean calculation or weighted average calculation to obtain a global reward parameter corresponding to the first sample production object in the current historical production cycle; processing the first workshop reward parameter, the actual workshop value parameter, the workshop-level reward parameter, and the global reward parameter according to the preset loss function to determine a second loss value corresponding to the current historical production cycle.

[0158] Among them, the second workshop reward parameter can be used to represent the immediate reward obtained by other sample production workshops in the current historical production cycle. The workshop-level reward parameter can be used to represent the average immediate reward situation of each agent in this level, this level is the level where the sample production workshop is located, and each agent represents each sample production workshop that produces the same batch of tobacco under the first sample production object.

[0159] In specific implementation, the second workshop reward parameter of other sample production workshops in the current historical production cycle can be obtained. Further, the first workshop reward parameter and the second workshop reward parameter can be processed through mean calculation or weighted mean calculation to obtain a workshop-level reward parameter corresponding to the current historical production cycle. Further, the object-level reward parameter corresponding to the first sample production object in the current historical production cycle and the equipment-level reward parameter corresponding to the sample production equipment in the current historical production cycle can be obtained, and the object-level reward parameter, the workshop-level reward parameter, and the equipment-level reward parameter can be processed through mean calculation or weighted average calculation to obtain a global reward parameter corresponding to the first sample production object in the current historical production cycle. Further, according to inputting the first workshop reward parameter, the actual workshop value parameter, the workshop-level reward parameter, and the global reward parameter into the preset loss function to obtain a loss value, this loss value can be used as the second loss value corresponding to the current historical production cycle.

[0160] Exemplarily, the preset loss function can be expressed based on the following formula:

[0161]

[0162] Among them, Q lw (s lw ,a lw) represents the actual workshop value parameter corresponding to the current historical production cycle; η represents the learning rate; r lw represents the first workshop reward parameter; μ3 represents the weight coefficient of the workshop-level reward parameter; R lw represents the workshop-level reward parameter; μ2 represents the weight coefficient of the global reward parameter; λ represents the discount factor; represents the sample production workshop in the next state s l ′ w under all possible actions a ′ lw the maximum value among the Q values, that is, the training objective of the sample production workshop in the current historical production cycle.

[0163] S470. For at least one sample production equipment corresponding to the current sample production workshop, according to the workshop tobacco production plan and the third reinforcement learning model corresponding to the current sample production equipment, determine the equipment tobacco production plan and the actual equipment value parameter corresponding to the current sample production equipment in the current historical production cycle, and determine the first equipment reward parameter corresponding to the current historical production cycle according to the preset equipment reward function and the equipment tobacco production plan.

[0164] Among them, the equipment tobacco production plan can be a production decision action determined by the third reinforcement learning model based on the equipment state information of the current sample production equipment in the current historical production cycle. The actual equipment value parameter can be a value parameter obtained after the state-action value network evaluates the equipment state information and the equipment tobacco production plan of the current sample production equipment in the current historical production cycle. In this embodiment, the equipment reward function is constructed based on the equipment production constraint conditions and the equipment production objective function corresponding to the sample production equipment. Optionally, the equipment production constraint conditions include task completion time constraints and production capacity constraints, etc.; the equipment production objective functions include production cost objective functions, delivery delay objective functions, and production capacity utilization objective functions.

[0165] Exemplarily, the workshop reward function can be expressed based on the following formula:

[0166]

[0167] Among them, λ t represents the weight coefficient of the production cost objective; represents the expected value of the unit production cost of the sample production equipment e in the historical production cycle t; ξ et is a random variable, representing the uncertainty of the unit production cost of the sample production workshop w in producing tobacco products in the historical production cycle t, and it is assumed to follow a normal distribution, that is represents the variance of the unit production cost of the sample production equipment e in the historical production cycle t; μt denotes the weight coefficient of the delivery delay target; K denotes the set of tobacco production tasks; f ekt denotes the time required for the sample production equipment e to complete the tobacco production task k in the historical production cycle t; d kt denotes the expected completion time of the tobacco production task k in the historical production cycle t; ν t denotes the weight coefficient of the production capacity utilization target; p et denotes the available time of the sample production equipment e in the historical production cycle t; φ1 denotes the weight coefficient of the production capacity constraint condition; ε et denotes the upper limit of the constraint violation probability allowed at the equipment level; φ2 denotes the weight coefficient of the task completion time constraint condition. P() denotes calculating the probability.

[0168] In this embodiment, after obtaining the workshop tobacco production plan output by the second reinforcement learning model, the workshop tobacco production plan can be input into the third reinforcement learning model corresponding to the current sample production equipment. In order to make the finally obtained equipment tobacco production plan conform to the production capacity of the current sample production equipment, the production association data associated with the current sample production equipment can also be input into the third reinforcement learning model. Furthermore, the workshop tobacco production plan and the production association data associated with the current sample production equipment can be processed according to the third reinforcement learning model to obtain the equipment tobacco production plan corresponding to the current sample production equipment in the current historical production cycle.

[0169] Optionally, according to the workshop tobacco production plan and the third reinforcement learning model corresponding to the current sample production equipment, determining the equipment tobacco production plan and the actual equipment value parameters corresponding to the current sample production equipment in the current historical production cycle includes: obtaining the first sample equipment production association data corresponding to the current sample production equipment in the current historical production cycle and the second sample equipment production association data corresponding to at least one other sample production equipment except the current sample production equipment in the current historical production cycle; processing the workshop tobacco production plan, the first sample equipment production association data, and the second sample equipment production association data according to the third reinforcement learning model corresponding to the current sample production equipment to obtain the equipment tobacco production plan and the actual equipment value parameters corresponding to the current sample production equipment in the current historical production cycle.

[0170] Among them, the first sample device production - related data can be used to evaluate the tobacco production efficiency of the current sample production device in the historical production cycle. The first sample device production - related data may include multiple data items that can evaluate the tobacco production efficiency of the current sample production device. Optionally, it includes tobacco production cost, delivery delay rate, production capacity utilization rate, etc. The second sample device production - related data can be used to evaluate the tobacco production efficiency of other sample production devices in the historical production cycle. The second sample workshop production - related data may include multiple data items that can evaluate the tobacco production efficiency of other sample production devices. Optionally, it includes tobacco production cost, delivery delay rate, production capacity utilization rate, etc.

[0171] In specific implementation, the device production constraint conditions and device production objective functions corresponding to the sample production device can be constructed in advance. Further, a device reward function can be constructed based on the device production constraint conditions and device production objective functions. Further, for multiple training samples, when the workshop tobacco production plan corresponding to the current sample production workshop is obtained, for at least one sample production device corresponding to the current sample production workshop, the first sample device production - related data corresponding to the current sample production device in the current historical production cycle and the second sample device production - related data corresponding to other sample production devices except the current sample production device among at least one sample production device in the current historical production cycle can be obtained. Further, the first sample device production - related data, the second sample device production - related data, and the workshop tobacco production plan can be input into the third reinforcement learning model, and the input data can be processed based on the third reinforcement learning model to obtain the device tobacco production plan and the actual device value parameters corresponding to the current sample production device in the current historical production cycle. Further, the first sample device production - related data for the next historical production cycle can be determined according to the device tobacco production plan and the first sample device production - related data. Furthermore, the first sample device production - related data and the second sample device production - related data for the next historical production cycle can be processed according to the device reward function to obtain the first device reward parameter corresponding to the current historical production cycle.

[0172] Among them, the first device reward parameter can be a numerical parameter used to characterize the degree of reward brought by the device tobacco production plan. The first device reward parameter is an immediate reward. In other words, the first device reward parameter can be understood as an evaluation or reward for the first sample device production - related data, the second sample device production - related data, and their corresponding device tobacco production plan in the current historical production cycle.

[0173] S480: Determine the third loss value corresponding to the current historical production cycle according to the preset loss function for the actual device value parameter and the first device reward parameter, and correct the model parameters in the third reinforcement learning model based on the third loss value.

[0174] Optionally, determine a third loss value corresponding to the current historical production cycle according to a preset loss function for the actual equipment value parameter and the first equipment reward parameter, including: obtaining a second equipment reward parameter corresponding to other sample production equipment in the current historical production cycle; determining an equipment level reward parameter corresponding to the current historical production cycle according to the first equipment reward parameter and the second equipment reward parameter; obtaining an object level reward parameter corresponding to the first sample production object in the current historical production cycle and a workshop level reward parameter corresponding to the sample production workshop in the current historical production cycle, and processing the object level reward parameter, the workshop level reward parameter and the equipment level reward parameter through mean calculation or weighted average calculation to obtain a global reward parameter corresponding to the first sample production object in the current historical production cycle; processing the first equipment reward parameter, the actual equipment value parameter, the equipment level reward parameter and the global reward parameter according to the preset loss function to determine a third loss value corresponding to the current historical production cycle.

[0175] Among them, the third workshop reward parameter can be used to represent the immediate reward obtained by other sample production equipment in the current historical production cycle. The equipment level reward parameter can be used to represent the average immediate reward situation of each agent in this level, this level is the level where the sample production equipment is located, and each agent represents each sample production equipment used to produce the same batch of tobacco under the current sample production workshop.

[0176] In a specific implementation, a second equipment reward parameter corresponding to other sample production equipment in the current historical production cycle can be obtained. Further, the first equipment reward parameter and the second equipment reward parameter can be processed through mean calculation or weighted mean calculation to obtain an equipment level reward parameter corresponding to the current historical production cycle. Further, an object level reward parameter corresponding to the first sample production object in the current historical production cycle and a workshop level reward parameter corresponding to the sample production workshop in the current historical production cycle can be obtained, and the object level reward parameter, the workshop level reward parameter and the equipment level reward parameter can be processed through mean calculation or weighted average calculation to obtain a global reward parameter corresponding to the first sample production object in the current historical production cycle. Further, according to the first equipment reward parameter, the actual equipment value parameter, the equipment level reward parameter and the global reward parameter are input into the preset loss function to obtain a loss value, and this loss value can be used as the third loss value corresponding to the current historical production cycle.

[0177] Exemplarily, the preset loss function can be represented based on the following formula:

[0178]

[0179] Among them, Q le (sle , a le ) represents the actual equipment value parameter corresponding to the current historical production cycle; η represents the learning rate; r le represents the first equipment reward parameter; μ4 represents the weight coefficient of the equipment level reward parameter; R le represents the equipment level reward parameter; μ2 represents the weight coefficient of the global reward parameter; λ represents the discount factor; represents the maximum value among the Q-values of all possible actions a l ′ e in the next state s of the sample production equipment, that is, the training objective of the sample production equipment in the current historical production cycle. ′ le The maximum value among the Q-values of all possible actions a

[0180] S490. When the preset training end condition is reached, obtain the first production strategy prediction model corresponding to the first sample production object, the second production strategy prediction model corresponding to the sample production workshop, and the third production strategy prediction model corresponding to the sample production equipment, so as to predict the tobacco production plan based on the first production strategy prediction model, the second production strategy prediction model, and the third production strategy prediction model.

[0181] In the technical solution of the embodiment of the present invention, for multiple training samples, the current training sample is input into the first reinforcement learning model to obtain the object tobacco production plan and the actual object value parameters corresponding to the current historical production cycle of the first sample production object, and the object tobacco production plan and the first sample object production association data are processed according to a preset object reward function to determine the object reward parameters corresponding to the current historical production cycle; wherein, the object reward function is constructed based on the object production constraint conditions and the object production objective function corresponding to the first sample production object; further, the object reward parameters and the actual object value parameters are processed according to a preset loss function to determine the first loss value corresponding to the current historical production cycle, and the model parameters in the first reinforcement learning model are corrected based on the first loss value; and for at least one sample production workshop, according to the object tobacco production plan and the second reinforcement learning model corresponding to the current sample production workshop, the workshop tobacco production plan and the actual workshop value parameters corresponding to the current historical production cycle of the current sample production workshop are determined, and the first workshop reward parameters corresponding to the current historical production cycle are determined according to the preset workshop reward function and the workshop tobacco production plan; wherein, the workshop reward function is constructed based on the workshop production constraint conditions and the workshop production objective function corresponding to the sample production workshop; further, the actual workshop value parameters and the first workshop reward parameters are processed according to a preset loss function to determine the second loss value corresponding to the historical production cycle, and the model parameters in the second reinforcement learning model are corrected based on the second loss value; and for at least one sample production device corresponding to the current sample production workshop, according to the workshop tobacco production plan and the third reinforcement learning model corresponding to the current sample production device, the device tobacco production plan and the actual device value parameters corresponding to the current historical production cycle of the current sample production device are determined, and the first device reward parameters corresponding to the current historical production cycle are determined according to the preset device reward function and the device tobacco production plan; wherein, the device reward function is constructed based on the device production constraint conditions and the device production objective function corresponding to the sample production device; further, the actual device value parameters and the first device reward parameters are processed according to a preset loss function to determine the third loss value corresponding to the current historical production cycle, and the model parameters in the third reinforcement learning model are corrected based on the third loss value, realizing the combination of the reinforcement learning algorithm and the multi-objective optimization algorithm, so that the finally obtained tobacco production plan can meet the product demand and the enterprise production capacity constraint while minimizing the production cost, the delivery delay rate and the deviation of the production capacity balance degree.

[0182] Figure 5 It is a schematic structural diagram of a production task allocation device provided by an embodiment of the present invention. As Figure 5 shown, the device includes: an object acquisition module 510 and a production plan determination module 520.

[0183] Among them, the object acquisition module 510 is used to acquire the first production object and the collaborative production object for producing the same batch of tobacco. Among them, the first production object includes at least one production workshop for producing the same batch of tobacco, and at least one production device for producing the same batch of tobacco is deployed in the production workshop. The production plan determination module 520 is used to, for at least one production device, according to the first production strategy prediction model corresponding to the first production object, the second production strategy prediction model corresponding to the current production workshop where the current production device is located, the third production strategy prediction model corresponding to the current production device, the first object production association data corresponding to the first production object in the first historical time range, and the second object production association data corresponding to the collaborative production object in the first historical time range, determine the first tobacco production plan corresponding to the current production device in the future time range, so that the current production device produces tobacco according to the first tobacco production plan. Among them, the first object production association data and the second object production association data are the model inputs of the first production strategy prediction model, the model output of the first production strategy prediction model is part of the model inputs of the second production strategy prediction model, and the model output of the second production strategy prediction model is part of the model inputs of the third production strategy model. The first object production association data is used to evaluate the tobacco production efficiency of the first production object within the first historical time range. The first tobacco production plan is used to represent the quantity of tobacco planned to be produced by the current production device within the future time range.

[0184] The technical solution of the embodiment of the present invention provides a data acquisition basis for subsequent distribution of tobacco production tasks by acquiring the first production object and the collaborative production object for producing the same batch of tobacco. Among them, the first production object includes at least one production workshop for producing the same batch of tobacco, and at least one production device for producing the same batch of tobacco is deployed in the production workshop, and reflects the application scenario of multiple production objects collaborating to produce the same batch of tobacco. Further, for at least one production device, according to the first production strategy prediction model corresponding to the first production object, the second production strategy prediction model corresponding to the current production workshop where the current production device is located, the third production strategy prediction model corresponding to the current production device, the first object production association data corresponding to the first production object, and the second object production association data corresponding to the collaborative production object, determine the first tobacco production plan corresponding to the current production device in the future time range, so that the current production device produces tobacco according to the first tobacco production plan, achieving the effect of determining the tobacco production plan based on the production strategy prediction models at the object level, workshop level, and equipment level in sequence, realizing the application of distributed collaborative optimization at multiple levels when formulating the tobacco production plan to obtain the optimal tobacco production plan, and being able to better handle the dynamically changing tobacco production supply chain environment based on the production strategy prediction model, with stronger adaptability and robustness.

[0185] Optionally, the production plan determination module 520 includes: a data acquisition unit, a second tobacco production plan determination unit, a third tobacco production plan determination unit, and a first tobacco production plan determination unit. Among them, the data acquisition unit is configured to acquire the first object production correlation data corresponding to the first production object in the first historical time range, and the second object production correlation data corresponding to the collaborative production object in the first historical time range; the second tobacco production plan determination unit is configured to process the first object production correlation data and the second object production correlation data according to the first production strategy prediction model to obtain the second tobacco production plan corresponding to the first production object in the future time range; the third tobacco production plan determination unit is configured to determine the third tobacco production plan corresponding to the current production workshop in the future time range according to the second tobacco production plan and the second production strategy prediction model corresponding to the current production workshop where the current production equipment is located; the first tobacco production plan determination unit is configured to determine the first tobacco production plan corresponding to the current production equipment in the future time range according to the third tobacco production plan and the third production strategy prediction model corresponding to the current production equipment.

[0186] Optionally, the third tobacco production plan determination unit includes: a data acquisition subunit and a third tobacco production plan determination subunit. Among them, the data acquisition subunit is configured to acquire the first workshop production correlation data corresponding to the current production workshop in the first historical time range, and the second workshop production correlation data corresponding to other production workshops except the current production workshop among at least one production workshop; wherein, the first workshop production correlation data is used to characterize the tobacco production efficiency of the current production workshop in the first historical time range; the third tobacco production plan determination subunit is configured to process the first workshop production correlation data, the second workshop production correlation data, and the second tobacco production plan according to the second production strategy prediction model to obtain the third tobacco production plan corresponding to the current production workshop in the future time range.

[0187] Optionally, the first tobacco production plan determination unit includes: a data acquisition subunit and a first tobacco production plan determination subunit. Among them, the data acquisition subunit is configured to acquire the first equipment production correlation data corresponding to the current production equipment in the first historical time range, and the second equipment production correlation data corresponding to other production equipment except the current production equipment among at least one production equipment; wherein, the first equipment production correlation data is used to characterize the tobacco production efficiency of the current production equipment in the first historical time range; the first tobacco production plan determination subunit is configured to process the first equipment production correlation data, the second equipment production correlation data, and the third tobacco production plan according to the third production strategy prediction model to obtain the first tobacco production plan corresponding to the current production equipment in the future time range.

[0188] The production task allocation device provided by the embodiments of the present invention can execute the production task allocation method provided by any embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the method.

[0189] Figure 6 It is a schematic structural diagram of a model training device provided by an embodiment of the present invention. As Figure 6 shown, the device includes: a sample object acquisition module 610, a training sample acquisition module 620, a model training module 630, and a policy prediction model determination module 640.

[0190] Among them, the sample object acquisition module 610 is used to acquire a first sample production object and a collaborative sample production object for producing the same batch of tobacco; wherein, the first sample production object includes at least one sample production workshop for producing the same batch of tobacco, and at least one sample production device for producing the same batch of tobacco is deployed in the sample production workshop; the training sample acquisition module 620 is used to acquire a plurality of training samples corresponding to the first sample production object; wherein, the training samples include first sample object production association data corresponding to the first sample tobacco production object in the historical production cycle and second sample object production association data corresponding to the collaborative sample production object in the historical production cycle; the model training module 630 is used to train a first reinforcement learning model corresponding to the first sample production object, a second reinforcement learning model corresponding to at least one sample production workshop, and a third reinforcement learning model corresponding to at least one sample production device under each sample production workshop according to the plurality of training samples; the policy prediction model determination module 640 is used to obtain a first production policy prediction model corresponding to the first sample production object, a second production policy prediction model corresponding to the sample production workshop, and a third production policy prediction model corresponding to the sample production device when the preset training end condition is reached, so as to predict the tobacco production plan based on the first production policy prediction model, the second production policy prediction model, and the third production policy prediction model.

[0191] The technical solution of the embodiment of the present invention includes obtaining a first sample production object and a collaborative sample production object for producing the same batch of tobacco; wherein, the first sample production object includes at least one sample production workshop for producing the same batch of tobacco, and at least one sample production device for producing the same batch of tobacco is deployed in the sample production workshop; further, obtaining a plurality of training samples corresponding to the first sample production object; wherein, the training samples include first sample object production association data corresponding to the first sample tobacco production object in the historical production cycle and second sample object production association data corresponding to the collaborative sample production object in the historical production cycle; further, training a first reinforcement learning model corresponding to the first sample production object, a second reinforcement learning model corresponding to at least one sample production workshop, and a third reinforcement learning model corresponding to at least one sample production device under each sample production workshop according to the plurality of training samples; further, when the preset training end condition is reached, obtaining a first production strategy prediction model corresponding to the first sample production object, a second production strategy prediction model corresponding to the sample production workshop, and a third production strategy prediction model corresponding to the sample production device, so as to predict the tobacco production plan based on the first production strategy prediction model, the second production strategy prediction model, and the third production strategy prediction model, achieving the effect of training production strategy prediction models at the object level, workshop level, and equipment level based on training samples at the object level, and the effect of applying the reinforcement learning algorithm to the tobacco production plan allocation scenario. Furthermore, the collaborative optimization between multi-level production strategy prediction models is realized during the prediction of the tobacco production plan to obtain the optimal tobacco production plan.

[0192] Optionally, the model training module 630 includes: an object tobacco production plan determination sub-module, a first loss value determination sub-module, a workshop tobacco production plan determination sub-module, a second loss value determination sub-module, an equipment tobacco production plan determination sub-module, and a third loss value determination sub-module.

[0193] The object tobacco production plan determination sub-module is configured to input the current training sample into the first reinforcement learning model for a plurality of training samples, obtain the object tobacco production plan and the actual object value parameters corresponding to the first sample production object in the current historical production cycle, and process the object tobacco production plan and the first sample object production association data according to a preset object reward function to determine the object reward parameters corresponding to the current historical production cycle; wherein, the object reward function is constructed based on the object production constraint conditions and the object production objective function corresponding to the first sample production object.

[0194] The first loss value determination sub-module is used to process the object reward parameter and the actual object value parameter according to a preset loss function, determine the first loss value corresponding to the current historical production cycle, and correct the model parameters in the first reinforcement learning model based on the first loss value; and,

[0195] The workshop tobacco production plan determination sub-module is used to, for at least one sample production workshop, determine the workshop tobacco production plan and the actual workshop value parameter corresponding to the current sample production workshop in the current historical production cycle according to the object tobacco production plan and the second reinforcement learning model corresponding to the current sample production workshop, and determine the first workshop reward parameter corresponding to the current historical production cycle according to the preset workshop reward function and the workshop tobacco production plan; wherein, the workshop reward function is constructed based on the workshop production constraint conditions and the workshop production objective function corresponding to the sample production workshop;

[0196] The second loss value determination sub-module is used to determine the second loss value corresponding to the historical production cycle according to the preset loss function for the actual workshop value parameter and the first workshop reward parameter, and correct the model parameters in the second reinforcement learning model based on the second loss value; and,

[0197] The equipment tobacco production plan determination sub-module is used to, for at least one sample production equipment corresponding to the current sample production workshop, determine the equipment tobacco production plan and the actual equipment value parameter corresponding to the current sample production equipment in the current historical production cycle according to the workshop tobacco production plan and the third reinforcement learning model corresponding to the current sample production equipment, and determine the first equipment reward parameter corresponding to the current historical production cycle according to the preset equipment reward function and the equipment tobacco production plan; wherein, the equipment reward function is constructed based on the equipment production constraint conditions and the equipment production objective function corresponding to the sample production equipment;

[0198] The third loss value determination sub-module is used to determine the third loss value corresponding to the current historical production cycle according to the preset loss function for the actual equipment value parameter and the first equipment reward parameter, and correct the model parameters in the third reinforcement learning model based on the third loss value.

[0199] Optionally, the first loss value determination sub-module includes: an object reward parameter acquisition unit, a hierarchical reward parameter determination unit, a global reward parameter determination unit, and a first loss value determination unit.

[0200] The object reward parameter acquisition unit is used to acquire the second object reward parameter corresponding to the collaborative sample production object in the current historical production cycle;

[0201] A hierarchical reward parameter determination unit, configured to determine an object hierarchical reward parameter corresponding to the current historical production cycle according to a first object reward parameter and a second object reward parameter;

[0202] A global reward parameter determination unit, configured to obtain a workshop hierarchical reward parameter corresponding to the current historical production cycle of the sample production workshop and an equipment hierarchical reward parameter corresponding to the current historical production cycle of the sample production equipment, and process the object hierarchical reward parameter, the workshop hierarchical reward parameter, and the equipment hierarchical reward parameter through mean calculation or weighted average calculation to obtain a global reward parameter corresponding to the first sample production object in the current historical production cycle;

[0203] A first loss value determination unit, configured to process the first object reward parameter, the actual object value parameter, the object hierarchical reward parameter, and the global reward parameter according to a preset loss function to determine a first loss value corresponding to the current historical production cycle.

[0204] Optionally, the workshop tobacco production plan determination sub-module includes: a data acquisition unit and a workshop tobacco production plan determination unit. Among them, the data acquisition unit is configured to acquire first sample workshop production association data corresponding to the current sample production workshop in the current historical production cycle and second sample workshop production association data corresponding to at least one other sample production workshop except the current sample production workshop in the current historical production cycle; the workshop tobacco production plan determination unit is configured to process the object tobacco production plan, the first sample workshop production association data, and the second sample workshop production association data according to a second reinforcement learning model corresponding to the current sample production workshop to obtain a workshop tobacco production plan corresponding to the current sample production workshop in the current historical production cycle and an actual workshop value parameter.

[0205] Optionally, the equipment tobacco production plan determination sub-module includes: a data acquisition unit and an equipment tobacco production plan determination unit. Among them, the data acquisition unit is configured to acquire first sample equipment production association data corresponding to the current sample production equipment in the current historical production cycle and second sample equipment production association data corresponding to at least one other sample production equipment except the current sample production equipment in the current historical production cycle; the equipment tobacco production plan determination unit is configured to process the workshop tobacco production plan, the first sample equipment production association data, and the second sample equipment production association data according to a third reinforcement learning model corresponding to the current sample production equipment to obtain an equipment tobacco production plan corresponding to the current sample production equipment in the current historical production cycle and an actual equipment value parameter.

[0206] The model training device provided by the embodiments of the present invention can execute the model training method provided by any embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the method.

[0207] Figure 7 FIG. 2 shows a schematic structural diagram of an electronic device 10 that can be used to implement an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0208] As Figure 7 shown, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. The memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0209] A plurality of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0210] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the production task allocation and model training methods.

[0211] In some embodiments, the production task allocation and model training method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the production task allocation and model training method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the production task allocation and model training method by any other suitable means (e.g., by means of firmware).

[0212] Various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard products (ASSP), systems-on-chip (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0213] The computer program for implementing the method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowchart and / or block diagram are implemented. The computer program can be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0214] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0215] To provide for interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0216] The systems and techniques described herein can be implemented in a computing system that includes backend components (such as, for example, a data server), or a computing system that includes middleware components (such as, for example, an application server), or a computing system that includes frontend components (such as, for example, a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (such as, for example, a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a target blockchain network, and the Internet.

[0217] A computing system may include a client and a server. The client and the server are generally far from each other and usually interact via a communication network. The client-server relationship is created by computer programs that run on respective computers and have a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0218] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and this is not limited herein.

[0219] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A production task allocation method, characterized in that, Including: Obtain a first production object and a collaborative production object for producing the same batch of tobacco; wherein, at least one production workshop for producing the same batch of tobacco is included under the first production object, and at least one production device for producing the same batch of tobacco is deployed in the production workshop; For at least one of the production devices, according to the first production strategy prediction model corresponding to the first production object, the second production strategy prediction model corresponding to the current production workshop where the current production device is located, the third production strategy prediction model corresponding to the current production device, the first object production association data corresponding to the first production object in the first historical time range, and the second object production association data corresponding to the collaborative production object in the first historical time range, determine the first tobacco production plan corresponding to the current production device in the future time range, so that the current production device conducts tobacco production according to the first tobacco production plan; Wherein, the first object production association data and the second object production association data are model inputs of the first production strategy prediction model, the model output of the first production strategy prediction model is a partial model input of the second production strategy prediction model, and the model output of the second production strategy prediction model is a partial model input of the third production strategy model; the first object production association data is used to evaluate the tobacco production efficiency of the first production object in the first historical time range; the first tobacco production plan is used to represent the quantity of tobacco planned to be produced by the current production device in the future time range.

2. The production task allocation method according to claim 1, wherein, The determining the first tobacco production plan corresponding to the current production device in the future time range according to the first production strategy prediction model corresponding to the first production object, the second production strategy prediction model corresponding to the current production workshop where the current production device is located, the third production strategy prediction model corresponding to the current production device, the first object production association data corresponding to the first production object, and the second object production association data corresponding to the collaborative production object includes: Obtain the first object production association data corresponding to the first production object in the first historical time range, and the second object production association data corresponding to the collaborative production object in the first historical time range; Process the first object production association data and the second object production association data according to the first production strategy prediction model to obtain a second tobacco production plan corresponding to the first production object in the future time range; Determine a third tobacco production plan corresponding to the current production workshop in the future time range according to the second tobacco production plan and the second production strategy prediction model corresponding to the current production workshop where the current production device is located; Determine the first tobacco production plan corresponding to the current production device in the future time range according to the third tobacco production plan and the third production strategy prediction model corresponding to the current production device.

3. The production task allocation method according to claim 2, wherein, Determining the third tobacco production plan corresponding to the current production workshop within the future time range according to the second production strategy prediction model corresponding to the second tobacco production plan and the current production workshop where the current production equipment is located, includes: Obtaining the first workshop production correlation data corresponding to the current production workshop within the first historical time range, and the second workshop production correlation data corresponding to other production workshops except the current production workshop among at least one of the production workshops; wherein, the first workshop production correlation data is used to characterize the tobacco production efficiency of the current production workshop within the first historical time range; Processing the first workshop production correlation data, the second workshop production correlation data, and the second tobacco production plan according to the second production strategy prediction model to obtain the third tobacco production plan corresponding to the current production workshop within the future time range.

4. The production task allocation method according to claim 2, wherein Determining the first tobacco production plan corresponding to the current production equipment within the future time range according to the third tobacco production plan and the third production strategy prediction model corresponding to the current production equipment, includes: Obtaining the first equipment production correlation data corresponding to the current production equipment within the first historical time range, and the second equipment production correlation data corresponding to other production equipment except the current production equipment among at least one of the production equipment; wherein, the first equipment production correlation data is used to characterize the tobacco production efficiency of the current production equipment within the first historical time range; Processing the first equipment production correlation data, the second equipment production correlation data, and the third tobacco production plan according to the third production strategy prediction model to obtain the first tobacco production plan corresponding to the current production equipment within the future time range.

5. A model training method, characterized in that, Includes: Obtaining a first sample production object and a collaborative sample production object for producing the same batch of tobacco; wherein, the first sample production object includes at least one sample production workshop for producing the same batch of tobacco, and at least one sample production equipment for producing the same batch of tobacco is deployed in the sample production workshop; Obtaining a plurality of training samples corresponding to the first sample production object; wherein, the training sample includes the first sample object production correlation data corresponding to the first sample tobacco production object within the historical production cycle and the second sample object production correlation data corresponding to the collaborative sample production object within the historical production cycle; Training a first reinforcement learning model corresponding to the first sample production object, a second reinforcement learning model corresponding to at least one of the sample production workshops, and a third reinforcement learning model corresponding to at least one of the sample production equipment under each of the sample production workshops according to the plurality of training samples; When the preset training end condition is reached, a first production strategy prediction model corresponding to the first sample production object, a second production strategy prediction model corresponding to the sample production workshop, and a third production strategy prediction model corresponding to the sample production equipment are obtained, so as to predict the tobacco production plan based on the first production strategy prediction model, the second production strategy prediction model, and the third production strategy prediction model.

6. The model training method according to claim 5, wherein The training of the first reinforcement learning model corresponding to the first sample production object, the second reinforcement learning model corresponding to at least one of the sample production workshops, and the third reinforcement learning model corresponding to at least one of the sample production equipment under each of the sample production workshops according to a plurality of the training samples includes: For a plurality of the training samples, input the current training sample into the first reinforcement learning model to obtain the object tobacco production plan and the actual object value parameter corresponding to the first sample production object in the current historical production cycle, and process the object tobacco production plan and the first sample object production association data according to a preset object reward function to determine the first object reward parameter corresponding to the current historical production cycle; wherein, the object reward function is constructed based on the object production constraint condition and the object production objective function corresponding to the first sample production object; Process the first object reward parameter and the actual object value parameter according to a preset loss function to determine the first loss value corresponding to the current historical production cycle, and correct the model parameters in the first reinforcement learning model based on the first loss value; and, For at least one of the sample production workshops, determine the workshop tobacco production plan and the actual workshop value parameter corresponding to the current sample production workshop in the current historical production cycle according to the object tobacco production plan and the second reinforcement learning model corresponding to the current sample production workshop, and determine the first workshop reward parameter corresponding to the current historical production cycle according to a preset workshop reward function and the workshop tobacco production plan; wherein, the workshop reward function is constructed based on the workshop production constraint condition and the workshop production objective function corresponding to the sample production workshop; Determine the second loss value corresponding to the historical production cycle according to the actual workshop value parameter and the first workshop reward parameter according to a preset loss function, and correct the model parameters in the second reinforcement learning model based on the second loss value; and, For at least one sample production device corresponding to the current sample production workshop, based on the workshop tobacco production plan and the third reinforcement learning model corresponding to the current sample production device, determine the device tobacco production plan and the actual device value parameters corresponding to the current historical production cycle of the current sample production device, and determine the first device reward parameter corresponding to the current historical production cycle according to a preset device reward function and the device tobacco production plan; wherein, the device reward function is constructed based on the device production constraint conditions and the device production objective function corresponding to the sample production device. According to a preset loss function, determine the third loss value corresponding to the current historical production cycle for the actual device value parameters and the first device reward parameter, and correct the model parameters in the third reinforcement learning model based on the third loss value.

7. The model training method according to claim 6, wherein The processing of the first object reward parameter and the actual object value parameter according to a preset loss function to determine the first loss value corresponding to the current historical production cycle includes: Obtain the second object reward parameter corresponding to the collaborative sample production object in the current historical production cycle. According to the first object reward parameter and the second object reward parameter, determine the object-level reward parameter corresponding to the current historical production cycle. Obtain the workshop-level reward parameter corresponding to the sample production workshop in the current historical production cycle and the device-level reward parameter corresponding to the sample production device in the current historical production cycle, and process the object-level reward parameter, the workshop-level reward parameter, and the device-level reward parameter through mean calculation or weighted average calculation to obtain the global reward parameter corresponding to the first sample production object in the current historical production cycle. According to a preset loss function, process the first object reward parameter, the actual object value parameter, the object-level reward parameter, and the global reward parameter to determine the first loss value corresponding to the current historical production cycle.

8. The model training method according to claim 6, wherein The determination of the workshop tobacco production plan and the actual workshop value parameters corresponding to the current sample production workshop in the current historical production cycle according to the object tobacco production plan and the second reinforcement learning model corresponding to the current sample production workshop includes: Obtain the first sample workshop production association data corresponding to the current sample production workshop in the current historical production cycle and the second sample workshop production association data corresponding to at least one other sample production workshop except the current sample production workshop in the current historical production cycle. Process the object tobacco production plan, the first sample workshop production association data, and the second sample workshop production association data according to the second reinforcement learning model corresponding to the current sample production workshop to obtain the workshop tobacco production plan and the actual workshop value parameters corresponding to the current sample production workshop in the current historical production cycle.

9. The model training method according to claim 6, wherein Determining the equipment tobacco production plan and actual equipment value parameters corresponding to the current sample production equipment in the current historical production cycle according to the workshop tobacco production plan and the third reinforcement learning model corresponding to the current sample production equipment, including: Obtaining the first sample equipment production association data corresponding to the current sample production equipment in the current historical production cycle and the second sample equipment production association data corresponding to at least one other sample production equipment except the current sample production equipment in the current historical production cycle; Processing the workshop tobacco production plan, the first sample equipment production association data, and the second sample equipment production association data according to the third reinforcement learning model corresponding to the current sample production equipment to obtain the equipment tobacco production plan and actual equipment value parameters corresponding to the current sample production equipment in the current historical production cycle.

10. A production task allocation device, characterized in that, Including: An object acquisition module, configured to acquire a first production object and a collaborative production object for producing the same batch of tobacco; wherein, under the first production object, there are at least one production workshop for producing the same batch of tobacco, and at least one production equipment for producing the same batch of tobacco is deployed in the production workshop; A production plan determination module, configured to, for at least one of the production equipment, determine a first tobacco production plan corresponding to the current production equipment in a future time range according to the first production strategy prediction model corresponding to the first production object, the second production strategy prediction model corresponding to the current production workshop where the current production equipment is located, the third production strategy prediction model corresponding to the current production equipment, the first object production association data corresponding to the first production object in a first historical time range, and the second object production association data corresponding to the collaborative production object in a first historical time range, so that the current production equipment performs tobacco production according to the first tobacco production plan; wherein, the first object production association data and the second object production association data are model inputs of the first production strategy prediction model, the model output of the first production strategy prediction model is part of the model inputs of the second production strategy prediction model, and the model output of the second production strategy prediction model is part of the model inputs of the third production strategy model; the first object production association data is used to evaluate the tobacco production efficiency of the first production object in the first historical time range; the first tobacco production plan is used to represent the quantity of tobacco planned to be produced by the current production equipment in the future time range.

11. A model training device, characterized in that Including: A sample object acquisition module, configured to acquire a first sample production object and a collaborative sample production object for producing the same batch of tobacco; wherein, the first sample production object includes at least one sample production workshop for producing the same batch of tobacco, and at least one sample production equipment for producing the same batch of tobacco is deployed in the sample production workshop; A training sample acquisition module for acquiring a plurality of training samples corresponding to the first sample production object; wherein the training samples include first sample object production association data corresponding to the first sample tobacco production object in a historical production cycle and second sample object production association data corresponding to the collaborative sample production object in the historical production cycle; A model training module for training a first reinforcement learning model corresponding to the first sample production object, a second reinforcement learning model corresponding to at least one of the sample production workshops, and a third reinforcement learning model corresponding to at least one of the sample production devices under each of the sample production workshops according to the plurality of training samples; A policy prediction model determination module for obtaining a first production policy prediction model corresponding to the first sample production object, a second production policy prediction model corresponding to the sample production workshop, and a third production policy prediction model corresponding to the sample production device when a preset training end condition is reached, so as to predict a tobacco production plan based on the first production policy prediction model, the second production policy prediction model, and the third production policy prediction model.

Images