Automated collaborative decision-making method, system, device, medium and product for enterprise resource planning

Abstract

The present invention relates to the field of data management technology, and discloses an automated collaborative decision-making method, system, device, medium, and product for enterprise resource planning to solve the problem of low efficiency of automatic decision-making in existing enterprise resource planning systems. The method includes: constructing a multi-agent reinforcement learning model, designing a global reward function and a local reward function, and constructing a potential function of the local reward function, and training a collaborative decision-making model based on training data; collecting work information of each collaborative workflow to obtain business data; calling the collaborative decision-making model to make collaborative decisions on the business data based on the global reward function and the local reward function, and at the same time reward shaping the local reward function based on the potential function to generate a business strategy; generating decision instructions based on the business strategy according to the division of labor of each executor in the enterprise resource system; each executor executes the corresponding decision instruction, monitors the execution process in real time, records the execution results, generates feedback information, and updates the work information.

Description

Technical Field

[0001] The present invention relates to the field of data management technology, and in particular to an automated collaborative decision-making method, system, device, medium and product for enterprise resource planning. Background Art

[0002] As the core management platform for modern enterprises, Enterprise Resource Planning (ERP) systems must coordinate decision-making information across multiple departments, including procurement, production, sales, and inventory, in real time. This ensures a comprehensive understanding of the entire enterprise process and optimizes resource planning. With intensifying market competition, traditional static decision-making models that rely on manual experience are no longer able to cope with dynamic challenges such as demand fluctuations and supply chain disruptions. Therefore, automating and intelligently implementing cross-departmental business decision-making is crucial for improving enterprise operational efficiency.

[0003] In existing technologies, enterprise resource planning decision-making solutions generally use static thresholds to trigger business operations. However, this mechanism generally relies more on historical data and cannot perceive sudden market fluctuations, which makes the decisions made generally lagging and difficult to adapt to situations with large changes. In addition, the decision-making system has a low level of automation and intelligence, and the decisions and actions made cannot respond to market changes in real time, resulting in low decision-making efficiency. Summary of the Invention

[0004] The main purpose of the present invention is to solve the technical problem that the decision-making method in the enterprise resource planning system in the prior art cannot respond to market fluctuations in a timely manner, resulting in low decision-making efficiency.

[0005] A first aspect of the present invention provides an automated collaborative decision-making method for enterprise resource planning, comprising:

[0006] Constructing a multi-agent reinforcement learning model, wherein the multi-agent reinforcement learning model includes a procurement agent, a production agent, and a sales agent;

[0007] Designing a global reward function for each agent included in the multi-agent reinforcement learning model, designing local reward functions for the procurement agent, production agent, and sales agent, respectively, and constructing latent functions of the local reward functions; training the multi-agent reinforcement learning model based on training data to obtain a collaborative decision-making model;

[0008] Collect work information of each collaborative workflow in the enterprise resource system in real time, integrate the work information and extract business data;

[0009] Calling the collaborative decision-making model to make collaborative decisions on the business data according to the global reward function and the local reward function, and simultaneously reward shaping the local reward function based on the potential function to generate a business strategy;

[0010] According to the division of labor of each executor in the enterprise resource system, generating decision instructions corresponding to each executor based on the business strategy;

[0011] Call each executor to execute the corresponding decision instruction, monitor the execution process in real time and record the execution results, generate feedback information through a closed-loop control mechanism based on the execution results, and update the work information of each workflow according to the feedback information.

[0012] Optionally, in a first implementation of the first aspect of the present invention, the potential function is expressed as:

[0013] = (s')- (s);

[0014] Among them, s represents the current state, s' represents the next state, represents the discount factor, (s) represents the potential function of the current state of the strategy, represents the shaping reward of the latent function.

[0015] Optionally, in a second implementation of the first aspect of the present invention, before designing a global reward function based on each agent included in the multi-agent reinforcement learning model, the method further includes:

[0016] The decision steps of each collaborative workflow in the acquisition enterprise resource system are obtained, the corresponding intelligent agents are configured according to the decision step information, and the local observation space and action space of each intelligent agent are defined.

[0017] Optionally, in a third implementation of the first aspect of the present invention, the global reward function is:

[0018] Global reward = sales price – total cost;

[0019] Total cost = purchase cost + production cost + inventory cost + other variable costs;

[0020] The sales price, purchase cost, inventory cost and other variable costs are determined based on the decision of the collaborative decision-making model.

[0021] Optionally, in a fourth implementation of the first aspect of the present invention, after updating the work information of each workflow according to the feedback information, the method further includes:

[0022] Collect inventory data in real time and calculate inventory turnover rate;

[0023] Determining whether to adjust the strategy update frequency according to the change status of the inventory turnover rate;

[0024] If the inventory turnover rate is less than the turnover rate update threshold, activating a high-frequency update mode, calculating a policy update interval, and updating the network policy parameters of the collaborative decision-making model based on the calculated policy update interval;

[0025] If the inventory turnover rate is not less than the turnover rate update threshold, the baseline update frequency is maintained, and the network policy parameters of the collaborative decision-making model are updated based on the policy update interval set based on the baseline update frequency.

[0026] Optionally, in a fifth implementation of the first aspect of the present invention, the calculation expression of the turnover rate update threshold is:

[0027] ;

[0028] Among them, the represents the turnover rate update threshold, μ is the mean of the historical inventory turnover rate, is the standard deviation, represents the departure adjustment coefficient, .

[0029] A second aspect of the present invention provides an automated collaborative decision-making system for enterprise resource planning, comprising:

[0030] a model training module for constructing a multi-agent reinforcement learning model, wherein the multi-agent reinforcement learning model includes a procurement agent, a production agent, and a sales agent; designing a global reward function based on each agent included in the multi-agent reinforcement learning model, designing local reward functions for each of the procurement agent, the production agent, and the sales agent, and constructing latent functions of the local reward functions; and training the multi-agent reinforcement learning model based on training data to obtain a collaborative decision-making model;

[0031] A data observation module is used to collect work information of each workflow in the enterprise resource system in real time, integrate the work information and extract business data;

[0032] a strategy adjustment module, configured to call a collaborative decision-making model to make collaborative decisions based on the business data and generate a business strategy, wherein the collaborative decision-making model is composed of multiple agents;

[0033] An automatic decision-making module, configured to generate a decision instruction corresponding to each executor based on the business strategy according to the division of labor of each executor in the enterprise resource system;

[0034] The action execution module is used to call each executor to execute the corresponding decision instructions, monitor the execution process in real time and record the execution results, generate feedback information through a closed-loop control mechanism based on the execution results, and update the work information of each workflow according to the feedback information.

[0035] The third aspect of the present invention provides an automated collaborative decision-making device for enterprise resource planning, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the automated collaborative decision-making device for enterprise resource planning executes the steps of the above-mentioned automated collaborative decision-making method for enterprise resource planning.

[0036] A fourth aspect of the present invention provides a computer-readable storage medium having instructions stored therein, which, when executed on a computer, enables the computer to execute the steps of the above-mentioned automated collaborative decision-making method for enterprise resource planning.

[0037] A fifth aspect of the present invention provides a computer program product comprising a computer program / instruction, which, when executed by a processor, implements the steps of the automated collaborative decision-making method for enterprise resource planning as described above.

[0038] In the technical solution provided by the present invention, a multi-agent reinforcement learning model is constructed, wherein the multi-agent reinforcement learning model includes a procurement agent, a production agent and a sales agent; a global reward function is designed based on each agent contained in the multi-agent reinforcement learning model, local reward functions are designed for the procurement agent, the production agent and the sales agent respectively, and potential functions of the local reward functions are constructed, and the multi-agent reinforcement learning model is trained based on the training data to obtain a collaborative decision-making model; work information of each collaborative workflow in the enterprise resource system is collected in real time, the work information is integrated and business data is extracted; the collaborative decision-making model is called to make collaborative decisions on business data based on the global reward function and the local reward function, and at the same time, the local reward function is reward-shaped based on the potential function to generate a business strategy; according to the division of labor of each executor in the enterprise resource system, decision instructions corresponding to each executor are generated based on the business strategy; each executor is called to execute the corresponding decision instructions, the execution process is monitored in real time and the execution results are recorded, feedback information is generated through a closed-loop control mechanism based on the execution results, and the work information of each workflow is updated according to the feedback information. This solution can make collaborative decisions based on multiple intelligent agents, automatically and intelligently generate immediate decision instructions based on work information in the enterprise resource system, and can respond to market fluctuations in real time to improve decision-making efficiency.

[0039] In addition, the automated collaborative decision-making system, equipment, medium and product for enterprise resource planning provided by the present invention also solve corresponding technical problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0041] Figure 1 This is a flow chart of a first embodiment of an automated collaborative decision-making method for enterprise resource planning according to an embodiment of the present invention;

[0042] Figure 2 This is a flow chart of a second embodiment of the automated collaborative decision-making method for enterprise resource planning according to an embodiment of the present invention;

[0043] Figure 3 This is a schematic diagram of business data transmission between various agents in a collaborative decision-making model in a second embodiment of an automated collaborative decision-making method for enterprise resource planning according to an embodiment of the present invention;

[0044] Figure 4 1. A flow chart of a calculation strategy update method in a second embodiment of an automated collaborative decision-making method for enterprise resource planning according to an embodiment of the present invention;

[0045] Figure 5 Schematic diagram of an embodiment of an automated collaborative decision-making system for enterprise resource planning in an embodiment of the present invention;

[0046] Figure 6 This is a schematic diagram of an embodiment of an automated collaborative decision-making device for enterprise resource planning in an embodiment of the present invention;

[0047] Figure 7 A schematic diagram of the principle of a computer-readable medium in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] Exemplary embodiments of the present invention will now be described more fully with reference to the accompanying drawings. However, exemplary embodiments can be implemented in various forms, and it should not be understood that the present invention is limited to the embodiments set forth herein. On the contrary, providing these exemplary embodiments enables the present invention to be more comprehensive and complete, making it easier to fully convey the inventive concept to those skilled in the art. In the figures, the same reference numerals represent the same or similar elements, components or parts, and thus their repeated description will be omitted.

[0049] Under the premise of being consistent with the technical concept of the present invention, the features, structures, characteristics or other details described in a specific embodiment do not exclude that they can be combined in one or more other embodiments in a suitable manner.

[0050] In the description of specific embodiments, the features, structures, characteristics, or other details of the present invention are described to enable those skilled in the art to fully understand the embodiments. However, this does not preclude those skilled in the art from practicing the technical solutions of the present invention without one or more of the specific features, structures, characteristics, or other details.

[0051] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0052] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0053] The term "and / or" or "and / or" includes all combinations of any one or more of the associated listed items.

[0054] See also Figure 1 The first embodiment of the automated collaborative decision-making method for enterprise resource planning in the embodiment of the present invention includes:

[0055] Before executing a specific automated collaborative decision-making method, it is first necessary to construct and pre-train the collaborative decision-making model. Specifically, it includes first obtaining multiple agents to construct a multi-agent reinforcement learning model, wherein the multi-agent reinforcement learning model includes a procurement agent, a production agent, and a sales agent; designing a global reward function based on each agent included in the multi-agent reinforcement learning model, designing local reward functions for the procurement agent, production agent, and sales agent respectively, and constructing a potential function of the local reward function; training the multi-agent reinforcement learning model based on the training data to obtain a collaborative decision-making model. After the collaborative decision-making model is trained, specific collaborative decisions can be executed based on the work information of each collaborative workflow in the real-time enterprise resource system.

[0056] S101. Collect work information of each collaborative workflow in the enterprise resource system in real time, integrate the work information and extract business data;

[0057] It is understood that the execution entity of the present invention can be an automated collaborative decision-making device for enterprise resource planning (ERP), a terminal, or a server, without limitation. The solutions in the embodiments of the present invention belong to the field of artificial intelligence technology and can be applied to data management and processing methods for administrative, commercial, financial, management, supervisory, or forecasting purposes. The embodiments of the present invention are described using a server as the execution entity.

[0058] When executing a specific automated collaborative decision-making method, the decision-making scheme in this embodiment is mainly based on the OODA (Observation - Orientation - Decision - Action) cycle idea to construct the overall workflow. This step is the Observe (data observation) step, which is mainly used to collect the work information of each collaborative workflow in the enterprise resource system in real time. Among them, the work information of each collaborative workflow can be order information, raw material consumption information, supplier quantity, market demand and other information in the procurement system. This information is generally in the decision-making process. In this embodiment, all this information will be collected and extracted to obtain actual business data.

[0059] S102: Invoke the collaborative decision-making model to make collaborative decisions on business data based on the global reward function and the local reward function, and simultaneously shape the local reward function based on the potential function to generate a business strategy.

[0060] After obtaining the business data based on the Observation step, this step begins to execute the specific Orientation step. Specifically in this embodiment, it is to call the collaborative decision-making model to make collaborative decisions based on the business data and determine what business strategy should be implemented.

[0061] In this embodiment, the specific method of collaborative decision-making for business strategies is based on the collaborative decision-making model pre-constructed and trained in the aforementioned steps. In particular, multiple agents in the collaborative decision-making model can make multiple decisions, but in order to make collaborative decisions and obtain the optimal business strategy, it is also necessary to adjust according to the preset reward function and the potential function of the reward function. Specifically, the reward function includes a global reward function and a local reward function, and the potential function is the potential function of the local reward function. Through the local reward function, each agent in the collaborative decision-making model can give the optimal local strategy for the current local decision range. By adjusting the global reward function and the potential function, the conflict between the local strategy and the global strategy is reduced, ensuring that the local reward does not change the global optimal strategy, thereby giving the optimal business strategy.

[0062] S103. Generate decision instructions corresponding to each executor based on the division of labor of each executor in the enterprise resource system and business strategy;

[0063] After determining the specific business strategy to be executed based on the Orientation step, this step begins to execute the specific Decision step. Specifically, in this embodiment, based on the division of labor of each executor in the enterprise resource system, it is determined to generate decision instructions corresponding to each executor based on the business strategy.

[0064] Based on the business strategy obtained in the above steps, a specific strategy solution is executed, and decision instructions corresponding to each actuator are generated based on the business strategy. For example, the decision instructions can be specific execution information, such as the purchasing agent deciding to purchase a certain amount of raw materials at a certain time, or the production agent deciding to start production for a certain number of hours at a certain time. These decision instructions are then sent to specific execution terminals based on the business strategy.

[0065] S104: Call each executor to execute the corresponding decision instruction, monitor the execution process in real time and record the execution results, generate feedback information through a closed-loop control mechanism based on the execution results, and update the work information of each workflow according to the feedback information.

[0066] After determining the specific decision instructions based on the Decision step, this step can call each executor to execute the corresponding decision instructions, that is, implement the Action step.

[0067] When executing specific decision-making instructions, the execution process will be monitored in real time and the execution results will be recorded. Feedback information will be generated through a closed-loop control mechanism based on the execution results, and the work information of each workflow will be updated according to the feedback information, so that the collaborative decision-making model can obtain the latest work data in real time and make the next collaborative decision in real time.

[0068] The solution in the embodiment of the present invention can make collaborative decisions based on multiple intelligent agents, automatically and intelligently generate immediate decision instructions based on work information in the enterprise resource system, and can respond to market fluctuations in real time to improve decision-making efficiency.

[0069] Please see Figure 2-Figure 4 The second embodiment of the automated collaborative decision-making method for enterprise resource planning in the embodiment of the present invention includes:

[0070] The solutions described in the embodiments of the present invention belong to the field of artificial intelligence technology and can be applied to data management and processing methods for administrative, commercial, financial, management, supervisory, or forecasting purposes. Specifically, they relate to an automated collaborative decision-making method for enterprise resource planning. Before executing the specific automated collaborative decision-making method, this method first models the system environment based on the enterprise resource planning system. Specifically, it first defines a global state space, including information such as inventory levels, order status, and production capacity.

[0071] S201. Construct an initial multi-agent reinforcement learning model, design a reward function based on each agent included in the multi-agent reinforcement learning model, train the initial multi-agent reinforcement learning model based on the training data, and obtain a collaborative decision-making model;

[0072] In this embodiment, an initial multi-agent reinforcement learning model is constructed based on the concept of multi-agent reinforcement learning (MARL). Specifically, the decision steps of each collaborative workflow in the enterprise resource acquisition system are obtained, the corresponding agents are configured based on the decision step information, and the local observation space and action space of each agent are defined.

[0073] The initial multi-agent reinforcement learning model includes multiple decision-making agents. For example, when the automated collaborative decision-making method for enterprise resource planning described in this embodiment requires decision-making and planning for multiple aspects such as procurement, sales, production, and inventory, the agents may include a procurement agent, a production agent, and a sales agent.

[0074] In this embodiment, the initial multi-agent reinforcement learning model can be trained using a centralized training for decentralized execution (CTDE) framework, enabling it to balance decision data based on multiple objectives, thereby providing subsequent decision results that prioritize both global and local information. Therefore, this embodiment employs a method of designing local and global reward functions. Based on historical data and feedback from the reward functions, the initial multi-agent reinforcement learning model is trained to produce a fully trained collaborative decision-making model.

[0075] In one specific embodiment, multiple different agents are first constructed based on the actual work information of multiple workflows existing in an actual enterprise resource planning system. An initial multi-agent reinforcement learning model is then constructed based on these agents. Taking the multi-agent reinforcement learning model as an example, which specifically includes a procurement agent, a production agent, and a sales agent, this step also includes designing a global reward function based on the procurement agent, the production agent, and the sales agent, while simultaneously designing local reward functions for each of these agents. The global reward function is used to balance the objectives of the enterprise resource planning system, while the local reward functions are used to balance the local objectives of each individual decision-making node. While the global objective of an enterprise resource planning system is typically to maximize corporate profits, the local objectives of each module (e.g., procurement, production, and sales) may conflict with the global objective. For example, the procurement module's pursuit of low costs may increase inventory pressure, impacting the inventory module. Therefore, in this embodiment, training data is also obtained to train each agent based on the global and local reward functions, adjusting the decision parameters, and ultimately obtaining a collaborative decision-making model. The training data in this embodiment may be actual historical data.

[0076] When constructing each agent, different agents are constructed through LSTM (Long Short-term Memory Networks), and an initial multi-agent reinforcement learning model is constructed based on the multi-agent reinforcement learning concept. The LSTM realizes the dynamic screening and memory of multi-dimensional time series features through the gate control unit. Each agent includes a feature extraction layer and a state memory unit. Taking the production agent as an example, the input features of the feature extraction layer include: (1) production rhythm: process interval time (such as 45 minutes between each batch of the stamping line), beat achievement rate; (2) equipment efficiency: utilization rate, mean time between failures (MTBF), maintenance response speed; (3) order attributes: delivery urgency, customer level weight, product profit margin; and filtering out short-term equipment downtime noise through the forget gate. The state memory unit stores long-term production rules through cell state.

[0077] S202: Collect work information of each collaborative workflow in the enterprise resource system in real time, integrate the work information and extract business data;

[0078] When executing a specific automated collaborative decision-making method, the decision-making scheme in this embodiment is mainly based on the OODA (Observation-Orientation-Decision-Action) cycle idea to construct the overall workflow. This step is the Observe (data observation) step, which is mainly used to collect the work information of each collaborative workflow in the enterprise resource system in real time. Among them, the work information of each collaborative workflow can be order information, raw material consumption information, supplier quantity, market demand and other information in the procurement system. This information is generally in the decision-making process. In this embodiment, all this information will be collected and extracted to obtain actual business data.

[0079] S203: Invoke the collaborative decision-making model to make collaborative decisions based on business data and generate business strategies;

[0080] After obtaining the business data based on the Observation step, this step begins to execute the specific Orientation step. Specifically in this embodiment, it is to call the collaborative decision-making model to make collaborative decisions based on the business data and determine what business strategy should be implemented.

[0081] In this embodiment, the specific method for collaborative decision-making on business strategies is based on the collaborative decision-making model pre-built and trained in the above steps. In this collaborative decision-making model, multiple agents can make multiple decisions, but in order to achieve the optimal business strategy through collaborative decision-making, adjustments must be made based on a preset reward function.

[0082] See also Figure 3 In a specific embodiment, the collaborative decision-making model obtained after adjustment based on a preset reward function can be shown in the figure. The collaborative decision-making model includes a production agent, a procurement agent, and a sales agent. The production agent is responsible for scheduling optimization and anomaly detection, the procurement agent is responsible for supplier evaluation and order generation, and the sales agent is responsible for pricing strategies and demand forecasting. Various business data are transmitted between the agents, and specific business strategies are generated based on this business data and the pre-established global and local reward functions.

[0083] To illustrate this with a specific example, a global reward function can be the overall profit performance of an enterprise. The global reward function can be expressed as:

[0084] Global reward = sales price – total cost;

[0085] Total cost = purchase cost + production cost + inventory cost + other variable costs;

[0086] The sales price, purchase cost, inventory cost and other variable costs are determined based on the decision of the collaborative decision-making model.

[0087] In a specific implementation, the net cash flow per day (or appropriate time step) can be defined as the global reward:

[0088] = - ;

[0089] in, represents the net cash flow, represents the revenue from all sales on day t, represents the cost spent on day t.

[0090] Furthermore, the The expression for (sales revenue) is:

[0091] = ;

[0092] in, Indicates the sales unit price, represents the total number of sales; both parameters are determined by the specific decision results of the sales agent included in the trained collaborative decision-making model.

[0093] Furthermore, The expression for (total cost) is:

[0094] ;

[0095] Among them, the represents the purchase cost, i.e., the cost of goods received on day t. This parameter is determined by the specific decision results of the purchase agent included in the trained collaborative decision-making model. represents the production cost, i.e., the cost of producing goods on day t. This parameter is determined by the specific decision results of the production agent included in the trained collaborative decision-making model (for example, according to the price of raw materials and labor costs); Represents inventory holding cost, which is calculated based on inventory value. A specific calculation method is:

[0096] =holding_cost_rate total_inventory_value;

[0097] Wherein, the holding_cost_rate represents the holding cost rate, and the total_inventory_value represents the total inventory value;

[0098] described Represents other variable costs, calculated based on marketing and transportation expenses.

[0099] Based on the above, all agents (purchase agents, production agents, sales agents, etc.) share the global reward ( ) to incentivize collaboration; the shared reward is implemented through a centralized training decentralized execution framework, and the central critic accesses the global state during training to assist the agent in learning.

[0100] Furthermore, global rewards can lead to inefficient learning due to latency or sparsity. Therefore, local rewards or reward shaping can be introduced to provide more frequent feedback. Local rewards must be consistent with the global goal to prevent the agent from pursuing local optimality at the expense of global performance.

[0101] When local rewards and reward shaping are introduced, this step specifically includes: calling the collaborative decision-making model to make collaborative decisions on business data based on the global reward function and the local reward function, and at the same time reward shaping the local reward function based on the potential function to generate a business strategy.

[0102] To illustrate this with a specific example, here is the local reward design for each agent in the collaborative decision-making model:

[0103] (1) Procurement:

[0104] Price reward: When the purchase price is lower than the reference price, to encourage low-price purchases, the local reward is set as:

[0105] =-(price - reference_price);

[0106] in, Indicates low-price purchase reward, price indicates purchase price, and reference_price indicates reference price;

[0107] Timely rewards: Provide on-time rewards when the goods arrive on time ( ), when the delay arrives, a delay reward is given ( ). However, in order to encourage on-time delivery, rewards are set < .

[0108] (2) Production part:

[0109] Efficiency bonus: When production costs are lower than expected, the bonus is set as:

[0110] =-(actual_cost - expected_cost);

[0111] in, Represents cost reward, actual_cost represents actual cost, and expected_cost represents expected cost;

[0112] Output matching reward: Positive rewards are given when production volume matches demand.

[0113] (3) Sales section:

[0114] Profit Margin Rewards: The rewards for each sale are:

[0115] =(selling_price - cost_price) quantity;

[0116] in, Indicates sales reward, selling_price is the actual selling price, cost_price is the actual cost price, and quantity is the sales quantity;

[0117] Customer satisfaction rewards: Customer satisfaction can be quantified through data such as order delivery timeliness. When customer satisfaction reaches a certain value, additional rewards can be given.

[0118] (4) Inventory:

[0119] Inventory Optimization Rewards: To encourage low inventory, the rewards are set as:

[0120] =-holding_cost inventory_level;

[0121] in, represents the low inventory reward, holding_cost represents the inventory cost, and inventory_level represents the inventory level rating;

[0122] Out-of-stock penalty: When out-of-stock occurs, the reward is set as:

[0123] =-large_number;

[0124] Among them, the is the out-of-stock penalty (reward), and large_number represents the out-of-stock quantity.

[0125] In addition to local rewards, to ensure that local rewards do not change the global optimal strategy, this embodiment also includes constructing a potential function for local rewards and using the potential function to shape rewards. Reward shaping accelerates learning by providing intermediate feedback while maintaining global optimality. Specifically, the expression for shaping rewards is:

[0126] = (s')- (s);

[0127] Among them, s represents the current state, s' represents the next state, (s) represents the potential function of the current state (s) of the strategy, which measures the contribution of the state to the global goal; s' represents the next state; Represents the discount factor.

[0128] Let's take a specific example to illustrate. In the inventory section, define:

[0129] (s) = - holding_cost inventory_level;

[0130] In the inventory section, when the inventory decreases, (s') increases, thus providing positive shaping incentives; in the procurement section: definition (s) = -delay_days, to encourage timely purchases.

[0131] In a preferred embodiment, the automated collaborative decision-making method for enterprise resource planning described in this embodiment often involves multiple reward objectives, such as profit maximization, cost minimization, customer satisfaction, etc. When there are multiple reward objectives, the following method can be used to perform multi-objective decision-making:

[0132] (1) Scalar method: Combine multiple objectives into a single reward. Let’s take a specific example to illustrate:

[0133] = · + · + ·( );

[0134] in, represents the final compound reward target, For profit reward targets, Reward targets for customer satisfaction, for cost incentive targets; 、 、 is the weight, which can be adjusted according to business priorities.

[0135] (2) Multi-objective reinforcement learning: Learn a set of non-dominated strategies, generate paths and methods to achieve Pareto optimality based on the Pareto optimal algorithm, and allow collaborative decision-making models to balance between different objectives.

[0136] S204: Generate decision instructions corresponding to each executor based on the business strategy according to the division of labor of each executor in the enterprise resource system;

[0137] After determining the specific business strategy to be executed based on the Orientation step, this step begins to execute the specific Decision step. Specifically, in this embodiment, based on the division of labor of each executor in the enterprise resource system, it is determined to generate decision instructions corresponding to each executor based on the business strategy.

[0138] Based on the business strategy obtained in the above steps, a specific strategy solution is executed, and decision instructions corresponding to each actuator are generated based on the business strategy. The decision instructions can be specific execution information, such as the purchasing agent deciding to purchase a certain amount of raw materials at a certain time, or the production agent deciding to start production for a certain number of hours at a certain time. These decision instructions are then sent to specific execution terminals based on the business strategy.

[0139] S205: Call each executor to execute the corresponding decision instruction, monitor the execution process in real time and record the execution results, generate feedback information based on the execution results through a closed-loop control mechanism, and update the work information of each workflow according to the feedback information;

[0140] After determining the specific decision instructions based on the Decision step, this step can call each executor to execute the corresponding decision instructions, that is, implement the Action step.

[0141] When executing specific decision instructions, the execution process is monitored in real time and the results are recorded. Feedback information is generated based on the execution results through a closed-loop control mechanism. The work information of each workflow is updated based on the feedback information, so that the collaborative decision-making model can obtain the latest work data in real time and make the next collaborative decision in real time. When executing specific decision steps, the OODA loop solution can be used to make cyclical decisions based on steps S202-S205 in this application.

[0142] S206. Collect inventory data in real time and calculate inventory turnover rate;

[0143] In a specific implementation, the technical solution in this embodiment, in addition to the aforementioned OODA loop, also includes updating the decision parameters of each intelligent agent contained in the pre-trained collaborative decision-making model, so as to be able to adaptively improve the response capability to emergencies when business fluctuates.

[0144] Specifically, in existing technologies, learning strategies are updated primarily at fixed intervals, while this embodiment primarily uses dynamic intervals. These intervals are calculated primarily based on business fluctuation data in the system, such as inventory turnover. This allows for dynamic perception of inventory turnover and real-time adaptive adjustment of the update frequency of the algorithm in the collaborative decision-making model. For example, if turnover drops by 10%, the update cycle is shortened from one hour to 15 minutes. This gives higher sampling weight to samples of abnormal inventory status, accelerating the policy network's ability to respond to emergencies.

[0145] Based on this, the solution in this embodiment also includes real-time collection of inventory data and calculation of inventory turnover rate. The inventory turnover rate can be calculated by dividing the sales cost by the inventory value.

[0146] S207: Determine whether to adjust the strategy update frequency according to the change status of the inventory turnover rate, and update the network strategy parameters of the collaborative decision-making model according to the adjusted update frequency.

[0147] See also Figure 4 In this embodiment, changes in inventory turnover rate are monitored in real time, and the policy update frequency is determined based on the change status. In a specific embodiment, an elastic threshold range can be set based on historical inventory turnover rate data (e.g., the mean μ and standard deviation σ over the past 30 days) to obtain the turnover rate update threshold.

[0148] In a specific embodiment, the calculation formula for the turnover rate update threshold may be:

[0149] ;

[0150] Among them, μ is the mean of the historical inventory turnover rate data in the past 30 days, σ is the standard deviation, and k is the trigger adjustment coefficient, which is .

[0151] If the inventory turnover rate is less than the turnover rate update threshold, the high-frequency update mode is activated, the policy update interval is calculated, and the network policy parameters of the collaborative decision-making model are updated based on the calculated policy update interval; if the inventory turnover rate is not less than the turnover rate update threshold, the baseline update frequency is maintained, and the network policy parameters of the collaborative decision-making model are updated based on the policy update interval set by the baseline update frequency.

[0152] By updating the network policy parameters of the collaborative decision-making model in this step, it is possible to store experience in the replay buffer and regularly update the policy and value network to adjust the hyperparameters or reward function.

[0153] In the solution of the embodiment of the present invention, the automated decision-making model includes multiple decision-making agents. These agents can consider a variety of local and global rewards, shaping rewards based on the potential function of the reward function to achieve collaborative decision-making. This automated decision-making model comprehensively considers multiple pieces of information based on these various local and global rewards to formulate practical strategies, enabling coupling between multiple businesses and the overall business. This intelligently generates instant decision instructions, enabling immediate response to market fluctuations, improving decision-making efficiency, and making decision results more reasonable and valuable.

[0154] The above describes the automated collaborative decision-making method for enterprise resource planning in an embodiment of the present invention. The following describes the automated collaborative decision-making system for enterprise resource planning in an embodiment of the present invention. Figure 5 An embodiment of an automated collaborative decision-making system for enterprise resource planning according to an embodiment of the present invention includes:

[0155] Model training module 501 is used to build a multi-agent reinforcement learning model, wherein the multi-agent reinforcement learning model includes a procurement agent, a production agent, and a sales agent; design a global reward function based on each agent included in the multi-agent reinforcement learning model, design local reward functions for the procurement agent, the production agent, and the sales agent respectively, and construct potential functions of the local reward functions; train the multi-agent reinforcement learning model based on training data to obtain a collaborative decision-making model;

[0156] The data observation module 502 is used to collect the work information of each workflow in the enterprise resource system in real time, integrate the work information and extract business data;

[0157] A strategy adjustment module 503 is configured to call the collaborative decision-making model to make collaborative decisions on business data based on the global reward function and the local reward function, and to shape the local reward function based on the potential function to generate a business strategy;

[0158] An automatic decision module 504 is configured to generate a decision instruction corresponding to each executor based on the business policy according to the division of labor of each executor in the enterprise resource system;

[0159] The action execution module 505 is used to call each executor to execute the corresponding decision instruction, monitor the execution process in real time and record the execution results, generate feedback information based on the execution results through a closed-loop control mechanism, and update the work information of each workflow according to the feedback information.

[0160] The system in the embodiment of the present invention can make collaborative decisions based on multiple intelligent agents, automatically and intelligently generate immediate decision instructions based on work information in the enterprise resource system, and can respond to market fluctuations in real time to improve decision-making efficiency.

[0161] In another embodiment of the present application, the potential function is expressed as:

[0162] = (s')- (s);

[0163] Among them, s represents the current state, s' represents the next state, represents the discount factor, (s) represents the potential function of the current state of the strategy, represents the shaping reward of the latent function.

[0164] In another embodiment of the present application, the model training module 501 is further configured to:

[0165] The decision steps of each collaborative workflow in the acquisition enterprise resource system are obtained, the corresponding intelligent agents are configured according to the decision step information, and the local observation space and action space of each intelligent agent are defined.

[0166] In another embodiment of the present application, the global reward function is:

[0167] Global reward = sales price – total cost;

[0168] Total cost = purchase cost + production cost + inventory cost + other variable costs;

[0169] The sales price, purchase cost, inventory cost and other variable costs are determined based on the decision of the collaborative decision-making model.

[0170] In another embodiment of the present application, the automated collaborative decision-making system for enterprise resource planning further includes a policy update module, which is specifically configured to:

[0171] Collect inventory data in real time and calculate inventory turnover rate;

[0172] Determining whether to adjust the strategy update frequency according to the change status of the inventory turnover rate;

[0173] If the inventory turnover rate is less than the turnover rate update threshold, activating a high-frequency update mode, calculating a policy update interval, and updating the network policy parameters of the collaborative decision-making model based on the calculated policy update interval;

[0174] If the inventory turnover rate is not less than the turnover rate update threshold, the baseline update frequency is maintained, and the network policy parameters of the collaborative decision-making model are updated based on the policy update interval set based on the baseline update frequency.

[0175] In another embodiment of the present application, the calculation expression of the turnover rate update threshold is:

[0176] ;

[0177] Among them, the represents the turnover rate update threshold, μ is the mean of the historical inventory turnover rate, is the standard deviation, represents the departure adjustment coefficient, .

[0178] In the system provided in the embodiments of the present invention, the automated decision-making model includes multiple decision-making agents, and these agents can consider a variety of local and global rewards to make collaborative decisions. This automated decision-making model comprehensively considers multiple pieces of information based on these and other local and global rewards to formulate practical strategies, enabling the coupling of multiple businesses and the overall business, intelligently generating real-time decision instructions, and enabling immediate response to market fluctuations, thereby improving decision-making efficiency.

[0179] Based on the same inventive concept, an embodiment of this specification also provides an electronic device for automated collaborative decision-making in enterprise resource planning. The electronic device for automated collaborative decision-making in enterprise resource planning in an embodiment of the present invention is described in detail below from the perspective of hardware processing.

[0180] Figure 6 This is a schematic diagram of the structure of an electronic device provided in the embodiment of this specification. Figure 6 The electronic device 600 according to this embodiment of the present invention will be described. Figure 6 The electronic device 600 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0181] like Figure 6 As shown, electronic device 600 is implemented as a general-purpose computing device. Components of electronic device 600 may include, but are not limited to, at least one processing unit 610, at least one storage unit 620, a bus 630 connecting various system components (including storage unit 620 and processing unit 610), and a display unit 640.

[0182] The storage unit stores program codes that can be executed by the processing unit 610, so that the processing unit 610 performs the steps according to various exemplary embodiments of the present invention described in the above processing method section of this specification. For example, the processing unit 610 can perform the following steps: Figure 1 or Figure 2 Steps shown.

[0183] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0184] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include an implementation of a network environment.

[0185] Bus 630 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0186] The electronic device 600 may also communicate with one or more external devices 100 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable a user to interact with the electronic device 600, and / or any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 through the bus 630. It should be understood that although Figure 6 Not shown, other hardware and / or software modules may be used in conjunction with electronic device 600, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0187] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the exemplary embodiments described in the present invention can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiment of the present invention can be embodied in the form of a software product, which can be stored in a computer-readable storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.) or on a network, and includes a number of instructions to enable a computing device (which can be a personal computer, server, or network device, etc.) to execute the above method according to the present invention. When the computer program is executed by a data processing device, the computer-readable medium is enabled to implement the above method of the present invention, that is: Figure 1 or Figure 2 The method shown.

[0188] Figure 7 A schematic diagram of a computer-readable medium provided in accordance with an embodiment of this specification.

[0189] accomplish Figure 1 or Figure 2 The computer program of the illustrated method can be stored on one or more computer-readable media. The computer-readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0190] The computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein the readable program code is carried. The data signal propagated may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or component. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.

[0191] In addition, the present invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the automated collaborative decision-making method for enterprise resource planning as described in any of the above embodiments.

[0192] Specifically, the program code for performing the operations of the present invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, as a standalone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0193] In summary, the present invention can be implemented in hardware, or as a software module running on one or more processors, or a combination thereof. Those skilled in the art will appreciate that, in practice, general-purpose data processing devices such as microprocessors or digital signal processors (DSPs) can be used to implement some or all of the functions of some or all of the components according to the embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., a computer program or computer program product) for performing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium or in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0194] The specific embodiments described above further illustrate the objectives, technical solutions, and beneficial effects of the present invention. It should be understood that the present invention is not inherently related to any specific computer, virtual device, or electronic device, and various general-purpose devices can also implement the present invention. The above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.

[0195] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0196] If the technical solution of this application involves personal information, the product that applies the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing personal information. If the technical solution of this application involves sensitive personal information, the product that applies the technical solution of this application has obtained the individual's separate consent before processing sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, a clear and prominent sign is set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that they agree to the collection of their personal information; or on the personal information processing device, when the personal information processing rules are notified by obvious signs / information, the individual's authorization is obtained through pop-up information or by asking the individual to upload their personal information; among which, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0197] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. An automated collaborative decision-making method for enterprise resource planning, characterized in that: include: Constructing a multi-agent reinforcement learning model, wherein the multi-agent reinforcement learning model includes a procurement agent, a production agent, and a sales agent; Designing a global reward function for each agent included in the multi-agent reinforcement learning model, designing local reward functions for the procurement agent, production agent, and sales agent, respectively, and constructing latent functions of the local reward functions; training the multi-agent reinforcement learning model based on training data to obtain a collaborative decision-making model; Collect work information of each collaborative workflow in the enterprise resource system in real time, integrate the work information and extract business data; Calling the collaborative decision-making model to make a collaborative decision on the business data according to the global reward function and the local reward function, and simultaneously shaping the local reward function based on the potential function so that the objective of the local reward function is consistent with the objective of the global reward function, thereby generating a business policy; According to the division of labor of each executor in the enterprise resource system, generating decision instructions corresponding to each executor based on the business strategy; Call each executor to execute the corresponding decision instruction, monitor the execution process in real time and record the execution results, generate feedback information through a closed-loop control mechanism based on the execution results, and update the work information of each workflow according to the feedback information.

2. The automated collaborative decision-making method for enterprise resource planning according to claim 1, characterized in that: The expression of the potential function is: = (s’)- (s); Among them, s represents the current state, s' represents the next state, represents the discount factor, (s) represents the potential function of the current state of the strategy, represents the shaping reward of the latent function.

3. The automated collaborative decision-making method for enterprise resource planning according to any one of claims 1 or 2, characterized in that: Before designing a global reward function based on each agent included in the multi-agent reinforcement learning model, the method further includes: The decision steps of each collaborative workflow in the acquisition enterprise resource system are obtained, the corresponding intelligent agents are configured according to the decision step information, and the local observation space and action space of each intelligent agent are defined.

4. The automated collaborative decision-making method for enterprise resource planning according to any one of claims 1 or 2, characterized in that: The global reward function is: Global reward = sales price – total cost; Total cost = purchase cost + production cost + inventory cost + other variable costs; The sales price, purchase cost, inventory cost and other variable costs are determined based on the decision of the collaborative decision-making model.

5. The automated collaborative decision-making method for enterprise resource planning according to any one of claims 1 or 2, characterized in that: After updating the work information of each workflow according to the feedback information, the method further includes: Collect inventory data in real time and calculate inventory turnover rate; Determining whether to adjust the strategy update frequency according to the change status of the inventory turnover rate; If the inventory turnover rate is less than the turnover rate update threshold, activating a high-frequency update mode, calculating a policy update interval, and updating the network policy parameters of the collaborative decision-making model based on the calculated policy update interval; If the inventory turnover rate is not less than the turnover rate update threshold, the baseline update frequency is maintained, and the network policy parameters of the collaborative decision-making model are updated based on the policy update interval set based on the baseline update frequency.

6. The automated collaborative decision-making method for enterprise resource planning according to claim 5, characterized in that: The calculation expression of the turnover rate update threshold is: ； Among them, the represents the turnover rate update threshold, μ is the mean of the historical inventory turnover rate, is the standard deviation of historical inventory turnover rate, Indicates the trigger adjustment coefficient, .

7. An automated collaborative decision-making system for enterprise resource planning, characterized in that: include: a model training module for constructing a multi-agent reinforcement learning model, wherein the multi-agent reinforcement learning model includes a procurement agent, a production agent, and a sales agent; designing a global reward function based on each agent included in the multi-agent reinforcement learning model, designing local reward functions for each of the procurement agent, the production agent, and the sales agent, and constructing latent functions of the local reward functions; and training the multi-agent reinforcement learning model based on training data to obtain a collaborative decision-making model; A data observation module is used to collect work information of each workflow in the enterprise resource system in real time, integrate the work information and extract business data; a strategy adjustment module, configured to call the collaborative decision-making model to make collaborative decisions on the business data according to the global reward function and the local reward function, and simultaneously shape the local reward function based on the potential function so that the objectives of the local reward function are consistent with the objectives of the global reward function, thereby generating a business strategy; An automatic decision-making module, configured to generate a decision instruction corresponding to each executor based on the business strategy according to the division of labor of each executor in the enterprise resource system; The action execution module is used to call each executor to execute the corresponding decision instructions, monitor the execution process in real time and record the execution results, generate feedback information through a closed-loop control mechanism based on the execution results, and update the work information of each workflow according to the feedback information.

8. An automated collaborative decision-making device for enterprise resource planning, characterized in that: The automated collaborative decision-making device for enterprise resource planning includes: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory to enable the automated collaborative decision-making device for enterprise resource planning to execute the steps of the automated collaborative decision-making method for enterprise resource planning according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the program / instructions are executed by a processor, the steps of the automated collaborative decision-making method for enterprise resource planning as claimed in any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the automated collaborative decision-making method for enterprise resource planning according to any one of claims 1 to 6 are implemented.

Images