Adaptive discrete label packet integration method and device, storage medium and computer program product

Abstract

The invention discloses a self-adaptive discrete label packet integration method and device, a storage medium and a computer program product, and belongs to the technical field of artificial intelligence and computer application. The method comprises the following steps: S1, a data preprocessing and state defining step: acquiring label packet data, preprocessing, converting attribute information into numeric features, defining a label packet division state, and constructing a state space of a reinforcement learning model; s2, an action and reward function definition step: defining an action space divided by a label packet and a composite reward function; s3, a reinforcement learning training step: initializing a Q value function, performing iterative training by using a deep Q-learning algorithm based on the state space, the action space and the composite reward function, and updating the Q value function until an optimal label packet division strategy is obtained; and S4, a label packet division application step: performing automatic division and integration on the to-be-divided label packet data. According to the scheme, dynamic self-adaption and full-process automation of label packet division are realized, and multiple targets such as cost, risk and efficiency can be optimized at the same time.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and computer application technology, and in particular to an adaptive discrete standard package integration method, device, storage medium and computer program product. Background Technology

[0002] In supply chain management, especially in centralized procurement scenarios, efficiently and rationally integrating numerous dispersed procurement demands (sub-bid packages) into several more manageable and tenderable bid packages is a complex and critical decision-making problem. Existing bid package partitioning methods largely rely on predefined rules or static algorithms. For example, patent CN202110824044.6 discloses a material requirements planning method and system that optimizes material supply and demand balance based on product BOM structure and inventory data. However, this method has fixed rules, making it difficult to adapt to dynamically changing procurement environments, and its optimization objective is singular, primarily focusing on inventory management. Another example is patent CN202111470404.3, which discloses a regional supply and demand balancing method based on reinforcement learning. This method applies reinforcement learning to optimize supply and demand allocation in just-in-time delivery scenarios, but its application scope is narrow, its optimization objective is singular (such as delivery efficiency), and it lacks the ability to comprehensively balance multiple objectives (such as cost, risk, and efficiency) in complex procurement environments. In summary, the existing technologies have the following main defects: (1) insufficient adaptability, unable to adjust strategies in real time according to the dynamic environment; (2) limited optimization scope, usually only focusing on a single objective; (3) weak dynamic optimization capability; (4) low level of automation, relying on manual intervention; (5) lack of consideration for long-term cumulative benefits. Summary of the Invention

[0003] This invention addresses some of the shortcomings mentioned in the background art by providing an adaptive discrete standard package integration method, device, storage medium, and computer program product.

[0004] In a first aspect, the present invention provides an adaptive discrete standard package integration method, the method comprising: S1. Data preprocessing and state definition steps: Obtain the label packet data to be divided, preprocess the label packet data, convert the attribute information of the label packet into numerical features, define the division state of the label packet, and construct the state space of the reinforcement learning model based on the numerical features and the division state.

[0005] S2. Action and Reward Function Definition Steps: Define the action space of the standard package partition, and the composite reward function used to evaluate the effect of the partitioned actions.

[0006] S3. Reinforcement learning training steps: Initialize the Q-value function, and iteratively train using the deep Q-learning algorithm based on the state space, action space, and composite reward function, updating the Q-value function until convergence, in order to obtain the optimal label splitting strategy.

[0007] S4. Label Packet Partitioning Application Steps: Utilize the optimal label packet partitioning strategy obtained through training to automatically partition and integrate the label packet data to be partitioned.

[0008] In one embodiment of the first aspect, in step S2, the action and reward function definition step, the actions in the action space include at least one of the following: grouping by item category, dividing by geographical location, and grouping by package amount.

[0009] In one embodiment of the first aspect, in step S3, the reinforcement learning training step, the composite reward function calculates the overall reward value by linearly weighting multiple optimization objective indicators; the optimization objective indicators include at least: package purity, supplier supply risk, and supplier historical supply matching degree.

[0010] In one embodiment of the first aspect, the purity of the bid package is calculated as follows: it is quantified based on the diversity of materials within the bid package or the consistency of their regional distribution; the supplier supply risk is quantified based on the frequency of historical adverse events; and the supplier's historical supply matching degree is quantified based on the correlation coefficient between the supplier's historical supply and the bid package demand.

[0011] In one embodiment of the first aspect, during the reinforcement learning training step, an ε-greedy strategy is employed for action selection to balance exploration and exploitation; and the ε value decays as the training process progresses.

[0012] In one embodiment of the first aspect, the Q-value function is updated according to the following formula: in, and These represent the current state and the action, respectively. This indicates the reward received. Indicates the next state. For learning rate, This is the discount factor.

[0013] In a second aspect, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method steps of the first aspect.

[0014] In a third aspect, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method steps of the first aspect.

[0015] In a fourth aspect, a computer program product is provided, which, when executed by a processor, implements the method steps of the first aspect.

[0016] Compared with the prior art, the present invention has at least the following beneficial effects: 1. Dynamic Adaptation: Through reinforcement learning models, the partitioning strategy can be dynamically adjusted according to real-time data and environmental changes, which significantly improves adaptability in complex and ever-changing scenarios.

[0017] 2. Multi-objective optimization: Through the designed composite reward function, multiple objectives such as package purity, supplier risk, and historical matching degree can be optimized simultaneously to achieve the global optimal decision.

[0018] 3. High automation: It realizes full automation from data input to standard package division result output, which greatly reduces manual intervention and improves efficiency.

[0019] 4. Long-term benefit considerations: The cumulative reward mechanism of reinforcement learning enables the model to consider not only immediate benefits but also long-term value when making decisions, thus avoiding short-sighted decisions.

[0020] 5. Strong generalization ability: This method does not rely on fixed rules in a specific domain. By adjusting the weight of the reward function, it can adapt to the standard package integration requirements of different industries and business scenarios. Attached Figure Description

[0021] Figure 1 This is a flowchart of the adaptive discrete standard package integration method provided in the embodiments of the present invention; Figure 2 This is a detailed flowchart of the data preprocessing and state definition steps in an embodiment of the present invention; Figure 3 This is a schematic diagram of the reinforcement learning training steps in an embodiment of the present invention; Figure 4 This is a schematic diagram of the label segmentation result of an application embodiment of the present invention. Detailed Implementation

[0022] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0023] This invention proposes a tag package ensemble optimization technique based on deep Q-learning reinforcement learning. The core innovations include the tag package definition format and reward mechanism. Specifically, by constructing a unified tag package definition format, tag package attribute information is transformed into a structured state matrix and partitioning state vectors, forming an extended state space suitable for reinforcement learning processing. Simultaneously, a composite reward mechanism is designed to quantify key indicators such as tag package purity, supplier risk, and matching degree, and combined with a dynamically updated reinforcement learning model, the partitioning strategy is optimized and adjusted. This achieves automated and adaptive tag package partitioning.

[0024] like Figure 1 As shown, the steps of the adaptive discrete standard package integration method of a specific embodiment of the present invention can be divided into four steps: Step S1 mainly involves data preprocessing and state definition. This stage includes three parallel processing modules: processing continuous discrete values ​​of the standard package, quantifying the attributes of the standard package, and defining the state of the sub-packages. The processing results of these modules are combined to form the standard package state matrix, laying the foundation for subsequent processing.

[0025] Steps S2 and S3 focus on defining the key components and parameters of the system. This phase includes three core parts: defining the standard package partitioning operation, defining the weighted reward function, and calculating the Q-value matrix. These components are connected through two important steps: state association and initialization, ultimately forming the initial configuration for Q-learning.

[0026] Step S4 is the system execution and optimization phase. In this phase, the system continuously optimizes the processing results through multiple iterations and experience caching. Specifically, it includes three key steps: Q-value updates, greedy pruning to accelerate convergence, and recording local optimal operations. These processes ultimately aggregate to form the globally optimal partitioning scheme and output the final standard package allocation result.

[0027] Specifically, the stage of quantitative description of package attributes includes: (1) Data preprocessing: The data of the standard packages to be divided are preprocessed to extract meaningful features and convert them into numerical data. For example, categorical variables such as item category and location can be converted into numerical variables using one-hot encoding.

[0028] The input data contains multiple attributes for each label, mainly including the following fields: Item categories (e.g., cables, steel pipes, transformers) Location (e.g., Zhengzhou or Luoyang in Henan Province) Unit price (e.g., 2 million yuan, 3 million yuan) Purchase quantity (e.g., 500 units, 1000 units) Subordinate municipal bureaus (such as Zhengzhou Bureau, Luoyang Bureau) Batch information (e.g., batch A, batch B) Let the set of each fine-grained sub-label be . ,in The attributes of each bid package include item category, unit price, and quantity purchased. .correspond This indicates the category to which the label belongs, such as the category of municipal bureau or batch information. Each value in this category is a discrete value. To facilitate model training and prediction, one-hot encoding is used to convert each discrete value into a unique binary vector. The length of this vector is equal to the total number of possible discrete values, and only the element corresponding to the discrete value is 1, with the rest being 0. For example, assuming the categorical variable of location has three values: Kunming, Guiyang, and Nanning, the corresponding three one-hot encoded vectors are as follows: Kunming: Guiyang: Nanning: For each package The algorithm assigns its properties One-hot encoding transformation yields the feature vector. Where k is the total number of all categorical variables, for the _th If there are categorical variables, then their corresponding values ​​are:

[0029] Then, for each package, its item category is converted into a binary vector using one-hot encoding. The length of the vector is equal to the total number of item categories, where only the element corresponding to that item category is 1, and the other elements are 0. For example, if an item category has 5 possible values, the one-hot encoded vector for each item category would be: .

[0030] The item category features of each package are transformed into a numerical vector. In the adaptive discrete package ensemble method based on deep Q-learning reinforcement learning, we can use these feature vectors for model training to improve the efficiency and accuracy of package ensemble. Simultaneously, to ensure consistent package feature dimensions, i.e., using the state matrix... storage Individual packages and There are 3 features, each with a different length. The filling matrix is... Size is Fill the elements of the matrix. Indicates the first The first sample The values ​​of each feature.

[0031] For continuous features such as package amount, floating-point numbers are used for storage. As a package amount feature in the state matrix, no additional preprocessing is required.

[0032] (2) State definition: In the package integration problem, the state consists of two parts: the attribute information (features) of the package: reflecting the static characteristics of the package, such as item category, location, unit price, etc.; and the division status of the package: reflecting whether the current package has been divided into a certain integrated package, represented by binary variables.

[0033] Data type: ● Categorical variables (such as item category, location): Convert to binary vectors using one-hot encoding.

[0034] ● Continuous variables (such as amount or quantity): Use normalization or standardization to convert them into dimensionless values.

[0035] After processing: ●State matrix S: Extract features and quantify them.

[0036] ● Divide the state vector C: Extract the partition state.

[0037] ● Expanding the state space: Combining the two as input to the reinforcement learning model.

[0038] The label package integration problem is abstracted into a state space. A feature extraction method is used to transform the label package description data into a set of feature vectors at each time step. The current feature vector is used as the state. Assume there is Each package contains [number] bid packages. If there are several attributes, then the attribute information for each packet can be represented as a single attribute. dimensional vector At the same time, all The attribute information of each target package is concatenated together to obtain a state matrix Used to record each sub-label packet:

[0039] Simultaneously, the partitioning status of each sub-package is recorded, i.e., whether it has been partitioned into a certain integrated package. The partitioning status of each sub-package can then be represented as a... dimensional vector ,in Indicates the first The partitioning status of each sub-label packet. The partitioning status can be represented by binary variables, for example: 0 indicates that it has not been partitioned, and 1 indicates that it has been partitioned. Further, the attribute information status matrix... and partitioning state vectors Combined, they form an extended state-space representation: At every moment The current feature vector can be used as the state. Then, based on this state, further decisions or operations are made. It's important to note that the dimension of the state space depends on the number of sub-labels. and the number of attributes in each sub-package When the number of sub-labels or the number of attributes changes, the dimensions of the state space will also be adjusted accordingly. Simultaneously, data structures can be designed to store the merged label information, such as: Ensemble Package Data Structure: i. package_id: A unique identifier for the integrated package; ii. properties: Used to store attribute information of the integrated package; it can be a dictionary or other appropriate data structure. iii. sub_packages: Used to store a list of subpackages that have been merged into this integration package.

[0040] Subpackage data structure: i. package_id: A unique identifier for the sub-package, which can be the same as the package_id of the integrated package or use other methods to determine its uniqueness; ii. properties: Used to store the attribute information of the sub-label package, which can be a dictionary or other appropriate data structure.

[0041] Specifically, the definition of the label-based reinforcement learning function includes: (1) Action definition: at each moment The system can select multiple actions. Each action represents a way of dividing the current state. For example, it could be divided according to item category, location, or bid price. Given the current state... Based on data structures, we can define a set of actions. These actions are used to divide the current state. In the scenario of merging packages, these actions can represent how the integrated packages are divided, such as by item category, location, package amount, etc. Below is an example action definition: Grouped by item category: i. Action Name: Classified by item category; ii. Action Description: Divide the integrated package according to the category of items in the sub-package, so that sub-packages of the same category are merged into the same integrated package.

[0042] By location: i. Action Name: Classified by Location; ii. Action Description: Divide the integrated package according to the location information of the items in the sub-package, so that sub-packages in the same location are merged into the same integrated package; Consolidated by package amount: i. Action Name: Classified by package amount; ii. Action Description: Divide the integrated bid package according to the amount information of the sub-bid packages, so that sub-bid packages with similar amounts are merged into the same integrated bid package.

[0043] Depending on specific business needs, decision-making actions derived from package attributes can be introduced to further segment integrated packages. However, the exact action definitions must be determined based on specific business scenarios and application conditions. At each moment... Users can adjust their current status Choose the appropriate action in the given situation. This allows for further segmentation and integration of the bid packages, enabling the manipulation and decision-making of the merged bid package information.

[0044] (2) Definition of reward function: Define a reward function. The reward function is used to evaluate the quality of the current state and the actions taken. For this scheme, rewards can be given based on factors such as the purity of the supplied goods in the bid package, the purity of the regions included in the bid package, the supplier's supply risk, and the matching degree of the supplier's historical supply volume. For example, if the bid package allocation results in high purity, low risk, and high historical supply volume matching, a positive reward value is given. The definition of the reward function plays a crucial role in the Q-learning algorithm; it is used to evaluate the current state. And take corresponding actions. The degree of quality. For the integrated package segmentation scheme, we can define the reward function based on factors such as package purity, supplier supply risk, and the matching degree of the supplier's historical supply volume. .

[0045] Specifically, this invention employs a linear combination approach to construct the reward function, multiplying each factor by its weight and summing the results to obtain the overall reward value. Assume the weight of the purity of the standard package is... The weight of supplier supply risk is The weight of the supplier's historical supply volume matching degree is: The reward function can then be calculated using a linear formula. The Bellman optimal equation is calculated as follows:

[0046] in, The purity of the label package representing the partitioning result. This indicates the supplier's supply risk. The supplier's historical supply matching degree is represented by the following calculation method: Standard package purity ( ): Package purity is a quality indicator for the classification results, used to assess the diversity and coverage of materials within a package. Typically, package purity is determined by calculating the ratio of the required number of material types to the total number of material types. The calculation method is as follows: = (Quantity of required material types / Total quantity of material types) 100%.

[0047] The method for calculating the purity of a package aims to quantify the degree of diversity of materials contained within the package. A higher purity value indicates that the package contains a richer and more comprehensive variety of materials.

[0048] Supplier risk ( ): Supplier risk is an indicator used to assess the potential threat a supplier poses to the stability and sustainability of business operations. A common calculation method is based on the ratio of the frequency of adverse events to the total number of orders. The calculation method is as follows: (Number of adverse events / Total number of orders) Supplier risk calculation aims to measure the problems and potential risks that suppliers may encounter during the delivery process. A higher risk value indicates that the supplier faces greater delivery risk.

[0049] Supplier matching degree ( ): Supplier-to-bid matching is an indicator that assesses the correlation between a supplier's historical supply volume and the bid demand. A common calculation method is to use a correlation coefficient to measure the relationship between supply and demand. The calculation method is as follows: Correlation coefficient (supply, demand).

[0050]

[0051] Due to the unique requirements of each subdivision scenario, i.e., different focuses, business personnel need to flexibly adjust the weights to adapt to the subdivision scenario:

[0052] The following is an example of purity reward quantification. Assume a package has the following characteristics: it involves 3 regions, for a total of 10 regions; it involves 5 product categories, for a total of 20 product categories; the regional purity weight w1 = 0.6, and the category purity weight w2 = 0.4. The quantification calculation steps are as follows: ●Regional purity: PR = 1 - (3 / 10) = 0.7 ●Category purity: PC = 1 - (5 / 20) = 0.75 ● Overall purity of standard package: P = 0.6 0.7 + 0.4 0.75 = 0.72 The purpose of calculating the supplier-to-package matching score is to measure the degree of fit between the supplier's past supply volume and the package demand. A higher matching score indicates that the supplier and package demand are more closely aligned.

[0053] According to the Bellman equation, the following formula can be used to update... :

[0054] in, It is the learning rate This determines the degree of trade-off between the old and new values ​​during each update; Discount factor This indicates the importance of future rewards; Indicates the next state China will take all possible actions. The obtained maximum value function estimate.

[0055] Through continuous iteration and updates The table allows the value function to gradually converge to the optimal value function, thus obtaining the optimal strategy. Generally, the Q-learning algorithm undergoes multiple rounds of training, continuously interacting with and updating the environment in each round. The table continues until the value function converges or a certain number of training iterations are reached.

[0056] First, we need to initialize the state space, which represents different states and their characteristics. Based on the specific circumstances, this study defines a state matrix. At the same time, we also need to initialize a partition state vector. This is used to record the partitioning state of each sub-label. The initial feature vector dimension is [dimensionality missing]. Secondly, the initial value for the state vector is the length. The zero vector, where Indicates the number of sub-labels. This indicates the number of attributes in each sub-item package.

[0057] Secondly, the action space needs to be initialized, representing the different action methods available. Depending on the specific scenario requirements, we can define multiple actions, such as grouping by item category, by location, or by package price. Each action has a name and description, which can be defined according to the actual situation. For example, initializing three decision action operations: Action 1: Merge by Item Category Description: Merge two items of the same category into the same package; Action 2: Location Division Description: Merge two bids located at the same location into the same bid; Action 3: Grouping by Package Amount Description: Group packages with similar amounts together; Finally, we need to initialize the Q-value function, which estimates the cumulative reward obtained by taking an action in the current state. According to the text description, we can use random initialization to ensure that each state-action pair has an initial value.

[0058] (3) Definition of Q-value function: This invention provides a method for defining a Q-value function. This method can be used to estimate the long-term cumulative reward of taking an action in the current state. The Q-value function is used to evaluate the long-term value of each state-action pair. This function is equivalent to a mapping whose input is the current state and the action pair, and whose output is the predicted cumulative reward. Initially, the Q-value function can be randomly initialized. First, define a Q-value function. This is used to estimate the long-term cumulative reward of taking an action in the current state. Initially, the Q-value function can be randomly initialized; that is, without prior knowledge, the Q-value of all state-action pairs is set to random values ​​to ensure the system has sufficient exploratory power. As the model iteratively learns from historical data, the Q-value function is updated based on empirical data to gradually approximate the true long-term reward.

[0059] Specifically, the label allocation reinforcement learning adaptive includes (1) Q-value function update: Updated according to the iterative formula of the Q-learning algorithm:

[0060] in, It's the learning rate. It is a discount factor. Indicates the current state and actions The next state obtained after the transition Indicates the next state Execute all actions in The maximum Q-value is obtained by iteratively updating the Q-value function. This allows us to gradually obtain the optimal mapping of execution actions for packet allocation under different resource states, reflecting the long-term value of various actions taken under different states.

[0061] (2) Definition of Learning Strategy: The learning strategy defines how the reinforcement learning algorithm explores and utilizes knowledge to find the optimal packet partitioning scheme. The main task of the strategy is to obtain the current optimal strategy while also retaining the opportunity to explore new actions, thus avoiding getting trapped in local optima during the learning process. At each time step... The system determines the current state. Choose an action using the Q-value function. To balance exploration and exploitation, it is commonly used - Greedy strategy, that is, using a certain probability Randomly select an action, to The probability of choosing the action with the highest Q-value function value is considered. In this invention, if there are certain decision biases during the label partitioning process, a greedy weight with decay can be used for pruning to accelerate the decision-making process: Define a variable with an initial value of 1.0. This is used to control the probability of randomly selecting an action. A decay rate is also set. Used for each update The value, specifically at each time step t (t is less than the maximum number of steps). The following operations are performed: i. Get the current state and get all available actions. ; ii. Use - Greedy strategy selects one action If one is in Random numbers within the range are less than Randomly select an action Otherwise, calculate the value of each action using the Q-value function and select the action with the maximum value; iii. Perform the action Obtain the next state and rewards ; iv. Update the Q-value function to adjust the action value based on the action taken and the reward received; v. Based on the set attenuation rate renew The value, which is about to Multiply ; vi. Time step .

[0062] (3) Training process During training, the Q-learning model continuously updates its Q-value function through interaction with real-world data, thereby gradually learning and optimizing the standard package ensemble partitioning strategy.

[0063] Q-learning models interact with real-world data. Specifically, during training, the Q-learning algorithm interacts with the actual labeled data in the following ways: (1) Input: The model receives actual bid package data as state input, including features such as the size of the bid package, geographical location, and material type.

[0064] (2) Action execution: The model selects an action (such as merging or splitting the package) based on the current strategy.

[0065] (3) Feedback acquisition: The model calculates the reward value based on the results after the action is performed (such as cost savings, changes in supplier risk, etc.).

[0066] (4) State update: After the action is executed, the model observes the new state of the target package.

[0067] (5) Q-value update: The model updates the Q-value table based on the reward obtained and the new state.

[0068] This interactive process is automated, driven by the Q-learning algorithm, and requires no manual intervention. To accelerate the learning process, the model first initializes the Q-value function using historical package partitioning and its corresponding results, ensuring the model has some experience at the start of training and avoiding random exploration from scratch. Similarly, the model can skip a large number of initial random exploration phases, approaching the optimal package partitioning strategy more quickly, thereby reducing training time and accelerating the convergence of the Q-value function. At each procurement task or package partitioning, the Q-value function is adjusted based on new data and the current state-action pair. After each action, the model updates the Q-value based on the reward received and the next state, allowing it to gradually optimize its decisions. Ultimately, the Q-value function converges to reflect the long-term value of each action taken under different states, enabling more informed partitioning decisions in future tasks. For example, within a procurement batch framework, there are multiple possible package partitioning strategies, such as grouping a certain type of accessory from different regions into different packages. The model evaluates the long-term benefits of each strategy through interaction with the data and selects the package partitioning strategy that maximizes overall utility. The final result is: Package Set: This is the set of package partitioning results after model optimization. For example, the package set may include different packages such as "Package A (including accessories for Region 1 and Region 2)" and "Package B (including accessories for Region 3)".

[0069] The Q-value function is logically designed with a bit table containing the following key information: First, state-action pairs: each possible state and action combination. Second, the Q-value: the expected long-term cumulative reward for each state-action pair, as shown in the bit table below, which represents the long-term value of performing an action in different states:

[0070] Policy Mapping: Based on the Q-table, we can derive the optimal policy mapping, which associates each state with the action that has the highest Q-value. For each unique state, find the action with the highest Q-value. Further pair this state with the optimal action to form the policy mapping. Example policy mapping:

[0071] The optimal packet partitioning strategy is obtained by mapping each state to it, which corresponds to the model's strategy output, i.e., the optimal packet partitioning decision scheme is obtained:

[0072] (4) Application of Optimal Partitioning Strategy: The learned optimal partitioning strategy is used to partition and integrate new data into sub-sub ... In Embodiment 1 of the present invention, refer to Figure 1 This embodiment provides an adaptive discrete standard package integration method, the specific steps of which are as follows: like Figure 2 As shown, step S1, data preprocessing and state definition: Enter the original bid package data, for example: `[{Location: "Zhengzhou City", Material: "Cable", Amount: 500, Risk: "Medium"}, ...]`.

[0073] Data cleaning and transformation: Categorical variables such as "location" and "materials" are one-hot encoded. For example, if "location" has three values: `Zhengzhou City, Luoyang City, and Kaifeng City`, then "Zhengzhou City" is encoded as `[1,0,0]`. Continuous variables such as "amount" are normalized to their minimum and maximum values, scaling them to the `[0,1]` range.

[0074] Construct the state matrix S: Stack all the feature vectors after processing the labels to form a matrix of \( n \times m \), where \( n \) is the number of labels and \( m \) is the total number of features.

[0075] Construct the partition state vector C: Initialize a zero vector of length \( n \) to indicate that all packets are initially unpartitioned.

[0076] Constructing extended states: Combine the state matrix \(S \) and the partitioned state vector \(C \) as the complete state input for the reinforcement learning model.

[0077] Step S2, Defining the Action and Reward Functions: Action space A: defined as `{Action 1: group by item category, Action 2: divide by location, Action 3: group by amount}`.

[0078] The reward function R is defined as \( R = w1 \cdot P + w2 \cdot (1 - Risk) + w3 \cdot M\).

[0079] Package purity: Calculates the consistency of material categories or regions within a package. For example, a package containing only "cables" has a higher purity than a package containing mixed materials.

[0080] Supplier risk: Calculated based on the supplier's historical adverse event rate.

[0081] Matching degree: Calculates the correlation coefficient between the supplier's historical supply capacity and the current demand for the tender package.

[0082] Weight configuration: In the "efficiency first" scenario, you can set `w1=0.6, w2=0.2, w3=0.2`; in the "risk sensitive" scenario, you can set `w1=0.2, w2=0.6, w3=0.2`.

[0083] like Figure 3 As shown, step S3, reinforcement learning model training: Initialize the Q-value table or Q-network.

[0084] Set the hyperparameters as follows: learning rate (alpha = 0.1), discount factor (gamma = 0.9), initial exploration rate (epsilon = 1.0), and decay rate (lambda = 0.99).

[0085] Multiple rounds of iterative training are performed. At each step, the model randomly selects an action (exploration) with a probability of \( \epsilon \) based on the current state \( s_t \), or selects the action with the largest current Q value (exploitation) with a probability of \( 1- \epsilon \).

[0086] Execute the action \( a_t \) (e.g., execute "split by location"), the environment transitions to the new state \( s_{t+1} \), and the reward \( r_t \) is calculated.

[0087] Update the Q value according to the Q-learning formula: \( Q(s_t, a_t) \leftarrow Q(s_t, a_t) + \alpha [r_t + \gamma \max_{a} Q(s_{t+1}, a) - Q(s_t, a_t)] \).

[0088] Decaying exploration rate: \( \epsilon \leftarrow \epsilon \times \lambda \).

[0089] Repeat the above steps until the Q value converges or the preset number of training rounds is reached.

[0090] like Figure 4 As shown, step S4, application of the optimal strategy: After training, save the optimal Q-value function. For any new state, simply select the action (a) that maximizes \(Q(s, a) \) to obtain the optimal split decision.

[0091] Applying this strategy to divide a procurement batch, the output is as follows: Figure 4 The optimized set of bid packages shown includes integrated region, materials, total price, and evaluation metrics for each package.

[0092] The method described in this embodiment enables efficient and intelligent integration of bid packages in the centralized procurement system of the supply chain, significantly improving procurement efficiency and the rationality of resource allocation.

[0093] The reward mechanism is the core of this technical solution. By quantifying the effect of the target package division, it provides a clear evaluation basis and feedback mechanism for the optimization model. The weight of the indicators can be adjusted according to different business scenarios, enabling the system to achieve a balance between cost, efficiency, and risk, and meet diverse needs.

[0094] The reward mechanism goes beyond simple metric score calculations. It comprehensively considers the performance of package segmentation across multiple dimensions, such as the purity and consistency within the package, the risk of supplier delivery, and the match between supply capacity and package requirements. The system transforms these factors into quantifiable reward values ​​and dynamically adjusts metric weights to adapt to different business scenarios. Furthermore, the reward mechanism considers long-term benefits. Through a cumulative reward design in reinforcement learning, the system can weigh short-term and long-term interests when evaluating current segmentation actions. After each segmentation, the system adjusts its strategy based on real-time feedback and gradually optimizes it through a dynamic update mechanism using reinforcement learning. This design not only enhances the system's adaptability but also enables the package integration strategy to flexibly respond to complex business needs and changes.

[0095] The definition format of the label package forms the basis of this invention. By extracting and formatting the label package attributes, a structured state space suitable for reinforcement learning models is constructed. Through this unified definition format, the system can efficiently process multi-source data, realize the structured expression of the label package, and adapt to complex business needs.

[0096] Specifically, the process begins by extracting core attribute information from the bid packages, including item category, geographical location, unit price, purchase quantity, and supplier characteristics. These attributes are then categorized into categorical and continuous variables. Categorical variables (such as item category and location) are converted into binary vectors using one-hot encoding to ensure a structured representation of discrete values; continuous variables (such as price and quantity) are normalized to eliminate dimensional differences. Next, the attribute information of each bid package is combined into a fixed-dimensional feature vector. These feature vectors from all bid packages are then used to construct a state matrix, which describes the current ensemble state of the bid packages. Simultaneously, a partitioning state vector records whether each bid package has been assigned to a specific ensemble. Finally, the state matrix and the partitioning state vectors are combined to form an extended state space, providing a comprehensive and dynamic input foundation for the reinforcement learning model.

[0097] The technical solution of the present invention has the following technical advantages: First, by introducing a reinforcement learning algorithm, the technical solution of this invention achieves dynamic adaptive capability for label packet partitioning and ensemble. The system can automatically adjust the partitioning strategy according to the real-time data environment and constraints, avoiding the limitations of fixed rules in the prior art and significantly improving the adaptability of label packet partitioning in complex and ever-changing scenarios.

[0098] Secondly, the technical solution of this invention designs a composite reward function that comprehensively considers multiple objectives such as package purity, supplier risk, and historical matching degree, and adapts to different scenarios through dynamic weight adjustment. Compared with the limitations of existing technologies that can only optimize a single objective, the technical solution of this invention can achieve global optimality in multi-objective scenarios, ensuring that the partitioning strategy simultaneously meets the requirements of cost minimization, efficiency maximization, and risk control.

[0099] Furthermore, the technical solution of this invention significantly improves the automation and efficiency of packet partitioning through state-action modeling and reward mechanisms using reinforcement learning. The system can quickly evaluate the long-term benefits of different partitioning strategies through multiple rounds of Q-value iteration, reducing computational overhead and human intervention while achieving full-process automation, thus overcoming the efficiency bottleneck of existing technologies in large-scale applications.

[0100] Finally, through the cumulative reward mechanism of reinforcement learning, the technical solution of this invention can balance short-term gains and long-term value, optimizing the overall effect of standard package ensemble. When making decisions, the system not only focuses on the current state but also dynamically adjusts its strategy based on the potential value of future states, improving partitioning accuracy and long-term gains. This capability enables the technical solution of this invention to significantly improve the overall effect and stability of standard package ensemble while adapting to complex constraints.

[0101] The technical solutions of the embodiments of the present invention have a wide range of applications, low application difficulty, and high feasibility.

[0102] Application Scope: The technical solution of this invention constructs a general standard package integration framework based on reinforcement learning technology, which is applicable to multiple fields including supply chain management, logistics optimization, and large-scale procurement segmentation. It has particularly high practical value in scenarios involving complex constraints (such as multiple suppliers, multiple material categories, and regional distribution). Furthermore, this framework can also be extended to industries such as manufacturing, construction, and centralized government procurement.

[0103] Difficulty of Application: The Q-learning algorithm and composite reward function design used in the technical solution of this invention rely on existing mature algorithms. Combined with state space construction and feature vector preprocessing methods, it is easy to integrate with the current business processes and data systems of enterprises. Its introduction and deployment do not require large-scale changes to the existing enterprise processes, so its application difficulty is low.

[0104] Feasibility: The automation and dynamic optimization capabilities of the technical solution of this invention can directly reduce manual intervention, lowering the time and labor costs for enterprises in bid package allocation. Simultaneously, by precisely optimizing the bid package allocation strategy, resource utilization efficiency and supply chain management quality can be improved, helping enterprises achieve cost reduction and efficiency improvement goals. In terms of technical implementation, reinforcement learning algorithms have extensive application experience, making system development and deployment highly feasible.

[0105] The technical solution of this invention has broad application prospects in the market, and is especially suitable for industries with complex data and urgent needs for benchmarking optimization. It can create direct economic value and improve management efficiency for enterprises.

[0106] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program that can run on the processor, and the processor executes the computer program to implement the above-described image processing method steps.

[0107] In one embodiment, a computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of an adaptive discrete package integration method.

[0108] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

Claims

1. An adaptive discrete standard package integration method, characterized in that, The method includes: S1. Data preprocessing and state definition steps: Obtain the label packet data to be divided, preprocess the label packet data, convert the attribute information of the label packet into numerical features, define the division state of the label packet, and construct the state space of the reinforcement learning model based on the numerical features and the division state. S2. Action and Reward Function Definition Steps: Define the action space partitioned by the standard package, and the composite reward function used to evaluate the effect of the partitioned actions; S3. Reinforcement learning training steps: Initialize the Q-value function, and based on the state space, action space and composite reward function, use the deep Q-learning algorithm to perform iterative training, update the Q-value function, until convergence, so as to obtain the optimal label packet partitioning strategy. S4. Label Packet Partitioning Application Steps: Utilize the optimal label packet partitioning strategy obtained through training to automatically partition and integrate the label packet data to be partitioned.

2. The adaptive discrete standard package integration method according to claim 1, characterized in that, In the S1 data preprocessing and state definition step, the attribute information includes categorical variables and continuous variables; The transformation of the attribute information includes: Perform one-hot encoding on the categorical variables and convert them into binary vectors; Normalize continuous variables; The attribute information of all processed packets is combined into a state matrix; A binary partition state vector is used to record whether each sub-label packet has been partitioned; The state matrix and the partitioned state vector are combined to form an extended state space, which serves as the input to the reinforcement learning model.

3. The adaptive discrete standard package integration method according to claim 1, characterized in that, In the S2 action and reward function definition step, the actions in the action space include at least one of the following: grouping by item category, dividing by geographical location, and grouping by package amount.

4. The adaptive discrete standard package integration method according to claim 1, characterized in that, In the S3 reinforcement learning training step, the composite reward function calculates the overall reward value by linearly weighting and combining multiple optimization objective indicators; The optimization target indicators include at least: standard package purity, supplier supply risk, and the matching degree of the supplier's historical supply volume.

5. The adaptive discrete standard package integration method according to claim 4, characterized in that, The purity of the standard package is calculated as follows: it is quantified based on the diversity of materials within the standard package or the consistency of regional distribution; the supplier supply risk is quantified based on the frequency of historical adverse events; and the supplier's historical supply matching degree is quantified based on the correlation coefficient between the supplier's historical supply and the demand for the standard package.

6. The adaptive discrete standard package integration method according to claim 1, characterized in that, In the reinforcement learning training steps, an ε-greedy strategy is used for action selection to balance exploration and exploitation; and the ε value decays as the training process progresses.

7. The adaptive discrete standard package integration method according to claim 1, characterized in that, The Q-value function is updated according to the following formula: in, and These represent the current state and the action, respectively. This indicates the reward received. Indicates the next state. For learning rate, This is the discount factor.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by the processor, they implement the method as described in any one of claims 1 to 7.

Images