Dynamic system collaborative fast consumer e-commerce pricing prediction method based on deep reinforcement learning

Abstract

The application relates to the technical field of artificial intelligence and decision optimization, in particular to a dynamic system cooperative fast-consumption e-commerce pricing prediction method based on deep reinforcement learning. The method proposes an end-to-end cooperative decision model. The model fuses system state multi-source data through a multi-modal feature extraction network, outputs initial prediction values of each decision variable by using a time series prediction network, and introduces an Actor-Critic framework-based reinforcement learning optimizer. The optimizer maximizes a hybrid objective function which combines a profit target and a supervision constraint, jointly optimizes and adjusts the variables, and finally outputs the cooperatively optimal decision values. The application realizes joint dynamic optimization of coupled variables, effectively improves the accuracy, training stability and overall profitability of the decision.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and decision optimization technology, specifically to a dynamic system collaborative fast-moving consumer goods e-commerce pricing prediction method based on deep reinforcement learning. Background Technology

[0002] In dynamic business systems, such as retail pricing and inventory management, price decisions and sales forecasting are two core and closely related processes.

[0003] In existing schemes, price and sales volume forecasts are often independent and do not take into account the impact of changes in sales volume and price, resulting in inaccurate forecasts. Summary of the Invention

[0004] In view of this, the embodiments of this application aim to provide a dynamic system collaborative fast-moving consumer goods e-commerce pricing prediction method based on deep reinforcement learning.

[0005] The first aspect of this application provides a dynamic system-based collaborative pricing prediction method for fast-moving consumer goods (FMCG) e-commerce based on deep reinforcement learning, including:

[0006] Acquire multi-source system state data for the target to be decided; The multi-source data of the system state are input into a pre-trained collaborative decision-making model; The collaborative decision-making model is used to process the data and output the values ​​of the first and second decision variables that are optimized collaboratively. The collaborative decision-making model is a deep neural network model trained using an end-to-end architecture, and includes: Feature extraction networks are used to fuse and learn representations of input data and output a unified state representation vector. A first prediction network is used to output an initial predicted value for a first decision variable based on the unified state representation vector; wherein the first decision variable is price; The second prediction network is used to output the initial predicted value of the second variable based on the unified state representation vector; wherein the second variable is sales volume. The reinforcement learning optimizer adjusts the first decision variable based on the unified state representation vector, the initial predicted value of the first decision variable, and the initial predicted value of the second variable, to obtain the first decision variable corresponding to the maximum objective function. The objective function includes: a first supervised loss term, a second supervised loss term, and a policy objective term; The first supervision loss term is inversely proportional to the first adjustment amount; the first adjustment amount is the adjustment between the adjusted first decision variable and the initial predicted value of the first decision variable; The second supervision loss term is inversely proportional to the second adjustment amount; the second adjustment amount is the adjustment between the adjusted second variable and the initial predicted values ​​of the second variable. The strategic objective is the profit corresponding to the first decision variable and the second variable.

[0007] Optionally, the system status multi-source data includes at least one of user characteristics, product characteristics, historical transaction sequence characteristics, and market environment context characteristics.

[0008] Optionally, the feature extraction network is a multimodal feature extraction network.

[0009] Optionally, both the first prediction network and the second prediction network are time series prediction networks.

[0010] Optionally, the reinforcement learning optimizer employs the Actor-Critic framework.

[0011] Optionally, the profit is calculated as: (first decision variable - cost) × second variable.

[0012] The second aspect of this application provides a dynamic system collaborative fast-moving consumer goods (FMCG) e-commerce pricing prediction device based on deep reinforcement learning, comprising: The acquisition module is used to acquire multi-source system state data of the target to be decided. The input module is used to input the multi-source data of the system state into a pre-trained collaborative decision-making model; The output module is used to process the collaborative decision-making model and output the values ​​of the first and second decision variables for collaborative optimization. The collaborative decision-making model is a deep neural network model trained using an end-to-end architecture, and includes: Feature extraction networks are used to fuse and learn representations of input data and output a unified state representation vector. A first prediction network is used to output the initial predicted value of the first decision variable based on the unified state representation vector; The second prediction network is used to output the initial predicted value of the second variable based on the unified state representation vector; The reinforcement learning optimizer adjusts the first decision variable based on the unified state representation vector, the initial predicted value of the first decision variable, and the initial predicted value of the second variable, to obtain the first decision variable corresponding to the maximum objective function. The objective function includes: a first supervised loss term, a second supervised loss term, and a policy objective term; The first supervision loss term is inversely proportional to the first adjustment amount; the first adjustment amount is the adjustment between the adjusted first decision variable and the initial predicted value of the first decision variable; The second supervision loss term is inversely proportional to the second adjustment amount; the second adjustment amount is the adjustment between the adjusted second variable and the initial predicted values ​​of the second variable. The strategic objective is the profit corresponding to the first decision variable and the second variable.

[0013] Optionally, the system status multi-source data includes at least one of user characteristics, product characteristics, historical transaction sequence characteristics, and market environment context characteristics.

[0014] Optionally, the feature extraction network is a multimodal feature extraction network; both the first prediction network and the second prediction network are time series prediction networks; and the reinforcement learning optimizer adopts the Actor-Critic framework.

[0015] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor.

[0016] The solution provided in this application generates a unified state representation vector through a feature extraction network, enabling the first prediction network (price) and the second prediction network (sales volume) to make initial predictions based on the same high-quality, multi-source data-integrated environmental perception, thus ensuring information consistency from the outset. More importantly, a reinforcement learning optimizer jointly optimizes the first decision variable (price) under a unified objective function. This optimization process simultaneously considers the resulting changes in the second variable (sales volume) (reflected in the second supervised loss term), achieving dynamic coupling and coordinated adjustment of price and sales volume. This joint optimization mechanism fundamentally avoids the suboptimal decision-making problem caused by input bias in staged models, outputting a collaboratively optimal solution under a global objective, thereby significantly improving the accuracy of the overall decision. This invention creatively introduces a first supervised loss term and a second supervised loss term into the objective function. These two terms respectively constrain the offset between the optimized decision variable and its initial predicted value. This design serves a dual purpose at the technical level: First, as a powerful regularizer, it significantly narrows the effective exploration space of the policy, guiding optimization to a reasonable neighborhood of the initial prediction (i.e., common-sense patterns implicit in historical data), thereby greatly improving the training speed, stability, and convergence reliability of the reinforcement learning model. Second, as a channel for injecting prior knowledge, it ensures that the final optimization decision combines the exploratory nature of "maximizing returns" with the robustness of "conforming to historical patterns," solving the technical pain points of poor interpretability and high deployment risk of pure reinforcement learning policies. It deeply integrates traditional supervised learning (prediction network) and deep reinforcement learning (optimizer) within the same end-to-end model architecture. The first and second prediction networks provide high-quality, interpretable decision benchmarks, upon which the reinforcement learning optimizer performs a reward-oriented fine-grained search. This architecture technically enables the entire process from environmental perception to collaborative decision-making to be completed in a single forward propagation, replacing the complex pipeline of traditional methods that require multiple models to be connected in series and rules to be designed manually. It not only improves decision-making efficiency, but also automatically discovers the complex dynamic balance between price and sales volume through a data-driven approach, providing a general and learnable automated technical solution for handling dynamic system decisions with multiple coupled variables. Attached Figure Description

[0017] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0018] Figure 1This is a flowchart illustrating a dynamic system collaborative pricing prediction method for fast-moving consumer goods e-commerce based on deep reinforcement learning, provided in one embodiment of this application. Figure 2 This is a schematic diagram of the structure of a collaborative decision-making model provided in one embodiment of this application; Figure 3 This is a schematic diagram of the architecture of a dynamic system collaborative fast-moving consumer goods e-commerce pricing prediction device based on deep reinforcement learning, provided in one embodiment of this application. Figure 4 This is a schematic diagram of an electronic device provided in one embodiment of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Figure 1 This is a flowchart illustrating a method for predicting FMCG e-commerce pricing based on a dynamic system using deep reinforcement learning, provided in one embodiment of this application. The method is applied to a device for predicting FMCG e-commerce pricing based on a dynamic system using deep reinforcement learning. Figure 1 As shown, the method includes the following:

[0021] Step S110: Obtain multi-source data on the system status of the target to be decided; This step is the starting point of the entire collaborative forecasting process, and its essence is to build a real-time, multi-dimensional, and heterogeneous environmental cognitive foundation for dynamic decision-making. In actual business scenarios, the decision target may be a specific product, service, or resource unit, while "system state multi-source data" is a comprehensive digital mapping around this target. This data includes at least four types of key information: user characteristics (such as purchasing power, price sensitivity, and historical behavior patterns), product characteristics (such as category, life cycle, and inventory level), historical transaction sequence characteristics (such as time-series sales and price fluctuation trajectories), and market environment context characteristics (such as competitor prices, holiday effects, and macroeconomic indicators). The acquisition process is not a simple data transfer, but involves real-time streaming collection from distributed business systems (such as user behavior logs, ERP, and CRM), followed by preprocessing operations such as timestamp alignment, robust imputation of missing values, and Z-score standardization on heterogeneous data, ultimately generating a unified format state tensor that conforms to the model input specifications. The technical value of this step lies in ensuring that the subsequent model receives not isolated, lagging single-point data, but a high-quality information carrier that has been structured and integrated and can reflect the overall dynamics of the market.

[0022] Step S120: Input the multi-source data of the system state into a pre-trained collaborative decision-making model; This step marks the transition from data perception to intelligent decision-making. The collaborative decision-making model employs an end-to-end deep neural network architecture, meaning that data propagation from input to output is completed within the same graph computation framework, eliminating the losses and error accumulation associated with interface conversions between modules in traditional pipeline systems. The input process is not a simple data delivery; instead, the state tensor generated in step S110 is loaded into GPU memory through vectorization encoding and tensor deformation operations, triggering the execution of the computation graph within the model. At this point, the data flows through a feature extraction network, which dynamically captures the nonlinear relationships between users, products, time series, and the market environment through a multimodal attention fusion mechanism, ultimately outputting a low-dimensional, dense, and information-compressed unified state representation vector. This vector serves as the shared foundation for all subsequent computations in the entire model; it carries both the accumulated historical patterns and encodes the dynamic changes in the real-time environment.

[0023] Step S130: Process the data through the collaborative decision-making model and output the values ​​of the first and second decision variables for collaborative optimization; Reference Figure 2 The collaborative decision-making model is a deep neural network model trained using an end-to-end architecture, which includes: Feature extraction network 1 is used to fuse and learn representations of the input data and output a unified state representation vector; The first prediction network 2 is used to output the initial predicted value of the first decision variable based on the unified state representation vector; wherein, the first decision variable is price; The second prediction network 3 is used to output the initial predicted value of the second variable based on the unified state representation vector; wherein, the second variable is sales volume; The reinforcement learning optimizer 4 adjusts the first decision variable based on the unified state representation vector, the initial predicted value of the first decision variable, and the initial predicted value of the second variable, to obtain the first decision variable corresponding to the maximum objective function. The objective function includes: a first supervised loss term, a second supervised loss term, and a policy objective term; The first supervision loss term is inversely proportional to the first adjustment amount; the first adjustment amount is the adjustment between the adjusted first decision variable and the initial predicted value of the first decision variable; The second supervision loss term is inversely proportional to the second adjustment amount; the second adjustment amount is the adjustment between the adjusted second variable and the initial predicted values ​​of the second variable. The strategic objective is the profit corresponding to the first decision variable and the second variable.

[0024] This is the core technology of the entire method, demonstrating the deep coupling between prediction and optimization. The processing unfolds in parallel within the model into three sub-processes: First, a unified state representation vector is simultaneously fed into both the first and second prediction networks. The first prediction network (responsible for price) outputs an initial price prediction based on the price formation patterns implicit in historical data; the second prediction network (responsible for sales volume) outputs an initial sales volume prediction based on historical sales volume patterns. These two predictions are not the final decision, but rather provide empirical anchors for the reinforcement learning optimizer, ensuring that subsequent optimizations do not deviate from the data-driven common-sense framework.

[0025] Subsequently, the reinforcement learning optimizer initiates a collaborative optimization loop. It receives a state representation vector and two initial prediction values, and within a predefined action space (e.g., the maximum price adjustment is 20% of the initial value), it iteratively explores the optimal price adjustment amount using an Actor-Critic framework. In each iteration, the optimizer generates a candidate price and re-inputs it into the second prediction network, enabling the sales forecasting model to dynamically respond to price changes and re-infer candidate sales volumes. This "prediction-optimization-re-prediction" inner-loop mechanism is the key technical design for achieving synergy in this invention. The optimizer continuously evaluates the objective function value calculated from the candidate decision combination. This objective function subtly balances three objectives: the first supervised loss term constrains the price adjustment amount through exponential decay, making it close to the initial prediction; the second supervised loss term similarly constrains the sales volume adjustment amount, maintaining prediction stability; and the strategy objective term directly maximizes profit. After hundreds of iterations, when the objective function value converges to its maximum and no longer significantly increases, the optimizer outputs the first decision variable and the corresponding second variable of the collaborative optimization. These two are no longer independent but rather a pair of coupled variables of the global optimal solution.

[0026] In the strategic objective, profit is specifically defined as: (First Decision Variable - Cost) × Second Decision Variable. "First Decision Variable - Cost": This represents the unit gross profit earned per item. It directly links the model's decision action (pricing) to a key business constraint (cost). "Multiplied by Second Decision Variable": This represents the total sales volume considering market response. The key significance is that the "Second Decision Variable" here is not an exogenously given fixed value, but rather the result of a dynamic correlation with the First Decision Variable (price) within the model. This sales volume value is synchronously determined in the model based on the optimized price.

[0027] The system status multi-source data includes at least one of the following: user characteristics, product characteristics, historical transaction sequence characteristics, and market environment context characteristics.

[0028] The intelligent decision-making system of this invention is based on a comprehensive and three-dimensional depiction of the decision-making environment for its perception and judgment. To this end, the "system state multi-source data" acquired and processed by the system is a comprehensive system that integrates multi-dimensional information, aiming to jointly construct an accurate "situation map" for decision-making from different perspectives.

[0029] Specifically, these data mainly include the following key dimensions: User characteristics: This is the core of understanding the "demand side." It may include static user attributes (such as age, region, membership level), dynamic preferences (such as recent browsing, clicks, and favorites), as well as spending power and historical sensitivity. This data helps the system determine "who" is most likely to engage in decision-related behavior in the current scenario.

[0030] Product characteristics: This is fundamental to describing the "supply side." It encompasses the inherent attributes of a product (such as category, brand, cost, inventory level, and life cycle stage) as well as marketing attributes (such as whether it is a new product or whether it participates in a specific event). These characteristics determine the product's basic positioning and constraints in the market.

[0031] Historical transaction sequence characteristics: This is key to capturing "dynamic patterns." It is typically presented in time series format, recording historical behavioral trajectories closely related to decision-making objectives, such as price fluctuations, sales fluctuations, promotional activity records, and user conversion rates over a past period (e.g., days or weeks). This data contains the system's own and users' reaction patterns to historical decisions and is the cornerstone for predicting future trends.

[0032] Market environment contextual characteristics: This is the sensory aid for perceiving the "external environment." It incorporates more macroscopic or immediate external factors, such as time context (whether it's a holiday, weekend, or specific season), macroeconomic indicators, industry trends, real-time pricing and promotional information of competitors, and even social media sentiment. These characteristics ensure that decisions can adapt to the constantly changing external environment, rather than simply being mechanical extrapolations based on internal historical data.

[0033] Importantly, these four categories of data do not all need to be complete to function. Depending on the specific decision-making scenario and data availability, the system can flexibly adopt at least one or more of them for combined analysis. For example, in a cold-start scenario with limited data, the system may primarily rely on product characteristics and basic market context (such as holidays) to make initial decisions; while in a mature scenario with abundant data, the system can integrate all four categories of features for more refined and precise collaborative optimization. This inclusive design allows the solution to adapt to a variety of practical application conditions, from simple to complex.

[0034] In some embodiments, the feature extraction network is a multimodal feature extraction network.

[0035] In a preferred embodiment of the present invention, the feature extraction network is specifically implemented as a multimodal feature extraction network. This design aims to efficiently address the inherent heterogeneity and modal differences of multi-source data in system states.

[0036] As mentioned earlier, the input data may include various forms such as numerical sequences (e.g., historical prices), categorical attributes (e.g., product categories), text descriptions (e.g., user review summaries), and even time-series behavioral patterns. These data can be considered as different "modalities." A simple fully connected network is insufficient to fully explore the complex relationships between these heterogeneous data.

[0037] The core technical advantage of using multimodal feature extraction networks lies in their ability to design or adapt specialized sub-encoders for different modalities of data. For example, for historical transaction sequences, Long Short-Term Memory (LSTM) networks or Temporal Convolutional Networks (TCNs) might be used to capture their temporal dependencies; for product and user category features, embedding layers might be used to map them into dense vectors; and for text descriptions, lightweight text encoders can be used to extract semantic information. Then, the network uses attention mechanisms, cross-modal fusion layers, or tensor concatenation followed by deep neural networks to deeply interact and fuse these separately encoded modal features. This divide-and-conquer, then fusion strategy can more effectively extract unified and information-rich state representation vectors. Through multimodal learning, the network can capture cross-modal associations that are not obvious in single-modal data. For example, it might learn complex patterns such as "a specific user group (user feature modality) will have a lower price sensitivity (historical sequence modality) to a certain type of luxury goods (product feature modality) during holidays (environmental context modality)." This makes the generated state representation vector more comprehensive and profound in reflecting the true state of the system, providing a more robust and information-rich input for subsequent prediction and optimization, thereby improving the accuracy and robustness of the entire collaborative decision-making model from a technical perspective.

[0038] Therefore, designating the feature extraction network as a multimodal feature extraction network is not a simple component replacement, but a key technical optimization made for the structured characteristics of the specific input data (multi-source system state data) of this invention, and is an important technical means to improve the performance of the entire solution.

[0039] Specifically, both the first prediction network and the second prediction network are time series prediction networks.

[0040] Furthermore, the historical transaction sequence features in the "system state multi-source data" (such as price and sales records over multiple consecutive days) are themselves typical time series data. Changes in price and sales are not isolated events, but are profoundly influenced by their own historical trends, cyclical fluctuations (such as seasonality and weekend effects), and long-term dependencies. Therefore, to output a meaningful initial prediction, the model must be able to capture and infer these complex time series patterns. Based on this, both the first and second prediction networks can be time series prediction networks. Employing time series prediction networks (such as LSTM, GRU, temporal convolutional networks, or Transformer-based time series models) brings the following key technical advantages: The network architecture is specifically designed for sequential data, enabling it to effectively memorize long-term historical information and understand its non-linear relationship with future states. For example, the network can learn dynamic patterns such as "after a series of small price increases, the sales growth rate may slow down" or "sales usually decline after a specific promotional event ends." This makes the generated initial price forecasts and initial sales forecasts no longer simple copies of the mean or recent values, but rather benchmark values ​​derived from historical patterns and consistent with business dynamics.

[0041] Providing high-quality "anchors" for the optimizer: Initial predictions play a dual role as both "stabilizers" and "starting points" in subsequent reinforcement learning optimization. An accurate and robust initial prediction generated solely by a time-series network means that it has deeply internalized the patterns of "what usually happens" in historical data. Based on this, the reinforcement learning optimizer can be more effectively guided to a reasonable policy space for exploration and fine-tuning, avoiding absurd decisions that completely contradict historical experience. This significantly accelerates training convergence at the technical level and ensures the commercial rationality of the final decision.

[0042] Achieving deep integration with state representation: The time series prediction network receives a unified state representation vector from the feature extraction network as input. This vector already incorporates real-time information from multiple modalities, including users, products, and the environment. The time series network then focuses on processing the inherent, time-evolution-related patterns and context, ultimately outputting a comprehensive prediction that integrates "historical longitudinal patterns" and "current horizontal state." This is more efficient and powerful than simply concatenating historical sequences with static features and feeding them into a regular fully connected network.

[0043] In summary, designating the prediction network as a time series prediction network is a core and necessary technical design choice made to address the inherent temporal, trend, and autocorrelation characteristics of decision variables such as price and sales volume. It ensures the scientific rigor of the initial predictions, lays a reliable foundation for the entire end-to-end collaborative optimization process, and is one of the key technical aspects of achieving high-precision decision-making in this method.

[0044] In the core component of this invention—the reinforcement learning optimizer—an Actor-Critic framework is preferably adopted to achieve intelligent adjustment and optimization of the first decision variable. This architectural choice is a key technical design made in response to the inherent complexity of collaborative decision-making problems in dynamic systems, aiming to efficiently and stably solve the aforementioned problem of maximizing the objective function.

[0045] The Actor-Critic framework is essentially a hybrid architecture that comprises two neural networks working together: Actor (Actor Network): Its responsibility is policy learning. It directly observes the current state (in this invention, the optimized state constructed based on the unified state representation vector, the initial predicted values ​​of the first and second decision variables), and outputs a specific action—an adjustment Δp to the first decision variable (price). It defines how to act in a given state.

[0046] The Critic network's role is value assessment. It evaluates the long-term expected value of the actions taken (or the actions followed) by the Actor network in the current state. Instead of directly outputting actions, it outputs a scalar value (such as the state-action value function Q or the state value function V) to judge the quality of the actions and provide guidance for Actor updates.

[0047] In a specific implementation of this invention, the workflow and advantages of this framework are as follows: The Actor network of this invention takes the fused state information (representation vector and initial predicted value) as input and outputs a continuous or discrete action distribution, thereby achieving fine-tuning of prices. The Critic network receives the same state information and the actions proposed by the Actors, and evaluates the expected total return that can be obtained in the future (i.e., after optimization) under this "state-action pair". This return directly corresponds to the objective function defined in this invention (which integrates the policy objective term and the supervision loss term). In this way, the Actor-Critic framework is seamlessly embedded into the entire collaborative decision-making model, becoming the "intelligent engine" driving collaborative optimization.

[0048] The core technical advantage of the Actor-Critic framework lies in its more stable and efficient learning. Pure "value learning" methods (such as Q-learning) or "policy gradient" methods are prone to training instability in high-dimensional continuous action spaces or complex reward functions. In this invention, the Critic network provides low-variance, guided gradient signals for policy updates by continuously evaluating the Actor's actions. This greatly alleviates the technical bottlenecks faced by traditional reinforcement learning in similar scenarios, such as low exploration efficiency, large training fluctuations, and convergence difficulties. The Actor updates its policy based on the evaluation signals provided by the Critic (using algorithms such as the policy gradient theorem), steadily improving its policy towards obtaining a higher Critic score (i.e., maximizing the objective function).

[0049] This framework facilitates the implementation of techniques such as the Advantage Function, which more accurately measures the merits of an action relative to the average level. This helps guide Actors to balance “utilizing” currently known effective strategies with “exploring” new strategies that may yield higher rewards, thus making it more likely to find the globally or near-globally optimal collaborative decision point.

[0050] The following section provides a specific example to further illustrate the solution provided in this application. Specifically, it uses a particular application scenario of dynamic pricing and sales forecasting in the retail industry to fully explain the overall workflow of the solution. Suppose a national supermarket chain wants to develop a daily pricing strategy for a best-selling carbonated beverage in its stores for the coming week to maximize total profit.

[0051] Step 1: Obtain multi-source system status data (corresponding to S110): After the business closes each day, the system automatically collects and integrates the following multi-source data as input for the next day's decision-making: User characteristics: Based on membership card data, analyze the spending power of customers around each store and their historical preference index for this beverage category.

[0052] Product characteristics: unit purchase cost, current inventory level, and remaining shelf life of the beverage.

[0053] Historical transaction sequence characteristics: the daily selling price, sales volume, and whether there were any promotional activities for this beverage at each store over the past 30 days.

[0054] Market environment context features: weather forecast for the next week (temperature significantly affects beverage demand), whether it is a holiday or weekend, and retail price monitoring data of major competitors during the same period.

[0055] After being cleaned and standardized, these heterogeneous data form a structured "system state snapshot".

[0056] Step 2: Input the collaborative decision-making model (corresponding to S120); The packaged multi-source data is then input into a pre-trained collaborative decision-making model. This model has already learned the complex relationship between price and sales volume, as well as optimization strategies, from massive amounts of historical sales data.

[0057] Step 3: Intra-model collaborative processing and output (corresponding to S130); End-to-end collaborative computation begins within the model: Feature Fusion and Representation: The multimodal feature extraction network begins its work. It uses LSTM to process historical price and sales sequences to capture trends, embedding layers to process store and product categories, and integrates numerical features such as weather temperature. The final output is a unified state representation vector. This vector encapsulates all the key information, such as "pre-summer weekend," "ample inventory," "competitors are having promotions," and "stable historical sales."

[0058] The initial forecast baseline is provided based on the aforementioned representation vector: The first forecast network (time series network) outputs an initial price forecast, such as 4.5 yuan. This can be understood as a "conventional" suggested price based on historical patterns and without considering extreme profit pursuit. The second forecast network (time series network) outputs an initial sales forecast at this "conventional" price, such as an estimated daily sales volume of 200 bottles per store.

[0059] Reinforcement Learning Collaborative Optimization: A reinforcement learning optimizer using the Actor-Critic framework takes over the work. The Actor examines the current state (representation vector) and the initial prediction value (4.5 yuan, 200 bottles). Its task is to propose a price adjustment action, such as "+0.3 yuan".

[0060] The Critic evaluates the potential consequences of the new price (4.8 yuan). It estimates that when the price rises to 4.8 yuan, sales volume might decrease to 185 bottles due to price elasticity (this value is determined internally by the model based on learned relationships). Then, it calculates the expected revenue based on the objective function: Strategy objective (profit): (4.8 - 3.0 cost) × 185 = 333 yuan. First supervised loss: Penalty for price adjustment |4.8 - 4.5| = 0.3 yuan. Second supervised loss: Penalty for predicted sales volume change |185 - 200| = 15 bottles. The weighted sum of the strategy objective, the first supervised loss, and the second supervised loss constitutes the objective function. The Critic feeds back the overall objective function value to the Actor, guiding it to adjust its strategy. After several such internal "trial and error" cycles, the optimizer eventually finds an equilibrium point: for example, setting the price at 4.7 yuan. At this price, the predicted sales volume is 190 bottles, and the calculated objective function value is maximized.

[0061] Final output: The model outputs a collaboratively optimized decision pair—suggested pricing: 4.7 yuan, expected daily sales: 190 bottles / store. This result is not isolated; it clearly reflects "collaboration aimed at maximizing overall profit": the price is the optimized price after considering sales elasticity, and the sales volume is the expected sales volume corresponding to the optimized price.

[0062] This application's solution dynamically incorporates the impact of price on sales volume into the decision-making loop through an end-to-end collaborative model. It outputs a set of self-consistent, jointly optimal decision variables, thereby achieving more accurate and profitable dynamic pricing in complex market environments. This process is also applicable to any dynamic system optimization scenario with coupled variables, such as ad bidding and click-through rate prediction, inventory level and replenishment quantity decisions.

[0063] The apparatus embodiments of this application can be used to execute the method embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the method embodiments of this application.

[0064] Reference Figure 3 The dynamic system collaborative fast-moving consumer goods e-commerce pricing prediction device based on deep reinforcement learning provided in this application includes: The acquisition module 31 is used to acquire multi-source data on the system status of the target to be decided. Input module 32 is used to input the multi-source data of the system state into a pre-trained collaborative decision-making model; Output module 33 is used to process the collaborative decision-making model and output the first decision variable value and the second variable value of collaborative optimization. The collaborative decision-making model is a deep neural network model trained using an end-to-end architecture, and includes: Feature extraction networks are used to fuse and learn representations of input data and output a unified state representation vector. A first prediction network is used to output the initial predicted value of the first decision variable based on the unified state representation vector; The second prediction network is used to output the initial predicted value of the second variable based on the unified state representation vector; The reinforcement learning optimizer adjusts the first decision variable based on the unified state representation vector, the initial predicted value of the first decision variable, and the initial predicted value of the second variable, to obtain the first decision variable corresponding to the maximum objective function. The objective function includes: a first supervised loss term, a second supervised loss term, and a policy objective term; The first supervision loss term is inversely proportional to the first adjustment amount; the first adjustment amount is the adjustment between the adjusted first decision variable and the initial predicted value of the first decision variable; The second supervision loss term is inversely proportional to the second adjustment amount; the second adjustment amount is the adjustment between the adjusted second variable and the initial predicted values ​​of the second variable. The strategic objective is the profit corresponding to the first decision variable and the second variable.

[0065] Furthermore, the system state multi-source data includes at least one of the following: user characteristics, product characteristics, historical transaction sequence characteristics, and market environment context characteristics.

[0066] Furthermore, the feature extraction network is a multimodal feature extraction network; both the first prediction network and the second prediction network are time series prediction networks; and the reinforcement learning optimizer adopts the Actor-Critic framework.

[0067] Below, for reference Figure 4 This describes an electronic device according to embodiments of the present application. Figure 4 A block diagram of an electronic device according to an embodiment of this application is illustrated.

[0068] like Figure 4 As shown, the electronic device 400 includes one or more processors 410 and memory 420.

[0069] The processor 410 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 400 to perform desired functions.

[0070] The memory 420 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 410 may execute the program instructions to implement the data security analysis methods of the various embodiments of this application described above and / or other desired functions. Various contents, such as category correspondences, may also be stored in the computer-readable storage medium.

[0071] In one example, the electronic device 400 may also include an input device 430 and an output device 440, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0072] In addition, the input device 430 may also include, for example, a keyboard, mouse, interface, etc. The output device 440 can output various information to the outside, including analysis results, etc. The output device 440 may include, for example, a display, speaker, printer, and communication network and its connected remote output devices, etc.

[0073] Of course, for the sake of simplicity, Figure 4 Only some of the components of the electronic device relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the electronic device may include any other suitable components depending on the specific application.

[0074] In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the data security analysis methods according to various embodiments of this application as described in the "Exemplary Methods" section of this specification.

[0075] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0076] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the data security analysis methods according to various embodiments of this application described in the "Exemplary Methods" section above.

[0077] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0078] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A dynamic system collaborative fast-moving consumer goods (FMCG) e-commerce pricing prediction method based on deep reinforcement learning, characterized in that, include: Acquire multi-source system state data for the target to be decided; The multi-source data of the system state are input into a pre-trained collaborative decision-making model; The collaborative decision-making model is used to process the data and output the values ​​of the first and second decision variables that are optimized collaboratively. The collaborative decision-making model is a deep neural network model trained using an end-to-end architecture, and includes: Feature extraction networks are used to fuse and learn representations of input data and output a unified state representation vector. A first prediction network is used to output an initial predicted value for a first decision variable based on the unified state representation vector; wherein the first decision variable is price; The second prediction network is used to output the initial predicted value of the second variable based on the unified state representation vector; wherein the second variable is sales volume. A reinforcement learning optimizer, based on the unified state representation vector, the initial predicted value of the first decision variable, and the initial predicted value of the second variable, adjusts the first decision variable to obtain the first decision variable corresponding to the maximum objective function; wherein, the reinforcement learning optimizer determines the second variable corresponding to the adjusted first decision variable as the adjusted second variable based on the adjusted first decision variable. The objective function includes: a first supervised loss term, a second supervised loss term, and a policy objective term; The first supervision loss term is inversely proportional to the first adjustment amount; the first adjustment amount is the adjustment between the adjusted first decision variable and the initial predicted value of the first decision variable; The second supervision loss term is inversely proportional to the second adjustment amount; the second adjustment amount is the adjustment between the adjusted second variable and the initial predicted values ​​of the second variable. The strategic objective is the profit corresponding to the first decision variable and the second variable.

2. The method for predicting FMCG e-commerce pricing based on deep reinforcement learning for dynamic systems, as described in claim 1, is characterized in that... The system status multi-source data includes at least one of the following: user characteristics, product characteristics, historical transaction sequence characteristics, and market environment context characteristics.

3. The method according to claim 1, characterized in that, The feature extraction network is a multimodal feature extraction network.

4. The method according to claim 1, characterized in that, Both the first prediction network and the second prediction network are time series prediction networks.

5. The method according to claim 1, characterized in that, The reinforcement learning optimizer uses the Actor-Critic framework.

6. The method according to claim 1, characterized in that, The profit is calculated as: (first decision variable - cost) × second variable.

7. A dynamic system collaborative fast-moving consumer goods (FMCG) e-commerce pricing prediction device based on deep reinforcement learning, characterized in that, include: The acquisition module is used to acquire multi-source system state data of the target to be decided. The input module is used to input the multi-source data of the system state into a pre-trained collaborative decision-making model; The output module is used to process the collaborative decision-making model and output the values ​​of the first and second decision variables for collaborative optimization; wherein, the reinforcement learning optimizer determines the second variable corresponding to the adjusted first decision variable as the adjusted second variable based on the adjusted first decision variable; The collaborative decision-making model is a deep neural network model trained using an end-to-end architecture, and includes: Feature extraction networks are used to fuse and learn representations of input data and output a unified state representation vector. A first prediction network is used to output the initial predicted value of the first decision variable based on the unified state representation vector; The second prediction network is used to output the initial predicted value of the second variable based on the unified state representation vector; The reinforcement learning optimizer adjusts the first decision variable based on the unified state representation vector, the initial predicted value of the first decision variable, and the initial predicted value of the second variable, to obtain the first decision variable corresponding to the maximum objective function. The objective function includes: a first supervised loss term, a second supervised loss term, and a policy objective term; The first supervision loss term is inversely proportional to the first adjustment amount; the first adjustment amount is the adjustment between the adjusted first decision variable and the initial predicted value of the first decision variable; The second supervision loss term is inversely proportional to the second adjustment amount; the second adjustment amount is the adjustment between the adjusted second variable and the initial predicted values ​​of the second variable. The strategic objective is the profit corresponding to the first decision variable and the second variable.

8. The dynamic system collaborative FMCG e-commerce pricing prediction device based on deep reinforcement learning according to claim 7, characterized in that, The system status multi-source data includes at least one of the following: user characteristics, product characteristics, historical transaction sequence characteristics, and market environment context characteristics.

9. The dynamic system collaborative FMCG e-commerce pricing prediction device based on deep reinforcement learning according to claim 7, characterized in that, The feature extraction network is a multimodal feature extraction network; both the first prediction network and the second prediction network are time series prediction networks; the reinforcement learning optimizer adopts the Actor-Critic framework.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 6.

Images