Policy generation method, system, device, and medium

Abstract

This application discloses a strategy generation method, system, device, and medium, relating to the technical fields of model reasoning. The method includes: for at least one asset pricing unit, obtaining state features used to construct the current state of the asset pricing unit; concatenating the state features in a preset order, and performing concatenation and normalization processing to generate a current state vector; inputting the current state vector into a preset strategy to generate an agent, and performing forward inference to output original action values; generating a price adjustment coefficient and recommended price corresponding to at least one asset pricing unit based on the original action values; and sending the price adjustment coefficient and recommended price to the business system, whereby the price adjustment coefficient and recommended price are used by the business system to update the price adjustment strategy. This improves the automation level of pricing decisions, significantly reduces labor costs, effectively enhances the accuracy and scientific nature of pricing, and can flexibly consider multiple business objectives, thus better meeting actual business needs.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence, machine learning and other technical fields, and in particular to a strategy generation method, system, device and medium. Background Technology

[0002] In supply-demand sensitive service industries, such as long-term rental apartments, hotels, and ride-sharing, dynamic pricing has become an effective means to improve asset operation efficiency and business returns. Generating a scientific pricing strategy is a complex system engineering project that requires comprehensive consideration of internal operational data, the external market environment, and multi-dimensional customer behavior characteristics to find the optimal balance between several mutually constraining goals, such as high revenue, high occupancy rates, and low operating costs.

[0003] However, passively predicting prices using algorithms such as Autoregressive Integrated Moving Average (ARIMA), Long Short-Term Memory (LSTM), or Gradient Boosting Decision Tree (GBDT) only provides static data references and lacks the ability to proactively generate optimal price adjustments based on real-time conditions. This reliance on human experience not only makes it difficult to achieve millisecond-level agility but also fails to effectively address the challenge of automating trade-offs among multiple business objectives, easily leading to lagging price adjustment strategies and missed opportunities for optimal returns. Summary of the Invention

[0004] The embodiments of this application aim to at least partially solve one of the technical problems in the related art. Therefore, the purpose of the embodiments of this application is to provide a strategy generation method, system, device, and medium.

[0005] This application provides a strategy generation method, including: For at least one asset pricing unit, obtain state features used to construct the current state of the asset pricing unit; The state features are concatenated in a preset order, and then concatenated and normalized to generate the current state vector. The current state vector is input into a preset strategy to generate an agent, and forward inference is performed to output the original action value. The price adjustment coefficient and recommended price corresponding to the at least one asset pricing unit are generated based on the original action value. The price adjustment coefficient and the recommended price are sent to the business system, and the price adjustment coefficient and the recommended price are used by the business system to update the price adjustment strategy.

[0006] In some embodiments, the state characteristics include at least one of the following: Price series with a previously preset time step; Market characteristic sequences with a pre-defined time step in the past; or, The sequence of price adjustment actions with a preset time step.

[0007] In some embodiments, generating the price adjustment coefficient and recommended price corresponding to the at least one asset pricing unit based on the original action value includes: Using a nonlinear activation function, the original action value is mapped and constrained to a normalized preset interval, and combined with a preset price adjustment range hyperparameter, the price adjustment coefficient is calculated. Extract the actual price of the at least one asset pricing unit from the previous period, and combine it with the price adjustment coefficient to determine the recommended price for the next period.

[0008] In some embodiments, the method further includes: Initialize the parameters of the agent's policy network and value network; The intelligent agent is controlled to interact with a preset pricing decision environment to generate and store multiple interaction data tuples in an experience buffer, wherein each interaction data tuple includes the current state, the action performed, the environmental feedback reward, and the next state after the transition; Interactive data tuples are sampled from the experience buffer, and the corresponding policy loss and value loss are calculated based on the Proximal Policy Optimization (PPO) algorithm. The parameters of the policy network and the value network are then updated respectively.

[0009] Repeatedly generate and store the interaction data tuples, and update the parameters of the policy network and the value network until the policy network converges, to obtain the preset policy generating agent.

[0010] In some embodiments, the step of calculating the corresponding policy loss and value loss based on the proximal policy optimization algorithm, and updating the parameters of the policy network and value network respectively, includes: The objective function of the near-end policy optimization algorithm is used to determine the policy loss corresponding to the interactive data tuple, and the value loss is calculated based on the value network prediction value and the target return value. Based on the policy loss, the parameters of the policy network are updated using a gradient optimization method to increase the value of the objective function; and based on the value loss, the parameters of the value network are updated using a gradient descent method to reduce the value loss.

[0011] In some embodiments, generating and storing multiple interactive data tuples into an experience buffer includes: In the current state, the policy network outputs the execution action; after the preset pricing decision environment executes the action, it feeds back the next state after the transition and calls the preset reward function to obtain the environment feedback reward; The environmental feedback reward function includes at least one of the following: An accuracy reward function is calculated based on the negative relative error between the recommended price and the future actual price; or... The stability reward function is calculated based on the absolute value of the difference between the current action and the action in the previous cycle.

[0012] In some embodiments, controlling the agent to interact with a preset pricing decision environment includes: Based on the at least one asset pricing unit, the preprocessed multidimensional time series data is encapsulated into the preset pricing decision environment; In response to interaction requests at any given time, the state features are extracted from the pricing decision environment. The state features include at least one of the following: historical price series, historical market feature series, or historical price adjustment action series. The state features are concatenated in a preset order to construct an environmental state vector corresponding to the preset pricing decision environment; Based on the environmental state vector, the agent is controlled to perform interactive operations with the preset pricing decision environment.

[0013] Another embodiment of this application provides a strategy generation system, the strategy generation system comprising: The acquisition module is used to acquire state features for constructing the current state of at least one asset pricing unit. The processing module is used to concatenate the state features in a preset order, and perform concatenation and normalization processing to generate the current state vector. The generation module is used to input the current state vector into a preset strategy to generate an agent, and perform forward inference to output the original action value; and generate the price adjustment coefficient and recommended price corresponding to the at least one asset pricing unit based on the original action value; The sending module is used to send the price adjustment coefficient and the recommended price to the business system, and the price adjustment coefficient and the recommended price are used by the business system to update the price adjustment strategy. Another embodiment of this application provides an electronic device having a computer program stored thereon, which, when executed by a processor, implements the steps of the method of any of the above embodiments.

[0014] Another embodiment of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method of any of the above embodiments.

[0015] In the above embodiments, the strategy generation method includes: for at least one asset pricing unit, obtaining state features used to construct the current state of the asset pricing unit; concatenating the state features in a preset order, and performing concatenation and normalization processing to generate a current state vector; inputting the current state vector into a preset strategy to generate an intelligent agent, and performing forward inference to output the original action value; generating a price adjustment coefficient and recommended price corresponding to at least one asset pricing unit based on the original action value; and sending the price adjustment coefficient and recommended price to the business system, whereby the price adjustment coefficient and recommended price are used by the business system to update the price adjustment strategy. This not only improves the automation level of pricing decisions but also connects the entire chain from data perception to price generation, significantly reducing labor costs while effectively enhancing the accuracy and scientific nature of pricing. Furthermore, the optimized price adjustment strategy can flexibly consider multiple business objectives, thus better meeting actual business needs. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A schematic flowchart of a strategy generation method provided for an embodiment of this application; Figure 2 A schematic diagram of another strategy generation method provided for an embodiment of this application; Figure 3 A detailed flowchart of a strategy generation method provided for embodiments of this application; Figure 4 A schematic diagram of a strategy generation system architecture provided for an embodiment of this application; Figure 5 A strategy generation system block diagram provided for another embodiment of this application; Figure 6 A block diagram of an electronic device provided for another embodiment of this application. Detailed Implementation

[0018] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0019] In supply-demand sensitive industries such as long-term rental apartments, hotels, and ride-sharing, dynamic pricing has become a key means to improve asset operation efficiency and business returns. A scientific pricing strategy is a complex system engineering project that requires comprehensive consideration of internal operational data, external market environment, and multi-dimensional customer behavior characteristics to find the optimal balance between multiple mutually constraining goals such as high revenue, high occupancy rate, and low operating costs.

[0020] The algorithms employed in related technologies, such as Autoregressive Integral Moving Average (ARIMA), Long Short-Term Memory (LSTM), or Gradient Boosting Tree (GBDT), are essentially passive predictions used to assist pricing. These methods only provide static data references and lack the ability to proactively generate optimal pricing actions based on real-time environments. This reliance on human experience not only makes it difficult to achieve millisecond-level agile responses but also fails to effectively address the challenge of automating trade-offs among multiple business objectives, easily leading to lagging pricing strategies and missing optimal profit windows.

[0021] In addition to the above, passively predictive pricing strategies also suffer from the following technical problems. First, the decision-making model is simplistic and lacks automation. Existing supervised learning models can only answer what the future price might be, lacking proactive planning capabilities and unable to generate decisions on how to adjust the current price to achieve optimal returns. Furthermore, these methods heavily rely on manual intervention for rule calibration, making it difficult to achieve automated adaptation in rapidly changing markets, resulting in low decision-making efficiency and slow response. Second, the decision-making objective is one-sided. Traditional methods typically focus on minimizing short-term prediction errors as the sole optimization objective, ignoring the crucial price stability in business practice. This easily leads to frequent and drastic fluctuations in pricing recommendations, violating business logic, damaging customer trust, and increasing operating costs. Finally, information utilization is insufficient. Existing models' state perception is usually limited to current market characteristics, failing to incorporate historical pricing actions into the decision-making process. This prevents the model from learning the long-term impact of its decisions on the market environment, resulting in a lack of continuity and necessary memory in the overall strategy.

[0022] In some embodiments, the passive prediction-based price adjustment strategy specifically includes: First, collecting and integrating multi-dimensional data such as historical transaction prices, property characteristics, operational indicators (e.g., occupancy rate, vacancy days), market competitor prices, and time characteristics (e.g., seasons, holidays) to construct a complete feature set, using future actual transaction prices as the target label. Then, using supervised learning regression algorithms such as gradient boosting decision trees and deep neural networks, the model is trained using historical data. By learning the mapping relationship from the feature set to the target label, the error (e.g., mean squared error) between the model's predicted value and the actual value is minimized. Finally, the latest feature data is input into the trained model to obtain price predictions for a future period, which are then used by business personnel for reference and combined with their experience to formulate the final price adjustment strategy.

[0023] However, the passively predicted price adjustment strategy in this embodiment has the following problems. First, the strategy lacks decision-making capabilities, resulting in low automation. Since the essence of supervised learning models is to fit the mapping relationship of historical data, their output is only a passive prediction, not an active decision like a 5% increase. Therefore, the model output still requires manual interpretation and judgment, failing to form an automated closed loop from data to execution, leading to a lengthy and inefficient decision-making process. Second, the optimization objective is singular, causing the strategy to deviate from business reality. Traditional models typically use minimizing prediction error as the sole optimization objective, failing to intrinsically balance multiple conflicting business metrics. For example, in pursuit of ultimate prediction accuracy, the model might suggest a significant price adjustment during minor market fluctuations, ignoring the increased operating costs and brand image damage caused by frequent price adjustments. This lack of a multi-objective trade-off mechanism makes the generated strategy technically accurate but commercially unfeasible. Finally, the limited state representation results in a lack of long-term vision in the strategy. Existing solutions typically only consider the objective state of the market when constructing input features, without including the system's own historical decision sequences. Since decision-making behavior itself has a reciprocal effect on the market (e.g., continuous price reductions may trigger competitors to follow suit or lower customer expectations), if the model cannot perceive its own historical actions, it cannot learn a time-coherent and smooth strategy. Its decision can only be a static optimum based on the current state, rather than a dynamic optimum that takes into account long-term effects.

[0024] In view of this, this application aims to address the aforementioned problems by proposing a strategy generation method. This method enables an agent to automatically generate pricing strategies that balance accuracy and stability, shifting from passive price prediction to proactive generation of optimal pricing actions, significantly improving the automation level of pricing decisions. Simultaneously, by incorporating historical action sequences into the state representation, the model can perform long-term planning, thereby generating more continuous and forward-looking pricing strategies. Furthermore, by designing a composite reward function that integrates price accuracy and stability, the generated decisions can effectively consider multiple business objectives and are more closely aligned with actual business needs.

[0025] Specifically, for at least one asset pricing unit, state features are obtained to construct the current state of that unit. These features are then concatenated in a preset order and normalized to generate a current state vector. This vector is input into a preset strategy to generate an agent, which performs forward inference to output raw action values. Based on these raw action values, a price adjustment coefficient and a recommended price are generated for at least one asset pricing unit. The adjustment coefficient and recommended price are then sent to the business system, where they are used to update the pricing strategy. This approach improves the automation level of pricing decisions, significantly reduces labor costs, and effectively enhances the accuracy and scientific rigor of pricing. It also allows for flexible consideration of multiple business objectives, thus better meeting actual business needs.

[0026] The following section will provide a detailed description of a strategy generation method provided in this application, with reference to the accompanying drawings.

[0027] In some embodiments, see Figure 1 The diagram illustrates a strategy generation method according to an embodiment of this application. The strategy generation method 100 provided in this embodiment includes steps S110-S140.

[0028] Step S110: For at least one asset pricing unit, obtain state features used to construct the current state of the asset pricing unit. It should be understood that an asset pricing unit is a core business entity in a dynamic pricing system that can independently generate cash flow, possesses independent operational characteristics, and serves as the smallest decision-making granularity. In other words, there can be one or multiple asset pricing units, determined based on actual business needs.

[0029] For example, in real-world scenarios such as long-term rental apartments, hotels, or ride-sharing, the granularity of the asset pricing unit can be flexibly defined: it can be a micro-entity such as a single property, a single room, or a specific vehicle, or it can be the overall capacity supply within an entire building, a specific store, or a certain geographic fence. This application does not make any specific limitations on this.

[0030] Furthermore, state features can precisely characterize the current state of an asset pricing unit in a quantitative form, thereby completing the model construction for that unit. Specifically, state features include at least one of the following: Internal operational characteristics that reflect the unit's own attributes; for example, internal operational characteristics include basic building features, current occupancy rate, number of vacancy days, etc. Capture the market environment characteristics of external supply and demand relationships; for example, market environment characteristics include competitor prices in the same area, macro market sentiment, etc. Time series features that reflect time patterns include seasonal cycles and holiday effects.

[0031] One possible design has state characteristics including at least one of the following: Price series with a previously preset time step; Market characteristic sequences with a pre-defined time step in the past; or, The sequence of price adjustment actions with a preset time step.

[0032] It should be understood that price series with preset time steps are used to depict the evolution of actual transaction prices or listing prices of assets within a specific historical window. By capturing this time-series data, the system can accurately perceive the inertial trend and cyclical patterns of market prices, providing a core basis for judging the relative position of the current price.

[0033] The market feature sequence with a pre-defined time step is used to characterize the dynamics of multi-dimensional indicators reflecting changes in the external market environment within the same historical time window. This sequence typically covers supply and demand indicators (such as page views, add-to-cart rate, and transaction volume), competitor price dynamics, inventory level changes, and macroeconomic or seasonal factors, which can help the model to deeply understand the market driving logic behind price changes.

[0034] The sequence of price adjustment actions over a preset time step is a complete record of all price adjustment strategies or magnitudes executed by the preset strategy-generating agent over a past period. This feature not only reflects the system's previous decision-making path and intervention intensity but also characterizes its past operational behavior trajectory.

[0035] Step S120: The state features are concatenated in a preset order and then concatenated and normalized to generate the current state vector.

[0036] It should be understood that in order for the model to accurately identify and interpret the meaning of each dimension in the state features, the state features can be concatenated in a preset order, so that the originally discrete state features are integrated into a structured initial current state vector.

[0037] Furthermore, since the numerical magnitudes and dimensions of different state features differ, directly inputting the initial current state vector into the model can lead to poor training results or even failure to converge. Therefore, the initial current state vector can be normalized. By compressing the initial current state vector to the same standard interval to generate the current state vector, the scaling bias between various indicators can be effectively eliminated, ensuring that the model can learn the importance of each state feature for pricing decisions fairly and in a balanced manner.

[0038] The standard range can be set to [0, 1] or other ranges that meet business requirements; no restrictions are imposed here.

[0039] For example, the status characteristics may include at least one of the following: historical price series, historical market characteristic series, or historical price adjustment action series.

[0040] Step S130: Input the current state vector into a preset strategy to generate an agent, perform forward inference to output the original action value, and generate the price adjustment coefficient and recommended price corresponding to the at least one asset pricing unit based on the original action value.

[0041] It should be understood that the decision-making model in the pre-defined policy generation agent is based on an (Actor-Critic) architecture. The pre-defined policy generation agent includes a policy network and a value network, where the policy network is used for action generation and the value network is used for value evaluation. Through their collaborative operation, the pre-defined policy generation agent can perform forward reasoning based on the current state vector and output the original action value.

[0042] Furthermore, to generate the price adjustment coefficient and recommended price for each asset pricing unit based on the original action values, the system first performs inverse mapping or decoding on the normalized dimensionless original action values ​​output by the strategy network, accurately restoring them to price adjustment coefficients with practical commercial significance. Subsequently, this price adjustment coefficient is fused with the real-time benchmark price of the asset pricing unit to derive the final recommended price. This process not only achieves an effective transformation from algorithm output to business implementation but also significantly improves the interpretability and operability of recommended prices in complex market environments.

[0043] One possible design is to generate a price adjustment factor and a recommended price, which may include: Using a nonlinear activation function, the original action value is mapped and constrained to a normalized preset interval, and combined with a preset price adjustment range hyperparameter, the price adjustment coefficient is calculated. Extract the actual price of the at least one asset pricing unit from the previous period, and combine it with the price adjustment coefficient to determine the recommended price for the next period.

[0044] It should be understood that the system uses a nonlinear activation function to map and constrain the original action values ​​output by the strategy network to a normalized preset range. Then, combined with preset price adjustment hyperparameters, it calculates a price adjustment coefficient with practical business significance. Next, the system extracts the actual price of each asset pricing unit from the previous period as a benchmark, integrates it with the aforementioned price adjustment coefficient, and derives the final recommended price for the next period, thus achieving a closed loop from the original action value to the final price adjustment strategy.

[0045] The nonlinear activation function may include at least one of the following: The S-shaped Sigmoid function can smoothly compress input values ​​to the (0,1) interval; Although the hyperbolic tangent Tanh function is similar to the Sigmoid function, it maps the output to the zero-centered interval (-1,1).

[0046] In this way, by using the interval constraints of the nonlinear activation function and adjusting the hyperparameters, the extreme pricing risk caused by excessively large or small model output values ​​is effectively avoided, ensuring that the price adjustment action is always within a safe and reasonable fluctuation range. At the same time, the dynamic iterative calculation based on the actual price of the previous period not only ensures the continuity and stability of price adjustments, but also significantly improves the interpretability and feasibility of intelligent price adjustment strategies in actual business scenarios.

[0047] Step S140: Send the price adjustment coefficient and the recommended price to the business system. The price adjustment coefficient and the recommended price are used by the business system to update the price adjustment strategy.

[0048] It should be understood that the system pushes the generated price adjustment coefficient and recommended price to the business system in real time through a standard interface, and the business system directly updates them to the front end for display, thereby realizing a complete closed loop from algorithm decision-making to business implementation.

[0049] Optionally, before updating the above data, the business system can first input the price adjustment coefficient and recommended price into a preset dynamic rule engine for compliance verification. After the verification is passed, the front-end display information will be updated. The compliance verification includes, but is not limited to, at least one of the following: checking the lower limit of gross profit margin and investigating mutually exclusive promotions.

[0050] Based on this, the strategy generation method provided in this application realizes the generation of automated price adjustment strategies across the entire chain, which greatly shortens the price adjustment cycle and enables the business system to respond to market supply and demand fluctuations in a timely manner. At the same time, through the secondary compliance verification of the business system, price incidents caused by model anomalies are effectively avoided.

[0051] In some embodiments, the preset strategy generation agent used to generate price adjustment strategies needs to be pre-trained so that the price adjustment strategies generated by the preset strategy generation agent meet the needs of the business system.

[0052] For details, see Figure 2 The diagram illustrates a strategy generation method flowchart provided in an embodiment of this application. The strategy generation method 200 provided in this embodiment includes steps S210-S240.

[0053] S210, initialize the parameters of the agent's policy network and value network.

[0054] It should be understood that through deep training of the agent, a pre-defined policy generation agent specifically designed to generate dynamic pricing strategies can be constructed. Specifically, in the initial stage of training, it is necessary to systematically initialize the parameters of the core policy network (Actor) and value network (Critic) within the agent, laying a solid foundation for subsequent efficient training.

[0055] The policy network establishes a mapping from environmental states to action spaces, then outputs specific action instructions or probability distributions to address the question of how to act. The value network predicts and quantifies the long-term expected returns of the current state or specific actions, thereby judging the merits of the current situation. Specifically, in the agent, the value network provides real-time evaluation and feedback on the decision-making effectiveness of the policy network, guiding the policy network to continuously optimize its parameters to pursue higher cumulative returns, thus achieving efficient collaboration and accurate decision-making between the two.

[0056] S220, control the agent to interact with the preset pricing decision environment to generate and store multiple interaction data tuples in the experience buffer, wherein each interaction data tuple includes the current state, the action performed, the environmental feedback reward, and the next state after the transition.

[0057] It should be understood that after the parameters of the policy network and value network in the agent are initialized, they need to interact with the preset pricing decision environment to generate interactive data tuples for subsequent model training.

[0058] The interactive data tuple includes one or more of the following: current state, executed action, environmental feedback reward, and next state after transition. Multiple interactive data tuples can form a dynamic historical experience pool, thus preserving the agent's trial and error trajectory in its entirety. This provides a data foundation for breaking the time-series correlation of data and achieving efficient offline batch training.

[0059] One possible design, to enable the agent to interact with the preset pricing decision environment, requires first constructing the preset pricing decision environment, and then controlling the interaction between the agent and the preset pricing decision environment. Specifically, this could include: Based on the at least one asset pricing unit, the preprocessed multidimensional time series data is encapsulated into the preset pricing decision environment; In response to interaction requests at any given time, the state features are extracted from the pricing decision environment. The state features include at least one of the following: historical price series, historical market feature series, or historical price adjustment action series. The state features are concatenated in a preset order to construct an environmental state vector corresponding to the preset pricing decision environment; Based on the environmental state vector, the agent is controlled to perform interactive operations with the preset pricing decision environment.

[0060] It should be understood that, firstly, using at least one asset pricing unit as a benchmark, the cleaned and standardized multidimensional time series data is encapsulated into a standardized preset pricing decision environment. That is to say, in the presence of multiple asset pricing units, each asset pricing unit can have its own corresponding preset pricing decision environment, or multiple asset pricing units can share a single preset pricing decision environment; no restrictions are imposed here.

[0061] Furthermore, when a system interaction request is triggered, the environment accurately extracts key state features, including historical prices, market characteristics, and past price adjustment actions, and concatenates these heterogeneous features in a strictly preset order to construct a unified-dimensional environment state vector. Finally, this environment state vector is input to the agent, driving it to complete inference calculations and output the corresponding dynamic price adjustment strategy.

[0062] It should be understood that determining the current state, the action performed, the environmental feedback reward, and the next state after the transition in the interactive data tuple is generated by the interaction between the agent and the preset pricing decision environment.

[0063] Specifically, the agent interacts with a pre-defined pricing decision environment, generating and storing multiple interaction data tuples in an experience buffer, which may include: In the current state, the policy network outputs the execution action; after the preset pricing decision environment executes the action, it feeds back the next state after the transition and calls the preset reward function to obtain the environment feedback reward.

[0064] It should be understood that, firstly, the preset pricing decision environment provides the agent with an environmental state vector representing the current state. After receiving this state, the policy network performs forward inference calculations and outputs the corresponding action. Subsequently, this action is executed in the preset pricing decision environment and triggers the evolution of underlying data. After the environment is dynamically updated, it not only generates and feeds back the new next state after the transition, but also synchronously calls the preset reward function to calculate a quantified instant reward based on the actual business effect brought about by the action. Finally, a complete interactive data tuple is formed, which provides core data samples for the agent's subsequent policy learning and parameter optimization.

[0065] The reward function includes at least one of the following: An accuracy reward function is calculated based on the negative relative error between the recommended price and the future actual price; or... The stability reward function is calculated based on the absolute value of the difference between the current action and the action in the previous cycle.

[0066] It should be understood that the accuracy reward function uses the negative relative error between the recommended price and the future true price as its core calculation indicator, guiding the agent to output the most forward-looking and profitable pricing decisions. By introducing the future true price as a reference benchmark, this function can accurately measure the deviation between the current recommended price and the ideal market price; driving the agent to continuously approach the future true high transaction price through continuous interaction, thereby maximizing overall returns in the long run.

[0067] The stability reward function is calculated based on the absolute value of the difference between the current action and the action in the previous cycle. It is primarily used to constrain the agent's output fluctuations, prevent drastic oscillations in the price adjustment strategy, and ensure the continuity of the actions themselves. This is achieved by regularizing the absolute value of the difference between two price adjustment actions. This encourages the agent to output smoother, more consistent price adjustment strategies, ensuring profitability while also considering the security of business implementation and a good user experience.

[0068] Optionally, the reward function can be further enhanced by incorporating more business metrics and increasing its risk sensitivity.

[0069] Specifically, key business metrics such as changes in occupancy rates and projected revenue can be taken into consideration. For example, by designing a segmented reward mechanism, additional positive incentives can be given when the recommended price can significantly boost occupancy rates or increase overall revenue, thereby directly transforming broader business objectives into the optimization direction of the model.

[0070] In addition, to avoid overly aggressive strategies, a risk-sensitive reward mechanism can be introduced. By adding a penalty term for return volatility to the reward function, or imposing additional penalties for price adjustments exceeding a certain threshold, the agent can be guided to learn to avoid high-risk pricing behaviors, thereby generating robust and low-risk price adjustment strategies.

[0071] S230, sample interactive data tuples from the experience buffer, calculate the corresponding policy loss and value loss based on the near-end policy optimization algorithm, and update the parameters of the policy network and value network respectively.

[0072] It should be understood that the system first randomly samples a batch of interactive data tuples from the experience buffer, thereby breaking the time series correlation of the data and improving the sample utilization rate.

[0073] Furthermore, deep computation is performed on this data based on a proximal policy optimization algorithm: by introducing a pruning mechanism to limit the deviation between the old and new policies, a policy loss aimed at maximizing long-term returns is calculated; and by minimizing the deviation between the predicted value and the actual return, a value loss for accurately evaluating the current state is calculated. Finally, the gradient signals generated by these two types of losses are used to backpropagate and update the parameters of the policy network and the value network, respectively.

[0074] This not only ensures the stability of the model when exploring new strategies and prevents performance crashes due to excessively large update steps, but also enables the agent to steadily evolve into a more accurate and robust dynamic pricing strategy through continuous iterative optimization.

[0075] Specifically, the step of calculating the corresponding policy loss and value loss based on the proximal policy optimization algorithm, and updating the parameters of the policy network and value network respectively, may include: The objective function of the near-end policy optimization algorithm is used to determine the policy loss corresponding to the interactive data tuple, and the value loss is calculated based on the value network prediction value and the target return value. Based on the policy loss, the parameters of the policy network are updated using a gradient optimization method to increase the value of the objective function; and based on the value loss, the parameters of the value network are updated using a gradient descent method to reduce the value loss.

[0076] It should be understood that during the training of the agent, the parameters of the policy network and the value network need to be continuously updated. Specifically, based on the proximal policy optimization algorithm, the system first uses a proxy objective function with a pruning mechanism to calculate the policy loss corresponding to the interaction data tuples, and then calculates the value loss by minimizing the mean squared error between the value network prediction and the target reward value estimated by generalized advantage.

[0077] Subsequently, for the policy network, gradient ascent is used to update parameters, maximizing the expected return of the objective function, thereby steadily improving pricing decision-making ability while limiting the deviation between old and new strategies. Simultaneously, for the value network, gradient descent is rigorously applied to update parameters to continuously reduce value loss, ensuring the model can accurately assess the true value of the current market state. This approach guarantees training stability when the agent explores better strategies while significantly improving its accuracy in judging value in complex dynamic environments.

[0078] S240, repeatedly execute the process of generating and storing the interaction data tuple, and updating the parameters of the policy network and the value network until the policy network converges, to obtain the preset policy generating agent.

[0079] It should be understood that the process involves iteratively executing two core steps: generating and storing interaction data tuples based on the current policy and environment, and updating the parameters of the policy network and value network using samples from the experience buffer. As training rounds deepen, the agent gradually optimizes its decision-making logic through repeated trial and error and learning, until the output of the policy network stabilizes and various performance indicators reach convergence criteria. Ultimately, the training is completed and solidified to obtain a pre-defined policy-generating agent capable of accurately responding to complex market changes.

[0080] In some embodiments, seeFigure 3 The document illustrates a detailed flowchart of a strategy generation method provided in an embodiment of this application. The method involves first training a preset strategy generation agent, and then generating a corresponding price adjustment strategy based on the preset strategy generation agent. The method includes: Step 1: Build a pricing decision environment.

[0081] The system receives preprocessed time series data and segments the time series data according to asset pricing units to construct a pricing decision environment (i.e., the preset pricing decision environment in the aforementioned embodiments).

[0082] Specifically, the system receives preprocessed time-series data containing multi-dimensional features and organizes it based on preset asset pricing units. Under this architecture, the independent historical data sequence of each asset pricing unit is encapsulated as a dedicated decision-making environment instance, serving as the core carrier supporting the interactive training of the decision-making agent.

[0083] Among them, the asset pricing unit can be a unique identifier generated by combining the store name and the unit type name.

[0084] Step 2: Construct a pricing decision environment.

[0085] A state vector is formed by concatenating historical price sequences, historical market characteristic sequences, and historical price adjustment action sequences. .

[0086] Specifically, at any time step The state vector of the environment It is composed of the following three parts of data assembled in a preset order: Historical price series: A time series consisting of price data of the asset pricing unit within a preset time window length (Context, CTX) in the past; for example, CTX can be set to 14 days.

[0087] Historical market feature series: Time series covering other relevant market and operational feature data within the same time window length (CTX); for example, feature data includes occupancy rate, vacancy days, competitor prices, and time characteristics.

[0088] Historical price adjustment action sequence: A time series recording the actual price adjustment actions performed by the decision-making agent within the past CTX time steps.

[0089] Step 3: Define the price adjustment action space.

[0090] The agent outputs raw action values. Mapped to through a non-linear activation function Further calculation of the price adjustment coefficient, combined with the price adjustment coefficient, determines the recommended price for the next period.

[0091] Specifically, the price adjustment decision is made in a continuous action space, through interaction between the agent and the preset pricing decision environment, to obtain several interaction data tuples. The specific implementation process is as follows: The policy network of the decision-making agent is based on the current state. Output a continuous raw action value. .

[0092] The original action value is mapped to a normalized interval through a non-linear activation function. Within. The nonlinear activation function can be the hyperbolic tangent function. .

[0093] Final price adjustment coefficient Through formula Perform calculations. Among them... This is a preset price adjustment range hyperparameter used to control the maximum allowable range of a single price adjustment. The price adjustment range hyperparameter can be set to 0.5.

[0094] Recommended price for the next period Based on the price of the previous period Multiplying by this coefficient yields: .

[0095] Step 4: Design a composite reward function.

[0096] First, the accuracy reward and stability reward are calculated separately. Then, the accuracy reward and stability reward are summed to obtain the composite reward (i.e., the environmental feedback reward in the aforementioned embodiment).

[0097] Specifically, in Perform actions at all times Subsequently, in order to guide the agent to learn a strategy that balances accuracy and stability, this invention designs a composite reward function. It is obtained by weighted summation of accuracy reward and stability reward: ,in and Preset weights.

[0098] Here, w1 and w2 are preset weighting coefficients. The specific definitions of the two parts of the reward are as follows: Accuracy Rewards This measures how closely the recommended price resembles the actual future market price. The calculation method is based on the recommended price. Compared to the actual market transaction price in the next cycle Negative relative error: .in, It is a very small positive number used to prevent the denominator from being zero. For example, it can be set to 7 days.

[0099] Stability Rewards : Price adjustments that are too drastic as a penalty. Calculated based on the current original action value. Compared with the original motion value of the previous cycle The negative of the absolute value of the difference, multiplied by a penalty system. : This reward encourages agents to generate smooth, continuous price adjustment strategies, avoiding meaningless and frequent fluctuations.

[0100] Step 5: Train the decision-making agent.

[0101] First, the policy network and value network of PPO are initialized. Then, the control agent interacts with the pricing decision environment to update network parameters and maximize cumulative rewards, thereby updating the parameters of the policy network and value network and finally obtaining the preset policy generating agent.

[0102] Specifically, a deep reinforcement learning algorithm suitable for continuous action spaces, such as the proximal policy optimization algorithm, is selected to train the decision agent.

[0103] Initialize the agent's policy network and value network.

[0104] The agent engages in multiple rounds of interaction in a pricing decision-making environment: in the current state Next, the policy network outputs actions. After the environment performs an action, it provides feedback on the new status. and rewards .

[0105] The intelligent agent will collect interaction data tuples Store in the experience buffer.

[0106] Data is periodically sampled from the buffer, and the parameters of the policy network and value network are updated according to the objective function of the selected algorithm, with the goal of maximizing the long-term cumulative discount reward.

[0107] Repeat the interaction and update process until the agent's policy converges and stabilizes. After training, save the final policy network model parameters to obtain a pre-defined policy-generating agent used to generate the optimal price adjustment policy.

[0108] Furthermore, after training the preset strategy generation agent, the agent can be used to determine the price adjustment strategy and output the optimal price adjustment strategy.

[0109] Specifically, the method may also include: Step 6: Application of Model Reasoning and Pricing Decisions After the agent is trained, its trained policy network is used in a real, automated pricing decision-making process. This process is executed at each point in time when a decision is needed, for example, daily, and the specific steps are as follows: Acquiring real-time status data: For a specific asset pricing unit requiring pricing decisions, the system retrieves the real-time and historical data needed to construct the current state from a database or other data sources. This includes: past The true price sequence at each time step.

[0110] past A sequence of real market characteristics at each time step.

[0111] past A sequence of actual price adjustment actions executed at each time step.

[0112] Constructing the current state vector: The system concatenates the three parts of data obtained in the exact same order and format as during training, and then normalizes them to construct the current state vector used for decision-making. .

[0113] Perform forward reasoning: convert the current state vector The input is fed into the pre-trained policy network. The policy network performs one forward inference and outputs a deterministic optimal original action value. This process does not involve backpropagation or parameter updates.

[0114] Generate price adjustment decision: The system generates the price adjustment decision based on the output of the original action value. Using the same action space definition as during training (as described in step three), the final adjustment factor is calculated. and recommended price .

[0115] Decision output and execution: The generated recommended price Or price adjustment coefficient Output. This output can be received by downstream business systems (such as asset management platforms) to automatically update online listing prices, or pushed as a price adjustment suggestion to operations personnel for final review. Simultaneously, the system records the actions performed in this instance. This is for use when constructing the state in the next decision cycle.

[0116] Through the above reasoning process, this invention achieves an end-to-end closed loop from real-time data input to the automated generation of executable pricing decisions, applying the trained agent strategy to real-world business scenarios.

[0117] In some embodiments, seeFigure 4 The illustration shows a schematic diagram of a policy generation system architecture provided by an embodiment of this application. The system includes a data input layer (also known as an acquisition module), a data preprocessing layer (also known as a processing module), a reinforcement learning core layer (also known as a generation module), and a decision application layer (also known as a sending module).

[0118] Specifically, in Figure 4 In this system, the first step is to acquire multi-source data stored in the database through the data input layer, covering information from multiple dimensions such as operations, marketing, and tenants. This data is then sent to the data preprocessing layer, where it undergoes standardization processing via data cleaning and feature engineering modules. The processed data is further used to construct the decision-making environment for the core reinforcement learning layer. In this environment, the decision-making agent completes interactive training by receiving state feedback and reward signals and outputting corresponding actions. Finally, the optimal strategy learned by the well-trained agent is deployed to the decision application layer. The decision output and execution modules at this layer can generate specific price adjustment instructions based on the latest market conditions and push them to downstream business systems for implementation.

[0119] In the embodiments provided in this application, a composite state representation method is first proposed. By integrating historical price sequences, historical market feature sequences, and the agent's own historical price adjustment action sequences, the model can perceive the continuity of decision-making and consider the long-term impact of its own behavior on the future, thereby generating a more forward-looking and planned dynamic price adjustment strategy.

[0120] Secondly, a dual-objective composite reward function was designed. By linearly combining the "accuracy reward" that measures the deviation of future real prices and the "stability reward" that measures drastic changes in penalty actions, the abstract multidimensional business objectives are transformed into a quantifiable mathematical form. This ensures that the model outputs smooth and business-logical executable suggestions while pursuing maximum returns.

[0121] Finally, it achieves a fundamental shift in problem modeling paradigm, innovatively upgrading dynamic pricing from the traditional supervised learning prediction paradigm to a reinforcement learning decision-making paradigm. It is no longer limited to a single price prediction, but learns an end-to-end mapping strategy from complex market states to optimal price adjustment actions through continuous interaction with the simulated environment, fundamentally improving the system's level of automated decision-making and its adaptability to dynamic environments.

[0122] In some embodiments, see Figure 5 This illustrates a block diagram of a strategy generation system provided in another embodiment of this application.

[0123] like Figure 5As shown, another embodiment of this application provides a strategy generation system 500, which includes: an acquisition module 510, a processing module 520, a generation module 530, and a sending module 540.

[0124] The acquisition module 510 is used to acquire state features for constructing the current state of at least one asset pricing unit. The processing module 520 is used to concatenate the state features in a preset order and perform concatenation and normalization processing to generate the current state vector. The generation module 530 is used to input the current state vector into a preset strategy to generate an intelligent agent, and perform forward inference to output the original action value; and generate the price adjustment coefficient and recommended price corresponding to the at least one asset pricing unit based on the original action value; The sending module 540 is used to send the price adjustment coefficient and the recommended price to the business system, and the price adjustment coefficient and the recommended price are used by the business system to update the price adjustment strategy. In some embodiments, the state characteristics include at least one of the following: Price series with a previously preset time step; Market characteristic sequences with a pre-defined time step in the past; or, The sequence of price adjustment actions with a preset time step.

[0125] In some embodiments, the generation module 530 is specifically used to map and constrain the original action value to a normalized preset range using a nonlinear activation function, and calculate the price adjustment coefficient in combination with a preset price adjustment range hyperparameter; extract the actual price of the at least one asset pricing unit in the previous period, and determine the recommended price for the next period in combination with the price adjustment coefficient.

[0126] In some embodiments, the policy generation system 500 further includes a training module 550, used to initialize the parameters of the agent's policy network and value network; control the agent to interact with a preset pricing decision environment to generate and store multiple interaction data tuples in an experience buffer, wherein each interaction data tuple includes the current state, the action performed, the environmental feedback reward, and the next state after the transition; sample interaction data tuples from the experience buffer, calculate the corresponding policy loss and value loss based on a proximal policy optimization algorithm, and update the parameters of the policy network and value network respectively. The generation and storage of the interaction data tuples, and the updating of the parameters of the policy network and the value network are repeated until the policy network converges, thus obtaining the preset policy generation agent.

[0127] In some embodiments, the training module 550 is specifically configured to determine the policy loss corresponding to the interactive data tuple based on the objective function of the proximal policy optimization algorithm; calculate the value loss based on the value network prediction value and the target reward value; update the parameters of the policy network using a gradient optimization method based on the policy loss to increase the value of the objective function; and update the parameters of the value network using a gradient descent method based on the value loss to reduce the value loss.

[0128] In some embodiments, the training module 550 is specifically used to: in the current state, the policy network outputs the execution action; after the preset pricing decision environment executes the action, it feeds back the next state after the transition and calls a preset reward function to obtain the environment feedback reward; wherein, the environment feedback reward function includes at least one of the following: an accuracy reward function calculated based on the negative relative error between the recommended price and the future real price; or, a stability reward function calculated based on the absolute value of the difference between the current action and the previous period action.

[0129] In some embodiments, the training module 550 is specifically configured to: encapsulate preprocessed multidimensional time series data into a preset pricing decision environment based on the at least one asset pricing unit; extract state features from the pricing decision environment in response to interaction requests at any time, the state features including at least one of the following: historical price series, historical market feature series, or historical price adjustment action series; concatenate the state features in a preset order to construct an environment state vector corresponding to the preset pricing decision environment; and control the agent to perform interactive operations with the preset pricing decision environment based on the environment state vector.

[0130] In the embodiments provided in this application, for at least one asset pricing unit, state features for constructing the current state of the asset pricing unit are obtained; the state features are concatenated in a preset order and subjected to concatenation and normalization processing to generate a current state vector; the current state vector is input into a preset strategy to generate an intelligent agent, and forward inference is performed to output the original action value; based on the original action value, a price adjustment coefficient and a recommended price corresponding to at least one asset pricing unit are generated; the price adjustment coefficient and the recommended price are sent to the business system, and the price adjustment coefficient and the recommended price are used by the business system to update the price adjustment strategy. This not only improves the automation level of pricing decisions but also connects the entire chain from data perception to price generation, significantly reducing labor costs while effectively enhancing the accuracy and scientific nature of pricing; furthermore, the optimized price adjustment strategy can flexibly take into account multiple business objectives, thus better meeting actual business needs.

[0131] Figure 6 A block diagram of an electronic device provided for another embodiment of this application.

[0132] Another embodiment of this application provides an electronic device having a computer program stored thereon, which, when executed by a processor, implements the steps of the method of any of the above embodiments.

[0133] like Figure 6 As shown, for ease of understanding, an embodiment of this application illustrates a specific electronic device 600.

[0134] Electronic device 600 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic device 600 may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0135] like Figure 6 As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. The RAM 603 may also store various programs and data required for the operation of the electronic device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0136] Multiple components in electronic device 600 are connected to input / output (I / O) interface 605. These components include: input unit 606, such as a keyboard or mouse; output unit 607, such as various types of displays or speakers; storage unit 608, such as a disk or optical disk; and communication unit 609, such as a network interface card (NIC), modem, or wireless transceiver. Communication unit 609 allows electronic device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0137] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods described above. For example, in some embodiments, any one or more of the various methods described above can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of any one or more of the various methods described above can be performed. Alternatively, in other embodiments, the computing unit 601 can be configured to perform any one or more of the various methods described above by any other suitable means (e.g., by means of firmware).

[0138] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method in any of the above embodiments.

[0139] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this application, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0140] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0141] In the description of this application, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this application, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0142] In the description of this application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicating the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0143] Furthermore, the terms "first," "second," etc., used in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying relative importance, or implicitly specifying the number of technical features indicated in this embodiment. Therefore, features defined with terms such as "first" and "second" in the embodiments of this application can explicitly or implicitly indicate that the embodiment includes at least one of those features. In the description of this application, the word "multiple" means at least two or more, such as two, three, four, etc., unless otherwise explicitly and specifically defined in the embodiments.

[0144] In this application, unless otherwise explicitly specified or limited in the embodiments, the terms "installation," "connection," "joining," and "fixing" appearing in the embodiments should be interpreted broadly. For example, a connection can be a fixed connection, a detachable connection, or an integral part; it can also be a mechanical connection, an electrical connection, etc. Of course, it can also be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal communication between two components, or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific implementation.

[0145] In this application, unless otherwise expressly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "on top of," and "over" the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0146] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A strategy generation method, characterized in that, The method includes: For at least one asset pricing unit, obtain state features used to construct the current state of the asset pricing unit; The state features are concatenated in a preset order, and then concatenated and normalized to generate the current state vector. The current state vector is input into a preset strategy to generate an agent, and forward inference is performed to output the original action value. The price adjustment coefficient and recommended price corresponding to the at least one asset pricing unit are generated based on the original action value. The price adjustment coefficient and the recommended price are sent to the business system, and the price adjustment coefficient and the recommended price are used by the business system to update the price adjustment strategy.

2. The method according to claim 1, characterized in that, The state characteristics include at least one of the following: Price series with a previously preset time step; Market characteristic sequences with a pre-defined time step in the past; or, The sequence of price adjustment actions with a preset time step.

3. The method according to claim 1 or 2, characterized in that, The step of generating the price adjustment coefficient and recommended price corresponding to the at least one asset pricing unit based on the original action value includes: Using a nonlinear activation function, the original action value is mapped and constrained to a normalized preset interval, and combined with a preset price adjustment range hyperparameter, the price adjustment coefficient is calculated. Extract the actual price of the at least one asset pricing unit from the previous period, and combine it with the price adjustment coefficient to determine the recommended price for the next period.

4. The method according to any one of claims 1-3, characterized in that, The method further includes: Initialize the parameters of the agent's policy network and value network; The intelligent agent is controlled to interact with a preset pricing decision environment to generate and store multiple interaction data tuples in an experience buffer, wherein each interaction data tuple includes the current state, the action performed, the environmental feedback reward, and the next state after the transition; The interaction data tuples are sampled from the experience buffer, and the corresponding policy loss and value loss are calculated based on the near-end policy optimization algorithm. The parameters of the policy network and the value network are then updated respectively. Repeatedly generate and store the interaction data tuples, and update the parameters of the policy network and the value network until the policy network converges, to obtain the preset policy generating agent.

5. The method according to claim 4, characterized in that, The calculation of the corresponding policy loss and value loss based on the near-end policy optimization algorithm, and the updating of the parameters of the policy network and value network respectively, includes: The objective function of the near-end policy optimization algorithm is used to determine the policy loss corresponding to the interactive data tuple, and the value loss is calculated based on the value network prediction value and the target return value. Based on the policy loss, the parameters of the policy network are updated using a gradient optimization method to increase the value of the objective function; and based on the value loss, the parameters of the value network are updated using a gradient descent method to reduce the value loss.

6. The method according to claim 4 or 5, characterized in that, The generation and storage of multiple interactive data tuples into the experience buffer includes: In the current state, the policy network outputs the execution action; after the preset pricing decision environment executes the action, it feeds back the next state after the transition and calls the preset reward function to obtain the environment feedback reward; The environmental feedback reward function includes at least one of the following: An accuracy reward function is calculated based on the negative relative error between the recommended price and the future actual price; or... The stability reward function is calculated based on the absolute value of the difference between the current action and the action in the previous cycle.

7. The method according to any one of claims 4-6, characterized in that, The control of the intelligent agent to interact with the preset pricing decision environment includes: Based on the at least one asset pricing unit, the preprocessed multidimensional time series data is encapsulated into the preset pricing decision environment; In response to interaction requests at any given time, the state features are extracted from the pricing decision environment. The state features include at least one of the following: historical price series, historical market feature series, or historical price adjustment action series. The state features are concatenated in a preset order to construct an environmental state vector corresponding to the preset pricing decision environment; Based on the environmental state vector, the agent is controlled to perform interactive operations with the preset pricing decision environment.

8. A strategy generation system, characterized in that, The system includes: The acquisition module is used to acquire state features for constructing the current state of at least one asset pricing unit. The processing module is used to concatenate the state features in a preset order, and perform concatenation and normalization processing to generate the current state vector. The generation module is used to input the current state vector into a preset strategy to generate an agent, and perform forward inference to output the original action value; and generate the price adjustment coefficient and recommended price corresponding to the at least one asset pricing unit based on the original action value; The sending module is used to send the price adjustment coefficient and the recommended price to the business system, and the price adjustment coefficient and the recommended price are used by the business system to update the price adjustment strategy.

9. An electronic device having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-7.

Images