Cross-border intelligent pricing optimization method based on reinforcement learning

Abstract

The invention discloses a cross-border intelligent pricing optimization method based on reinforcement learning, and belongs to the cross technical field of artificial intelligence and cross-border electronic commerce. The method comprises the following steps: acquiring and preprocessing multi-source data in a cognitive direct-connected manufacturing network, and constructing a state vector of a fusion structure conservation mapping constraint; a bidirectional reinforcement learning structure is established on the basis, and benefits and risks are collaboratively optimized; an exchange rate sensitive gating unit and an economic constraint condition are introduced to ensure the robustness and compliance of pricing; and finally, cross-market rapid strategy adaptation is realized through migration reinforcement learning. As a key link of a CDM cognitive direct-connection manufacturing technology system, the dynamic pricing problem in a direct-connection scene is effectively solved, and the overall income and market competitiveness of a platform are improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence and cross-border e-commerce, and particularly to a method and system for optimizing cross-border intelligent pricing based on multi-source data fusion and reinforcement learning in a cognitive direct-connect manufacturing network. Background Technology

[0002] With the rapid development of cross-border e-commerce, commodity pricing is influenced by multiple factors, including exchange rate fluctuations, tariff policies, logistics delays, regional consumption levels, and competitive landscape. To cope with this complex market environment, existing cross-border e-commerce platforms typically utilize pricing models based on statistical regression or machine learning. These models fit the relationship between price and demand using historical sales data, thereby achieving automated pricing within a certain range. While these methods are highly applicable in a single market environment, their pricing results are easily affected by external disturbances under conditions of multiple regions, multiple currencies, and dynamic policies.

[0003] Most existing cross-border pricing methods employ static rules or single-layer supervised learning models, which have limited responsiveness to exchange rate fluctuations, market heterogeneity, and inventory dynamics. In multi-market scenarios, consumer behavior, policy restrictions, and logistics costs vary significantly across different regions, making it difficult for traditional pricing models to maintain consistency across multiple factors. Furthermore, while some reinforcement learning pricing research possesses adaptive learning capabilities, it generally lacks cross-market factor modeling mechanisms and fails to implement structured controls for exchange rate fluctuation risks and economic constraints. This leads to models that are prone to price volatility or non-compliant pricing results in practical deployments.

[0004] In particular, pricing strategies face unprecedented challenges in cognitive direct-connect manufacturing platforms centered on the "super factory direct-to-consumer" model. The traditional multi-tiered distribution system is disrupted, and pricing must directly reflect dynamically changing factory costs, instantaneous production capacity, and globally dispersed personalized demands. However, existing pricing models cannot effectively integrate and understand this cross-domain, cross-modal real-time data, making it impossible to achieve a dynamic optimal balance between ensuring reasonable factory profits and providing consumers with the ultimate cost-effectiveness. This has become one of the key bottlenecks restricting the large-scale development of the "CDM model." Therefore, how to achieve intelligent pricing that is dynamic, cognitive, and robust in complex cross-border direct-connection scenarios has become a problem that urgently needs to be solved by those skilled in the art.

[0005] Therefore, how to provide a cross-border intelligent pricing optimization method based on reinforcement learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a reinforcement learning-based intelligent pricing optimization method for cross-border transactions, addressing the issues of poor pricing adaptability and weak risk control in existing technologies within cognitive direct-connection manufacturing scenarios. This invention, by introducing a bidirectional reinforcement learning structure, structural conservation mapping constraints, and exchange rate-sensitive gating and economic constraint mechanisms, achieves intelligent pricing with self-awareness and self-optimization capabilities, working in conjunction with the company's cognitive direct-connection system and seller credit assessment system. This is a core component in constructing a closed-loop intelligent business decision-making system across the entire "factory-to-consumer" supply chain.

[0007] The cross-border intelligent pricing optimization method based on reinforcement learning according to embodiments of the present invention includes the following steps:

[0008] Collect multi-source data from different regions and perform time synchronization, outlier removal and normalization to obtain a standardized cross-border dataset;

[0009] Based on the standardized cross-border dataset, economic features, market features and behavioral features are mapped to a shared latent space through a hierarchical embedding network. A structure conservation mapping constraint layer is introduced to output the state vector after structure conservation constraints.

[0010] A bidirectional reinforcement learning structure is constructed based on the state vector, the structure including a forward reinforcement learning network and a backward reinforcement learning network, to generate the target price;

[0011] The composite reward value is calculated based on the target price, and the weights of each target are dynamically adjusted according to market feedback to output a comprehensive reward signal;

[0012] In the reinforcement learning model, a currency-sensitive gating unit is introduced to adjust the output amplitude of the strategy by using the exchange rate volatility as input, and outputs a controlled action.

[0013] Imposing economic constraints during the reinforcement learning and training process;

[0014] The constrained optimal strategy after training convergence is deployed to the cross-border e-commerce pricing module, and online inference is performed by inputting the updated state vector in real time.

[0015] Based on cross-market return performance, transfer reinforcement learning updates are performed. When a set condition is detected, the strategy parameters are transferred to the new market model and incrementally fine-tuned.

[0016] Furthermore, the multi-source data includes:

[0017] The data includes historical sales data from cross-border e-commerce platforms, real-time exchange rate data for different currencies, tariff rate parameters for various countries or regions, logistics and transportation cost and time delay data, market demand index data, competitor product prices and promotional activity data, as well as user click-through rate and conversion rate data. All types of data are accompanied by a unified timestamp.

[0018] Furthermore, the steps for generating the standardized cross-border dataset include:

[0019] The collected multi-source data is linearly interpolated and aligned with the time window based on the timestamp;

[0020] An outlier data sample was identified and removed using a statistical detection method based on the three-times standard deviation principle.

[0021] Perform Z-score standardization on the remaining data to make the mean of each feature distribution zero and the standard deviation one.

[0022] The processed data is aggregated in multiple dimensions according to product identifier, market region, and time index to form a structured and standardized cross-border dataset.

[0023] Furthermore, the step of generating the state vector includes:

[0024] The standardized cross-border dataset is input into the economic embedding layer, the market embedding layer, and the behavioral embedding layer, respectively.

[0025] The economic embedding layer performs numerical encoding on exchange rate, tariff rate and logistics cost features and extracts economic feature vectors through a fully connected neural network.

[0026] The market embedding layer performs embedding transformations on the market demand index, competitive prices, and inventory level features to generate a market feature vector.

[0027] The behavior embedding layer performs time-series modeling of user click-through rate and conversion rate features to generate behavior feature vectors;

[0028] The three types of feature vectors are concatenated by the fusion layer and then input into the structure conservation mapping constraint layer. In the structure conservation mapping constraint layer, the difference between the original feature correlation matrix and the embedded feature correlation matrix is ​​calculated and its Frobenius norm is used as the constraint loss.

[0029] The fused latent space representation output by the constraint layer is transformed by nonlinear mapping and normalized projection to generate a fixed-length state vector.

[0030] Furthermore, the construction steps of the bidirectional reinforcement learning structure include:

[0031] The state vector is input into the forward reinforcement learning network and the backward reinforcement learning network;

[0032] The forward reinforcement learning network uses an algorithm based on deterministic policy gradients to generate the optimal pricing action in the current market.

[0033] The reverse reinforcement learning network estimates the long-term risk cost based on future market states;

[0034] The two networks share a state embedding layer and jointly update parameters during training using action consistency constraints;

[0035] During the strategy decoupling phase, the price range action output by the forward network is set by the decoupling layer to set the action value range of the lower network. The lower network outputs price fine-tuning action within the range and superimposes it with the forward action to generate the target price of the corresponding product.

[0036] By performing weighted learning on historical samples through an experience replay mechanism and a time discount factor, the collaborative optimization of the two-way strategy is achieved.

[0037] Furthermore, the calculation steps for the composite reward value include:

[0038] Calculate profit and revenue indicators based on target price and actual sales results;

[0039] Market share growth indicators are calculated based on product sales volume and total market sales volume.

[0040] Calculate risk penalties based on exchange rate volatility and inventory change rate;

[0041] The profit and revenue indicators, market share growth indicators, and risk penalty items are multiplied by dynamic weight parameters and linearly combined to form a composite reward value.

[0042] The dynamic weight parameters are adaptively updated through a genetic algorithm in multiple training iterations. The resulting composite reward value is fed back to the forward network and the backward network as a joint optimization signal to simultaneously update the profit strategy and risk strategy parameters.

[0043] Furthermore, the implementation steps of the exchange rate sensitive gating unit include:

[0044] Extract the exchange rate volatility component from the state vector as a gating input signal;

[0045] The standard deviation of the exchange rate change series is calculated using a sliding time window to characterize the current exchange rate volatility.

[0046] The exchange rate volatility is input into a gating function. When the exchange rate volatility is higher than a preset threshold, the output of the gating function approaches zero.

[0047] The gated output is applied simultaneously to the policy outputs of both the forward reinforcement learning network and the backward reinforcement learning network to suppress the price adjustment range and generate controlled price actions with limited range.

[0048] The gated output participates in action regulation during both model training and online inference phases.

[0049] Furthermore, the implementation steps of the economic constraints include:

[0050] During the reinforcement learning training phase, the tariff ceiling parameter, minimum price parameter, and inventory capacity parameter are input into the constraint modules of the forward network and the backward network.

[0051] The upper limit of the target price is determined by the tariff ceiling parameter, the lower limit of the target price is determined by the minimum price parameter, and the price adjustment range is determined by the inventory capacity parameter.

[0052] Add an economic constraint regularization term and a structural conservation constraint term to the loss function, and calculate the penalty value when the target price deviates from the constraint range.

[0053] During the online inference phase, an interval truncation operation is performed on the predicted output to ensure that the pricing output generated by the bidirectional network conforms to the feasible interval defined by the constraint parameters.

[0054] Furthermore, the strategy deployment and online inference steps include:

[0055] The optimal strategy parameters, which have converged after bidirectional training, are loaded into the cross-border e-commerce pricing server.

[0056] During operation, it periodically receives real-time exchange rate data, commodity sales data, and inventory status data;

[0057] The real-time data is converted into a state vector and input into the forward and backward networks to perform synchronous inference.

[0058] During the inference output phase, the target price value is calculated by weighted fusion of the price actions output by the two networks.

[0059] The target price value is formatted and then written to the pricing database for front-end access.

[0060] Furthermore, the transfer reinforcement learning update step includes:

[0061] During the operation phase, cross-market return and risk indicators are monitored. When any indicator of return decline rate, market share change rate, or risk variance exceeds the threshold, a migration update is triggered.

[0062] During the migration and update phase, the policy parameters of the forward network and the backward network are migrated to the target market model as initialization parameters, respectively.

[0063] An incremental training set is constructed based on historical transaction data and the latest state vector of the target market;

[0064] In the incremental training set, the mini-batch stochastic gradient descent algorithm is used to perform synchronous fine-tuning of the parameters of the two networks;

[0065] The fine-tuned model parameters are written into the pricing module on the deployment server.

[0066] The beneficial effects of this invention are:

[0067] This invention introduces a bidirectional reinforcement learning structure into cross-border e-commerce pricing scenarios, enabling the forward and backward networks to achieve a synergistic optimization relationship in maximizing returns and minimizing risks. The forward network outputs the optimal pricing action based on real-time market conditions, while the backward network estimates long-term risk costs based on future market trends. The two networks jointly update their parameters through action consistency constraints. This design maintains the stability and forward-looking nature of the pricing strategy even during periods of significant market volatility or changes in external factors. Compared to traditional unidirectional reinforcement learning structures, it effectively reduces price decision oscillations and overfitting risks, improving the model's robustness and generalization performance.

[0068] This invention incorporates a structure-conserving mapping constraint layer during the state vector generation stage. By minimizing the difference between the original feature correlation matrix and the embedded feature correlation matrix, it ensures that the statistical correlation structure of the embedding space remains consistent with the original feature space. This constraint mechanism guarantees the structural mapping relationship between economic features, market features, and behavioral features in the latent space, preventing embedding representation distortion and thus improving the fusion accuracy and model interpretability of multi-source heterogeneous data. Compared to ordinary deep embedding networks without structure-conserving constraints, the state representation of this invention is more stable and can more accurately reflect dynamic market relationships.

[0069] This invention further introduces an exchange rate-sensitive gating unit and an economic constraint module. It uses exchange rate volatility as the gating input to dynamically adjust the strategy output amplitude and constructs a multi-constraint mechanism by combining economic constraint parameters such as tariff ceilings, minimum quoted prices, and inventory capacity. This design enables the reinforcement learning model to generate a controlled target price even under highly volatile exchange rates or policy constraints, preventing abnormal price jumps or exceeding limits. Simultaneously, by combining a multi-objective adaptive weighted average with a composite reward function and a transfer reinforcement learning update mechanism, this invention achieves rapid cross-market strategy transfer and dynamic adaptive optimization, significantly improving the stability, economic compliance, and return equilibrium of cross-border smart pricing. Attached Figure Description

[0070] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0071] Figure 1 This is a flowchart of the cross-border intelligent pricing optimization method based on reinforcement learning proposed in this invention;

[0072] Figure 2 This is the state vector generation graph for the cross-border intelligent pricing optimization method based on reinforcement learning proposed in this invention.

[0073] Figure 3 This is a diagram illustrating the bidirectional reinforcement learning structure of the cross-border intelligent pricing optimization method based on reinforcement learning proposed in this invention. Detailed Implementation

[0074] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0075] refer to Figure 1 - Figure 3 The cross-border intelligent pricing optimization method based on reinforcement learning includes the following steps: collecting historical sales data, exchange rate data, tariff rates, logistics delays, market demand intensity, competitor pricing information and consumer conversion rate data from different regions, and performing time synchronization, outlier removal and normalization processing to obtain a standardized cross-border dataset.

[0076] Based on the standardized cross-border dataset, economic features, market features, and behavioral features are mapped to a shared latent space through a hierarchical embedding network. A structure conservation mapping constraint layer is introduced during the embedding process, and the state vector after structure conservation constraint is output.

[0077] A bidirectional reinforcement learning structure is constructed based on the state vector. The structure includes a forward reinforcement learning network and a backward reinforcement learning network. The forward network predicts the optimal pricing action based on the current state, and the backward network estimates the long-term risk cost based on the future market state. The two networks share a state embedding layer and are jointly optimized through an action consistency constraint term to generate a target price.

[0078] The composite reward value is calculated based on the target price, and the weights of each target are dynamically adjusted according to market feedback to output a comprehensive reward signal;

[0079] In the reinforcement learning model, a currency-sensitive gating unit is introduced to adjust the output amplitude of the strategy by using the exchange rate volatility as input, and outputs a controlled action.

[0080] Imposing economic constraints during intensive learning and training, including tariff caps, minimum price limits, and inventory capacity restrictions;

[0081] The constrained optimal strategy after training convergence is deployed to the cross-border e-commerce pricing module, and online inference is performed by inputting the updated state vector in real time.

[0082] Based on cross-market return performance, transfer reinforcement learning updates are performed. When a set condition is detected, the strategy parameters are transferred to the new market model and incrementally fine-tuned.

[0083] In this embodiment, the multi-source data includes: historical sales data of cross-border e-commerce platforms, real-time exchange rate data of different currencies, tariff rate parameters of various regions, logistics and transportation cost and time delay data, market demand index data, competitor's product prices and promotional activity data, and user click-through rate and conversion rate data. All types of data have a unified timestamp.

[0084] In practical implementation, historical sales data from cross-border e-commerce platforms are periodically exported through the platform's backend interface. Data fields include product identifier, sales time, transaction price, order quantity, and region code. Real-time exchange rate data is updated minute-by-minute or hourly by external financial data service interfaces, with exchange rate units using standardized ISO 4217 currency codes. Tariff rate parameters for each region are synchronously obtained from the customs data center or public tax interfaces and stored as a parameter table indexed by region code. Logistics and transportation cost and delay data are obtained through the API of logistics service providers, recording transportation methods, average delivery time, unit cost, and historical delay rates. Market demand index data is provided by third-party market monitoring agencies, updated weekly or daily, reflecting the overall consumption trends in the target market. Competitor product prices and promotional activity data are updated in real time through the platform's price comparison interface, recording price changes and promotional time windows of target products in similar markets in JSON format. User click-through rate and conversion rate data are collected by platform tracking points, and sales data are linked through session identifiers and timestamps to achieve a mapping of behavioral information.

[0085] To ensure consistency across heterogeneous data sources over time, all data types undergo unified timestamp standardization after collection, using Coordinated Universal Time (UTC) and recording with second-level precision. For features with different update frequencies, alignment is achieved on a unified timeline through interpolation or by maintaining the latest value, ensuring the synchronization of input features at the same time step. All data sources are aggregated and stored in a time-series database in a key-value structure, providing a unified data interface for subsequent time window segmentation and feature embedding.

[0086] In this implementation, to ensure consistency in time and numerical scales across heterogeneous multi-source data, all timestamped data is segmented and aligned using a unified time window. A preset time step is used as the window unit, and linear interpolation is performed to fill in missing records from different data sources. When a missing record exists within the time window, it is filled using linear interpolation based on known values ​​from adjacent time points. If the time interval between adjacent records exceeds a preset time span limit, the data is marked as uninterpolable and removed in subsequent processing. This method ensures the continuity and comparability of data from different sources on a unified timeline.

[0087] To eliminate the impact of outlier data samples, a three-standard-deviation detection method based on statistical features is used to identify outliers. For each feature class, the mean and standard deviation of that feature class are calculated. When a data sample deviates from the mean by more than three standard deviations, the sample is identified as an outlier and removed. For features that change over time, such as exchange rates, sales figures, and inventory levels, the detection threshold is dynamically updated within a sliding time window to improve the accuracy and stability of outlier identification.

[0088] Standardization is performed on the data after anomaly detection. By shifting and scaling the values ​​of each feature, the mean of their distribution is zero and the standard deviation is one, thereby eliminating the influence of differences in the units and ranges of different features and ensuring the stability of gradient changes during the subsequent training of the embedding model.

[0089] The processed data is aggregated according to three dimensions: product identification, market region, and time index, forming structured multidimensional data units. Each data unit contains sales characteristics, exchange rate characteristics, tax rate characteristics, logistics characteristics, market characteristics, competition characteristics, and user behavior characteristics within a fixed time window, and is stored in a time-series database in a unified data structure format, forming a standardized cross-border dataset. This dataset provides a unified and stable source of input features for subsequent hierarchical embedding networks.

[0090] In this embodiment, the state vector generation step is used to transform standardized multi-source cross-border data into a unified learnable representation to provide input features for the reinforcement learning model. This step includes five stages: economic embedding, market embedding, behavioral embedding, feature fusion, and structural conservation mapping constraints.

[0091] Feature extraction and mapping are performed on data reflecting the macroeconomic environment. Economic features include exchange rates, tariff rates, transportation costs, and their trends over time. After preprocessing, all numerical economic features are mapped to a unified dimensional interval to eliminate scale differences between different economic indicators. For continuously fluctuating features, multi-level linear and non-linear transformations are performed to extract high-dimensional semantic representations; for discrete economic variables, embedding encoding is performed to generate trainable dense representations. All features are processed by feature cross-interaction units to generate combined terms reflecting the interactions between variables, outputting an economic feature vector.

[0092] Features related to supply and demand dynamics are encoded and fused. Input features include market demand index, competitive price, inventory level, and sales cycle. Discrete market identifiers are embedded to obtain a dense representation, while numerical features are normalized and trend extracted to obtain variation magnitude and direction features. An attention weighting mechanism is introduced during the embedding process to dynamically adjust feature weights based on feature fluctuation frequency and competitive activity. The fused feature vector is then output.

[0093] Modeling is performed on the temporal features describing user interaction behavior. Input data includes click-through rate, conversion rate, browsing duration, and repeat visit frequency. The behavior sequence is divided into segments based on a fixed time span, and statistical indicators such as average level, growth trend, and fluctuation intensity are extracted within each segment. A time-dependent modeling structure is used to capture the dynamic relationships between adjacent segments, and a gating mechanism is used to adjust the strength of time dependencies to extract the behavioral features with the main influence within the current time period. The output is a behavioral feature vector after multi-layer temporal modeling.

[0094] The economic feature vector, market feature vector, and behavioral feature vector are concatenated and normalized. The fusion layer extracts cross-domain interaction features through multi-layer nonlinear mapping and imposes constraints on the feature dimension distribution during the mapping process to maintain a stable distribution in the latent space.

[0095] Constraints are applied to the structural consistency of the fused features. This stage establishes statistical correlations between the original input features to describe the dependency structure of each feature in the original space; simultaneously, it establishes the correspondence structure between the embedded features and measures the differences between them. When the structural differences increase, a corresponding structure-preserving signal is generated to guide the embedding layer to adjust the mapping weights during training. After this stage, the output fused latent space representation undergoes nonlinear mapping and normalized projection to form a fixed-length state vector.

[0096] This state vector comprehensively reflects three types of information: economic, market, and behavioral information, and is used as input data for reinforcement learning models.

[0097] In this embodiment, a bidirectional reinforcement learning structure is used to establish reward-oriented and risk-oriented policy models within the same training framework, thereby forming a decision structure with bidirectional update capabilities. The structure includes a forward reinforcement learning network and a backward reinforcement learning network, which share a state vector input and update their parameters through action consistency constraints during training.

[0098] During the training phase of the feedforward reinforcement learning network, the cross-border market state, represented by a state vector, is input into the network. The network generates pricing actions based on a deterministic policy, where each pricing action represents the direction and magnitude of price adjustments under the current market state. Network training is primarily profit-driven, calculating the impact of the current action on expected profit through a policy gradient-based update method and performing gradient corrections based on historical experience samples. To enhance learning stability, an experience sample replay method is used during training, drawing samples from past state-action-reward sequences to balance the distribution of new and old data. After multiple rounds of updates, the feedforward network generates a profit-oriented pricing strategy.

[0099] During the training phase of the inverse reinforcement learning network, the input is the same state vector as the forward network. This network builds a long-term risk estimation model based on changes in market state and future return predictions, used to calculate potential risk losses under different pricing strategies. During training, backtracking is used to update the parameters of the risk assessment part by using the return volatility of subsequent time steps as a monitoring signal. After multiple rounds of training, the inverse network can output risk signals related to pricing actions, describing the impact of these actions on long-term stability.

[0100] Constraint associations are applied to the output actions of the forward and backward networks. This stage calculates the difference between the outputs of the two networks and generates constraint signals to ensure that the output actions of both networks maintain a consistent direction or a difference within an acceptable range when they are in the same state. These constraint signals are added as an additional term to the parameter updates during training, enabling synchronized adjustments between reward-oriented and risk-oriented strategies within the weight space.

[0101] The price range action output by the forward network serves as a range reference, limiting the value range of the price fine-tuning action output by the reverse network. The forward action determines the macro price range, and the reverse action performs fine-tuning adjustments within this range. The decoupled result forms the target price action, used to describe the final pricing decision under the current market conditions.

[0102] Based on the reward signal calculated by the composite reward function, the parameters of the forward and backward networks are updated synchronously. The forward network adjusts the policy output direction according to the reward signal in the payoff component, while the backward network corrects the prediction model according to the risk signal. Both are executed iteratively within the same training loop, enabling the payoff-oriented and risk-oriented policies to converge simultaneously. After collaborative updating, the bidirectional reinforcement learning structure completes training and outputs a joint policy model that can be used for target price generation.

[0103] In this implementation, the calculation of the composite reward value is based on three quantitative indicators: revenue, market share, and risk. These indicators are used to generate a unified reward signal that can be invoked by the reinforcement learning model during training. Each pricing cycle is considered a training period. During this period, the system collects sales records, market summary information, and external economic parameters. Feature extraction and statistical processing are performed on this information to generate the three basic indicators.

[0104] Revenue calculation is based on selling price, unit cost, and transaction volume. For each transaction within the specified period, its profit value is first calculated, and then averaged over the period to obtain the average profit level. To avoid bias from extreme samples, a Windsor truncation process is performed on the profit series, replacing outliers exceeding quantile boundaries with boundary values. If a time decay mechanism is required, a time weight can be used when calculating the average, giving a higher weight to the contribution of the most recent transaction records to the revenue result. The average profit level obtained after the above processing is the revenue indicator.

[0105] Market share is calculated using the product's sales volume and the total market sales volume of similar products as input. First, the market share ratio for the current period is calculated, then compared with the market share for the previous period to obtain the change in share. To mitigate abnormal fluctuations caused by short-term promotions or stockouts, the change in share sequence is smoothed using a fixed-length sliding window, ensuring the input data exhibits a continuous time trend. The smoothed change in share serves as the market share indicator, representing the market performance of the current pricing strategy relative to competing products.

[0106] Risk measurement uses exchange rate changes and inventory fluctuations as input data. Exchange rate data is recorded in time series form, and the level of exchange rate volatility is obtained by calculating the relative rate of change between adjacent time points and statistically analyzing the dispersion within a sliding window. Inventory data is based on continuous inventory quantity records, and the level of inventory volatility is obtained by calculating the relative rate of change between adjacent time points and similarly statistically analyzing the dispersion. After standardization, the two volatility levels are weighted at a fixed ratio to obtain a single risk measure value, which is used to characterize the combined uncertainty of the external economic environment and the internal inventory status.

[0107] After calculating the three types of indicators, the results for revenue, market share, and risk are standardized to ensure they are on a comparable numerical scale. A weighted composite operation is then performed, adding the revenue and market share indicators according to their respective weights, and then subtracting the risk indicator after weighting it according to its own weight, resulting in the composite reward value for that period. All three weights are non-negative and their sum is limited to one. The weight parameters are updated using a genetic algorithm. The algorithm uses candidate weight combinations as a population, calculates the cumulative reward as fitness within the validation window, and generates new candidate combinations through selection, crossover, and fine-tuning. Normalization is performed after each iteration to maintain the constraints. When the fitness converges or reaches a predetermined number of generations, the optimal weight combination is selected for the current training cycle.

[0108] During the calculation process, if sales data is incomplete, missing items are skipped; if there are short-term gaps in market summary data or foreign exchange data and the span does not exceed the set threshold, linear interpolation is used to complete them; segments exceeding the threshold are not included in the current period's statistics. All substitution and elimination operations are recorded for subsequent review.

[0109] After calculating the composite reward value, this scalar value is input as the reward signal for the same training period into the forward and backward networks of the reinforcement learning framework. The forward network directly uses this reward signal to calculate the reward-oriented gradient when updating policy parameters, while the backward network adjusts the estimated weights based on the risk component implicit in the same signal when updating risk assessment parameters. Both are updated synchronously within the same training cycle, thus achieving parameter linkage between maximizing reward and mitigating risk. The composite reward sequence generated through this process is used for continuous optimization in subsequent training iterations and online inference phases.

[0110] In this implementation, the exchange rate sensitive gating unit uses the exchange rate features in the state vector as input to numerically limit the magnitude of price movements. This unit remains enabled during both the training and online inference phases, ensuring that the policy network is consistently constrained in all operating states.

[0111] Exchange rate change records corresponding to the target currency and the base currency are extracted from the state vector to form a continuous time series. This series is then sorted by time and divided into fixed-length sliding windows. Within each window, the rate of change of the exchange rate between adjacent time points is calculated. The rate of change data within each window is then subjected to median filtering to remove single-point outliers, and the dispersion index of the rate of change is calculated based on the filtering result. This dispersion index reflects the current exchange rate volatility and serves as the input to the gating function.

[0112] The gating function is a single-input, single-output continuous mapping unit. The input is the exchange rate volatility, and the output is the gating coefficient. The gating function's output is close to one in the low volatility range, gradually decreasing as volatility increases, and approaching zero after reaching a set threshold. The gating function can be a logistic or exponential continuous function. The smoothing parameter is set during model initialization and remains constant during training. The gating coefficient's value is limited to between zero and one.

[0113] During the training phase, the pricing actions generated by the reinforcement learning network are multiplied by gating coefficients before entering the environment for interaction. This operation is performed simultaneously at the outputs of both the forward and backward reinforcement learning networks, ensuring that the gating units impose the same amplitude constraints on the action signals of both networks. The gating-adjusted action signals then serve as the final execution signals in the interaction, generating corresponding feedback. This feedback data is then input into the model update loop.

[0114] During the online inference phase, real-time exchange rate data is fed into the sliding window in the same manner to update the exchange rate change rate sequence and calculate the current volatility intensity. A gating coefficient is obtained based on the latest volatility intensity, and this coefficient is used to scale the action values ​​output by the policy network during the inference phase. Gradient calculation is not involved in this phase; the gating output only serves as an arithmetic scaling factor in action generation.

[0115] The gating threshold is set based on statistical results of historical exchange rate sequences, using a multiple of the long-term average volatility level as the standard. Thresholds for different markets can be configured independently according to their respective exchange rate characteristics. The gating unit maintains output suppression when continuously detecting high volatility until the exchange rate volatility intensity falls below the set threshold for several consecutive periods, at which point it resumes normal output.

[0116] Throughout the training and inference process, the gating unit does not change the structure and loss form of the reinforcement learning network, but only applies numerical scaling to the action result at the output layer.

[0117] In this implementation, economic constraints are applied throughout the training and inference phases of the reinforcement learning model. These constraints limit the feasible range of the target price and prevent outputs that exceed business and regulatory boundaries. The constraints are established based on three types of parameters: tariff ceiling, minimum list price, and inventory capacity. During model training, these constraints are used as regularization terms in loss calculations, and during inference, they are executed as numerical truncation conditions.

[0118] During the training phase, three types of parameters—tariff ceiling, minimum price, and inventory capacity—are input into the constraint modules of the forward and backward reinforcement learning networks, respectively. The tariff ceiling parameter limits the upper bound of the price output, the minimum price parameter limits the lower bound of the price output, and the inventory capacity parameter limits the allowed price adjustment range. Before the training data enters the loss calculation unit, the price deviation value is calculated based on the above three parameters. If the target price exceeds the upper bound or falls below the lower bound, the difference in value of the deviation is taken as the constraint deviation. All constraint deviations are weighted and summed to form the economic constraint regularization term, which is used together with the strategy loss and value loss to calculate the overall loss function of the model. In this way, the constraint module applies continuous constraints to the network parameters during gradient backpropagation, causing the model to gradually converge to the feasible pricing range during the optimization phase.

[0119] During the inference phase, economic constraints are executed using explicit interval truncation. After the model outputs the target price, it checks whether the price exceeds the upper bound of the allowable range based on the tariff ceiling parameter; if so, it is truncated to the upper bound value. It then checks whether the price is below the lower bound based on the minimum quoted price parameter; if so, it is adjusted to the lower bound value. Finally, it checks whether the price adjustment exceeds the allowable range based on the inventory capacity parameter; if so, it is corrected according to the boundary values. The price value after these three types of constraint processing is the final output, which is then used for subsequent data storage and pricing invocation.

[0120] To maintain consistency in the constraint mechanisms between the training and inference phases, the constraint module uses the same parameter source and value caliber in both phases. Parameters are loaded uniformly before each batch training begins and updated periodically during online runtime to reflect external factors such as policy changes, inventory changes, or tariff adjustments. All parameter updates are recorded in the operation log, ensuring that the pricing output at any given time is traceable to the corresponding constraint state.

[0121] The calculation order of the economic constraint module and the exchange rate sensitive gating unit is fixed as follows: first, gating adjustment is performed, and then interval constraint truncation is performed. This order ensures that the feasible interval is checked after the price action is scaled, thus maintaining the determinism of the operation order in terms of numerical value.

[0122] In this embodiment, the strategy deployment and online inference steps include:

[0123] The optimal strategy parameters, which have converged after bidirectional training, are loaded into the cross-border e-commerce pricing server.

[0124] During operation, it periodically receives real-time exchange rate data, commodity sales data, and inventory status data;

[0125] The real-time data is converted into a state vector and input into the forward and backward networks to perform synchronous inference.

[0126] During the inference output phase, the target price value is calculated by weighted fusion of the price actions output by the two networks.

[0127] The target price value is formatted and then written to the pricing database for front-end access.

[0128] In this embodiment, the transfer reinforcement learning update step includes:

[0129] During the operation phase, cross-market return and risk indicators are monitored. When any indicator of return decline rate, market share change rate, or risk variance exceeds the threshold, a migration update is triggered.

[0130] During the migration and update phase, the policy parameters of the forward network and the backward network are migrated to the target market model as initialization parameters, respectively.

[0131] An incremental training set is constructed based on historical transaction data and the latest state vector of the target market;

[0132] In the incremental training set, the mini-batch stochastic gradient descent algorithm is used to perform synchronous fine-tuning of the parameters of the two networks;

[0133] The fine-tuned model parameters are written into the pricing module on the deployment server.

[0134] Example 1:

[0135] To verify the feasibility of this invention in practice, it was applied to pricing training and inference for three categories of goods on a cross-border e-commerce platform in three regions over an eight-week historical period and a two-week online period. The product categories were electronic accessories, clothing accessories, and small home appliances, and the regions were identified as R1, R2, and R3. The data granularity was daily, the training data spanned the first eight weeks, and the online inference evaluation spanned the following two weeks.

[0136] Data preparation utilized historical sales details from the platform, real-time exchange rate sequences from external financial interfaces, tariff rate tables from public relations data, logistics costs and transit delays from carrier interfaces, market demand intensity from third-party indices, competitive pricing and promotional windows from price comparison interfaces, and click-through rates and conversion rates from event tracking data. All data was recorded with Coordinated Universal Time (UTC) timestamps and aligned to a daily index. Missing data was imputed using linear interpolation for gaps not exceeding three days; gaps exceeding three days resulted in the removal of the corresponding samples. Anomaly detection employed a three-standard-deviation principle to remove outliers. Standardization ensured that each feature had zero mean and one standard deviation within the training set. The standardized features were then aggregated based on product identifier, region code, and date to obtain a standardized cross-border dataset.

[0137] State vector generation is performed in the order of economic embedding, market embedding, and behavioral embedding: Economic embedding performs multi-layer nonlinear mapping and feature crossing on exchange rates, tariff rates, logistics costs, and their daily variation characteristics, outputting an economic feature vector; Market embedding performs discrete embedding and numerical feature fusion on regional identifiers, market demand indices, competitive prices, inventory levels, and sales cycles, and introduces attention weighting to reflect the weight of high-volatility areas on the day, outputting a market feature vector; Behavioral embedding extracts the mean, trend, and volatility intensity from the time window sequences of click-through rate, conversion rate, browsing duration, and repeat visit frequency, uses time-dependent structure to model the correlation between adjacent windows, and outputs a behavioral feature vector. The three types of vectors are concatenated in the fusion layer and nonlinearly mapped. The structure-conserving mapping constraint layer compares the differences between the original feature correlations and the embedded correlations and outputs constraint signals. The constrained latent representation is then normalized and projected to form a fixed-length state vector.

[0138] The bidirectional reinforcement learning structure takes a state vector as input. The feedforward network generates daily pricing actions based on a deterministic policy; the back-end network learns long-term risk costs based on the volatility characteristics of future return sequences. Both networks share a state embedding layer, and the difference in output magnitude is controlled by action consistency constraints. Policy decoupling employs a two-level "range-fine-tuning" structure: the feedforward network outputs the endpoints of a price range, and the fine-tuning network provides continuous fine-tuning within that range, which are then summed to obtain the target price. The composite reward value is a single scalar formed by weighting and summing the daily average profit, the difference in market share between adjacent weeks, and the synthetic risk metric, and subtracting the risk term. The weights are adaptively updated by a genetic algorithm within a rolling validation window. The exchange rate sensitivity gating takes the dispersion of the exchange rate change rate within a fixed-length window as input and outputs a gating coefficient between zero and one, which numerically scales the daily action magnitude; when the dispersion exceeds a threshold, the coefficient approaches zero. Economic constraints participate in the loss function as a regularization term during training and are used as an interval truncation to test the upper and lower bounds and the daily price adjustment magnitude of the output price during inference. Training was conducted using eight weeks of offline data for multiple iterations, with soft updates and experience replay enabled for the target network; inference was run daily for the following two weeks, with gating and constraints continuously in effect.

[0139] Table 1 Summary of key indicators for the two-week online period

[0140] area Product Categories Average daily profit before training (in equivalent currency) Average daily profit after training (in equivalent currency) Weekly market share increase (percentage points) Risk measurement (unitized) Gating trigger rate Constraint violation rate (before training → after training) R1 Electronic accessories 12,480 14,050 +1.8 0.62 0.29 6.4% →1.2% R1 Clothing accessories 8,320 9,110 +1.1 0.58 0.24 5.1% →1.0% R1 Small home appliances 15,940 16,730 +0.9 0.67 0.33 7.0% →1.5% R2 Electronic accessories 9,760 10,420 +1.3 0.49 0.18 4.6% →0.9% R2 Clothing accessories 6,480 6,920 +0.7 0.45 0.16 3.9% →0.8% R2 Small home appliances 11,210 11,980 +1.0 0.52 0.20 5.2% →1.1% R3 Electronic accessories 10,540 11,360 +1.5 0.71 0.41 7.6% →1.8% R3 Clothing accessories 7,020 7,540 +0.9 0.68 0.37 6.8% →1.6% R3 Small home appliances 12,880 13,610 +1.2 0.73 0.44 8.1% →2.0%

[0141] Table 1 lists the main observations in the three regions during the two-week online period and compares them with the pre-training baseline. The pre-training baseline is the historical benchmarking result of the platform's original pricing logic within the same window; the post-training baseline is the statistical result of daily inference for two weeks after the implementation of the method of this invention. The average daily profit is denominated in the same currency, the share increment is the difference in percentage compared to the previous week, the risk measure is the standardized weighted sum of exchange rate and inventory dispersion, the gating trigger rate is the percentage of days with a gating coefficient lower than the set threshold, and the constraint violation rate is the percentage of days exceeding the upper bound, lower bound, or single-day amplitude limit before truncation.

[0142] Across all regions and product combinations, average daily profit was higher than the pre-training baseline during the two-week online period, ranging from approximately 6% to 15% per category. This difference corresponds to a positive change in the revenue component of the composite reward, with the revenue weight relatively increasing after the weight update. Weekly share increment was positive, ranging from 0.7% to 1.8%. The contribution of share to the composite reward was more pronounced in electronic accessories and regions R1 and R3, consistent with the larger fluctuations in competitive prices and demand intensity in these combinations. Risk metrics were generally higher in R3 than in R1 and R2, with a relatively higher gating trigger rate. Under these circumstances, action scaling was more frequent, and the constraint violation rate decreased from approximately 4-8 percentage points before training to approximately 1-2 percentage points under the combined effect of gating and interval truncation. This change is highly consistent with the daily percentage of threshold triggers. Despite high risk metrics, small household appliances in the three regions still maintained positive market share growth, indicating that under the two-tier structure of price range and fine-tuning, the fine-tuning layer effectively transmits the response of the fine-tuning layer to competitive price and inventory signals to the target price within the restricted range. The share and profit weights of the composite reward are adjusted to the appropriate combination by the genetic algorithm within the rolling window. For electronic accessories in R1 and R3, the gating trigger rates are 29 and 41 per thousand, respectively. The difference in risk metrics and the difference in trigger rates are in the same direction, indicating that when the exchange rate window dispersion is high, the action scaling mechanism has a wider impact on the original output. The decrease in the violation rate is mainly concentrated in the two categories of over-limit and lower limit default. The training log shows that within two weeks of online period, the weight vector converges to a stable solution within three to four generations, and the sum of profit, share, and risk weights remains one, satisfying the composition constraint.

[0143] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A cross-border intelligent pricing optimization method based on reinforcement learning, applied to a cognitive direct connection manufacturing platform, characterized in that, The method comprises the following steps: Collecting multi-source data in different regions, and performing time synchronization, outlier elimination and normalization processing to obtain a standardized cross-border data set; Based on the standardized cross-border data set, economic characteristics, market characteristics and behavior characteristics are mapped to a shared latent space through a hierarchical embedding network, a structure-conserving mapping constraint layer is introduced, and a state vector after structure-conserving constraint is output; A bidirectional reinforcement learning structure is constructed based on the state vector, the structure includes a forward reinforcement learning network and a reverse reinforcement learning network, and a target price is generated; Based on the target price, a composite reward value is calculated, and each target weight is dynamically adjusted according to market feedback to output a comprehensive reward signal; A exchange rate sensitive gate unit is introduced in the reinforcement learning model, and the exchange rate volatility is taken as the input to gate adjust the output amplitude of the strategy, and a controlled action is output; Economic constraints are applied in the reinforcement learning training process; The limited optimal strategy after training is deployed to the cross-border e-commerce pricing module, and the updated state vector is input to perform online inference in real time; Based on the cross-market performance, the transfer reinforcement learning is updated, and when the set conditions are detected, the strategy parameters are transferred to the new market model and incrementally fine-tuned.

2. The method of claim 1, wherein, The multi-source data includes: The historical sales data of cross-border e-commerce platforms, real-time exchange rate data of different currencies, tariff rate parameters of each country or region, logistics transportation cost and time delay data, market demand index data, competitor product price and promotion activity data, and user click rate and conversion rate data, all types of data have a unified timestamp. 3.The method of claim 1, wherein, The generation steps of the standardized cross-border data set include: The collected multi-source data is linearly interpolated and time window aligned according to the timestamp; Anomaly data samples are identified and eliminated using a statistical detection method based on the standard deviation three times rule; The remaining data is subjected to Z-score standardization processing to make the mean of each feature distribution zero and the standard deviation one; The processed data is multi-dimensionally aggregated according to product identification, market area and time index to form a structured standardized cross-border data set. 4.The method of claim 1, wherein, The generation steps of the state vector include: The standardized cross-border data set is input into the economic embedding layer, the market embedding layer and the behavior embedding layer respectively; the economic embedding layer performs numerical coding on the exchange rate, tariff rate and logistics cost characteristics and extracts the economic feature vector through a fully connected neural network; The market embedding layer performs embedding transformation on the market demand index, competitive price and inventory level characteristics to generate a market feature vector; The behavior embedding layer performs time series modeling on the user click rate and conversion rate characteristics to generate a behavior feature vector; The three types of feature vectors are spliced through a fusion layer and input into a structure-conserving mapping constraint layer, which calculates the difference between the original feature correlation matrix and the embedded feature correlation matrix and takes the Frobenius norm as the constraint loss; The fusion latent space representation output by the constraint layer is subjected to nonlinear mapping transformation and normalization projection to generate a fixed-length state vector. 5.The method of claim 1, wherein, The construction steps of the bidirectional reinforcement learning structure include: inputting the state vector into a forward reinforcement learning network and a backward reinforcement learning network; the forward reinforcement learning network generates an optimal pricing action for the current market using an algorithm based on deterministic policy gradient; the backward reinforcement learning network estimates long-term risk cost based on future market states; the two networks share a state embedding layer and jointly update parameters during training with an action consistency constraint term; in the policy decoupling stage, the price interval action output by the forward network sets the action value range of the lower network through the decoupling layer, and the lower network outputs a price fine-tuning action within the range and superimposes it with the forward action to generate the target price of the corresponding product; through the experience replay mechanism and the time discount factor, weighted learning is performed on historical samples to realize the collaborative optimization of the bidirectional policy. 6.The method of claim 1, wherein, the calculation step of the composite reward value comprises: calculating a profit yield indicator according to the target price and the actual sales result; calculating a market share growth indicator according to the product sales and the total market sales; calculating a risk penalty term according to the exchange rate fluctuation rate and the inventory change rate; multiplying the profit yield indicator, the market share growth indicator and the risk penalty term by dynamic weight parameters respectively and linearly combining them into a composite reward value; the dynamic weight parameters are updated adaptively in multiple rounds of training iterations through a genetic algorithm, and the obtained composite reward value is fed back to the forward network and the backward network as a joint optimization signal to update the profit policy and the risk policy parameters simultaneously.

7. The method of claim 1, wherein, the implementation steps of the exchange rate sensitive gate unit comprise: extracting the exchange rate fluctuation rate component in the state vector as a gate input signal; calculating the standard deviation of the exchange rate change rate sequence through a sliding time window to represent the current exchange rate fluctuation intensity; inputting the exchange rate fluctuation intensity into a gate function, which outputs a value close to zero when the exchange rate fluctuation rate is higher than a preset threshold; the gate output simultaneously acts on the policy output of the forward reinforcement learning network and the backward reinforcement learning network to suppress the price adjustment amplitude and generate controlled price actions with limited amplitude; the gate output participates in action adjustment in both model training and online inference stages. 8.The method of claim 1, wherein, the implementation steps of the economic constraint condition comprise: in the reinforcement learning training stage, inputting the tariff upper limit parameter, the minimum price parameter and the inventory capacity parameter into the constraint module of the forward network and the backward network; limiting the upper bound of the target price value according to the tariff upper limit parameter, limiting the lower bound of the target price value according to the minimum price parameter, and limiting the price adjustment range according to the inventory capacity parameter; adding economic constraint regularization terms and structure conservation constraint terms to the loss function respectively, and calculating a penalty value when the target price deviates from the constraint interval; in the online inference stage, performing interval truncation operation on the prediction output to make the pricing output generated by the bidirectional network conform to the feasible interval defined by the constraint parameters. 9.The method of claim 1, wherein, the policy deployment and online inference steps comprise: loading the optimal policy parameters converged after bidirectional training into the cross-border e-commerce pricing server; periodically receiving real-time exchange rate data, commodity sales data and inventory state data during operation; converting the real-time data into a state vector and inputting it into the forward and backward networks for synchronous inference; In the inference output stage, the price actions output by the two networks are weighted and fused to calculate a target price value; After performing format processing on the target price value, it is written into a pricing database for front-end calling. 10.The method of claim 1, wherein, The migration reinforcement learning update step includes: In the running stage, the cross-market return and risk indicators are monitored. When any of the indicators such as the return decline rate, the market share change rate, or the risk variance exceeds the threshold, the migration update is triggered; In the migration update stage, the forward network and the backward network strategy parameters are migrated to the target market model as initialization parameters respectively; According to the historical transaction data and the latest state vector of the target market, an incremental training set is constructed; In the incremental training set, a small batch random gradient descent algorithm is used to perform synchronous fine-tuning on the parameters of the two networks; The model parameters after fine-tuning are written into the pricing module in the deployment server.

Images