A precious metal trend prediction method and system based on a transformer architecture pre-training model

Abstract

The application discloses a precious metal trend prediction method and system based on a Transformer architecture pre-training model, and the method comprises the following steps: constructing a Transformer pre-training model based on foreign exchange market historical data to generate a base model with generalization capability; performing multi-dimensional feature extraction and standardization processing on precious metal market historical data to construct a structured training sample set containing price sequences, technical indicators and market sentiment factors; using the structured training sample set to fine-tune the base model, and using transfer learning and dynamic learning rate adjustment to adapt the model to the characteristics of the precious metal market to generate a special prediction model; inputting real-time precious metal market data into the special prediction model to output price prediction sequences and volatility estimates at multiple time steps in the future for transaction strategy optimization. By using the embodiment of the application, the accuracy and generalization capability of precious metal trend prediction can be improved, and the problem of model overfitting in a small sample scenario can be solved.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, and in particular to a method and system for predicting the trend of precious metals based on a pre-trained model of the Transformer architecture. Background Technology

[0002] As an important safe-haven asset and investment target, precious metals are influenced by a complex interplay of factors, including macroeconomics, geopolitics, and market sentiment, exhibiting highly nonlinear, non-stationary, and long-term dependent characteristics. Traditional forecasting methods, such as time series analysis (ARIMA) and machine learning models (SVM, random forest), typically rely on manual feature engineering, making it difficult to fully capture the deep interaction patterns and long-term dependencies within massive amounts of market data. In recent years, deep learning technology, especially the Transformer architecture, has demonstrated strong potential in time series forecasting. However, directly applying it to the precious metals market faces two major challenges: first, the scarcity of specialized labeled data for precious metals makes it difficult to train high-generalization deep models from scratch; second, the rich time series patterns inherent in general financial models (such as foreign exchange forecasting models) have not been effectively transferred to the precious metals scenario, resulting in a waste of data resources and knowledge representation. Summary of the Invention

[0003] The purpose of this invention is to provide a method and system for predicting precious metal price trends based on a pre-trained model using the Transformer architecture, in order to address the shortcomings of existing technologies, improve the accuracy and generalization ability of precious metal price trend prediction, and solve the problem of model overfitting in small sample scenarios.

[0004] One embodiment of this application provides a method for predicting precious metal price trends based on a Transformer architecture pre-trained model, the method comprising: A Transformer pre-trained model is built based on historical foreign exchange market data. Through self-supervised learning and multi-task training, the model learns general financial time series patterns and generates a base model with generalization ability. The historical foreign exchange market data includes at least foreign exchange prices, trading volume, and market sentiment indicators. Multi-dimensional feature extraction and standardization processing are performed on historical data of the precious metals market to construct a structured training sample set containing price series, technical indicators and market sentiment factors. The historical data of the precious metals market includes at least the price, trading volume and market sentiment indicators of precious metal varieties. The base model is fine-tuned using the structured training sample set, and the model is adapted to the characteristics of the precious metals market through transfer learning and dynamic learning rate adjustment to generate a dedicated prediction model. Real-time precious metals market data is input into the dedicated prediction model, which outputs a price prediction sequence and volatility estimate for multiple future time steps, for use in trading strategy optimization.

[0005] Optionally, the Transformer pre-trained model is constructed based on historical foreign exchange market data. Through self-supervised learning and multi-task training, the model learns general financial time series patterns, generating a base model with generalization capabilities. The historical foreign exchange market data includes at least foreign exchange prices, trading volumes, and market sentiment indicators, including: Historical data of the foreign exchange market were collected from multiple international financial data sources, including price series, trading volume series and market sentiment indicators of major currency exchanges, to generate the original foreign exchange multimodal dataset. The original foreign exchange multimodal dataset is cleaned and aligned, outliers and missing values ​​are removed, and data of different frequencies are uniformly interpolated to the same timestamp to generate a cleaned and aligned foreign exchange time series dataset. Based on the cleaned and aligned foreign exchange time series dataset, a self-supervised learning task is designed. The time series masking reconstruction method is used to randomly mask some time steps of the data, forcing the model to learn contextual dependencies to predict the masked values, and generating a self-supervised training task set. A multi-task training framework is constructed. Based on the self-supervised reconstruction task, two auxiliary tasks, foreign exchange price direction prediction and volatility estimation, are added. An adaptive weight algorithm is used to balance the losses of different tasks. The Transformer model is trained until convergence, and finally a financial time series base model with generalization ability is generated.

[0006] Optionally, the step of performing multi-dimensional feature extraction and standardization on historical precious metals market data to construct a structured training sample set containing price series, technical indicators, and market sentiment factors, wherein the historical precious metals market data includes at least the price, trading volume, and market sentiment indicators of precious metal varieties, including: Historical data from the precious metals market was collected, including London gold and silver fixing prices, minute-level candlestick data for futures, ETF trading volume, and news sentiment analysis scores and social media sentiment indices extracted from financial news and social media, to generate a raw multi-source dataset of precious metals. Multi-dimensional features are extracted from the original precious metal multi-source dataset. Technical indicators, including at least moving averages, relative strength index, and Bollinger Bands, are calculated based on price and volume sequences. Sentiment dictionary features and topic distribution features are extracted from text data to generate the original feature pool. All features in the original feature pool are standardized and stabilized. The Z-score method is used to remove the dimensions, and the price series is log-differenced to meet the stationarity requirement, generating a standardized feature matrix. The standardized feature matrix is ​​organized into continuous time window samples in chronological order. Each sample contains features from the past N time steps as input and the real prices from the next M time steps as the prediction target. A structured training sample set is constructed for model fine-tuning.

[0007] Optionally, the step of using the structured training sample set to fine-tune the base model, adapting the model to the characteristics of the precious metals market through transfer learning and dynamic learning rate adjustment, and generating a dedicated prediction model includes: Load a financial time series base model with generalization capabilities, freeze some parameters of its underlying Transformer encoder, retain its learned general time series representation capabilities, and generate a fine-tuned initial model with partially frozen parameters. The structured training sample set is input into the initial model for fine-tuning. A transfer learning strategy is adopted to add a fully connected layer adapted for precious metal prediction at the top layer of the model. Only the newly added layer and some high-level attention layers are trained to generate the initial fine-tuned model. A dynamic learning rate adjustment strategy is implemented. Based on the loss changes of the model on the precious metal validation set, the cosine annealing algorithm is used to dynamically adjust the learning rate of different network layers, so as to promote the efficient convergence of the initial fine-tuned model to the precious metal data distribution and generate the optimized fine-tuned model. The prediction accuracy and generalization ability of the optimized fine-tuned model are evaluated on an independent precious metals test set. The model parameters are further fine-tuned through backpropagation algorithm to minimize the prediction error, and finally a special prediction model for the precious metals market is generated.

[0008] Optionally, the step of inputting real-time precious metals market data into the dedicated prediction model and outputting a price prediction sequence and volatility estimate for multiple future time steps for trading strategy optimization includes: It collects the latest precious metals market data in real time, including real-time quotes, trading volume, and market news flow, and generates real-time standardized feature vectors. The real-time standardized feature vectors are input into a dedicated prediction model. The model captures long-term dependencies through its Transformer encoder and performs multi-step forward inference via a fine-tuned prediction head to generate a price point prediction sequence for multiple future time steps. Based on the internal attention weight matrix and hidden layer state of the dedicated prediction model, Monte Carlo Dropout technique is used to perform multiple forward inferences, calculate the standard deviation of the prediction results to estimate the volatility of future prices, and generate a volatility estimation sequence. Integrating future price point prediction sequences and volatility estimation sequences across multiple time steps, the system outputs a structured forecast report containing predicted values, confidence intervals, and risk measures to optimize trading strategy parameters and risk control logic.

[0009] Another embodiment of this application provides a precious metal price trend prediction system based on a Transformer architecture pre-trained model, the system comprising: The module is used to build a Transformer pre-trained model based on historical foreign exchange market data. Through self-supervised learning and multi-task training, the model learns general financial time series patterns and generates a base model with generalization ability. The historical foreign exchange market data includes at least foreign exchange prices, trading volume, and market sentiment indicators. The extraction module is used to extract and standardize multi-dimensional features from historical data of the precious metals market, and to construct a structured training sample set containing price series, technical indicators and market sentiment factors. The historical data of the precious metals market includes at least the price, trading volume and market sentiment indicators of precious metal varieties. The adjustment module is used to fine-tune the base model using the structured training sample set, and adapt the model to the characteristics of the precious metals market through transfer learning and dynamic learning rate adjustment to generate a dedicated prediction model. The output module is used to input real-time precious metals market data into the dedicated prediction model and output a price prediction sequence and volatility estimate for future multiple time steps for trading strategy optimization.

[0010] Another embodiment of this application provides a storage medium storing a computer program, wherein the computer program is configured to execute the method described in any of the preceding claims when running.

[0011] Another embodiment of this application provides an electronic device including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the method described in any of the preceding claims.

[0012] Compared with existing technologies, the present invention provides a method for predicting precious metal trends based on a pre-trained model of the Transformer architecture, which can improve the accuracy and generalization ability of precious metal trend prediction and solve the problem of model overfitting in small sample scenarios. Attached Figure Description

[0013] Figure 1 A hardware structure block diagram of a computer terminal for a precious metal trend prediction method based on a Transformer architecture pre-trained model, provided in an embodiment of the present invention. Figure 2 A flowchart illustrating a method for predicting precious metal price trends based on a Transformer architecture pre-trained model, provided in an embodiment of the present invention. Figure 3This is a schematic diagram of the structure of a precious metal trend prediction system based on a Transformer architecture pre-trained model, provided in an embodiment of the present invention. Figure 4 This is a schematic diagram of a Transformer model architecture provided in an embodiment of the present invention. Detailed Implementation

[0014] The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0015] This invention first provides a method for predicting the price trend of precious metals based on a pre-trained model of the Transformer architecture. This method can be applied to electronic devices, such as computer terminals, specifically ordinary computers.

[0016] The following detailed explanation uses a computer terminal as an example. Figure 1 This is a hardware structure block diagram of a computer terminal for a precious metal price trend prediction method based on a Transformer architecture pre-trained model, provided as an embodiment of the present invention. Figure 1 As shown, the computer device includes a processor, memory, and network interface connected via a system bus, wherein the memory may include non-volatile storage media and internal memory.

[0017] See Figure 2 and Figure 4 The present invention provides a method for predicting precious metal price trends based on a Transformer architecture pre-trained model, which may include the following steps: S201, Construct a Transformer pre-trained model based on historical foreign exchange market data, and enable the model to master general financial time series patterns through self-supervised learning and multi-task training, generating a base model with generalization ability. The historical foreign exchange market data includes at least foreign exchange prices, trading volume, and market sentiment indicators. Specifically, historical data of the foreign exchange market can be collected from multiple international financial data sources, including price series, trading volume series and market sentiment indicators of major currency exchanges, to generate a raw foreign exchange multimodal dataset. The core of this step is to collect comprehensive and authentic multimodal historical foreign exchange data to provide sufficient data support for model pre-training. The specific implementation method is as follows: The data sources collected cover multiple international financial data channels, focusing on acquiring relevant data for major global currency pairs to ensure data diversity and representativeness, and avoid bias caused by a single data source. Major currency pairs include six core pairs: EUR / USD, GBP / USD, USD / JPY, and USD / CHF. These pairs have huge trading volumes, high market liquidity, and their price fluctuations exhibit typical financial time-series characteristics, effectively helping the model learn general financial patterns.

[0018] The collected data types are divided into three categories, corresponding to price series, trading volume series, and market sentiment indicators, forming a multimodal dataset. The price series includes the opening price, closing price, highest price, and lowest price at each time step, fully reflecting the price fluctuations of the currency pair; the trading volume series records the total trading volume at each time step, reflecting market activity; market sentiment indicators include the fear and greed index, the bullish / bearish ratio of the currency pair, and the institutional holding ratio, used to capture the sentiment tendencies of market participants and help the model understand the driving factors of price fluctuations.

[0019] The data collection period was set at the past 5 years, with time granularity divided into daily and hourly levels. Daily data was used to capture long-term market trends, while hourly data was used to capture short-term fluctuation patterns. Approximately 500,000 time-series data records were collected. During the collection process, to ensure data authenticity, data from each data source was cross-validated. If the deviation of data from different channels at the same time step exceeded 0.001 (exchange rate units), the abnormal data was removed, and data with high consistency was retained. In the example, the collected hourly data for EUR / USD showed an opening price of 1.0920, a closing price of 1.0935, a high of 1.0940, a low of 1.0915, a trading volume of 1200 lots, a fear and greed index of 65 (leaning towards greed), and a bullish / bearish ratio of 1.2. After integration, a single multimodal data record was formed. All records were then aggregated to generate the original forex multimodal dataset, labeled with the data collection timestamp and currency pair identifier to ensure traceability.

[0020] The original foreign exchange multimodal dataset is cleaned and aligned, outliers and missing values ​​are removed, and data of different frequencies are uniformly interpolated to the same timestamp to generate a cleaned and aligned foreign exchange time series dataset. The core of this step is to improve data quality, eliminate data noise and temporal bias, and ensure that the data meets the requirements for model training. The specific implementation method is as follows: First, outlier removal is performed. Outliers mainly include data that does not conform to market patterns, such as sudden price changes, abnormal surges or drops in trading volume. The 3σ principle is used for identification and removal. The 3σ principle is an outlier detection method based on the normal distribution of data. Its core is that when a data value exceeds the mean ± 3 times the standard deviation, it is considered an outlier. The specific calculation process is as follows: first, calculate the mean μ and standard deviation σ for each currency pair and each data type (such as closing price and trading volume), and set the outlier threshold to [μ-3σ, μ+3σ]. Data exceeding this range is considered outliers and removed. In the example, the mean μ of the USD / JPY closing price is 138.5, the standard deviation σ is 2.1, and the outlier threshold is [138.5-6.3, 138.5+6.3] = [132.2, 144.8]. A certain data point has a closing price of 145.2, which exceeds the threshold, is identified as an outlier, and is removed to ensure that the data conforms to the normal fluctuation patterns of the market.

[0021] Next, missing value handling is performed. Missing values ​​mainly originate from data source interruptions or collection failures. Linear interpolation is used to fill in the missing values. The core of linear interpolation is to calculate the missing value based on the two valid data points before and after the missing value through linear fitting, ensuring that the supplemented data is continuous and closely matches the original fluctuation trend. In the example, the hourly closing price data for the Euro / USD exchange rate is missing. The previous valid data point is 1.0930 (timestamp t-1), and the next valid data point is 1.0940 (timestamp t+1). The interpolated closing price at time t is calculated as (1.0930 + 1.0940) / 2 = 1.0935, which is then added to the dataset. If more than three consecutive time steps are missing, the consecutive missing data is removed to avoid excessive interpolation deviation.

[0022] Finally, data frequency alignment is performed. The original data contains both daily and hourly frequencies, which need to be uniformly interpolated to hourly timestamps to ensure consistent time granularity across all data, facilitating the model's capture of temporal dependencies. When interpolating daily data to hourly data, a uniform interpolation method is used, evenly distributing daily closing prices, trading volumes, and other data across the 24-hour timestamps of that day, while preserving the trend characteristics of the daily data. Hourly data remains unchanged, with only the timestamp format aligned. In the example, the daily closing price for a currency pair is 1.0950. When interpolating to the 24-hour timestamps of that day, the closing price for each time step is set to 1.0950, and the trading volume is evenly distributed across each hourly time step based on the total daily trading volume. After all processing is complete, the data is integrated to form a cleaned and aligned forex time-series dataset. The data is sequential, without anomalies or missing data, and can be directly used for subsequent self-supervised task design.

[0023] Based on the cleaned and aligned foreign exchange time series dataset, a self-supervised learning task is designed. The time series masking reconstruction method is used to randomly mask some time steps of the data, forcing the model to learn contextual dependencies to predict the masked values, and generating a self-supervised training task set. The core of this step is to design a reasonable self-supervised task, allowing the model to autonomously learn the contextual dependencies of financial time series data under unlabeled conditions. The specific implementation method is as follows: The time series masking reconstruction method is the core of self-supervised learning. Its core principle is to randomly mask the feature values ​​of some time steps in the dataset, use the masked sequence as the model input, let the model predict the masked feature values, optimize the model parameters through the prediction error, and force the model to learn the inherent rules and contextual relationships of the time series data. No manual labeling is required, which reduces the training cost.

[0024] There are two masking strategies: random point masking and continuous segment masking. These two strategies are used in combination to ensure that the model can learn both local context and capture long-term temporal dependencies. Random point masking randomly selects 15% of a single time step for masking. The masking method involves setting all feature values ​​(price, volume, market sentiment indicators) of that time step to 0 and adding a masking marker to inform the model that the position is a masked value. Continuous segment masking randomly selects 5% of a continuous time period (each time period contains 3-5 time steps) for masking. Similarly, all feature values ​​of that segment are set to 0 and a masking marker is added to simulate a scenario where short-term market data is missing.

[0025] The prediction task after masking is set as follows: the model predicts all feature values ​​of the masked time steps based on the unmasked context data. The prediction accuracy is measured by the mean squared error loss function to ensure that the model can accurately capture the fluctuation patterns of the time series data. In the example, a EUR / USD time series data with 20 time steps is selected. Random point masking is used to mask 3 individual time steps, and continuous segment masking is used to mask 1 continuous segment containing 4 time steps, for a total of 7 time steps masked. The 13 effective time steps after masking are used as input, and the model predicts the closing price, trading volume, and market sentiment indicators for the 7 masked time steps. For example, the predicted closing price for a certain masked time step is 1.0932, and the actual value is 1.0935. The model is optimized by calculating the mean squared error between the two.

[0026] All cleaned and aligned foreign exchange time series datasets were processed according to the masking strategy and prediction task described above. Each time series data segment generated a corresponding self-supervised training sample. The sample contained the input sequence (masked data), the masking location identifier, and the target sequence (masked real value). After summarizing all samples, a self-supervised training task set was generated. The task set was divided into a self-supervised training set and a self-supervised validation set in an 8:2 ratio for subsequent model training and validation, ensuring that the model can fully learn general financial time series patterns.

[0027] A multi-task training framework is constructed. Based on the self-supervised reconstruction task, two auxiliary tasks, foreign exchange price direction prediction and volatility estimation, are added. An adaptive weight algorithm is used to balance the losses of different tasks. The Transformer model is trained until convergence, and finally a financial time series base model with generalization ability is generated.

[0028] The core of this step is to improve the model's generalization ability through multi-task training, enabling the model to simultaneously master multiple financial time series prediction capabilities. The specific implementation method is as follows: The multi-task training framework centers on a self-supervised masking reconstruction task, with two additional auxiliary tasks and three tasks working together to train the model. This allows the model to learn financial time-series features from different perspectives, enhancing its generalization ability. Specifically, the self-supervised masking reconstruction task trains the model to capture temporal contextual dependencies, the foreign exchange price direction prediction task trains the model to determine price trends, and the volatility estimation task trains the model to predict price fluctuations. The combination of these three tasks comprehensively covers the core features of financial time series data.

[0029] The specific setup for the foreign exchange price direction prediction task is as follows: based on data from N time steps prior to a given time step, predict the price direction at that time step. If the closing price is higher than the previous time step's closing price, it is marked as an increase (label 1); if it is lower, it is marked as a decrease (label 0). The cross-entropy loss function is used to measure the prediction accuracy. In the example, based on the EUR / USD data from the previous 10 time steps, the model predicts the price direction at the 11th time step. The model predicts an increase (label 1), and the actual closing price is higher than the previous time step, so the prediction is correct. The cross-entropy loss function optimizes the model's ability to judge trends.

[0030] The volatility estimation task is specifically designed to predict the price volatility at a given time step, based on data from N time steps prior to that step. Volatility is calculated using a rolling window standard deviation, with a window size of 5 time steps. The volatility is defined as σ = √[Σ(x_i-μ)^2 / 5], where x_i is the closing price within the window, and μ is the average closing price within the window. The mean squared error loss function is used to measure prediction accuracy. In the example, based on data from the previous 10 time steps, the model predicts the volatility for the 11th time step. The predicted value is 0.0012, while the actual calculated value is 0.0011. The mean squared error loss function optimizes the model's ability to predict volatility amplitude.

[0031] An adaptive weighting algorithm is employed to balance the losses of the three tasks, preventing the loss of any one task from dominating model training. The core of this algorithm is to dynamically adjust the weights based on the loss of each task on the validation set; the greater the loss, the larger the weight, ensuring the model achieves good training results on each task. The weight update cycle is set to once every 100 iterations, and the total weight sum is always 1. In the example, in a certain iteration, the loss for the masking reconstruction task is 0.02, the loss for the price direction prediction task is 0.05, and the loss for the volatility estimation task is 0.03. The calculated weights for these three tasks are 0.2, 0.5, and 0.3, respectively. The total loss is obtained by weighted summation and used for updating model parameters.

[0032] The Transformer model's training parameters are set as follows: 6 encoder layers, 8 attention heads, 128 hidden layers, 300 training iterations, and a batch size of 64. The model is considered converged when the total loss on the validation set no longer decreases after 50 consecutive iterations, and the decrease is less than 1e-5. After convergence, all model parameters are saved, generating a generalized financial time-series foundation model. This model can grasp common financial time-series patterns and can be used for transfer learning and fine-tuning in the precious metals market.

[0033] S202, Perform multi-dimensional feature extraction and standardization processing on historical data of the precious metals market, and construct a structured training sample set containing price series, technical indicators and market sentiment factors. The historical data of the precious metals market includes at least the price, trading volume and market sentiment indicators of precious metal varieties. Specifically, historical data from the precious metals market can be collected, including London gold and silver fixing prices, minute-level candlestick data for futures, ETF trading volume, and news sentiment analysis scores and social media sentiment indices extracted from financial news and social media, to generate a raw multi-source dataset of precious metals. The core of this step is to collect comprehensive, multi-dimensional historical data on the precious metals market, covering three core dimensions: price, trading volume, and market sentiment. This provides sufficient and targeted raw material for subsequent feature extraction. The specific implementation method is as follows: The data sources collected are both authoritative and diverse, ensuring the authenticity, completeness, and representativeness of the data. These primarily include data interfaces from international precious metals trading platforms, mainstream global financial data service channels, authoritative financial media news databases, and mainstream social media data interfaces. During the collection process, data from each data source is cross-validated to avoid data bias or omissions from a single source. The precious metals collected focus on the two core commodities with the highest market attention and trading volume—gold and silver. Gold data primarily uses the London Gold Fixing price and gold futures data, while silver data primarily uses the London Silver Fixing price and silver futures data. This is supplemented with corresponding ETF trading volume data to comprehensively reflect the trading activity and capital flows in the precious metals market.

[0034] The collected data types are divided into four main categories, forming a complete original multi-source dataset of precious metals. Each category has clearly defined collection standards and physical meaning. The first category is fixing price data, including the morning and afternoon fixing prices for London gold and London silver. The collection timeframe is set at the past 6 years, with a daily granularity. The fixing price is published daily at fixed times by international precious metals pricing agencies and is a core indicator reflecting the fair price of precious metals. The unit is uniformly US dollars per ounce (gold) and US dollars per ounce (silver), rounded to two decimal places. The second category is futures minute-level candlestick data, collecting 5-minute candlestick data for gold and silver futures, covering major global trading sessions (London session, New York session, and Asian session). The collection timeframe is the past 4 years. Each candlestick data includes five core indicators: opening price, closing price, highest price, lowest price, and trading volume. The unit is consistent with the fixing price, and the trading volume unit is lots (1 lot of gold futures = 100 ounces, 1 lot of silver futures = 5000 ounces). The third category is ETF trading volume data, which collects daily trading volume data from the world's largest gold and silver ETFs. The time frame is consistent with the fixing price, and the unit is shares. This data reflects the investment sentiment of ordinary investors and institutions towards the precious metals market. The fourth category is market sentiment indicators, including news sentiment analysis scores and social media sentiment indices. The news sentiment analysis scores are extracted from authoritative financial media news databases, collecting no fewer than 50 financial news items related to precious metals daily. A sentiment analysis algorithm is used to calculate the sentiment score for each news item (range 0-100, where 0 represents extremely negative and 100 represents extremely positive). The average of all news scores for the day is taken as the daily news sentiment analysis score. The social media sentiment index is extracted from mainstream social media platforms, collecting topics and comments related to precious metals investment. A text mining algorithm is used to calculate the sentiment index (range -50 to 50, where -50 represents extremely bearish and 50 represents extremely bullish), with a daily time granularity, aligned with the fixing price and ETF trading volume data.

[0035] During the data collection process, deviation thresholds were set for different types of data to verify consistency. The deviation threshold for fixing price data was set at $0.5 / ounce, the deviation threshold for futures candlestick data was set at $0.3 / ounce, and the deviation threshold for trading volume data was set at 8%. If the deviation of data from different channels at the same time step exceeded the corresponding threshold, the data was removed, and data with high consistency was retained. In the example, the afternoon fixing price of London gold on a certain trading day was $1925.30 / ounce. The opening price of gold futures on a 5-minute candlestick chart at a certain time step was $1924.80, the closing price was $1925.50, the highest price was $1926.20, the lowest price was $1924.30, the trading volume was 2300 lots, the trading volume of gold ETF on that day was 5 million shares, the news sentiment analysis score was 72 (leaning positive), and the social media sentiment index was 35 (leaning bullish). After integration, a single multi-source data record was formed, labeled with the data collection timestamp, precious metal variety identifier, and data source. After all records were summarized, the original precious metal multi-source dataset was generated, with a data volume of approximately 850,000 records, ensuring that the data is traceable and verifiable.

[0036] Multi-dimensional features are extracted from the original precious metal multi-source dataset. Technical indicators, including at least moving averages, relative strength index, and Bollinger Bands, are calculated based on price and volume sequences. Sentiment dictionary features and topic distribution features are extracted from text data to generate the original feature pool. The core of this step is to extract multi-dimensional features that reflect the trends of the precious metals market from multi-source raw data. These features cover two core categories: technical and sentiment aspects, constructing a comprehensive raw feature pool to provide effective feature support for subsequent model fine-tuning. The specific implementation method is as follows: Feature extraction is divided into two main modules: technical indicator feature extraction and text sentiment feature extraction. These two modules work together to ensure that the extracted features can comprehensively capture the price fluctuation patterns and market sentiment changes in the precious metals market, avoiding model prediction bias caused by a single type of feature.

[0037] Technical indicator feature extraction is based on the price series (fixing price, futures candlestick opening price, closing price, highest price, lowest price) and trading volume series in the original data. Three types of core technical indicators are calculated, each with a clear calculation logic and physical meaning, closely reflecting the volatility characteristics of the precious metals market. The first type is the moving average (MA), used to capture long-term price trends. Three commonly used periods are selected: MA5 (5-day moving average), MA10 (10-day moving average), and MA20 (20-day moving average). The calculation formula is MA_n=(Σx_i) / n, where x_i is the closing price over the past n time steps, and n is the moving average period. The larger the MA value, the stronger the long-term price trend. In the example, to calculate the MA5 for a certain trading day in London gold, the closing prices of the previous 5 trading days were selected as 1920.10, 1922.30, 1924.50, 1923.70, and 1925.30. Substituting these prices into the formula, we get MA5 = (1920.10 + 1922.30 + 1924.50 + 1923.70 + 1925.30) / 5 = 1923.18 USD / ounce, reflecting a slight upward trend in gold prices over the past 5 trading days. The second type is the Relative Strength Index (RSI), used to determine the overbought or oversold state of the precious metals market. A 14-day period is selected, and the calculation formula is RSI = 100 - [100 / (1+RS)], where RS is the ratio of the average increase over the past 14 trading days to the average decrease over the past 14 trading days. The RSI value ranges from 0 to 100. Generally, RSI > 70 indicates an overbought market, and RSI < 30 indicates an oversold market. In the example, calculating the 14-day RSI for gold futures, there were 8 trading days of gains with an average increase of $1.2 per ounce, and 6 trading days of losses with an average decrease of $0.8 per ounce. RS = 1.2 / 0.8 = 1.5. Substituting into the formula, we get RSI = 100 - [100 / (1+1.5)] = 60, indicating that the market is in a normal state and there is no overbought or oversold condition. The third type is Bollinger Bands (BOLL), used to reflect the price fluctuation range and trend strength. It consists of three lines: upper band, middle band, and lower band. The middle band is the 20-day moving average (MA20). The upper band = MA20 + 2 × σ (where σ is the standard deviation of the closing prices over the past 20 time steps), and the lower band = MA20 - 2 × σ. The wider the gap between the upper and lower bands, the more volatile the price fluctuations. In the example, the MA20 of gold futures at a certain time step is $1923.50 / ounce, and the standard deviation of the closing prices over the past 20 time steps, σ, is $2.3 / ounce. The calculated upper band = $1923.50 + 2 × 2.3 = $1928.10 / ounce, and the lower band = $1923.50 - 2 × 2.3 = $1918.90 / ounce. The Bollinger Band width is $9.2 / ounce, reflecting that the current price fluctuation is moderate.

[0038] Text sentiment feature extraction is based on news sentiment analysis scores and social media sentiment indices from the original data. Two core features are extracted to quantify the impact of market sentiment on precious metal prices. The first type is the sentiment lexicon feature, constructing a sentiment lexicon specifically for precious metals, including positive words (such as "rise," "positive," "increase in holdings"), negative words (such as "fall," "negative," "reduction in holdings"), and neutral words. The proportion of positive and negative words in daily financial news texts is statistically analyzed. The calculation formula is: positive word proportion = number of positive words / total number of words; negative word proportion = number of negative words / total number of words, with values ​​ranging from 0 to 1. A higher proportion indicates a stronger corresponding sentiment. In the example, among 50 gold-related news articles collected on a certain trading day, 28 contained positive words and 12 contained negative words. With a total of 8000 words, there were 320 positive words and 160 negative words. The calculated positive word proportion = 320 / 8000 = 0.04, and the negative word proportion = 160 / 8000 = 0.02, indicating that the news sentiment on that day was predominantly positive. The second category is thematic distribution characteristics. Thematic modeling algorithms are used to analyze financial news and social media texts, extracting five core themes (geopolitics, economic policy, inflation expectations, dollar trend, and market fund flows). The distribution percentage of each theme in the daily text data is calculated, ranging from 0 to 1. A higher percentage indicates a greater impact of the theme on the market. In the example, in precious metals-related texts on a certain trading day, the geopolitical theme accounted for 0.35%, the economic policy theme for 0.25%, the inflation expectations theme for 0.20%, the dollar trend theme for 0.15%, and the market fund flows theme for 0.05%, indicating that geopolitical factors had the greatest impact on the precious metals market that day.

[0039] All extracted technical indicator features (MA5, MA10, MA20, RSI, Bollinger Band upper band, Bollinger Band middle band, Bollinger Band lower band) and text sentiment features (news sentiment analysis score, social media sentiment index, percentage of positive words, percentage of negative words, and percentage of distribution of 5 themes) are summarized to generate an original feature pool containing 15 core features. Each feature is labeled with its corresponding calculation logic and physical meaning to ensure the interpretability and usability of the features.

[0040] All features in the original feature pool are standardized and stabilized. The Z-score method is used to remove the dimensions, and the price series is log-differenced to meet the stationarity requirement, generating a standardized feature matrix. The core of this step is to eliminate the dimensional differences between features, ensuring that all features are on the same order of magnitude. It also addresses the non-stationarity of the price series, meeting the input data requirements of the Transformer model and improving model training efficiency and prediction accuracy. The specific implementation method is as follows: First, standardization is performed using the Z-score standardization method. This removes dimensional differences among all features in the original feature pool, ensuring that each feature has a mean of 0 and a standard deviation of 1. This guarantees that features of different types and magnitudes are treated equally by the model, preventing features with larger magnitudes from dominating model training. The Z-score standardization formula is x_norm=(x-μ) / σ, where x is the original feature value, μ is the mean of all samples for that feature, σ is the standard deviation of all samples for that feature, and x_norm is the standardized feature value, typically ranging from -3 to 3. Values ​​outside this range are considered outliers and require re-standardization. In the example, the gold MA5 feature in the original feature pool has a mean μ = $1920.50 / oz and a standard deviation σ = $15.30 / oz for all samples. The original MA5 value for one sample is $1923.18 / oz. Substituting these values ​​into the formula, we get x_norm = (1923.18 - 1920.50) / 15.30 ≈ 0.175, indicating that the standardized feature value is within a reasonable range. For the news sentiment analysis score feature (original values ​​0-100), one sample has an original value of 72. The mean μ = 65 and the standard deviation σ = 12. After standardization, x_norm = (72 - 65) / 12 ≈ 0.583, achieving dimensional consistency with other features. The standardization process covers all 15 features in the original feature pool, with the mean and standard deviation calculated separately for each feature to ensure accuracy and avoid interference between different features.

[0041] Secondly, a stabilization process is performed, focusing on price series (fixing price, futures closing price). Since price series in financial time series data often exhibit trends and non-stationarity, directly inputting them into the model can lead to difficulties in capturing inherent patterns and even spurious regression. Therefore, the logarithmic differencing method is used to stabilize the price series, eliminating trends and ensuring it meets the stationarity requirement. The formula for logarithmic differencing is Δln(x_t) = ln(x_t) - ln(x_{t-1}), where x_t is the price value at the current time step, x_{t-1} is the price value at the previous time step, ln is the natural logarithm, and Δln(x_t) is the stabilized price series value. This value reflects the relative rate of price change, effectively eliminating absolute price trends while preserving the regularity of price fluctuations. In the example, the closing price of London gold on a certain trading day is x_t = $1925.30 / ounce, and the closing price of the previous trading day is x_{t-1} = $1923.70 / ounce. Substituting these values ​​into the formula, we get Δln(x_t) = ln(1925.30) - ln(1923.70) ≈ 7.562 - 7.561 = 0.001. The stabilized value reflects a slight increase in the gold price on that day, eliminating the magnitude effect of the absolute price. For other non-price features (such as RSI and sentiment score), since they are inherently stationary, there is no need for logarithmic differencing; only the standardized results are retained.

[0042] After all features have been standardized and stationary, they are integrated in chronological order into a standardized feature matrix. Rows in the matrix correspond to each time step, and columns correspond to each processed feature. The matrix dimension is (T×F), where T is the total number of time steps (approximately 780,000) and F is the number of features (15). Each element in the matrix is ​​a standardized or stationary feature value, labeled with its corresponding timestamp and feature identifier to ensure the matrix's order and traceability. The resulting standardized feature matrix has no outliers, no dimensionless differences, and the price series meets the stationarity requirement, making it directly usable for constructing subsequent time window samples.

[0043] The standardized feature matrix is ​​organized into continuous time window samples in chronological order. Each sample contains features from the past N time steps as input and the real prices from the next M time steps as the prediction target. A structured training sample set is constructed for model fine-tuning.

[0044] The core of this step is to convert the standardized feature matrix into structured training samples adapted for fine-tuning the Transformer model. By dividing the time window, a correlation is established between the input features and the prediction target, ensuring that the samples can effectively train the model to capture the time-series price trends of precious metals. The specific implementation method is as follows: First, the time window parameters were determined, including the input window size N and the prediction window size M. The parameter settings were tailored to the volatility characteristics of the precious metals market and the model's prediction needs. After multiple adjustments and optimizations, the input window size N=60 and the prediction window size M=5 were determined. N=60 means that the input for each sample is all the features of the past 60 time steps (corresponding to 5 hours for 5-minute data and 60 days for daily data in gold futures), which can fully capture the short-term and medium-term time series dependencies of precious metal prices. M=5 means that the prediction target for each sample is the actual price in the next 5 time steps, which can meet the needs of trading strategy optimization for short-term price prediction, while avoiding the decrease in prediction accuracy caused by an excessively large prediction window.

[0045] The time window is divided using a sliding window approach, continuously extracting samples from the standardized feature matrix in chronological order. The sliding step size is set to 1, meaning that each adjacent sample differs by only one time step, ensuring the continuity and completeness of the samples and maximizing the use of existing data to improve the sufficiency of model training. Specifically, the division process is as follows: the first training sample is formed by using time steps 1 to 60 of the standardized feature matrix as input features and the stationary price series from time steps 61 to 65 as the prediction target; the second training sample is formed by using time steps 2 to 61 as input features and the stationary price series from time steps 62 to 66 as the prediction target; and so on, until the entire standardized feature matrix is ​​traversed, generating all training samples.

[0046] Each training sample consists of two parts: input features and a prediction target. The input features are an N×F matrix (60×15), representing 15 standardized features from the past 60 time steps. The prediction target is an M×1 vector (5×1), representing the stationary true price for the next 5 time steps, with timestamps added to distinguish samples from different time periods. In the example, the input features of a training sample are the standardized values ​​of 15 features, including MA5, RSI, Bollinger Bands, and sentiment score, from the past 60 time steps. The prediction target is the stationary gold price sequence for the next 5 time steps (0.001, 0.002, -0.001, 0.003, 0.002, respectively), reflecting a slight upward trend in gold prices over the next 5 time steps.

[0047] After sample generation, all samples are screened to remove those with missing or outlier values ​​in the input features or prediction targets, ensuring sample quality. This results in a structured training sample set of approximately 779,000 samples. To meet the training, validation, and testing needs of model fine-tuning, the structured training sample set is divided into training, validation, and test sets in a 7:2:1 ratio. The training set is used for fine-tuning model parameters, the validation set is used to adjust model hyperparameters (such as learning rate and batch size), and the test set is used to evaluate the model's final prediction accuracy. The data in these three sets are non-overlapping, ensuring the objectivity of the evaluation results. Simultaneously, data augmentation is performed on the training set by randomly shuffling the feature order within a time window (while maintaining temporal dependencies), increasing sample diversity and preventing model overfitting. This results in a complete structured training sample set that can be directly used for targeted fine-tuning of the subsequent base model.

[0048] S203, The base model is fine-tuned using the structured training sample set, and the model is adapted to the characteristics of the precious metals market through transfer learning and dynamic learning rate adjustment to generate a dedicated prediction model. Specifically, a financial time series base model with generalization capabilities can be loaded, some parameters of its underlying Transformer encoder can be frozen, the general time series representation capabilities it has learned can be retained, and a fine-tuned initial model with partially frozen parameters can be generated. The core of this step is to reuse the general financial time series patterns learned by the financial time series foundation model, and to prevent the destruction of underlying general features by freezing parameters, while laying the foundation for subsequent targeted fine-tuning. The specific implementation method is as follows: First, a financial time-series baseline model was loaded. This baseline model is based on historical foreign exchange market data and converged after self-supervised learning and multi-task training. This model has mastered the contextual dependencies and price fluctuation patterns of financial time-series data and can be directly applied to transfer learning in the precious metals market. During the loading process, all model parameters were fully read, including the attention layer parameters of the Transformer encoder, the parameters of the feedforward neural network, and the parameters related to the multi-task loss function, ensuring the integrity of the model and that its generalization ability is not lost. After loading, the model parameters were initially verified to confirm that there was no damage or missing parameters and that the model could run normally.

[0049] The core purpose of parameter freezing is to preserve the general financial time series representation capabilities learned by the base model's underlying layers, preventing excessive modification of the underlying parameters during subsequent fine-tuning that could destroy the mastered general patterns. Simultaneously, it reduces the computational cost of fine-tuning, improves training efficiency, and focuses on adapting to the specific characteristics of the precious metals market. The freezing scope targets the bottom layer of the Transformer encoder. Based on the base model's structure (the encoder has 8 layers), all parameters of the bottom 4 Transformer encoder layers are frozen. These 4 layers are primarily responsible for capturing the fundamental characteristics of financial time series data (such as time series continuity, short-term volatility correlations, and other general patterns). These characteristics are universal in financial markets such as foreign exchange and precious metals, and do not require retraining.

[0050] The parameters that are not frozen include the parameters of the top four layers (layers 5-8) of the Transformer encoder and the output layer parameters of the top layer of the model. The top-layer encoder is mainly responsible for capturing the specific features of the market and is reserved for subsequent fine-tuning to adapt to the precious metals market. The output layer parameters are set for multi-task training of the foreign exchange market and cannot be directly adapted to the precious metals price prediction task, so they are not frozen to facilitate subsequent replacement and adjustment. The parameter freezing is implemented by setting all parameters of the bottom four layers of the encoder to a non-trainable state, that is, the gradients of these parameters are not updated or their values ​​are not changed during training, only their original values ​​are retained. The top-layer parameters and the output layer parameters are set to a trainable state for subsequent fine-tuning.

[0051] In the example, the loaded financial time series base model has an 8-layer encoder, 256 hidden layers, and 12 attention heads. After loading, all parameters of the encoder layers 1-4 (including attention weight matrices, bias parameters, etc.) are frozen, and the trainability flag for these parameters is set to "non-trainable." The encoder parameters of layers 5-8 and the output layer parameters are set to "trainable." After verification, a fine-tuned initial model with partially frozen parameters is generated. This initial model retains the ability to represent general financial time series data while also having adjustable space adapted to the precious metals market, and can be directly used for subsequent transfer learning training.

[0052] The structured training sample set is input into the initial model for fine-tuning. A transfer learning strategy is adopted to add a fully connected layer adapted for precious metal prediction at the top layer of the model. Only the newly added layer and some high-level attention layers are trained to generate the initial fine-tuned model. The core of this step is to use transfer learning to train the adjustable parameters of the model using a structured training sample set (adapted to the precious metals market), allowing the model to gradually adapt to the specific characteristics of the precious metals market. The specific implementation method is as follows: The core of the transfer learning strategy is "reusing the general and fine-tuning the specific," which means reusing the general financial time series features at the bottom of the base model, and training adjustable parameters to allow the model to learn the differences between the precious metals market and the foreign exchange market (such as the specific pattern that precious metal prices are more significantly affected by geopolitics and inflation expectations), thus adapting the model from general financial time series to precious metals-specific time series. The structured training sample set as input is the sample set constructed in claim 3, containing nearly 779,000 samples. Each sample contains multi-dimensional features (technical indicators, market sentiment factors, etc.) from the past 60 time steps as input, and the real price of the next 5 time steps as the prediction target. The samples have been standardized and stabilized and can be directly used for model training.

[0053] A new fully connected layer adapted for precious metals prediction has been added to the top layer of the model, replacing the original output layer adapted for multi-task training in forex. The structure of the new fully connected layer is tailored to the requirements of precious metals price prediction tasks, ensuring that the output results accurately correspond to price predictions for multiple future time steps. The new fully connected layer adopts a two-layer structure. The first layer is a hidden layer with a dimension of 128, using the ReLU function (to introduce non-linear features and improve the model's ability to capture non-linear fluctuations in precious metal prices). The second layer is the output layer with a dimension of 5 (corresponding to a prediction window M=5, i.e., the price prediction value for the next 5 time steps), using a linear activation function (to meet the regression task requirements of price prediction and avoid compressing the output value).

[0054] The focus of training is clearly defined on the newly added fully connected layers and some high-level attention layers (layers 7-8) of the Transformer encoder. The reason for this setting is that the newly added fully connected layers are the core part that directly adapts to precious metal price prediction and need to be trained from scratch to ensure that they can accurately map the correlation between precious metal features and price prediction results. The high-level attention layers (layers 7-8) are responsible for capturing market-specific time-series features. Training this part in a focused manner can allow the model to focus on learning the unique fluctuation patterns of the precious metal market, while avoiding excessive modification of the general features of the middle layers (layers 5-6).

[0055] During training, the gradient descent algorithm is used to update the adjustable parameters. The training batch size is set to 128, and the number of training iterations is initially set to 100 epochs. The mean squared error loss function is used (to quantify the deviation between the model's predicted price and the actual price, aligning with the requirements of the regression task). The model's prediction error on the validation set is monitored in real time during training to ensure the model gradually adapts to the distribution of precious metal data. In the example, the training set (approximately 545,000 samples) of the structured training sample set is input into the initial model for fine-tuning. The focus is on training the newly added fully connected layer and the 7th-8th attention layers. During training, the parameters of the newly added fully connected layer are gradually updated from random initialization, and the parameters of the higher-level attention layers are fine-tuned based on the original general features. When training reaches 100 epochs and the validation set loss tends to stabilize, this stage of training is stopped, generating the preliminary fine-tuned model. This preliminary fine-tuned model already possesses a certain ability to predict precious metal prices, but further optimization of the convergence effect is still needed through dynamic learning rate adjustment.

[0056] A dynamic learning rate adjustment strategy is implemented. Based on the loss changes of the model on the precious metal validation set, the cosine annealing algorithm is used to dynamically adjust the learning rate of different network layers, so as to promote the efficient convergence of the initial fine-tuned model to the precious metal data distribution and generate the optimized fine-tuned model. The core of this step is to address issues such as training oscillations, slow convergence, and overfitting that can easily occur with a fixed learning rate by dynamically adjusting the learning rate. This allows the initially fine-tuned model to converge efficiently to the precious metal data distribution, improving the model's prediction accuracy. The specific implementation method is as follows: The core of the dynamic learning rate adjustment strategy is to adjust the learning rate of different network layers in real time based on the loss changes of the model on the precious metal validation set, so that the learning rate matches the model training state: in the early stage of model training, the validation set loss decreases rapidly, so a relatively high learning rate is used to accelerate the parameter update speed; in the later stage of training, the validation set loss decreases slowly, so a relatively low learning rate is used to finely adjust the parameters and avoid oscillations. At the same time, differentiated learning rates are set for different network layers, with the highest learning rate for newly added fully connected layers (which need to converge quickly), followed by the learning rate for high-level attention layers, to ensure that the parameters of each layer are synchronously adapted to the distribution of precious metal data.

[0057] Cosine annealing is used to achieve dynamic learning rate adjustment. The core principle of this algorithm is to simulate the learning rate change curve as half a period of a cosine function, gradually decreasing the learning rate from the initial level to the minimum level, and then increasing it back to the initial level, iterating in a loop. This ensures rapid convergence in the early stages of training while avoiding overfitting in the later stages, and also alleviates the gradient vanishing problem. The key parameters of the cosine annealing algorithm are set as follows: initial learning rate lr_base = 0.001 (set according to the model parameter size and sample size to ensure a moderate initial update speed), minimum learning rate lr_min = 0.0001 (to avoid training stagnation due to an excessively low learning rate), annealing period T_max = 100 rounds (i.e., every 100 rounds completes the process of decreasing from the initial learning rate to the minimum learning rate), and the learning rate update formula is lr_t = lr_min + 0.5 × (lr_base - lr_min) × (1 + cos(π × t / T_max)), where t is the current iteration round and lr_t is the learning rate in the current round.

[0058] Different learning rates are set for different network layers: the learning rate of the newly added fully connected layer is 1 times the cosine annealing value of the current round (lr_fc=lr_t), ensuring that the layer converges quickly and is accurately adapted to the precious metal prediction task; the learning rate of the attention layers of the Transformer encoder layers 7-8 is 0.5 times the value of the current round (lr_attn=0.5×lr_t), to avoid the parameters of this layer being updated too quickly, which would disrupt the balance between the original general features and the newly added specific features; the encoder parameters of layers 5-6 are not adjusted in a major way, and the learning rate is set to 0.1 times the value of the current round (lr_mid=0.1×lr_t), with only slight fine-tuning to retain general features while adapting to precious metal data.

[0059] In the example, the model is initially fine-tuned and enters this training stage with an initial learning rate of lr_base=0.001, lr_min=0.0001, and T_max=100 epochs. In the early stage of training (t=10 epochs), lr_t=0.001+0.5×(0.001-0.0001)×(1+cos(π×10 / 100))≈0.00095. At this time, the learning rate of the newly added fully connected layer is 0.00095, the learning rate of the 7th-8th attention layer is 0.000475, and the learning rate of the 5th-6th layer is 0.000095, which speeds up parameter updates. In the later stage of training (t=80 epochs), lr_t=0.001+0.5×0.0009×(1+cos(π×80 / 100))≈0.00015, and the learning rate of each layer is reduced synchronously to finely adjust the parameters. During training, the validation set loss is calculated every 20 iterations. If the validation set loss does not decrease for 30 consecutive iterations and the decrease is less than 1e-6, the model is considered converged, training is stopped, and an optimized fine-tuned model is generated. This optimized fine-tuned model has fully converged to the precious metal data distribution, and its prediction accuracy and generalization ability are significantly improved compared to the initial fine-tuned model.

[0060] The prediction accuracy and generalization ability of the optimized fine-tuned model are evaluated on an independent precious metals test set. The model parameters are further fine-tuned through backpropagation algorithm to minimize the prediction error, and finally a special prediction model for the precious metals market is generated.

[0061] The core of this step is to evaluate the model's performance using an independent test set, identify its shortcomings, and further fine-tune and optimize it to ensure that the model has excellent predictive and generalization capabilities for precious metal prices. The final result is a dedicated prediction model, implemented as follows: The independent precious metals test set used for evaluation was obtained from the structured training sample set at a 1:10 ratio, totaling approximately 78,000 samples. This test set has no overlap with the training and validation sets, and the time range of the samples covers different fluctuation cycles (upswing, downswing, and oscillations) in the precious metals market. This allows for a comprehensive evaluation of the model's predictive accuracy and generalization ability, avoiding model overfitting (which only fits the training set data and cannot adapt to new data). Three core evaluation metrics were selected to align with the requirements of regression prediction tasks: Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Coefficient of Determination (R²). 2 Each indicator has a clear calculation logic and evaluation criteria.

[0062] Mean Absolute Error (MAE) measures the average absolute deviation between model predictions and actual values. The formula is MAE = (1 / N) × Σ|y_pred - y_true|, where N is the number of samples in the test set, y_pred is the model's predicted price, and y_true is the actual price. A smaller MAE value indicates a more accurate prediction. In precious metals prediction tasks, the target MAE is ≤ 0.002 (corresponding to a stable price series). Root Mean Square Error (RMSE) measures the square root of the average of the squared deviations between model predictions and actual values. The formula is RMSE = √[(1 / N) × Σ(y_pred - y_true)]. 2 RMSE is more sensitive to larger deviations; the target RMSE is ≤0.003; the coefficient of determination (R²) 2 R is used to measure the ability of a model's predicted values ​​to explain the true values, and is calculated using the formula R. 2 =1-[Σ(y_true-y_pred) 2 / Σ(y_true-μ) 2 ], where μ is the mean of the true prices, R 2 The value range is 0-1, R 2 The closer the value is to 1, the stronger the model's generalization ability, and the better the target R. 2 ≥0.85.

[0063] During the evaluation process, the input features of the independent test set are input into the optimized fine-tuned model to obtain the model's predicted price series. This predicted price series is then compared with the actual price series of the test set, and the specific values ​​of the three evaluation metrics are calculated. In the example, the evaluation results are MAE=0.0018, RMSE=0.0027, and R... 2 =0.89, all three indicators have reached the preset target, indicating that the model has good prediction accuracy and generalization ability, but there are still some samples with large prediction bias (such as during periods of sharp fluctuations in precious metal prices), which need to be further fine-tuned and optimized.

[0064] Further fine-tuning employs the backpropagation algorithm. The core of this algorithm is to calculate the gradient of the prediction error on the test set and backpropagate it to the adjustable parameters of the model (adding fully connected layers and high-level attention layers). The parameters are then fine-tuned to minimize the prediction error, with a focus on optimizing parameters for samples with large prediction biases. During fine-tuning, the learning rate is reduced (set to 0.00005) to decrease the parameter update amplitude and avoid model oscillations. The training batch size is changed to 64, and the number of iterations is set to 50. The mean squared error loss function is still used, while L2 regularization (regularization coefficient = 0.001) is introduced to prevent overfitting.

[0065] After fine-tuning, the model performance is evaluated again on an independent test set. If the evaluation metric does not decrease significantly and the number of samples with large prediction bias decreases (the proportion of samples with a bias greater than 0.005 ≤ 5%), the fine-tuning is considered complete. If the requirements are not met, the fine-tuning process is repeated until the target is met. Finally, all the fine-tuned model parameters are saved to generate a dedicated prediction model for the precious metals market. This model retains the general financial time series representation capabilities of the base model and, through targeted fine-tuning, accurately adapts to the price fluctuation patterns and characteristics of the precious metals market. It can efficiently output future multi-time-step price prediction sequences and volatility estimates, providing reliable support for trading strategy optimization.

[0066] S204. Input real-time precious metals market data into the dedicated prediction model and output a price prediction sequence and volatility estimate for future multi-time steps for trading strategy optimization.

[0067] Specifically, it can collect the latest precious metals market data in real time, including real-time quotes, trading volume and market news flow, and generate real-time standardized feature vectors; The core of this step is to acquire timely and comprehensive real-time data on precious metals. Through standardization, the raw real-time data is converted into feature vectors that are adapted to the input format of a dedicated prediction model, ensuring the consistency and validity of the model's input data. The specific implementation method is as follows: The data sources for real-time data acquisition are authoritative, timely, and diverse, covering international real-time precious metal trading interfaces, mainstream financial data real-time push channels, authoritative financial news real-time broadcasting platforms, and social media real-time data interfaces. Data verification mechanisms are implemented during the acquisition process to ensure data transmission is lossless and unbiased. The precious metals collected focus on the two core commodities, gold and silver, maintaining consistency with the commodities in the structured training sample set mentioned earlier, ensuring the compatibility between the model input and training data.

[0068] The collected data strictly corresponds to the multi-dimensional features extracted earlier and is divided into three main categories. The collection frequency is set at once per minute to meet the real-time prediction needs, ensuring data timeliness while avoiding redundancy and computational pressure caused by overly intensive collection. The first category is real-time price quotes, including the real-time buy price, real-time sell price, and latest transaction price of London Gold and London Silver, all in USD / ounce, rounded to two decimal places. The latest transaction price serves as the core price indicator, while the buy and sell prices are used to help determine market liquidity. The second category is real-time trading volume data, including the real-time trading volume of gold and silver futures (1 lot of gold futures = 100 ounces, 1 lot of silver futures = 5000 ounces) and the real-time trading volume of ETFs (in shares), reflecting real-time market trading activity and capital flows. The third category is real-time market news stream data, collecting no fewer than 5 breaking news, policy announcements, and industry dynamics related to precious metals every minute. It also collects real-time comments and topic popularity data related to precious metal investment on social media platforms to capture changes in market sentiment in real time.

[0069] The collected data needs to undergo real-time preprocessing and standardization. Preprocessing is mainly used to quickly remove abnormal data (such as sudden price changes, abnormal surges in trading volume, etc.). A simplified version of the 3σ principle is adopted, and the mean μ and standard deviation σ are calculated based on the real-time data of the past 10 minutes. Data that exceeds the range of [μ-2σ,μ+2σ] is judged as outliers and temporarily replaced with valid data from the previous minute to ensure data continuity. The standardization process strictly follows the method set in claim 3 above to ensure that the data input to the model is consistent with the dimensions and distribution of the training samples. Specifically, it is divided into two categories of processing: price features (real-time transaction price, bid price, ask price) are processed using logarithmic differencing, with the calculation formula being Δln(x_t)=ln(x_t)-ln(x_{t-1}), where x_t is the price value of the current minute and x_{t-1} is the price value of the previous minute, eliminating the non-stationarity of the price series and the influence of dimensions; non-price features (real-time transaction volume, news sentiment score, social media sentiment index) are standardized using the Z-score method, with the calculation formula being x_norm=(x-μ) / σ, where μ and σ are the mean and standard deviation of the corresponding features in the structured training sample set above, ensuring uniform standardization and avoiding model adaptation bias.

[0070] In the example, the latest transaction price of London gold in the current minute (timestamp t) is x_t = $1926.50 / ounce, and the transaction price in the previous minute is x_{t-1} = $1926.20 / ounce. After logarithmic difference processing, Δln(x_t) = ln(1926.50) - ln(1926.20) ≈ 7.563 - 7.562 = 0.001. The real-time trading volume of gold futures in the current minute is 250 lots. The mean of the trading volume in the training sample set is μ = 200 lots, and the standard deviation is σ = 50 lots. After Z-score standardization, x_norm = (250 - 200) / 50 = 1.0. At the same time, 5 real-time news items were collected. The mean score calculated by sentiment analysis is 75. The sentiment score of the news items in the training sample is μ = 65, σ = 12. After standardization, the score is (75 - 65) / 12 ≈ 0.833. All standardized real-time features (15 in total, consistent with the original feature pool mentioned above) are integrated in a fixed order to generate a real-time standardized feature vector with a dimension of 1×15. The acquisition timestamp (accurate to the second) is labeled to ensure that the vector is traceable, and this step is completed.

[0071] The real-time standardized feature vectors are input into a dedicated prediction model. The model captures long-term dependencies through its Transformer encoder and performs multi-step forward inference via a fine-tuned prediction head to generate a price point prediction sequence for multiple future time steps. The core of this step is to utilize the feature extraction and inference capabilities of a dedicated prediction model (generated through fine-tuning above) to convert real-time standardized feature vectors into price prediction results for multiple future time steps, thereby achieving accurate prediction of precious metal price movements. The specific implementation method is as follows: The input real-time standardized feature vectors need to be formatted. Since the dedicated prediction model uses features from the past 60 time steps as input (input window N=60) during training, the currently generated real-time standardized feature vectors need to be concatenated with the standardized feature vectors from the previous 59 minutes to form a 60×15 input matrix. This ensures that the input format is consistent with that used during model training, providing data support for the model to capture long-term dependencies. During the concatenation process, if there are a small number of anomalous substitute data from the previous 59 minutes, the anomalous locations will be marked. During model inference, an attention mechanism will be used to reduce the weight of anomalous data to avoid affecting prediction accuracy.

[0072] The Transformer encoder in the dedicated forecasting model is the core for capturing long-term dependencies. Through a multi-head attention mechanism, it calculates the association weights of each time-step feature in the input matrix with features from all other time-steps. It focuses on historical time steps that have a significant impact on current prices (such as recent 10-20 minute market sentiment and technical indicator features) while retaining long-term trend features (such as price fluctuation patterns over the past 60 minutes), effectively addressing the difficulty of traditional time-series models in capturing long-term dependencies. The calculation of attention weights incorporates precious metal market characteristics learned during the fine-tuning process, assigning higher weights to time steps corresponding to geopolitical and inflation expectation-related features, thus enhancing forecast targeting.

[0073] The fine-tuned prediction head consists of two newly added fully connected layers. The first hidden layer has a dimension of 128 and uses the ReLU activation function to introduce non-linear mapping and capture the complex correlation between precious metal prices and multi-dimensional features. The second output layer has a dimension of 5 (prediction window M=5) and uses a linear activation function to adapt to the regression task of price prediction, ensuring that the output is not compressed and can accurately correspond to the price fluctuation range of the next 5 time steps. During the multi-step forward inference process, the model first extracts features from the 60×15 input matrix through the Transformer encoder to generate a 256-dimensional temporal feature vector. This vector is then input into the prediction head, processed by two fully connected layers, and outputs the stationary price prediction value (log-differenced value) for the next 5 time steps.

[0074] In the example, the standardized feature vectors of the current time and the previous 59 minutes are concatenated to form a 60×15 input matrix. After being input into the dedicated prediction model, the Transformer encoder focuses on the news sentiment features and trading volume features of the past 15 minutes through a multi-head attention mechanism, extracting the core information of the current market sentiment being positive and the increase in capital inflow, and generating a 256-dimensional time-series feature vector. The prediction head operates on this feature vector and outputs the stationary price prediction values ​​for the next 5 time steps (1 minute each) as 0.0012, 0.0015, 0.0008, 0.0021, and 0.0017, respectively. Subsequently, through inverse logarithmic differencing (x_t=exp(Δln(x_t)+ln(x_{t-1}))), the stationary forecast value is converted into the actual price forecast value. Combined with the current minute-by-minute transaction price of $1926.50 / ounce, the price point forecast sequence for the next 5 time steps is calculated as [1926.73, 1927.02, 1927.18, 1927.58, 1927.91] / ounce, completing the multi-step forward inference.

[0075] Based on the internal attention weight matrix and hidden layer state of the dedicated prediction model, Monte Carlo Dropout technique is used to perform multiple forward inferences, calculate the standard deviation of the prediction results to estimate the volatility of future prices, and generate a volatility estimation sequence. The core of this step is to use Monte Carlo Dropout technology to quantify the uncertainty of the forecast results, namely the volatility of future precious metal prices, to provide data support for risk control of trading strategies. The specific implementation method is as follows: Monte Carlo Dropout is a highly efficient volatility estimation method. Its core principle is to maintain the Dropout layer (a component used during model training to prevent overfitting, with a dropout rate of 0.2, consistent with training) during model inference. Multiple forward inferences generate multiple predictions, and the dispersion (standard deviation) of these predictions measures the uncertainty of the forecast, thereby estimating future price volatility. This method requires no additional model training, directly utilizing the internal structure of a dedicated prediction model, balancing computational efficiency and estimation accuracy, and is suitable for real-time prediction needs.

[0076] The attention weight matrix and hidden layer states within the dedicated prediction model provide support for volatility estimation. The attention weight matrix reflects the model's attention to different time steps and features, and can help identify the core driving factors of increased volatility (e.g., if volatility is high at a certain time step, the feature with higher attention weight is geopolitical news, indicating that geopolitical factors may lead to increased price volatility). The hidden layer states contain all the time-series feature information extracted by the model, ensuring that the results of multiple forward inferences are representative and avoiding the randomness of single inferences.

[0077] In the specific implementation, the number of forward inference iterations is set to 100. Too many iterations would increase computational pressure and affect real-time performance, while too few iterations would lead to large deviations in volatility estimation. 100 iterations strike a balance between real-time performance and accuracy. The 60×15 input matrix from step two is repeatedly input into the dedicated prediction model for 100 forward inference iterations with the Dropout layer enabled. Each inference iteration generates a price prediction sequence for the next 5 time steps, resulting in a total of 100 prediction sequences. For each future time step, the standard deviation of the 100 prediction values ​​is calculated. This standard deviation is the volatility estimate for that time step. A larger standard deviation indicates more volatile prices and higher volatility in the future; conversely, a smaller standard deviation indicates lower volatility.

[0078] In the example, for the first future time step, the predicted values ​​from 100 forward inferences are distributed between $1926.60 and $1926.86 per ounce. The mean of these 100 predicted values ​​is $1926.73 (consistent with the point prediction value in step two), and the standard deviation is $0.065 per ounce, meaning the volatility estimate for this time step is $0.065 per ounce. For the second future time step, the 100 predicted values ​​are distributed between $1926.88 and $1927.16 per ounce, with a mean of $1927.02 and a standard deviation of $0.068 per ounce. This pattern continues, resulting in a volatility estimate sequence of [0.065, 0.068, 0.072, 0.078, 0.082] per ounce for the next five time steps. Meanwhile, combined with the analysis of the attention weight matrix, it was found that the volatility was highest in the fifth time step, and the feature with the highest attention weight was the geopolitical theme. This indicates that the price fluctuations in this time step may be mainly affected by geopolitical factors, providing a reference for subsequent risk control.

[0079] Integrating future price point prediction sequences and volatility estimation sequences across multiple time steps, the system outputs a structured forecast report containing predicted values, confidence intervals, and risk measures to optimize trading strategy parameters and risk control logic.

[0080] The core of this step is to integrate the price predictions and volatility estimates obtained from model inference, supplement them with confidence intervals and risk metrics, and form a structured forecast report. This provides specific and actionable support for adjusting trading strategy parameters and controlling risk. The specific implementation method is as follows: The integration process centers on the price point prediction series and volatility estimation series, supplemented by two key components: a 95% confidence interval and a risk measurement indicator, ensuring the report's completeness and usability. The 95% confidence interval is a commonly used reliability indicator in financial forecasting, reflecting the high-probability range of future price fluctuations. The formula is: Confidence Interval = [y_pred - 2 × σ, y_pred + 2 × σ], where y_pred is the predicted price point value at a certain time step, σ is the estimated volatility value at that time step, and two standard deviations correspond to a 95% confidence level, meaning that the probability of future prices falling within this interval is 95%, facilitating investors' assessment of the forecast's reliability.

[0081] Two core risk metrics are selected to align with the optimization needs of precious metals trading strategies: volatility risk coefficient and maximum potential drawdown. Volatility risk coefficient = estimated volatility at the current time step / average volatility over the past 60 minutes, ranging from 0 to 2. A value greater than 1.5 indicates that the current volatility is significantly higher than the recent average, indicating high market risk; 1.0-1.5 indicates moderate risk; and less than 1.0 indicates low risk. Maximum potential drawdown = (maximum predicted price point over multiple future time steps - minimum predicted price point) / maximum predicted price point, reflecting the maximum possible price decline over a future period. It is used to set stop-loss levels and control trading risk.

[0082] The structured forecast report is concise and clear, highlighting key points. It mainly includes four parts: basic forecast information (forecast time range, data collection timestamp, precious metal commodity), core forecast results (price point forecast sequence, volatility estimation sequence, 95% confidence interval for each time step), risk measurement results (volatility risk coefficient, maximum potential drawdown), and strategy optimization suggestions (position adjustment, stop-loss and take-profit recommendations based on forecast results and risk measurements). The report format is standardized, facilitating automatic reading by trading systems and manual analysis, enabling seamless integration of real-time forecasting and strategy optimization.

[0083] In the example, the integrated structured forecast report shows: the forecast time range is the next 5 minutes (5 time steps), the data collection timestamp is the current minute (accurate to the second), and the commodity is London Gold; in the core forecast results, the price point forecast sequence is [1926.73, 1927.02, 1927.18, 1927.58, 1927.91] USD / oz, the volatility estimation sequence is [0.065, 0.068, 0.072, 0.078, 0.082] USD / oz, and the 95% confidence interval for the first future time step is [1926.73 - 2 × 0.065, 1926.73 + 2 × 0.065] = [1926.60, 1926.86] USD / oz; in the risk measurement results, volatility... The risk coefficient is 0.072 / 0.058≈1.24 (the average volatility over the past 60 minutes is 0.058), which is considered medium risk. The maximum potential drawdown is (1927.91-1926.73) / 1927.91≈0.00061, or 0.061%, which is manageable. The recommended strategy optimization is to adjust the trading position to 50% (suitable for medium risk), set the stop-loss at $1926.50 / ounce (below the lower limit of the confidence interval for the first time step), and set the take-profit at $1928.00 / ounce (close to the upper limit of the confidence interval for the fifth time step). If any unexpected geopolitical news emerges, promptly re-collect data, update the forecast report, and adjust the trading strategy to maximize returns while ensuring manageable risk. This structured forecast report can be directly used to optimize trading strategy parameters and adjust risk control logic, completing the entire forecasting process.

[0084] Another embodiment of the present invention provides a precious metal price trend prediction system based on a Transformer architecture pre-trained model, see [link to relevant documentation]. Figure 3 The system may include: Module 301 is used to build a Transformer pre-trained model based on historical foreign exchange market data. Through self-supervised learning and multi-task training, the model learns general financial time series patterns and generates a base model with generalization ability. The historical foreign exchange market data includes at least foreign exchange prices, trading volume, and market sentiment indicators. Extraction module 302 is used to perform multi-dimensional feature extraction and standardization processing on historical data of the precious metals market, and to construct a structured training sample set containing price series, technical indicators and market sentiment factors. The historical data of the precious metals market includes at least the price, trading volume and market sentiment indicators of precious metal varieties. The adjustment module 303 is used to fine-tune the base model using the structured training sample set, and adapt the model to the characteristics of the precious metals market through transfer learning and dynamic learning rate adjustment to generate a dedicated prediction model. The output module 304 is used to input real-time precious metal market data into the dedicated prediction model and output a price prediction sequence and volatility estimate for future multi-time steps for trading strategy optimization.

[0085] This invention also provides a storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when running.

[0086] This invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0087] Specifically, the aforementioned electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the aforementioned processor, and the input / output device is connected to the aforementioned processor.

[0088] The above description, based on the embodiments shown in the figures, details the structure, features, and effects of the present invention. The above description is only a preferred embodiment of the present invention, but the present invention is not limited to the scope of implementation shown in the figures. Any changes made in accordance with the concept of the present invention, or equivalent embodiments modified to have equivalent changes, that do not exceed the spirit covered by the specification and figures, should be within the protection scope of the present invention.

Claims

1. A method for predicting precious metal price trends based on a pre-trained model using the Transformer architecture, characterized in that, The method includes: A Transformer pre-trained model is built based on historical foreign exchange market data. Through self-supervised learning and multi-task training, the model learns general financial time series patterns and generates a base model with generalization ability. The historical foreign exchange market data includes at least foreign exchange prices, trading volume, and market sentiment indicators. Multi-dimensional feature extraction and standardization processing are performed on historical data of the precious metals market to construct a structured training sample set containing price series, technical indicators and market sentiment factors. The historical data of the precious metals market includes at least the price, trading volume and market sentiment indicators of precious metal varieties. The base model is fine-tuned using the structured training sample set, and the model is adapted to the characteristics of the precious metals market through transfer learning and dynamic learning rate adjustment to generate a dedicated prediction model. Real-time precious metals market data is input into the dedicated prediction model, which outputs a price prediction sequence and volatility estimate for multiple future time steps, for use in trading strategy optimization.

2. The method according to claim 1, characterized in that, The Transformer pre-trained model, constructed based on historical foreign exchange market data, employs self-supervised learning and multi-task training to enable the model to master general financial time-series patterns, generating a base model with generalization capabilities. The historical foreign exchange market data includes at least foreign exchange prices, trading volumes, and market sentiment indicators, including: Historical data of the foreign exchange market were collected from multiple international financial data sources, including price series, trading volume series and market sentiment indicators of major currency exchanges, to generate the original foreign exchange multimodal dataset. The original foreign exchange multimodal dataset is cleaned and aligned, outliers and missing values ​​are removed, and data of different frequencies are uniformly interpolated to the same timestamp to generate a cleaned and aligned foreign exchange time series dataset. Based on the cleaned and aligned foreign exchange time series dataset, a self-supervised learning task is designed. The time series masking reconstruction method is used to randomly mask some time steps of the data, forcing the model to learn contextual dependencies to predict the masked values, and generating a self-supervised training task set. A multi-task training framework is constructed. Based on the self-supervised reconstruction task, two auxiliary tasks, foreign exchange price direction prediction and volatility estimation, are added. An adaptive weight algorithm is used to balance the losses of different tasks. The Transformer model is trained until convergence, and finally a financial time series base model with generalization ability is generated.

3. The method according to claim 2, characterized in that, The process involves multi-dimensional feature extraction and standardization of historical precious metals market data to construct a structured training sample set containing price series, technical indicators, and market sentiment factors. This historical precious metals market data includes at least the price, trading volume, and market sentiment indicators of the precious metals varieties, including: Historical data from the precious metals market was collected, including London gold and silver fixing prices, minute-level candlestick data for futures, ETF trading volume, and news sentiment analysis scores and social media sentiment indices extracted from financial news and social media, to generate a raw multi-source dataset of precious metals. Multi-dimensional features are extracted from the original precious metal multi-source dataset. Technical indicators, including at least moving averages, relative strength index, and Bollinger Bands, are calculated based on price and volume sequences. Sentiment dictionary features and topic distribution features are extracted from text data to generate the original feature pool. All features in the original feature pool are standardized and stabilized. The Z-score method is used to remove the dimensions, and the price series is log-differenced to meet the stationarity requirement, generating a standardized feature matrix. The standardized feature matrix is ​​organized into continuous time window samples in chronological order. Each sample contains features from the past N time steps as input and the real prices from the next M time steps as the prediction target. A structured training sample set is constructed for model fine-tuning.

4. The method according to claim 3, characterized in that, The process of fine-tuning the base model using the structured training sample set, and adapting the model to the characteristics of the precious metals market through transfer learning and dynamic learning rate adjustment, to generate a dedicated prediction model, includes: Load a financial time series base model with generalization capabilities, freeze some parameters of its underlying Transformer encoder, retain its learned general time series representation capabilities, and generate a fine-tuned initial model with partially frozen parameters. The structured training sample set is input into the initial model for fine-tuning. A transfer learning strategy is adopted to add a fully connected layer adapted for precious metal prediction at the top layer of the model. Only the newly added layer and some high-level attention layers are trained to generate the initial fine-tuned model. A dynamic learning rate adjustment strategy is implemented. Based on the loss changes of the model on the precious metal validation set, the cosine annealing algorithm is used to dynamically adjust the learning rate of different network layers, so as to promote the efficient convergence of the initial fine-tuned model to the precious metal data distribution and generate the optimized fine-tuned model. The prediction accuracy and generalization ability of the optimized fine-tuned model are evaluated on an independent precious metals test set. The model parameters are further fine-tuned through backpropagation algorithm to minimize the prediction error, and finally a special prediction model for the precious metals market is generated.

5. The method according to claim 4, characterized in that, The process of inputting real-time precious metals market data into the dedicated prediction model and outputting a multi-time-step price prediction sequence and volatility estimate for trading strategy optimization includes: It collects the latest precious metals market data in real time, including real-time quotes, trading volume, and market news flow, and generates real-time standardized feature vectors. The real-time standardized feature vectors are input into a dedicated prediction model. The model captures long-term dependencies through its Transformer encoder and performs multi-step forward inference via a fine-tuned prediction head to generate a price point prediction sequence for multiple future time steps. Based on the internal attention weight matrix and hidden layer state of the dedicated prediction model, Monte Carlo Dropout technique is used to perform multiple forward inferences, calculate the standard deviation of the prediction results to estimate the volatility of future prices, and generate a volatility estimation sequence. Integrating future price point prediction sequences and volatility estimation sequences across multiple time steps, the system outputs a structured forecast report containing predicted values, confidence intervals, and risk measures to optimize trading strategy parameters and risk control logic.

6. A precious metal price trend prediction system based on a Transformer architecture pre-trained model, characterized in that, The system includes: The module is used to build a Transformer pre-trained model based on historical foreign exchange market data. Through self-supervised learning and multi-task training, the model learns general financial time series patterns and generates a base model with generalization ability. The historical foreign exchange market data includes at least foreign exchange prices, trading volume, and market sentiment indicators. The extraction module is used to extract and standardize multi-dimensional features from historical data of the precious metals market, and to construct a structured training sample set containing price series, technical indicators and market sentiment factors. The historical data of the precious metals market includes at least the price, trading volume and market sentiment indicators of precious metal varieties. The adjustment module is used to fine-tune the base model using the structured training sample set, and adapt the model to the characteristics of the precious metals market through transfer learning and dynamic learning rate adjustment to generate a dedicated prediction model. The output module is used to input real-time precious metals market data into the dedicated prediction model and output a price prediction sequence and volatility estimate for future multiple time steps for trading strategy optimization.

7. The system according to claim 6, characterized in that, The building module is specifically used for: Historical data of the foreign exchange market were collected from multiple international financial data sources, including price series, trading volume series and market sentiment indicators of major currency exchanges, to generate the original foreign exchange multimodal dataset. The original foreign exchange multimodal dataset is cleaned and aligned, outliers and missing values ​​are removed, and data of different frequencies are uniformly interpolated to the same timestamp to generate a cleaned and aligned foreign exchange time series dataset. Based on the cleaned and aligned foreign exchange time series dataset, a self-supervised learning task is designed. The time series masking reconstruction method is used to randomly mask some time steps of the data, forcing the model to learn contextual dependencies to predict the masked values, and generating a self-supervised training task set. A multi-task training framework is constructed. Based on the self-supervised reconstruction task, two auxiliary tasks, foreign exchange price direction prediction and volatility estimation, are added. An adaptive weight algorithm is used to balance the losses of different tasks. The Transformer model is trained until convergence, and finally a financial time series base model with generalization ability is generated.

8. The system according to claim 7, characterized in that, The extraction module is specifically used for: Historical data from the precious metals market was collected, including London gold and silver fixing prices, minute-level candlestick data for futures, ETF trading volume, and news sentiment analysis scores and social media sentiment indices extracted from financial news and social media, to generate a raw multi-source dataset of precious metals. Multi-dimensional features are extracted from the original precious metal multi-source dataset. Technical indicators, including at least moving averages, relative strength index, and Bollinger Bands, are calculated based on price and volume sequences. Sentiment dictionary features and topic distribution features are extracted from text data to generate the original feature pool. All features in the original feature pool are standardized and stabilized. The Z-score method is used to remove the dimensions, and the price series is log-differenced to meet the stationarity requirement, generating a standardized feature matrix. The standardized feature matrix is ​​organized into continuous time window samples in chronological order. Each sample contains features from the past N time steps as input and the real prices from the next M time steps as the prediction target. A structured training sample set is constructed for model fine-tuning.

9. A storage medium, characterized in that, The storage medium stores a computer program, wherein the computer program is configured to execute the method of any one of claims 1-5 when it is run.

10. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the method of any one of claims 1-5.

Images