Transaction risk early warning method and system based on big data and periodic function

Abstract

The invention provides a transaction risk early warning method and system based on big data and a periodic function, and relates to the technical field of transaction risk early warning, and the transaction risk early warning method based on the big data and the periodic function comprises the steps: obtaining historical transaction data, related macroeconomic data and related text data of commodity futures; extracting periodic features, non-periodic factors and key features of the commodity future price sequence; constructing a periodic function model and a risk early warning model, and inputting the periodic function model and the aperiodic factors into the risk early warning model; accessing real-time transaction data, related macroeconomic data and related text data; performing risk assessment on the real-time data; and when the evaluation result exceeds a preset risk threshold value, outputting an early warning signal. The method not only can well fit the periodicity of the commodity futures, but also can accurately and timely analyze the influence of various factors on the value of the commodity futures based on big data.

Description

Technical Field

[0001] The present invention relates to the technical field of trading risk early warning, and particularly relates to a trading risk early warning method and system based on big data and periodic functions. Background Art

[0002] Commodity futures are a type of standardized contract that allows both buyers and sellers to buy or sell a specific commodity at a certain price at a specific future time. Such contracts are traded on organized futures exchanges, which provide a tool for producers, processors, distributors, and investors of commodities to manage price risks and hedge. There are many types of commodity futures, generally including agricultural product futures (such as grain futures, cash crop futures, oilseed futures, and livestock product futures, etc.), metal futures (such as precious metal futures and industrial metal futures, etc.), and energy futures (such as crude oil futures, natural gas futures, and heating oil futures, etc.).

[0003] For commodity futures, their value has a certain periodicity. However, they are also affected by multiple factors at the same time. That is to say, on the basis of showing certain periodic fluctuations, the value of commodity futures will also suddenly change due to multiple factors. Moreover, futures trading is a type of trading with a relatively high leverage. Therefore, the value fluctuations of commodity futures will significantly affect the gains or losses of investors.

[0004] However, in the related art, the current trading risk early warning scheme for commodity futures is still not perfect, especially in terms of big data processing and periodic fluctuations. It can neither fit well with the periodicity of commodity futures themselves, nor accurately analyze the impact of multiple factors on the value of commodity futures based on big data. Summary of the Invention

[0005] In order to solve the problems in the related art, the present invention provides a trading risk early warning method and system based on big data and periodic functions. The trading risk early warning method based on big data and periodic functions of the present invention can not only fit well with the periodicity of commodity futures themselves, but also can more accurately and timely analyze the impact of multiple factors on the value of commodity futures based on big data, providing accurate and timely early warnings for investors.

[0006] To achieve the above object, the technical solutions adopted by the present invention include:

[0007] According to the first aspect of the present invention, there is provided a trading risk early warning method based on big data and periodic functions, including the following steps:

[0008] Step S1: Obtaining historical transaction data, relevant macroeconomic data and relevant text data of commodity futures, wherein the historical transaction data includes the opening price, closing price, highest price, lowest price, trading volume and open interest of the commodity futures, the relevant macroeconomic data includes agricultural policies, climate conditions of production areas and supply and demand conditions related to the commodity futures, and the relevant text data includes agricultural news, market reports and emergencies related to the commodity futures;

[0009] Clean the acquired data to obtain basic transaction data;

[0010] Step S2: extracting the periodic features of the commodity futures price series in the basic transaction data; extracting the non-periodic factors that affect the commodity futures prices; extracting the key features in the basic transaction data, the key features including changes in agricultural policies, changes in climatic conditions in production areas, changes in supply and demand, changes in agricultural news, changes in market reports, and changes in emergencies;

[0011] Step S3: constructing a periodic function model based on the periodic features extracted in step S2; evaluating the degree of fit of the periodic function model to the commodity futures price fluctuations;

[0012] Step S4: constructing a risk warning model using a long short-term memory network, and inputting the periodic function model and non-periodic factors into the risk warning model;

[0013] Use historical data to train risk warning models, evaluate model performance, adjust model parameters, and optimize warning effects;

[0014] Step S5: Connect the real-time transaction data of commodity futures, relevant macroeconomic data and relevant text data to the risk warning model; use the risk warning model to conduct risk assessment on the real-time data;

[0015] When the assessment result exceeds the preset risk threshold, a warning signal is output.

[0016] Optionally, the data cleaning of the acquired data specifically includes:

[0017] Step S1-1: De-noising the historical transaction data, removing abnormal values ​​and erroneous values, and then dividing the remaining data into time windows to obtain first transaction data for periodic analysis;

[0018] Step S1-2: Standardize the relevant macroeconomic data and the relevant text data to obtain second transaction data that can be used for model analysis.

[0019] Optionally, the step S1 further includes:

[0020] Step S1-0-1: Check the authority and reliability of the data source; and eliminate untrustworthy data;

[0021] Step S1-0-2: Check whether there are missing values, outliers or duplicate records in the dataset; and fill in the missing values, eliminate the outliers and duplicate records;

[0022] Step S1-0-3: Check the formats and types of all data items; and unify the formats and types of the data items;

[0023] Step S1-0-4: Check the internal logical relationships of all data; and eliminate the data that does not conform to the internal logical relationships;

[0024] Step S1-0-5: Verify whether the data conforms to the expected distribution type, and eliminate the data that does not conform to the expected distribution type;

[0025] Step S1-0-6: Check the continuity of the timestamps of the time series data, and adjust the time series data so that the timestamps of the time series data are continuous and there are no time jumps;

[0026] Step S1-0-7: For multi-source data, check the synchronization between different data sources to ensure data consistency.

[0027] Optionally, step S1 further includes:

[0028] Step S1-0-8: Compile a data verification report, recording the verification process, the problems found and their handling methods.

[0029] Optionally, step S3 specifically includes:

[0030] Step S3-1: Gaussian process regression model construction: P t ~GP(m(t),k(t,t ′ ))

[0031] In the formula, P t is the actual price, GP(·) is the Gaussian process, t and t ′ are different time variables, m(t) is the mean function, k(t,t ′ ) is the covariance function; where,

[0032] m(t)=β0+β1cos(2πft)+β2cos(2πft)

[0033]

[0034] In the formula, β0 is the constant term, β1 and β2 are the periodic coefficients, f is the fundamental frequency, σ 2 is the variance, and l is the length scale parameter;

[0035] Step S3-2: Estimate the model parameters σ 2 , l, β0, β1, and β2 by the maximum likelihood estimation method;

[0036] Step S3-3: Use the Gaussian process regression model to fit the commodity futures price series to obtain the predicted values and prediction uncertainties;

[0037] Step S3-4: Evaluation of the fitting degree:

[0038]

[0039] where P t is the actual price, is the predicted price, N is the number of data points, MSE is the mean squared error, and RMSE is the root mean squared error.

[0040] Optionally, the step S4 specifically includes:

[0041] Step S4-1: Design the long short-term memory network structure;

[0042] Determine the number of network layers and the number of neurons in each layer; select the activation function; determine the dropout ratio;

[0043] Step S4-2: Use the periodic features and non-periodic factors output by the periodic function model as input features;

[0044] Step S4-3: Define the internal mechanism of the long short-term memory network unit using the following formula:

[0045] Forget gate: F t = σ(W f ·[h t-1 , x t + b f ), where F t is the output of the forget gate at time step t, σ(·) is the activation function, W f is the weight matrix of the forget gate, h t-1 is the hidden state of the previous time step, x t is the input of the current time step, b f is the bias term of the forget gate, and [h t-1 , x t represents the vector obtained by concatenating the previous hidden state and the current input;

[0046] Input gate: i t = σ(W i ·[h t-1 , x t + b i ), where i tis the input gate output at time step t, W i is the weight matrix of the input gate, b i is the bias term of the input gate;

[0047] Output gate: o t = σ(W o · [h t-1 , x t + b o ), where o t is the output gate output at time step t, W o is the weight matrix of the output gate, b o is the bias term of the output gate;

[0048] Cell state candidate value: where is the cell state candidate value at time step t, tanh(·) is the hyperbolic tangent function, W C is the weight matrix of the cell state candidate value, b C is the bias term of the cell state candidate value;

[0049] Cell state update: where C t is the cell state at time step t, C t-1 is the cell state at the previous time step;

[0050] Hidden state output: h t = o t · tanh(C t ), where h t is the hidden state output at time step t;

[0051] Step S4-4: Add a fully connected layer for classification or regression prediction:

[0052] y t = W d · h t + b d

[0053] where y t is the output of the fully connected layer at time step t, W d is the weight of the fully connected layer, b d is the bias term of the fully connected layer;

[0054] Step S4-5: Calculate the loss function according to the following formula:

[0055] L A = μ1 · L B + μ2 · L C

[0056]

[0057] Wherein, μ1 and μ2 are weight coefficients, and L B is the mean square error loss function, and L C is the cross-entropy loss function, y t is the actual output, is the predicted output;

[0058] Select the RMSprop optimizer and set the learning rate;

[0059] Step S4-6: Train the model using historical data and update the weights through the following formula:

[0060]

[0061] Wherein, W new is the updated weight, W old is the weight before update, α is the learning rate, is the gradient of the loss function with respect to the weight;

[0062] Step S4-7: Evaluate the performance of the model using the validation set, and calculate the accuracy, recall rate, and F1 score;

[0063] Step S4-8: Adjust the network structure, learning rate, and dropout ratio according to the performance feedback of the validation set.

[0064] Optionally, the specific steps of step S5 include:

[0065] Step S5-1: Use a data collection tool to collect data from the data source in real time;

[0066] Step S5-2: Clean, format, and perform preliminary analysis on the data through a stream processing framework;

[0067] Step S5-3: Use a message queue to ensure real-time synchronization of data between processing nodes;

[0068] Step S5-4: Update the data storage system regularly or based on event triggers;

[0069] Step S5-5: Design the form of the warning signal according to the habits and urgency of the recipients; and customize message templates according to different types of warnings; automatically push the warning signal;

[0070] Step S5-6: After the warning is sent, collect the feedback from the recipients and perform statistical analysis on the feedback data to evaluate the accuracy of the warning signal, and adjust the risk assessment model and warning strategy according to the feedback results.

[0071] According to a second aspect of the present invention, there is also provided a trading risk warning system based on big data and periodic functions, which is applied to the trading risk warning method based on big data and periodic functions described in any one of the technical solutions of the first aspect of the present invention, and includes:

[0072] A data acquisition and cleaning module, configured to acquire historical trading data, relevant macroeconomic data, and relevant text data of commodity futures; and configured to perform data cleaning on the acquired data;

[0073] A data processing module, configured to extract the periodic characteristics of the commodity futures price series in the basic trading data; and configured to extract the non-periodic factors affecting the commodity futures price; and configured to extract the key characteristics in the basic trading data;

[0074] A periodic function model construction module, configured to construct a periodic function model according to the extracted periodic characteristics; and configured to evaluate the fitting degree of the periodic function model to the commodity futures price fluctuation;

[0075] A risk warning model construction module, configured to construct a risk warning model according to a long short-term memory network;

[0076] A real-time data access module, configured to access real-time trading data, relevant macroeconomic data, and relevant text data of commodity futures;

[0077] A warning module, configured to output a warning signal externally.

[0078] According to a third aspect of the present invention, there is also provided a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the computer program, it can implement the steps of the trading risk warning method based on big data and periodic functions described in any one of the technical solutions of the first aspect of the present invention.

[0079] According to a fourth aspect of the present invention, there is also provided a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it can implement the steps of the trading risk warning method based on big data and periodic functions described in any one of the technical solutions of the first aspect of the present invention.

[0080] Beneficial effects:

[0081] 1. Through the above technical solutions, the method of the present invention collects periodic data related to the periodicity of the value of commodity futures (specifically, analyzes the periodicity of the value of commodity futures through historical trading data) and aperiodic data related to the mutation of the value of commodity futures (specifically, analyzes the mutation of the value of commodity futures through relevant macroeconomic data and relevant text data). At the same time, by inputting the periodic function model constructed with periodic characteristics and aperiodic factors into the risk warning model, a risk warning model that fits the actual change of the value of commodity futures is trained and obtained. Then, by connecting real-time trading data, relevant macroeconomic data, and relevant text data to the trained risk warning model, in this way, the established risk warning model can not only fit well with the periodicity of commodity futures itself, but also can analyze the accurate impact of various factors on the value of commodity futures based on big data. Thus, not only can timely and accurate warning information be provided for investors to help them effectively avoid market risks and reduce losses, but also it can promote the stable development of the financial market to a certain extent.

[0082] 2. Other beneficial effects or advantages of the present invention will be described in detail in the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or in the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those skilled in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0084] Wherein:

[0085] Figure 1 is a schematic flow chart of the steps of a trading risk warning method based on big data and periodic functions provided by an exemplary embodiment of the present invention. SPECIFIC IMPLEMENTATION MANNER

[0086] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention.

[0087] Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0088] In addition, the terms "including" and "having" and any variations thereof mentioned in the description of the present invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally further include other unlisted steps or units, or may optionally further include other steps or units inherent to these processes, methods, products, or devices. It should also be noted that in the embodiments of the present invention, words such as "exemplary" or "for example" are used to represent examples, illustrations, or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present invention should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present relevant concepts in a specific manner.

[0089] To facilitate a clearer and more accurate understanding of the technical solution of the present invention by relevant technical personnel, the reasons for the periodic changes in the value of commodity futures and the reasons for sudden changes in value will be described below taking agricultural product futures as an example.

[0090] For agricultural product futures, the main reasons for the periodic changes in their value mainly include:

[0091] First, the influence of the growth cycle. Crops (such as wheat, corn, and soybeans, etc.) have specific growth cycles, and it takes a certain amount of time from sowing, harvesting to consumption. This entire period of time can be divided into sowing season time period, growing season time period, harvesting season time period, and consumption season time period.

[0092] Among them, during the sowing season time period, the price of agricultural product futures may rise due to market uncertainty about the yield of new crops and the reduction of current supply. During the growing season time period, the price of agricultural product futures may fluctuate due to market expectations of the growth conditions of crops. Forecasts of adverse weather may lead to price increases, while favorable weather may lead to price decreases. During the harvesting season time period, the price of agricultural product futures may fall due to the listing of new crops. However, if the actual yield is lower than expected, the futures price may remain stable or increase. During the consumption season time period, the price of certain agricultural product futures may rise due to the increase in expected demand (for example, during the Chinese Spring Festival, food manufacturing raw materials related to the festival, such as soybeans).

[0093] Second, the influence of seasonal weather changes. For agricultural product futures, the price in the futures market is very sensitive to seasonal weather changes. For example, if the forecast shows that there will be weather unfavorable to crop growth, the futures price may rise due to the expected reduction in supply. On the contrary, if the weather conditions are good and favorable to crop growth, the futures price may fall.

[0094] Third, the impact of the economic cycle. For agricultural futures, changes in economic growth and consumer income will affect the demand for agricultural products, thereby affecting futures prices. During the economic expansion period, futures prices may rise; during the economic recession period, futures prices may fall.

[0095] In addition, for agricultural futures, the main reasons for sudden changes in their value mainly include changes in supply and demand relationships (changes in output, such as crop production reduction caused by natural disasters; changes in demand, such as increased demand for agricultural products in emerging markets or sudden growth in demand in the original market), policy changes (changes in trade policies, such as changes in trade barriers, tariff adjustments, export restrictions or import quotas, etc.; changes in subsidy policies, such as changes in the government's subsidy policies for agricultural products), changes in market sentiment (speculative behavior, such as expectations of future market trends by investors may trigger speculative trading, leading to price fluctuations; information dissemination, such as the market's reaction to new information, such as weather forecasts, production forecasts, policy changes, etc.), and so on.

[0096] In the existing related technologies, especially in terms of big data processing and periodic fluctuations, it is neither possible to well fit the periodicity of commodity futures themselves, nor is it easy to accurately analyze the impact of multiple factors on the value of commodity futures based on big data. Therefore, the present invention proposes a brand-new technical solution, that is, a trading risk warning method based on big data and periodic functions, to solve the above problems and provide more accurate and timely warnings for investors.

[0097] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings.

[0098] Please refer to Figure 1 , according to the first aspect of the present invention, this embodiment provides a trading risk warning method based on big data and periodic functions, including the following steps:

[0099] Step S1: Obtain the historical trading data, relevant macroeconomic data and relevant text data of the commodity futures. Among them, the historical trading data includes the opening price, closing price, highest price, lowest price, trading volume and open interest of the commodity futures, the relevant macroeconomic data includes agricultural policies, local climate conditions and supply and demand situations related to the commodity futures, and the relevant text data includes agricultural news, market reports and emergencies related to the commodity futures;

[0100] Clean the obtained data to obtain the basic trading data;

[0101] Step S2: Extract the periodic characteristics of the commodity futures price series in the basic transaction data; extract the non-periodic factors affecting the commodity futures price; extract the key features in the basic transaction data, where the key features include changes in agricultural policies, changes in production area climate conditions, changes in supply and demand, changes in agricultural news, changes in market reports, and changes in emergencies;

[0102] Step S3: Construct a periodic function model based on the periodic characteristics extracted in Step S2; evaluate the fitting degree of the periodic function model to the fluctuations of the commodity futures price;

[0103] Step S4: Construct a risk warning model with a long short-term memory network, and input the periodic function model and non-periodic factors into the risk warning model;

[0104] Use historical data to train the risk warning model, evaluate the model performance, adjust the model parameters, and optimize the warning effect;

[0105] Step S5: Connect the real-time transaction data, relevant macroeconomic data, and relevant text data of the commodity futures to the risk warning model; use the risk warning model to conduct a risk assessment on the real-time data;

[0106] When the evaluation result exceeds the preset risk threshold, output a warning signal.

[0107] Through the above technical solution, the method of the present invention collects periodic data related to the periodicity of the commodity futures value (specifically, analyzes the periodicity of the commodity futures value through historical transaction data) and non-periodic data related to the mutation of the commodity futures value (specifically, analyzes the mutation of the commodity futures value through relevant macroeconomic data and relevant text data). At the same time, by inputting the periodic function model constructed based on the periodic characteristics and non-periodic factors into the risk warning model, a risk warning model that fits the actual change of the commodity futures value is trained. Then, by connecting the real-time transaction data, relevant macroeconomic data, and relevant text data to the trained risk warning model, in this way, the established risk warning model can not only fit well with the periodicity of the commodity futures itself, but also can analyze the accurate impact of various factors on the commodity futures value based on big data. Thus, it can not only provide timely and accurate warning information for investors to help them effectively avoid market risks and reduce losses, but also can promote the stable development of the financial market to a certain extent.

[0108] In addition, it can be understood that, first, the trading risk warning method based on big data and periodic functions of the present invention can not only be applied to the agricultural product futures market, but also to other futures markets with obvious periodic fluctuations, such as energy futures or metal futures. The present invention does not make specific limitations on this.

[0109] Second, based on the solution disclosed in the present invention, the parameters and structure of the periodic function model and risk warning model involved in the method can be adjusted according to different actual conditions and market characteristics to adapt to the risk warning needs of different financial products.

[0110] In one embodiment of the present invention, in step S2, data cleaning of the acquired data may specifically include:

[0111] Step S1-1: De-noising the historical transaction data, removing abnormal values ​​and erroneous values, and then dividing the remaining data into time windows to obtain first transaction data for periodic analysis;

[0112] Step S1-2: Standardize the relevant macroeconomic data and the relevant text data to obtain second transaction data that can be used for model analysis.

[0113] In this way, the steps of data denoising and elimination of outliers and erroneous values ​​are introduced, which can effectively ensure the accuracy and reliability of the acquired data. At the same time, the historical transaction data with obvious periodic characteristics and the related macroeconomic data and related text data without obvious periodic characteristics are processed separately, which can better perform periodic analysis on the first transaction data and model analysis on the second transaction data, so as to ensure the accuracy and reliability of data analysis.

[0114] In this embodiment, it is understood that the acquisition of historical transaction data can adopt the API interface provided by the futures exchange to obtain accurate and comprehensive historical transaction data of commodity futures. In addition, for relevant macroeconomic data, the macroeconomic data interface provided by the third-party data service provider can be used to obtain it. For relevant text data, technologies such as web crawlers can be used to collect and summarize.

[0115] In one embodiment of the present invention, step S1 of the present invention may further include:

[0116] Step S1-0-1: Check the authority and reliability of the data source; and eliminate unreliable data; in this embodiment, the authority of the data source means that the data is from an official exchange or an authoritative data service provider to ensure the authenticity and authority of the data and reduce information bias.

[0117] Step S1-0-2: Check whether there are missing values, outliers or duplicate records in the data set; fill in the missing values, remove outliers and duplicate records; in this way, the integrity and accuracy of the data set can be effectively improved, and excessive deviations in subsequent data processing can be avoided.

[0118] Step S1-0-3: Check the formats and types of all data items; and unify the formats and types of data items; in this way, the data formats and types can be effectively unified, facilitating subsequent processing and analysis, and avoiding data conversion errors.

[0119] Step S1-0-4: Check the internal logical relationships of all data; and eliminate the data that does not conform to the internal logical relationships; in this way, the logical rationality of the data can be effectively ensured, improving the data quality. For example, the trading volume should not be less than 0, and the price fluctuation should be within a reasonable range (it will not occur that the price is small at the first moment, huge at the second moment, and small again at the third moment).

[0120] Step S1-0-5: Verify whether the data conforms to the expected distribution type, and eliminate the data that does not conform to the expected distribution type; in this implementation, statistical methods (such as descriptive statistics, distribution test, etc.) can be used to analyze the basic statistical characteristics of the data, and check whether the data conforms to the expected statistical distribution, such as normal distribution, uniform distribution, etc., so that the anomalies in the data distribution can be effectively identified, providing a more accurate data basis for subsequent data processing (such as modeling).

[0121] Step S1-0-6: Check the continuity of the timestamps of the time series data, and adjust the time series data so that the timestamps of the time series data are continuous and there are no time jumps; in this implementation, for time series data, it is possible to ensure the correct order of the time series data, without inversion or confusion, by checking whether its timestamps are continuous and whether there are time jumps, and then ensure the continuity and sequentiality of the time series data for more accurate periodic analysis and trend prediction.

[0122] Step S1-0-7: For multi-source data, check the synchronization between different data sources to ensure data consistency. In this way, the consistency of multi-source data can be ensured, avoiding analysis errors caused by data asynchronization.

[0123] In summary, through this implementation, it is possible to effectively ensure that the acquired data is accurate and reliable before being used for model construction, thereby improving the overall performance and prediction accuracy of the trading risk warning model.

[0124] In an implementation of the present invention, step S1 of the present invention may further include:

[0125] Step S1-0-8: Compile a data verification report, recording the verification process, the problems found and their handling methods.

[0126] In this way, the process of data verification can be accurately and completely recorded, and the documentation record (data verification report) of data verification can be provided at any time for subsequent tracking and auditing.

[0127] In one embodiment of the present invention, step S3 of the present invention may specifically include:

[0128] Step S3-1: Construction of Gaussian process regression model:

[0129] Assume that the commodity futures value sequence P t can be modeled by a Gaussian process, then this process can be expressed as:

[0130] P t ~GP(m(t), k(t, t ′ ))

[0131] In the formula, P t is the actual price, GP(·) is the Gaussian process, t and t ′ are different time variables, m(t) is the mean function, k(t, t ′ ) is the covariance function, which is used to describe the correlation between different time points t and t ′ .

[0132] For the above Gaussian process, a suitable mean function m(t) should be selected to be applicable to the periodic characteristics. The mean function selected in the present invention is:

[0133] m(t) = β0 + β1cos(2πft) + β2cos(2πft)

[0134] In the formula, β0 is the constant term, β1 and β2 are the periodic coefficients, and f is the fundamental frequency.

[0135] In the present invention, the Gaussian function is selected as the covariance function k(t, t ′ ) to describe the correlation between different time points. Among them, the covariance function k(t, t ′ ) can be expressed as:

[0136]

[0137] In the formula, σ 2 is the variance, and l is the length scale parameter, which is used to control the smoothness of the function.

[0138] Step S3-2: Then, the model parameters, including σ 2 , l, β0, β1 and β2, can be estimated by the maximum likelihood estimation (MLE) method. In this way, the optimal parameter values can be found by optimizing the log marginal likelihood function.

[0139] Step S3-3: Subsequently, the Gaussian process regression model can be used to fit the commodity futures price sequence to obtain the predicted values and prediction uncertainties;

[0140] Step S3-4: Finally, the fitting degree of the model can be evaluated by calculating the mean square error or root mean square error between the predicted value and the actual value.

[0141] Specifically, the mean square error MSE can be expressed as:

[0142]

[0143] In the formula, P t is the actual price, is the predicted price, and N is the number of data points.

[0144] The root mean square error RMSE can be expressed as:

[0145]

[0146] In this way, through the above steps, a periodic function model can be constructed to accurately and effectively capture the periodic characteristics of the commodity futures price series and provide important inputs for the risk warning model.

[0147] In addition, it should be noted that in this embodiment, the Gaussian process regression model is adopted because it can provide an estimate of the uncertainty of the prediction, which is convenient for the subsequent risk warning model to more accurately combine real-time data for price trend prediction.

[0148] In an embodiment of the present invention, step S4 of the present invention may specifically include:

[0149] Step S4-1: Design a long short-term memory (LSTM) network structure;

[0150] Determine the number of network layers and the number of neurons in each layer;

[0151] Select an appropriate activation function. For example, ReLU or Tanh;

[0152] Determine the dropout ratio to prevent overfitting.

[0153] In this way, a neural network structure capable of capturing the long-term dependence relationship of time series data can be established.

[0154] Step S4-2: Define the input layer of the periodic function model and the non-periodic factors:

[0155] Take the periodic characteristics and non-periodic factors output by the periodic function model as input features;

[0156] In this way, it can be ensured that the model can simultaneously consider the influences of periodic and non-periodic factors to achieve more accurate data prediction.

[0157] Step S4-3: Construct an LSTM network:

[0158] Define the internal mechanism of the LSTM cell using the following formula:

[0159] Forget gate: F t = σ(W f · [h t-1 , x t + b f ), where F t is the output of the forget gate at time step t, σ(·) is the activation function, W f is the weight matrix of the forget gate, h t-1 is the hidden state of the previous time step, x t is the input of the current time step, b f is the bias term of the forget gate, and [h t-1 , x t represents the vector obtained by concatenating the previous hidden state and the current input;

[0160] Input gate: i t = σ(W i · [h t-1 , x t + b i ), where i t is the output of the input gate at time step t, W i is the weight matrix of the input gate, and b i is the bias term of the input gate;

[0161] Output gate: o t = σ(W o · [h t-1 , x t + b o ), where o t is the output of the output gate at time step t, W o is the weight matrix of the output gate, and b o is the bias term of the output gate;

[0162] Cell state candidate value: where is the cell state candidate value at time step t, tanh(·) is the hyperbolic tangent function, W C is the weight matrix of the cell state candidate value, and b C is the bias term of the cell state candidate value;

[0163] Cell state update: where C t is the cell state at time step t, C t-1is the unit state at the previous time step;

[0164] Hidden state output: h t = o t ·tanh(C t ), where h t is the hidden state output at time step t;

[0165] In this way, through the complex gating mechanism of the LSTM unit, the model can learn the long-term dependencies in the data to achieve a better fit to the periodic changes and aperiodic mutations in the value of commodity futures, thus enabling more accurate predictions.

[0166] Step S4-4: Add a fully connected layer:

[0167] Add a fully connected layer after the LSTM layer for classification or regression prediction:

[0168] y t = W d ·h t + b d

[0169] where y t is the output of the fully connected layer at time step t, W d is the weight of the fully connected layer, and b d is the bias term of the fully connected layer;

[0170] to convert the output of the LSTM layer into the final prediction result.

[0171] Step S4-5: Define the loss function and optimizer:

[0172] Combine the mean squared error loss function and the cross-entropy loss function to establish a comprehensive loss function. Specifically, the loss function can be:

[0173] L A = μ1·L B + μ2·L C

[0174]

[0175] where μ1 and μ2 are weight coefficients, L B is the mean squared error loss function, L C is the cross-entropy loss function, y t is the actual output, is the predicted output;

[0176] Select the RMSprop optimizer (alternatively, the Adam optimizer can also be selected), and set the learning rate;

[0177] In this way, through the loss function designed in this manner, the prediction accuracy of the model can be further effectively improved.

[0178] Step S4-6: Model training:

[0179] Use historical data to train the model and update the weights through the following formula:

[0180]

[0181] In the formula, W new is the updated weight, W old is the weight before update, α is the learning rate, is the gradient of the loss function with respect to the weight;

[0182] In this way, through backpropagation and weight update, the constructed model can learn from the data and gradually improve its prediction ability.

[0183] Step S4-7: Evaluate model performance:

[0184] Use the validation set to evaluate the performance of the model and calculate metrics such as accuracy, recall, and F1 score;

[0185] To ensure that the model has good generalization ability and can perform well on unknown data.

[0186] Step S4-8: Adjust model parameters:

[0187] According to the performance feedback of the validation set, adjust parameters such as the network structure, learning rate, and dropout ratio.

[0188] In this way, through parameter adjustment, the prediction effect of the model can be effectively optimized to reduce overfitting or underfitting.

[0189] Through the above steps, step S4 of the present invention can construct a model that can comprehensively consider periodic and non-periodic factors and can effectively improve the accuracy and effectiveness of commodity futures trading risk warning.

[0190] In an embodiment of the present invention, step S5 of the present invention may specifically include:

[0191] Step S5-1: Use a data collection tool (e.g., Flume or Kafka) to collect data from the data source in real time;

[0192] Step S5-2: Through a stream processing framework (e.g., Apache Flink or Spark Streaming), clean, format, and perform preliminary analysis on the data;

[0193] Step S5-3: Use a message queue (such as RabbitMQ, Kafka) to ensure real-time synchronization of data among various processing nodes;

[0194] Step S5-4: Update the data storage system regularly or triggered by events; for example, a real-time database or a data warehouse.

[0195] In this way, not only can the real-time nature of data be effectively ensured, minimizing the time delay from data generation to processing and meeting the requirements of real-time monitoring. Moreover, the consistency of data can also be effectively ensured, ensuring the consistency of data among nodes through a synchronization mechanism and avoiding monitoring errors caused by data deviation.

[0196] Step S5-5: Design the form of warning signals according to the habits and urgency of the recipients; for example, email, SMS, application push, mobile phone;

[0197] Customize message templates according to different types of warnings, thus ensuring the effectiveness, accuracy, and clarity of warning information transmission;

[0198] Automatically push warning signals; to ensure that warning signals can be transmitted in a timely manner.

[0199] In this way, it can be ensured that warning signals can be quickly conveyed to relevant personnel, improving the emergency response speed. At the same time, the needs of different users can also be met through multiple notification methods, improving the user experience.

[0200] Step S5-6: After the warning is sent, feedback from the recipients can be collected through methods such as questionnaires and online feedback systems, and the feedback data can be statistically analyzed to evaluate the accuracy of the warning signal. Adjust the risk assessment model and warning strategy according to the feedback results.

[0201] In this way, the model can be continuously optimized through the feedback mechanism, improving the accuracy and reliability of the warning system. Enhancing users' trust in the warning system and increasing the acceptance and usage rate of the system.

[0202] According to the second aspect of the present invention, there is also provided a trading risk warning system based on big data and periodic functions, which is applied to the trading risk warning method based on big data and periodic functions in any one of the technical solutions of the first aspect of the present invention, including:

[0203] A data acquisition and cleaning module, configured to acquire historical trading data, relevant macroeconomic data, and relevant text data of commodity futures; and configured to perform data cleaning on the acquired data;

[0204] A data processing module, configured to extract the periodic characteristics of the commodity futures price series in the basic trading data; and configured to extract non-periodic factors affecting the commodity futures price; and configured to extract key features in the basic trading data;

[0205] A periodic function model construction module, configured to construct a periodic function model according to the extracted periodic features; and configured to evaluate the fitting degree of the periodic function model to the price fluctuation of commodity futures;

[0206] A risk warning model construction module, configured to construct a risk warning model according to a long short-term memory network;

[0207] A real-time data access module, configured to access real-time trading data, relevant macroeconomic data, and relevant text data of commodity futures;

[0208] A warning module, configured to output warning signals externally.

[0209] In this way, through the above technical solution, the trading risk warning system based on big data and periodic functions configured in this way can process periodic data related to the periodicity of commodity futures value and aperiodic data related to the mutation of commodity futures value. At the same time, by inputting the periodic function model constructed based on periodic features and aperiodic factors into the risk warning model to train and obtain a risk warning model that fits the actual change of commodity futures value, and then by accessing real-time trading data, relevant macroeconomic data, and relevant text data into the trained risk warning model, in this way, the established risk warning model can not only fit well with the periodicity of commodity futures itself, but also can analyze the accurate impact of various factors on the value of commodity futures based on big data. In this way, not only can timely and accurate warning information be provided to investors to help investors effectively avoid market risks and reduce losses, but also the stable development of the financial market can be promoted to a certain extent.

[0210] According to the third aspect of the present invention, there is also provided a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it can implement the steps of the trading risk warning method based on big data and periodic functions in any one of the technical solutions in the first aspect of the present invention.

[0211] It can be understood that in this embodiment, the memory may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as read-only memory, flash memory, hard disk, or solid-state drive; in addition, the memory may also include a combination of the above types of memory. The present invention does not make specific limitations thereto.

[0212] Similarly, the processor may implement or execute various exemplary logical steps described in connection with the disclosure of the present invention. The processor may be a central processing unit, a general-purpose processor, a digital signal processor, an application specific integrated circuit, a field programmable gate array or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It may implement or execute various exemplary logical steps described in connection with the disclosure of the present invention. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0213] According to a fourth aspect of the present invention, there is also provided a computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, it can implement the steps of the trading risk warning method based on big data and periodic functions in any one of the technical solutions in the first aspect of the present invention.

[0214] In this embodiment, the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a register, a hard disk, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above, or any other form of computer-readable storage medium well known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium may also be a component of the processor. The processor and the storage medium may be located in an application specific integrated circuit (ASIC). In the embodiments of the present invention, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0215] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A transaction risk early warning method based on big data and periodic functions, characterized in that: The risk warning for commodity futures trading includes the following steps: Step S1: Obtaining historical transaction data, relevant macroeconomic data and relevant text data of commodity futures, wherein the historical transaction data includes the opening price, closing price, highest price, lowest price, trading volume and open interest of the commodity futures, the relevant macroeconomic data includes agricultural policies, climate conditions of production areas and supply and demand conditions related to the commodity futures, and the relevant text data includes agricultural news, market reports and emergencies related to the commodity futures; Clean the acquired data to obtain basic transaction data; Step S2: extracting the periodic features of the commodity futures price series in the basic transaction data; extracting the non-periodic factors that affect the commodity futures prices; extracting the key features in the basic transaction data, the key features including changes in agricultural policies, changes in climatic conditions in production areas, changes in supply and demand, changes in agricultural news, changes in market reports, and changes in emergencies; Step S3: constructing a periodic function model based on the periodic features extracted in step S2; evaluating the degree of fit of the periodic function model to the commodity futures price fluctuations; Step S4: constructing a risk warning model using a long short-term memory network, and inputting the periodic function model and non-periodic factors into the risk warning model; Use historical data to train risk warning models, evaluate model performance, adjust model parameters, and optimize warning effects; Step S5: Connect the real-time transaction data of commodity futures, relevant macroeconomic data and relevant text data to the risk warning model; use the risk warning model to conduct risk assessment on the real-time data; When the assessment result exceeds the preset risk threshold, a warning signal is output.

2. The transaction risk early warning method based on big data and periodic functions according to claim 1 is characterized in that: The data cleaning of the acquired data specifically includes: Step S1-1: De-noising the historical transaction data, removing abnormal values ​​and erroneous values, and then dividing the remaining data into time windows to obtain first transaction data for periodic analysis; Step S1-2: Standardize the relevant macroeconomic data and the relevant text data to obtain second transaction data that can be used for model analysis.

3. The transaction risk early warning method based on big data and periodic functions according to claim 2 is characterized in that: The step S1 further comprises: Step S1-0-1: Check the authority and reliability of the data source; and remove untrustworthy data; Step S1-0-2: Check whether there are missing values, outliers or duplicate records in the data set; fill in missing values ​​and remove outliers and duplicate records; Step S1-0-3: Check the format and type of all data items; and unify the format and type of data items; Step S1-0-4: Check the internal logical relationship of all data; and remove the data that does not conform to the internal logical relationship; Step S1-0-5: Verify whether the data conforms to the expected distribution type, and remove data that does not conform to the expected distribution type; Step S1-0-6: Check the continuity of the timestamps of the time series data, and adjust the time series data so that the timestamps of the time series data are continuous and there is no time jump; Step S1-0-7: For multi-source data, check the synchronization between different data sources to ensure data consistency.

4. The transaction risk early warning method based on big data and periodic functions according to claim 3 is characterized in that: The step S1 further comprises: Step S1-0-8: Write a data verification report to record the verification process, problems found and their solutions.

5. The transaction risk early warning method based on big data and periodic functions according to claim 1 is characterized in that: The step S3 specifically includes: Step S3-1: Gaussian process regression model construction: P t ~GP(m(t),k(t,t ′ )) Where P t is the actual price, GP(·) is the Gaussian process, t and t ′ are different time variables, m(t) is the mean function, k(t,t ′ ) is the covariance function; where, m(t)=β0+β1cos(2πft)+β2cos(2πft) Where β0 is a constant term, β1 and β2 are periodic coefficients, f is the fundamental frequency, and σ 2 is the variance, l is the length scale parameter; Step S3-2: Estimate model parameters σ by maximum likelihood estimation method 2 , l, β0, β1 and β2; Step S3-3: Fit the commodity futures price series using the Gaussian process regression model to obtain the predicted value and prediction uncertainty; Step S3-4: Fit evaluation: Where P t is the actual price, is the predicted price, N is the number of data points, MSE is the mean square error, and RMSE is the root mean square error.

6. The transaction risk early warning method based on big data and periodic functions according to claim 1 is characterized in that: The step S4 specifically includes: Step S4-1: Designing a long short-term memory network structure; Determine the number of network layers and the number of neurons in each layer; select the activation function; determine the dropout ratio; Step S4-2: taking the periodic features and non-periodic factors output by the periodic function model as input features; Step S4-3: Define the internal mechanism of the LSTM network unit using the following formula: Forget Gate: F t =σ(W f ·[h t-1 ,x t ]+b f ), where F t is the output of the forget gate at time step t, σ(·) is the activation function, and W f is the weight matrix of the forget gate, h t-1 is the hidden state of the previous time step, x t is the input of the current time step, b f is the bias term of the forget gate, [h t-1 ,x t ] represents the vector that concatenates the previous hidden state and the current input; Input gate: i t =σ(W i ·[h t-1 ,x t ]+b i ), where i t is the input gate output at time step t, W i is the weight matrix of the input gate, b i is the bias term of the input gate; Output gate: o t =σ(W o ·[h t-1 ,x t ]+b o ), where o t is the output gate output at time step t, W o is the weight matrix of the output gate, b o is the bias term of the output gate; Cell status candidate values: In the formula, is the candidate value of the cell state at time step t, tanh(·) is the hyperbolic tangent function, W C is the weight matrix of the candidate values ​​of the unit state, b C is the bias term of the candidate value of the unit state; Unit status update: In the formula, C t is the cell state at time step t, C t-1 is the cell state at the previous time step; Hidden state output: h t =o t tanh(C t ), where h t is the hidden state output at time step t; Step S4-4: Add a fully connected layer for classification or regression prediction: y t =W d ·h t +b d In the formula, y t is the fully connected layer output at time step t, W d is the weight of the fully connected layer, b d is the bias term of the fully connected layer; Step S4-5: Calculate the loss function according to the following formula: L A =μ1·L B +μ2·L C In the formula, μ1 and μ2 are weight coefficients, L B is the mean square error loss function, L C is the cross entropy loss function, y t is the actual output, is the predicted output; Select the RMSprop optimizer and set the learning rate; Step S4-6: Use historical data to train the model and update the weights using the following formula: Where W new is the updated weight, W old is the weight before update, α is the learning rate, is the gradient of the loss function with respect to the weight; Step S4-7: Use the validation set to evaluate the performance of the model and calculate the precision, recall and F1 score; Step S4-8: Adjust the network structure, learning rate, and dropout ratio based on the performance feedback of the validation set.

7. The transaction risk early warning method based on big data and periodic functions according to claim 1 is characterized in that: The step S5 specifically includes: Step S5-1: using a data collection tool to collect data from a data source in real time; Step S5-2: Clean, format and preliminarily analyze the data through the stream processing framework; Step S5-3: Using message queues to ensure real-time synchronization of data between various processing nodes; Step S5-4: updating the data storage system periodically or based on event triggers; Step S5-5: Design the form of the warning signal according to the habits and urgency of the recipient; customize the message template according to different types of warnings; and automatically push the warning signal; Step S5-6: After the warning is sent, feedback from the recipients is collected and statistical analysis is performed on the feedback data to evaluate the accuracy of the warning signal and adjust the risk assessment model and warning strategy based on the feedback results.

8. A transaction risk early warning system based on big data and periodic functions, characterized in that: The transaction risk early warning method based on big data and periodic functions applied to any one of claims 1 to 7 comprises: Data acquisition and cleaning module, used to acquire historical commodity futures trading data, relevant macroeconomic data and relevant text data; and to clean the acquired data; A data processing module is used to extract the periodic characteristics of the commodity futures price series in the basic transaction data; and to extract the non-periodic factors affecting the commodity futures prices; and to extract the key features in the basic transaction data; A periodic function model building module is used to build a periodic function model based on the extracted periodic features; and to evaluate the degree of fit of the periodic function model to the commodity futures price fluctuations; A risk warning model construction module is used to construct a risk warning model based on the long short-term memory network; Real-time data access module, used to access real-time trading data of commodity futures, relevant macroeconomic data and related text data; The early warning module is used to output early warning signals to the outside.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it can implement the steps of the transaction risk early warning method based on big data and periodic functions described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it can implement the steps of the transaction risk early warning method based on big data and periodic functions described in any one of claims 1 to 7.

Images