Risk early warning method based on financial numeralization Transformer-GPT4 model

Through the risk warning method based on the numerical financial Transformer-GPT4 model, the problem of insufficient data in financial warning and insufficient real-time macro-control information is solved, more efficient financial market warning is achieved, and the accuracy and intelligence of early warning are improved. degree.

CN119027245APending Publication Date: 2024-11-26EAST CHINA NORMAL UNIV
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411239578.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-05
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The existing financial early warning methods rely on historical data analysis, and there are problems such as data inaccuracy and insufficient real-time macro-control information, making it difficult to accurately predict the trends of the financial market.

Method used

Using the risk warning method based on the financial numerical Transformer-GPT4 model, by obtaining financial data from the Internet, performing preprocessing and feature engineering, a convolutional Transformer risk warning model for financial numericals is constructed, and data analysis and model are used for GPT4 large language model. Fine-tuning to achieve real-time risk warning.

Benefits of technology

Simplifies the data preprocessing process, improves the accuracy and intelligence of early warnings, can better adapt to real-time updates of financial multimodal information, enhances the applicability and generalization capabilities of the model, and helps decision makers to better respond. Economic risks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027245A_ABST
    Figure CN119027245A_ABST
Patent Text Reader

Abstract

The invention discloses a risk early warning method based on a financial numeralization Transformer-GPT4 model. The method is realized through the following steps: a) obtaining various financial system data and constructing a historical database; b) preprocessing system data by using a GPT4 large model, and performing feature value generation and feature value dimensionality reduction by using feature engineering; c) constructing a convolutional Transform depth early warning model oriented to financial numerical values and training the convolutional Transform depth early warning model; and d) completing risk early warning based on multi-modal financial data. The method is characterized in that the GPT4 model and the financial value-oriented convolution Transform model are comprehensively applied, so that the data preprocessing process is simplified, the data source types are enriched, and the extraction and early warning precision of financial data features is improved. Compared with a traditional method, the method can dynamically capture the complex dynamic state of the financial market more accurately, provides a more reliable risk early warning result, and provides better decision support for financial institutions and investors.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of software development technology, and in particular to a method based on a financial numerical Transformer-GPT4 model (abbreviated as NUMT model, i.e. Num Eric-Oriented T ransformer-GPT4 Model for FinancialPrediction) risk warning method. Background Art

[0002] In today's society, people are very interested in economic early warning and analysis, especially quantitative trading analysis based on artificial intelligence. In the context of the evolution of the large asset management industry towards net value and technology, investors are paying more and more attention to quantitative strategies. According to the statistics of a securities company in recent years, the total scale of quantitative funds in the current market is about hundreds of billions. However, the impact of macroeconomic regulation on the financial trend is also crucial. Various crises and rapidly changing macroeconomic conditions have made the economic outlook extremely uncertain. Under the current circumstances, all countries have introduced a series of economic stimulus policies, so macroeconomic regulation information is a crucial link for the subsequent economic development.

[0003] In terms of the development of artificial intelligence technology, the advent of ChatGPT has led to the explosion of global language models in recent years and ignited the craze for generative AI. The number of visitors to the OpenAI website has reached nearly 1 billion, equivalent to more than 10% of the world's population, and has increased six times compared to the data just after it went online. The advent of ChatGPT has not only played an important role in the development of artificial intelligence, but also promoted the improvement of productivity and brought new impetus to the economic development of various countries. It has a very broad market prospect and will give birth to a number of new models and new formats. Therefore, its impact is profound and extensive.

[0004] At present, many financial early warning work relies on historical data analysis. However, many financial databases have a certain time difference compared to the current ones, and in actual economic life, major events or regulatory policy changes often seriously affect the future direction of the economy or the stock market. Therefore, extracting relevant financial data from real-time text information and then using fine-tuning mechanisms to optimize the early warning model will greatly improve the accuracy of the early warning results. Fortunately, with the emergence of technologies such as ChatGPT, text processing and analysis capabilities have been greatly improved. Through these technologies, macro-control information, news reports, and social opinions can be analyzed and understood more conveniently and intelligently, and the model can be fine-tuned in time, so as to more accurately warn of financial market trends. These text understanding technologies can be regarded as influencing factors that supplement the traditional financial early warning model, so that the early warning direction of the financial model can be adjusted to optimize the early warning results of the traditional financial model.

[0005] The GPT series of models are large language models based on the Transformer architecture. Their core idea is to learn language rules and knowledge from a large amount of text data through pre-training and fine-tuning. The GPT model uses a self-attention mechanism, which allows them to capture long-range dependencies, thereby achieving powerful generation capabilities.

[0006] The Transformer architecture is the basis of the GPT series of models. It uses a self-attention mechanism, which allows the model to capture long-range dependencies. The Transformer architecture is mainly composed of the following components: Input encoder, which converts the input text into a fixed-length vector sequence. Self-attention mechanism, which is used to calculate the importance of each word in the sequence to generate a context vector. Position encoding, in order to capture position information, the position encoding is added to the input vector. Multi-head attention, through parallel calculation of multiple attention heads, improves the expressiveness of the model. Feedforward neural network, in order to capture more complex grammatical and semantic rules, a multi-layer perceptron is used as a subsequent hierarchical structure.

[0007] In the Transformer architecture, the self-attention mechanism is a key component. The calculation formula of the self-attention mechanism is as follows:

[0008]

[0009] Where Q, K, and V represent query vector, key vector, and value vector, respectively. k Represents the dimension of the key vector. The softmax function is used for normalization so that the importance of each word in the sequence is balanced.

[0010] Then, the training process of GPT can be divided into the following steps:

[0011] 1. Pre-training: Use a large amount of text data for unsupervised learning to allow the model to learn language rules and knowledge.

[0012] 2. Fine-tuning: Use conversation data for supervised learning to adapt the model to conversation tasks.

[0013] 3. Evaluation: Use conversation data to evaluate the performance of the model.

[0014] In summary, the research on financial early warning and analysis is heating up, but there are still many problems and challenges in terms of the impact of macro-control information on financial markets and the inaccuracy of data. Therefore, there is an urgent need for a method that can better analyze multimodal information through advanced technologies such as large models and bring a new analysis process to the early warning of the financial market. Summary of the invention

[0015] The purpose of this invention is to propose a risk warning method based on the financial numerator Transformer-GPT4 model, which simplifies the data preprocessing process and improves the accuracy and intelligence of the warning.

[0016] A risk warning method based on the financial numerical Transformer-GPT4 model includes the following steps:

[0017] Step 1: Obtain public financial information from the Internet, including internal, external and macro data of the financial system. The data type is text or structured data, and build a historical financial database;

[0018] Step 2: Use the GPT4 large model to preprocess the database data, and then use feature engineering to generate feature values ​​and reduce feature values ​​to generate embedded layer data;

[0019] Step 3: Build a convolutional Transformer risk warning model NUMT for financial numerical values. The model includes: embedding layer, encoder layer, decoder layer and convolution layer, and then complete model training and performance evaluation;

[0020] Step 4: Based on the latest financial data and prompt words input by the user, use the already constructed NUMT model to complete the risk warning based on financial data.

[0021] The present invention proposes a risk warning method based on the financial numerical Transformer-GPT4 model. Step 1 obtains financial data from the Internet and constructs a historical financial database, which includes the following steps:

[0022] Step A1: Use Python crawler framework to obtain financial data from public websites and public databases of financial websites. The data acquisition platforms include Eastmoney.com, government official websites, the official website of the National Bureau of Statistics, China Financial Yearbook and open source financial database Akshare;

[0023] Step A2: Use the Pandas framework to clean the data and filter out errors and missing data;

[0024] Step A3: Use MongoDB to build a financial history database to ensure data consistency and persistence;

[0025] The constructed financial history database contains a variety of financial historical data selected from inside and outside the financial system. The historical data outside the financial system include: GDP per capita, treasury bond issuance / GDP, inflation rate, industrial output growth rate, M2 / GDP, urban registered unemployment rate, CPI, fixed asset investment completion growth rate, fiscal deficit / GDP, real estate development comprehensive prosperity index, US interest rate, WTI crude oil growth rate, exchange rate volatility, export growth rate, import growth rate, foreign debt balance / foreign exchange reserves, foreign exchange reserves growth rate, foreign exchange reserves / import amount, current account balance / GDP, foreign direct investment / GDP; financial system The internal data include: M1 growth rate, M2 growth rate, M2 / foreign exchange reserves, financial institutions deposit balance growth rate, stock market yield, stock market price volatility, credit / GDP, real interest rate, deposit-lending spread, bank non-performing loan rate; macro-control data include: central bank benchmark interest rate, refinancing rate, deposit rate, central bank holdings of treasury bonds, deposit reserve ratio, credit quota, financial institutions leverage ratio limit, rediscount rate, government public investment, government budget deficit, civil servants salary and welfare benefits expenditure, government emergency expenditure, personal income tax revenue, corporate tax revenue, consumption tax revenue, commercial banks' capital adequacy ratio, capital and liquidity reserves.

[0026] The present invention proposes a risk warning method based on the financial numerical Transformer-GPT4 model. Step 2 preprocesses the system data and uses feature engineering to generate feature values ​​and reduce feature values. The method for generating embedded layer data includes the following steps:

[0027] Step B1: Use the GPT4 large language model to summarize the text information and extract professional financial data for each piece of financial historical data to obtain a summary description of the text and structured historical financial data;

[0028] Step B2: Use the Pandas framework to perform a second data cleansing on the financial data extracted by GPT4 in step B1, filter out errors and missing data, and then store them in the financial database;

[0029] Step B3: Create a fast labeling algorithm based on a greedy strategy, called the clustering similarity labeling algorithm CSL. Based on this algorithm and the GPT4 large language model, generate a labeling program for fully automated labeling to mark the correct regression results and feature classification of the original data;

[0030] Step B4: The structured historical financial indicator data is further standardized and normalized using the entropy weight method, and then the time dependency is processed. Then the feature matrix of each type of financial indicator is established, and the embedding layer feature vector is obtained using the entropy weight method.

[0031] The present invention proposes a risk warning method based on the financial numerical Transformer-GPT4 model. The cluster similarity labeling algorithm CSL in step B3 is expressed in the following mathematical formula:

[0032]

[0033] Among them, b represents the total number of labels, l i represents the i-th label, T(P) represents the result set, P represents the original data set, and P′ represents the remaining unlabeled data set.

[0034] 1. The present invention proposes a risk warning method based on the financial numerical Transformer-GPT4 model. Step three constructs a convolutional Transformer risk warning model NUMT for financial numerical values. The model includes: an embedding layer, an encoder layer, a decoder layer and a convolution layer; the embedding layer includes four steps: normalization operation, numerical weight calculation operation, information entropy calculation and weight calculation; the encoder layer contains a two-layer structure sub-layer consisting of N multi-head convolutional self-attention layers and a two-dimensional convolutional feedforward network; the decoder is composed of a stack of N identical layers, each stack consists of three sub-layers: a multi-head convolutional self-attention layer, a query self-attention layer and a two-dimensional convolutional feedforward network; the convolution layer contains only one feedforward neural network for the final synthesis of the financial risk probability map. The model is expressed in mathematical formula as follows:

[0035] (Q(t),K(t),V(t))=H(J(t)+PosMap(t))

[0036]

[0037] Among them, t is a certain moment in history, J(t) represents the embedding layer vector of the training data corresponding to that moment, PosMap(t) represents the position encoding information at that moment, S is the feedforward neural network function, n represents the dimension of the data, and the Attention function is the self-attention mechanism.

[0038] The present invention proposes a risk warning method based on the financial numerical Transformer-GPT4 model. Step 3 includes the following steps based on constructing a convolutional Transformer risk warning model for financial numerical values:

[0039] Step C1: Divide the preprocessed data set into a training set, a validation set, and a test set. The training set is used to train the model and fit the data sample. The test set is used to verify the generalization effect of the model and evaluate the model performance. The data is divided according to the principle of 8:1:1, that is, 80% of the data is divided into a training set, 10% is divided into a validation set, and 10% is divided into a test set.

[0040] Step C2: training the NUMT model, including model initialization, forward propagation, loss calculation, parameter update, and iterative training;

[0041] Step C3: During the training process, the validation set is used to verify the performance and adjust the hyperparameters of the NUMT model. After the training is completed, the final performance evaluation and verification of the model is performed on the test set.

[0042] The present invention proposes a risk warning method based on the financial numerical Transformer-GPT4 model. Step 4 is based on the constructed NUMT model to complete the risk warning based on financial data, including the following steps:

[0043] Step D1: Establish HTTP service, communication method and interactive interface based on NUMT model;

[0044] Step D2: Obtain input financial data, pre-process it using the GPT4 large model, obtain structured financial indicator data, and use it as the input of the model;

[0045] Step D3: Use the generated NUMT model to start early warning and analysis of the input financial indicator data;

[0046] Step D4: Continuously try financial early warning. If the risk assessment results output by the model are consistent and reliable in multiple warnings, that is, the warning results are stable, the warning ends and the risk assessment results of the model are returned; otherwise, return to step 3 to modify the hyperparameters of the model and re-train the NUMT model.

[0047] The beneficial effects of the present invention include: realizing a risk warning method using the GPT4 large language model for data preprocessing combined with the financial numerical Transformer model, simplifying the data preprocessing process, and improving the accuracy and intelligence of the warning. At the same time, through adaptive pre-training and fine-tuning mechanisms, it can better adapt to different types of real-time updated financial multimodal information, improving the applicability and generalization ability of the model. This can help decision makers better cope with the impact of economic risks, take corresponding measures in advance, and reduce potential risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a flow chart of a method for analyzing financial risks using the NUMT financial early warning model of the present invention;

[0049] Figure 2 It is a pseudo code diagram of the main algorithm of data cleaning in the present invention;

[0050] Figure 3It is a pseudo code schematic diagram of the fast marking algorithm CSL algorithm designed in the present invention;

[0051] Figure 4 It is a framework diagram of the NUMT model constructed in the present invention. DETAILED DESCRIPTION

[0052] The present invention is further described in detail with reference to the following specific examples and drawings. The process, conditions, experimental methods, etc. for implementing the present invention, except for the contents specifically mentioned below, are all common knowledge and common common sense in the art and are not particularly limited by the present invention.

[0053] like Figure 1 As shown, a risk warning method based on the financial numerical Transformer-GPT4 model of the present invention includes the following steps:

[0054] Step S1: Obtain public financial information from the Internet, including internal, external and macro data of the financial system. The data type can be text or structured data, and build a historical financial database;

[0055] Step S2: Use the GPT4 large model to preprocess the database data, and then use feature engineering to generate feature values ​​and reduce feature values ​​to generate embedded layer data;

[0056] Step S3: Construct a convolutional Transformer risk warning model NUMT for financial numerical values. The model includes the following parts:

[0057] Embedding layer, encoder layer, decoder layer and convolution layer, and then complete model training and performance evaluation;

[0058] Step S4: Based on the latest financial data and prompt words input by the user, the risk warning based on financial data is completed using the constructed NUMT model.

[0059] Each step is further described below.

[0060] Step S1 of the present invention comprises the following steps:

[0061] Step A1: Use Python crawler framework to obtain financial data from public websites and public databases of financial websites. The data acquisition platforms include Eastmoney.com, government official websites, the official website of the National Bureau of Statistics, China Financial Yearbook and open source financial database Akshare;

[0062] Step A2: Use the Pandas framework to clean the data and filter out errors and missing data;

[0063] Step A3: Use MongoDB to build a financial history database to ensure data consistency and persistence.

[0064] Before using the Python crawler framework to obtain various economic data, it is necessary to search the Internet for financial-related websites and corresponding publication addresses, and then organize them into a multimodal financial information acquisition list. On the other hand, it is necessary to query financial data indicators from the National Bureau of Statistics and the open source financial database Akshare as a data set.

[0065] The main steps for data cleaning in step A2 are:

[0066] First, use the request architecture to request data, and then store the acquired data in the local MongoDB database and local files, waiting for the next step of processing;

[0067] Then, the Pandas framework is used to perform corresponding filtering on the above stored text data. The specific algorithm is as follows:

[0068] 1) First, you need to filter out the abnormal data such as failed downloads, empty data, and default data;

[0069] 2) Text data source type, define the corresponding text extraction regular expression according to the text characteristics. For example, the financial data published on the government official website is sorted in the form of "one", "two", "three", etc., and the rest is irrelevant descriptions or links. For financial websites, comments exist in the corresponding DIV container of the page, so links and other unnecessary information can be filtered out through DOM operations and regular expressions;

[0070] 3) The regular expression is passed as a parameter to the processing function, which first reads all stored pages and then traverses and processes each page of data one by one;

[0071] 4) After obtaining the main content of the page text, obtain the location of the main text information in the article according to the originally stored data format;

[0072] 5) Return the processed data to facilitate replacement of native data and database storage.

[0073] The main algorithm of step A2 is as follows Figure 2 shown.

[0074] In step A3, the processed data are stored in the local MongoDB database according to the classification of the above financial indicators. Finally, the MongoDB database stores structured data containing the following financial indicators as feature values ​​of the entire data. The features are mainly divided into three categories: internal financial system, external financial system, and macroeconomic regulation.

[0075] The variable indicator table of the systemic risk early warning indicator system is as follows:

[0076]

[0077]

[0078]

[0079] Step S2 uses the GPT4 large model to preprocess the system data, and uses feature engineering to generate and reduce the eigenvalues. Generating the embedding layer data includes the following steps:

[0080] Step B1: In order to improve the accuracy and availability of data, it is necessary to ensure that each piece of information is correctly divided and cleaned. This makes it easier to analyze and compare the data, and to better extract the key financial indicator data. However, at present, the GPT4 large language model has token restrictions, making it difficult to uniformly train long texts of more than ten or twenty pages. In order to solve this problem, some optimization measures have been taken. Under the current circumstances, the most appropriate solution is to divide the long text into shorter paragraphs or sentences, and then input them into the large language model for training, and standardize the output result format through prompt words, so that it outputs key information text and the financial digital indicators involved in the text. Through the above optimization measures, it is possible to more effectively use the large language model to train long texts, and obtain more accurate and comprehensive text summary information, providing better support for subsequent training.

[0081] Step B2: Match and classify the text data obtained in step B1 and the structured financial data, call the GPT4 large language model interface again for batch labeling, and return the corresponding csv file.

[0082] Step B3: After obtaining the data generated in step B2, since the content generated by GPT4 is not necessarily correct, although it greatly relieves the work of repeated labeling, there are still cases of wrong labeling and missing labels. In view of the huge amount of data, it is impossible to manually check one by one. Therefore, a clustering similarity labeling algorithm based on a greedy strategy is designed, called the CSL algorithm, to assist in rapid retrieval of data and then perform rapid classification inspection and labeling. Specifically, the CSL algorithm can gradually select the optimal solution based on a series of rules or indicators to achieve the purpose of the optimal solution. The specific execution steps of the CSL algorithm are as follows:

[0083] First, the algorithm initializes an empty result set T, then processes the unlabeled models in a group loop, manually checks and assigns labels, and adds them to the result set. Next, the algorithm uses a stack V to track unvisited models, pops the model, searches for similar unlabeled models, manually assigns the same label and adds it to the result set, and finally adds the related models to the stack for the next round of processing. This continues until all data in the dataset are labeled, and the algorithm ends. The specific pseudo code is as follows Figure 3 shown.

[0084] From the data perspective, suppose there are n data that can be labeled with t labels. Let the text data set be P = {p 1 ,p 1 ,…,p n}, the result set is T(P)={(p 1 ,l 1 ),…,(p n ,l t )}. The overall problem can be expressed as follows:

[0085]

[0086] Among them, P′ represents the remaining unlabeled text data set.

[0087] Finally, based on this algorithm and the GPT4 large language model, a labeling program is generated for fully automated labeling to mark the correct regression results and feature classification of the original data.

[0088] Step B4: After the text data is correctly labeled, it is still necessary to further standardize and normalize the structured financial indicator data using the entropy weight method, process the time dependency, and then establish the feature matrix of various financial indicators, and use the entropy weight method to obtain the embedding layer feature vector. The specific steps are as follows:

[0089] First, the original data matrix X is composed of m samples and n indicators, where n = 47, and m is a function of time t, denoted as m = m(t). The original data can be expressed as: X = (X ij ) m(t)*n .

[0090] Next, create a unique-hot encoding for the originally classified data to obtain a new pure data sample; then all the data needs to be normalized:

[0091] For the internal and external data of the financial system, since they are positive indicators relative to financial risks, they are normalized using positive indicators:

[0092]

[0093] However, for the macro-control impact indicator data, on the contrary, negative indicators are used for normalization:

[0094]

[0095] After processing, we get a new data matrix X 1 .

[0096] On this basis, it is necessary to calculate the numerical weight P of the i-th item of the j-th indicator ij :

[0097] Then, calculate the entropy value e of the jth indicator j :

[0098] Next, we can find the variation index d of the jth indicator: j =1-e j ;

[0099] Finally, for each indicator, its weight

[0100] With the weights, we can calculate the final feature vector of the embedding layer:

[0101]

[0102] Among them, J(t) is the feature vector processed by the feature embedding network.

[0103] The next step is to build the NUMT model. The specific framework of the model is as follows Figure 4 As shown in the figure, it mainly consists of four layers: embedding layer, encoder, decoder, and convolution layer. The embedding layer has been specifically explained in the above steps. The following mainly involves the steps of encoder, decoder, and convolution layer. The specific implementation steps of this part are as follows:

[0104] Step C1: Divide the preprocessed data set into a training set and a test set. The training set is used to train the model and fit the data sample. The test set is used to verify the generalization effect of the model and evaluate the model performance. The steps to generate standard data are as follows: Split the data into a training set and a test set. The principle of splitting is the "80-20 principle". This can prepare for the accuracy of the verification model and further optimize it. The verification adopts the k-fold cross-validation method (a machine learning model evaluation method).

[0105] Step C2: Based on the data set described in step C1 and the data model structure, a comprehensive financial stress index is constructed, and then the NUMT model is created; this mainly includes the data processing flow of the data embedding layer and the encoder layer in the architecture, which includes the following three steps: (1) position encoding; (2) multi-head convolution self-attention mechanism; (3) output synthesis.

[0106] (1) Encode the data position, that is, provide the model with time series information so that the model can understand the passage of time. This step helps the model capture the time order and understand the dynamic characteristics of the time series. The specific formula is as follows:

[0107] Z(t)=J(t)+PosMap(t)

[0108] Among them, Z(t) is the input feature vector with position encoding, which is used to generate the Q, K, and V vectors required by the Transformer model. PosMap is the position encoding, which uses a combination of sine and cosine functions to represent the position information at time t. The specific definition is as follows:

[0109]

[0110] Among them, t is the time index and i is the dimension index.

[0111] (2) Use the multi-head convolutional self-attention mechanism of the Transformer model to process time series data and extract temporal dependencies.

[0112] First, the input feature vector Z(t) with position encoding generated in the previous step is used to generate the input required by the Transformer model: query, key, and value mapping. The specific formula is as follows:

[0113] (Q i ,K i ,V i )=H(Z i ),i∈[1,…,n]

[0114] where H is applied to the eigenvector Z i The convolutional subnetwork is used to generate the corresponding Q, K, V. Q, K, V are query, key, and value mappings respectively.

[0115] Then, the data is connected to the Transformer model to get the corresponding attention score:

[0116]

[0117] Among them, the softmax function is a normalized exponential function, d k is the vector dimension of query and key.

[0118] Finally, we get the output feature vector O of the Transformer model. i =∑ j Attention + (Q i ,K i ,V i )*V j

[0119] (3) The last step is to convert the output of the attention mechanism into the final financial risk pressure index through a feedforward neural network. The specific formula is as follows:

[0120] Y(t)=S(O(t))

[0121] Among them, S represents a feedforward neural network, which is used to convert the output into a financial stress index for early warning, and Y(t) is the financial comprehensive stress index for early warning, which can be used for decision support and risk management.

[0122] At this point, the construction of the NUMT model has been completed. After the construction is completed, the gradient descent algorithm is used to tune the model parameters and obtain the corresponding results.

[0123] After the model is built, the trained model needs to be evaluated using the test set. The specific steps are as follows:

[0124] (1) Load the model saved in the training phase to provide early warning for test data.

[0125] (2) Use the trained model to warn the test set and obtain the output of the model.

[0126] (3) Use the confusion matrix to calculate key performance indicators such as accuracy, precision, and recall of the model.

[0127] (4) Calculate the true positive rate (TPR) and false positive rate (FPR) of the test results, draw the ROC curve using different thresholds, and then calculate the area under the ROC curve.

[0128] After the NUMT model is trained, it is necessary to perform financial analysis and early warning tasks based on the model in actual applications, and then regularly optimize the latest financial data using the fine-tuning mechanism and iterate the model. The specific steps are as follows: Step D1: Establish HTTP services, communication methods, and interactive interfaces based on the NUMT model;

[0129] Step D2: Obtain input financial data, pre-process it using the GPT4 large model, obtain structured financial indicator data, and use it as the input of the model;

[0130] Step D3: Use the generated NUMT model to start early warning and analysis of the input financial indicator data;

[0131] Step D4: Continue to try financial early warning. If the risk assessment results output by the model are consistent and reliable in multiple warnings, that is, the warning results are stable, the warning ends and the risk assessment results of the model are returned. Otherwise, return to step 3 to modify the model's hyperparameters and retrain the NUMT model.

[0132] The protection content of the present invention is not limited to the above embodiments. Without departing from the spirit and scope of the inventive concept, changes and advantages that can be thought of by those skilled in the art are included in the present invention and are protected by the attached claims.

Claims

1. A risk warning method based on the financial numerical Transformer-GPT4 model, characterized by: The steps include: Step 1: Obtain public financial information from the internet, including internal, external, and macroeconomic data of the financial system. The data type is text or structured data, and build a historical financial database. Step 2: Use the GPT4 large model to preprocess the database data, and then use feature engineering to generate feature values ​​and reduce feature value dimensions to generate embedding layer data; Step 3: Build a convolutional Transformer risk warning model (NUMT) for financial numerical data. The model includes an embedding layer, an encoder layer, a decoder layer, and a convolutional layer. Then complete model training and performance evaluation. Step 4: Based on the latest financial data and prompt words input by the user, use the established NUMT model to complete the risk warning based on financial data.

2. The risk warning method according to claim 1, characterized in that: The step 1 specifically includes the following steps: Step A1: Use a Python crawler framework to obtain financial data from public websites and public databases on financial websites. Data acquisition platforms include Eastmoney.com, government websites, the National Bureau of Statistics website, the China Financial Yearbook, and the open source financial database Akshare. Step A2: Use the Pandas framework to clean the data and filter out errors and missing data; Step A3: Use MongoDB to build a financial history database to ensure data consistency and persistence; The constructed financial history database contains a variety of financial historical data selected from inside and outside the financial system. The historical data outside the financial system include: GDP per capita, treasury bond issuance / GDP, inflation rate, industrial output growth rate, M2 / GDP, urban registered unemployment rate, CPI, fixed asset investment completion growth rate, fiscal deficit / GDP, comprehensive real estate development prosperity index, US interest rate, WTI crude oil growth rate, exchange rate volatility, export growth rate, import growth rate, foreign debt balance / foreign exchange reserves, foreign exchange reserve growth rate, foreign exchange reserves / import amount, current account balance / GDP, foreign direct investment / GDP; financial system The internal data include: M1 growth rate, M2 growth rate, M2 / foreign exchange reserves, growth rate of deposit balances of financial institutions, stock market yield, stock market price volatility, credit / GDP, real interest rate, deposit-lending spread, bank non-performing loan rate; macroeconomic control data include: central bank benchmark interest rate, refinancing rate, deposit rate, number of treasury bonds held by the central bank, reserve requirement ratio, credit quota, leverage ratio limit of financial institutions, rediscount rate, government public investment, government budget deficit, civil servants' salary and welfare benefits expenditure, government emergency expenditure, personal income tax revenue, corporate tax revenue, consumption tax revenue, capital adequacy ratio of commercial banks, capital and liquidity reserves.

3. The risk warning method according to claim 1, characterized in that: The step 2 specifically includes the following steps: Step B1: Use the GPT4 large language model to summarize the text information and extract professional financial data for each piece of financial historical data, obtaining a summary description of the text and structured historical financial data; Step B2: Use the Pandas framework to perform a second data cleansing on the financial data extracted using GPT4 in Step B1, filtering out errors and missing data before storing it in a financial database. Step B3: Create a fast tagging algorithm based on a greedy strategy, called the Cluster Similarity Labeling algorithm (CSL). Based on this algorithm and the GPT4 large language model, a tagging program is generated to perform fully automated tagging, marking the correct regression results and feature classification of the original data; Step B4: Use the entropy weight method to further standardize and normalize the structured historical financial indicator data, then process the time dependency, establish the feature matrix of each type of financial indicator, and use the entropy weight method to obtain the embedding layer feature vector.

4. The risk warning method according to claim 1, characterized in that: The cluster similarity labeling algorithm CSL described in step B3 is expressed as follows: Among them, b represents the total number of labels, l i represents the i-th label, T(P) represents the result set, P represents the original data set, and P′ represents the remaining unlabeled data set.

5. The risk warning method according to claim 1, characterized in that: Step 3 constructs a convolutional Transformer risk warning model NUMT for financial numerical values. The model includes: an embedding layer, an encoder layer, a decoder layer, and a convolution layer. The embedding layer includes four steps: normalization operation, numerical weight calculation operation, information entropy calculation, and weight calculation. The encoder layer contains a two-layer structure consisting of N multi-head convolutional self-attention layers and a two-dimensional convolutional feedforward network. The decoder consists of a stack of N identical layers, each stack consisting of three sub-layers: a multi-head convolutional self-attention layer, a query self-attention layer, and a two-dimensional convolutional feedforward network. The convolution layer only contains one feedforward neural network for the final synthesis of the financial risk probability map. The model is expressed in mathematical formulas as follows: (Q(t),K(t),V(t))=H(J(t)+PosMap(t)) Where t is a certain moment in history, J(t) represents the embedding layer vector of the training data corresponding to that moment, PosMap(t) represents the position encoding information at that moment, S is the feedforward neural network function, n represents the dimension of the data, and the Attention function is the self-attention mechanism.

6. The financial risk early warning method according to claim 1, characterized in that: Step 3 completes model training and performance evaluation, including the following steps: Step C1: Divide the preprocessed dataset into a training set, a validation set, and a test set. The training set is used to train the model and fit the data samples, while the test set is used to verify the generalization effect of the model and evaluate the model performance. The data is divided according to the ratio of 8:1:1, that is, 80% of the data is divided into the training set, 10% into the validation set, and 10% into the test set. Step C2: Train the NUMT model, including model initialization, forward propagation, loss calculation, parameter update, and iterative training; Step C3: Use the validation set to perform performance verification and hyperparameter adjustment on the NUMT model during training. After training, perform final performance evaluation and verification on the test set.

7. The risk warning method according to claim 1, characterized in that: The step 4 specifically includes the following steps: Step D1: Establish HTTP service, communication method and interactive interface based on NUMT model; Step D2: Obtain input financial data, preprocess it using the GPT4 large model, and obtain structured financial indicator data as input to the model; Step D3: Use the generated NUMT model to start early warning and analysis of the input financial indicator data; Step D4: Continuously try financial early warning work. If the risk assessment results output by the model are consistent and reliable in multiple warnings, that is, the warning results are stable, the warning ends and the risk assessment results of the model are returned; otherwise, it falls back to step 3 to modify the model's hyperparameters and re-train the NUMT model.

Citation Information

Cited By

  • Intelligent private placement net value management system

    CN120070061A