Financial data risk analysis method and device based on reinforcement learning, and medium

Abstract

The application discloses a financial data risk analysis method and device based on reinforcement learning and a medium, relates to the technical field of data risk analysis, and comprises the following steps: performing time sequence processing on splicing indexes of financial related data to determine a financial time sequence vector; performing time period division feature extraction on the financial time sequence vector to obtain a financial feature vector; determining a financial plan risk value based on the financial feature vector by learning the financial plan risk analysis of an intelligent agent; obtaining a trained reinforcement learning intelligent agent through parameterized learning of a stochastic gradient descent update according to the financial plan risk value; inputting the financial time sequence vector into the reinforcement learning intelligent agent to determine a risk value function, and determining a financial data risk analysis report through plan feature analysis based on the risk value function. The application solves the technical problems of insufficient timeliness and adaptability and dependence on manual processing in the prior art by using the above method.

Description

Technical Field

[0001] This application relates to the field of data risk analysis technology, and in particular to a method, device and medium for financial data risk analysis based on reinforcement learning. Background Technology

[0002] Financial data is a crucial quantitative reflection of the rationality of a company's production and operational decisions and plans. Therefore, the identification and analysis of financial data risks can serve as a key basis for corporate managers to formulate subsequent business plans. However, influenced by the dynamic changes in the global economic environment and the continuously increasing complexity of corporate management, the types and scale of financial data risks faced by enterprises are showing a rapid growth trend. In this context, the inability to identify financial data risks in a timely and accurate manner and adjust business plans accordingly will directly hinder the long-term sustainable development of enterprises.

[0003] In the field of financial data risk analysis, current mainstream methods still rely on traditional manual statistical analysis or neural network prediction, both of which have significant limitations. First, traditional manual statistical analysis struggles to efficiently process massive amounts of financial data, resulting in insufficient timeliness and impacting the comprehensiveness and accuracy of risk assessment. Second, while neural network models can handle the analysis of large volumes of data, they are ill-suited to adapting to dynamic changes in the market environment. Neural network models can only identify anomalies within a known range based on historical data patterns, lacking the ability to perceive and warn of entirely new risks. Furthermore, existing methods still require manual review of reports after completing financial data risk analysis. Due to differences in the professional backgrounds and judgment logic of different analysts, risk response plans are difficult to fully implement. Summary of the Invention

[0004] This application provides a method, device, and medium for financial data risk analysis based on reinforcement learning, which solves the technical problems of insufficient timeliness and adaptability of financial data risk analysis and reliance on manual processing in the prior art.

[0005] In a first aspect, embodiments of this application provide a method for financial data risk analysis based on reinforcement learning. The method includes: acquiring financial-related data and performing time-series processing on the financial-related data to determine a financial time-series vector; extracting features from the financial time-series vector by dividing it into time periods to obtain a financial feature vector; determining a financial planning risk value based on the financial feature vector through financial planning risk analysis of a learning agent; obtaining a trained reinforcement learning agent by parameterized learning updated via stochastic gradient descent based on the financial planning risk value; inputting the financial time-series vector into the reinforcement learning agent to determine a risk value function, and determining a financial data risk analysis report based on the risk value function through planning feature analysis.

[0006] In one implementation of this application, time-series processing of concatenated indicators is performed on financial-related data to determine a financial time-series vector. Specifically, this includes: preprocessing the financial-related data to obtain normalized financial-related data; wherein the data preprocessing includes: data deduplication, outlier correction, and data normalization; dividing the normalized financial-related data to determine the data types of financial-related data; wherein the data types of financial-related data include: static financial data and dynamic financial data; sorting the static financial data using fixed indicators to obtain a first financial time-series vector; concatenating multiple time-series indicators from the dynamic financial data to obtain a second financial time-series vector; and determining the final financial time-series vector based on the first and second financial time-series vectors.

[0007] In one implementation of this application, feature extraction is performed on the financial time series vector by dividing it into time periods to obtain a financial feature vector. Specifically, this includes: performing data correlation analysis on the financial feature vector to obtain financial data correlation indicators; wherein, the data correlation analysis includes: correlation analysis and importance analysis; based on the financial data correlation indicators, feature extraction is performed on the financial feature vector to determine the feature vector to be divided; and the feature vector to be divided is divided into period intervals to obtain the financial feature vector.

[0008] In one implementation of this application, the financial plan risk value is determined based on the financial feature vector and through financial plan risk analysis by a learning agent. Specifically, this includes: executing the financial plan corresponding to the financial feature vector through a preset learning agent to obtain the current feature risk value; executing the financial plan after a predetermined time period based on the current feature risk value to determine the updated feature risk value; and determining the financial plan risk value by using the optimal risk value judgment based on the current feature risk value and the updated feature risk value.

[0009] In one implementation of this application, a trained reinforcement learning agent is obtained by parametric learning updated through stochastic gradient descent based on the financial planning risk value. Specifically, this includes: performing learning parameter generalization processing on the financial planning risk value to determine risk function pairs; obtaining a set of learning parameters based on the risk function pairs through feature basis design; performing stochastic gradient descent processing on the learning parameter set to determine an updated parameter set; and training the current learning agent until the model converges based on the updated parameter set to obtain the trained reinforcement learning agent.

[0010] In one implementation of this application, a financial data risk analysis report is determined based on a risk value function and through planning feature analysis. Specifically, this includes: aligning the risk value function with time intervals to determine the risk value within the same time interval; comparing the risk values ​​within the same time interval to obtain the risk data characteristics corresponding to the risk values; determining the risk analysis data based on the risk data characteristics and through risk data status marking; and generating the financial data risk analysis report based on the risk analysis data using a planning template.

[0011] In one implementation of this application, after determining the financial data risk analysis report based on the risk value function and through planning feature analysis, the method further includes: perceiving the financial environment status of the financial data risk analysis report to determine the financial risk environment status; obtaining the financial data risk threshold through dynamic threshold optimization based on the financial risk environment status; and updating the financial data risk analysis report according to the financial data risk threshold.

[0012] In one implementation of this application, a financial data risk threshold is obtained based on the financial risk environment state through dynamic threshold optimization. Specifically, this includes: performing reinforcement learning on the financial risk environment state to optimize the risk threshold and obtain a threshold reward function; and obtaining the financial data risk threshold through exponentially weighted moving average processing based on the threshold reward function.

[0013] Secondly, embodiments of this application also provide a financial data risk analysis device based on reinforcement learning, characterized in that the device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to: acquire financial-related data and perform time-series processing of the financial-related data by concatenating indicators to determine a financial time-series vector; extract features from the financial time-series vector by dividing it into time periods to obtain a financial feature vector; determine a financial planning risk value based on the financial feature vector through financial planning risk analysis of a learning agent; obtain a trained reinforcement learning agent by parameterized learning updated through stochastic gradient descent according to the financial planning risk value; input the financial time-series vector into the reinforcement learning agent to determine a risk value function, and determine a financial data risk analysis report based on the risk value function through planning feature analysis.

[0014] Thirdly, embodiments of this application also provide a non-volatile computer storage medium for financial data risk analysis based on reinforcement learning, storing computer-executable instructions. The computer-executable instructions are configured to: acquire financial-related data and perform time-series processing on the financial-related data to determine a financial time-series vector; extract features from the financial time-series vector by dividing it into time periods to obtain a financial feature vector; determine a financial planning risk value based on the financial feature vector through financial planning risk analysis of a learning agent; obtain a trained reinforcement learning agent by parameterized learning updated through stochastic gradient descent based on the financial planning risk value; input the financial time-series vector into the reinforcement learning agent to determine a risk value function, and determine a financial data risk analysis report based on the risk value function through planning feature analysis.

[0015] This application provides a method, device, and medium for financial data risk analysis based on reinforcement learning. By learning the financial planning risk analysis of intelligent agents, training the corresponding intelligent agents, and analyzing the planning characteristics of financial data risk values, it solves the technical problems of insufficient timeliness and adaptability of financial data risk analysis and reliance on manual processing in the prior art. It realizes the intelligent generation of subsequent risk response plans and improves the adaptability of financial data risk analysis. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0017] Figure 1 A flowchart illustrating a financial data risk analysis method based on reinforcement learning, provided for embodiments of this application;

[0018] Figure 2 A flowchart illustrating the state learning and update process of a reinforcement learning agent is provided in this embodiment of the application.

[0019] Figure 3 A flowchart illustrating the construction of a risk value database is provided for an embodiment of this application.

[0020] Figure 4 A structural diagram of a financial data risk analysis system based on reinforcement learning is provided for embodiments of this application;

[0021] Figure 5 This is a schematic diagram of the internal structure of a financial data risk analysis device based on reinforcement learning, provided for an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] This application provides a method, device, and medium for financial data risk analysis based on reinforcement learning. By learning the financial planning risk analysis of intelligent agents, training the corresponding intelligent agents, and analyzing the planning characteristics of financial data risk values, it solves the technical problems of insufficient timeliness and adaptability of financial data risk analysis and reliance on manual processing in the prior art. It realizes the intelligent generation of subsequent risk response plans and improves the adaptability of financial data risk analysis.

[0024] The technical solutions proposed in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0025] Figure 1 This document provides a flowchart of a financial data risk analysis method based on reinforcement learning, as illustrated in an embodiment of this application. Figure 1 As shown in the figure, the financial data risk analysis method based on reinforcement learning provided in this application embodiment specifically includes the following steps:

[0026] Step 101: Obtain financial-related data and perform time-series processing on the financial-related data to determine the financial time-series vector.

[0027] For example, financial data can be obtained from publicly available financial reports, third-party professional databases, or accounting firms of financial institutions, or directly from financial data and financial plans provided by the target user company. This application achieves normalization and standardization of financial data by splicing indicators into time-series processing, thus providing a data foundation for training intelligent agents.

[0028] Specifically, the financial-related data undergoes time-series processing to determine a financial time-series vector. This includes: preprocessing the financial-related data to obtain normalized financial-related data; whereby data preprocessing includes: data deduplication, outlier correction, and data normalization; dividing the normalized financial-related data to determine the types of financial-related data; whereby the types of financial-related data include: static financial data and dynamic financial data; sorting the static financial data using fixed indicators to obtain a first financial time-series vector; concatenating multiple time-series indicators from the dynamic financial data to obtain a second financial time-series vector; and determining the final financial time-series vector based on the first and second financial time-series vectors.

[0029] In one embodiment, financial data, financial plan data, and historical risk data of relevant companies within the target user's professional field are obtained and preprocessed. This data can be obtained from publicly available financial reports, third-party professional databases, and financial institutions / accounting firms.

[0030] Then, the data is preprocessed, including removing duplicate data, identifying and handling outliers, correcting data inconsistencies, and standardizing and normalizing the data.

[0031] Finally, for static data, the preprocessed data can be directly arranged into a vector according to a fixed index order; for time series data, the time dimension needs to be included in the vector, and a time series vector is constructed by splicing multiple period indicators and building a feature matrix, so as to determine the financial time series vector.

[0032] Step 102: Extract features from the financial time series vector by dividing it into time periods to obtain the financial feature vector.

[0033] For example, this application extracts features from the financial time series vector by dividing it into time periods to obtain a financial feature vector, thereby compressing the high-dimensional financial time series vector into a low-dimensional financial time series vector, retaining the core features of the data, and eliminating redundant indicators.

[0034] Specifically, feature extraction is performed on the financial time series vector by dividing it into time periods to obtain financial feature vectors. This includes: performing data correlation analysis on the financial feature vectors to obtain financial data correlation indicators; wherein, data correlation analysis includes: correlation analysis and importance analysis; based on the financial data correlation indicators, feature extraction is performed on the financial feature vectors to determine the feature vectors to be divided; and the feature vectors to be divided are divided into period intervals to obtain financial feature vectors.

[0035] In one embodiment, the size of the interval can be set manually, such as a day, a week, or a month. Assuming the overall time interval is one year, and within that year... Divide the data into weekly periods, and denot the resulting feature vectors for financial data, financial planning data, and historical risk data as follows: , , subscript This indicates that it belongs to the first A time interval.

[0036] For feature vectors of static data, a separate vector storage area can be allocated and stored in each time period for immediate use.

[0037] Step 103: Based on the financial feature vector, determine the financial planning risk value by learning the agent's financial planning risk analysis.

[0038] Specifically, based on financial feature vectors, the financial plan risk value is determined through financial plan risk analysis by a learning agent, including: executing the financial plan corresponding to the financial feature vector through a pre-set learning agent to obtain the current feature risk value; executing the financial plan after a predetermined time period based on the current feature risk value to determine the updated feature risk value; and determining the financial plan risk value by using the optimal risk value judgment based on the current feature risk value and the updated feature risk value.

[0039] Figure 2 This is a flowchart illustrating the state learning and updating process of a reinforcement learning agent, as provided in an embodiment of this application.

[0040] Figure 3 A flowchart illustrating the construction of a risk value database is provided for an embodiment of this application.

[0041] In one embodiment, such as Figure 2 As shown, in the first Within a given time period, the current state of the agent is characterized by financial data features. The intelligent agent performs financial planning features The corresponding plan can then be obtained Corresponding risk value ,implement Following the corresponding plan, the agent enters a new state. .

[0042] It is important to note that Corresponding risk value It is obtained from a risk value database.

[0043] like Figure 3 As shown, the process of constructing the risk value database is as follows: First, using the feature vectors of historical risk data... According to each vector The corresponding risk data determines the first Loss categories within a time interval, such as reputational and trust losses, operational and strategic losses, etc.

[0044] Then, the magnitude of the impact of each type of loss is determined and classified into major losses, moderate losses, and general losses. The risk value is obtained by weighting the loss classification within each time interval and the importance of different types of losses.

[0045] Finally, the risk value for each time interval is calculated using the above steps and denoted as follows: All existing to and Perform a match.

[0046] For a certain risk value If the first It appeared in the interval. Yes, that means This will cause risk value At this time, remember Risk value ,in These are manually set integer parameters, which can be configured based on factors such as the size of the time interval segment and the characteristics of the company's industry. The purpose is to avoid misjudging risk values ​​due to excessively small time intervals. If the above process... For each occurrence, the corresponding risk value increases proportionally with the number of occurrences.

[0047] Furthermore, if there is no precise value in the risk database... Corresponding data risk value This is similar to reinforcement learning. Greedy strategy, finding and matching closest For the corresponding risk value, in terms of probability The probability of randomly selecting a risk value, while using The probability is used with the closest approximation. For the corresponding data risk value, where 0 It is usually large at the beginning of the iteration and gradually decreases as training progresses.

[0048] Step 104: Based on the financial plan risk value, obtain the trained reinforcement learning agent through parameterized learning updated by stochastic gradient descent.

[0049] Specifically, based on the financial planning risk value, a trained reinforcement learning agent is obtained through parameterized learning updated by stochastic gradient descent. This includes: performing parameter generalization processing on the financial planning risk value to determine risk function pairs; obtaining a set of learning parameters based on the risk function pairs through feature basis design; performing stochastic gradient descent processing on the learning parameter set to determine the updated parameter set; and training the current learning agent until the model converges based on the updated parameter set to obtain the trained reinforcement learning agent.

[0050] Figure 4 This is a structural diagram of a financial data risk analysis system based on reinforcement learning, provided for an embodiment of this application.

[0051] In one embodiment, the goal of a reinforcement learning agent is to select a policy. To minimize the risk of financial data over the entire time period, the strategy is... It refers to the first Plans will be implemented for each time period. The set, i.e. .

[0052] Furthermore, set a strategy In state Timely implementation plan The risk is Then the risk minimization function is defined as follows: Its representation is in the state. Timely implementation plan The total risk value of consistently making decisions according to the optimal strategy. The strategy that makes the above equation hold. Recorded as , which represents the optimal strategy.

[0053] During training, in order to address the issue that it is impossible to directly store all states and actions in a large-scale state and action space... The problem of values ​​can be solved by learning them using a parameterized approach. Its core idea is: to Represented as a parameter-dependent function By learning parameters Thus, it generalizes to unseen. Yes, not for each Storing values ​​separately. One approach is to approximate them using a linear function. Its core is: firstly, based on the characteristics of the enterprise domain... Design feature base .

[0054] Then, Represented as .

[0055] in, These are the parameters to be learned. However, when the state and action spaces are extremely complex, or when the characteristics of the enterprise's domain are diverse, linear parameterization cannot meet the requirements. In such cases, nonlinear function approximation is needed. Another approach is to use a neural network as a function approximator, setting the weights and biases of the neural network as the parameters to be optimized. .

[0056] Furthermore, the parameters are updated using stochastic gradient descent. The following formula will be used to explain this.

[0057] (1)

[0058] (2)

[0059] Among them, parameters The weighted discount for future risks means that risks further in the future have a smaller impact on the present state; parameters Represents the parameters of the target network, each Updated once per iteration, in the [number]th iteration. Next iteration update It is not updated in other iterations. This is a manually set interval, designed to avoid underestimating the risks associated with financial data; parameters This is the learning rate.

[0060] It's important to note that when unexpected risks arise in the environment, reinforcement learning agents do not need to be retrained from scratch. This is mainly because they continuously learn online and dynamically update their policies.

[0061] When a sudden policy change causes a sharp increase in a certain type of risk, traditional neural network models, with their fixed parameters, will still assess the risk according to the old logic and need to be retrained with data under the new policy to correct their behavior. In contrast, reinforcement learning agents will immediately receive a negative reward for "the loss caused by assessing the risk according to the old strategy". Based on this, adjust the parameters directly. The updates allow for adjustments to subsequent strategies without requiring a halt to retraining.

[0062] Divide the target company's financial data feature vectors into time periods. The time intervals here should be consistent with the time intervals of the reinforcement learning agent.

[0063] The characteristics of the target company's financial data after being divided by time are as follows: Financial planning data characteristics are By inputting the feature vectors of the target company's financial data and financial plan data, divided by time, into a reinforcement learning agent, the agent can generate a risk value function for the target company's financial data. .

[0064] Specifically, the above This refers to The input is fed into a reinforcement learning agent that has been trained and set to not update its parameters for the time being, and the risk value function is generated by the agent.

[0065] Will The optimal policy can be obtained by inputting the data into a reinforcement learning agent that has been trained and set to update its parameters. ;in, It refers to the optimal financial plan data features generated autonomously by the reinforcement learning agent within each time period.

[0066] Will and By comparing and analyzing the risk values ​​generated by the two methods in different time intervals, we can better understand their respective characteristics. The size of the time period Financial data features with low risk values ​​are marked as risky data features, and a financial data risk analysis report is generated.

[0067] Specifically, data characteristics can be categorized into high-risk, medium-risk, and low-risk data characteristics based on the target company's needs and risk levels. The distinction between high-risk, medium-risk, and low-risk data is determined by the actual situation of Company A's industry. For example, for Company A, if within a certain timeframe there is a significant drop in profit margin exceeding 10%, inventory turnover rate deteriorating to below 4%, and accounts receivable days extending by more than 80 days, the data within that timeframe is considered high-risk. Conversely, if revenue growth slows to 4%, profit margin slightly decreases by 2%, and accounts receivable days extend to 50 days while other indicators remain stable, this falls within the medium-risk range, and the data within that timeframe is also considered medium-risk. If all indicators consistently outperform the industry average, cash flow is ample, and operational efficiency improves, this falls within the low-risk range, and the data within that timeframe is also considered low-risk. Further, a data risk analysis report is needed. The data is generated based on the data within the aforementioned risky time period. The generation process is as follows: First, the risk data is preprocessed, including cleaning, integration, and transformation. Then, based on the processed data, anomalies or trends that may trigger risks are identified through methods such as clustering and visualization. Next, the identified risk points are compiled into a list, specifying the "risk name, associated business scenario, and abnormal data behavior." Finally, the identified risk points are evaluated from two dimensions: "probability of occurrence" and "degree of impact," to determine the risk level (high-risk, medium-risk, and low-risk), thereby generating a financial data risk analysis report.

[0068] Within the time interval corresponding to the above-mentioned risky data characteristics The financial planning data characteristics are labeled, and corresponding risk response plans are generated.

[0069] Specifically, for each of the risk points in the aforementioned financial data risk analysis report, their corresponding time periods are utilized. The plan data generates a response plan for each risk point.

[0070] Furthermore, if the types of risk points and corresponding response plans are similar across different time periods, operations such as merging, eliminating, and sorting can be used to make the final response plan more focused and well-organized.

[0071] Step 105: Input the financial time series vector into the reinforcement learning agent to determine the risk value function, and based on the risk value function, determine the financial data risk analysis report through planning feature analysis.

[0072] Specifically, based on the risk value function, a financial data risk analysis report is determined through planning characteristic analysis, including: aligning the risk value function with time intervals to determine the risk value within the same time interval; comparing the risk values ​​within the same time interval to obtain the risk data characteristics corresponding to the risk values; determining the risk analysis data based on the risk data characteristics and through risk data status marking; and generating the financial data risk analysis report based on the risk analysis data using a planning template.

[0073] Furthermore, after determining the financial data risk analysis report based on the risk value function and through planning characteristic analysis, the method also includes: perceiving the financial environment status of the financial data risk analysis report to determine the financial risk environment status; obtaining the financial data risk threshold through dynamic threshold optimization based on the financial risk environment status; and updating the financial data risk analysis report according to the financial data risk threshold.

[0074] Furthermore, based on the financial risk environment status, a financial data risk threshold is obtained through dynamic threshold optimization. Specifically, this includes: optimizing the risk threshold by reinforcement learning on the financial risk environment status to obtain a threshold reward function; and obtaining the financial data risk threshold through exponentially weighted moving average processing based on the threshold reward function.

[0075] In one embodiment, firstly, the risk values ​​for Company A over the past four quarters are calculated based on a risk value function, and time interval alignment is performed to ensure that all data is uniformly aligned by quarter. The risk values ​​for the four quarters are then numerically compared, and Company A's current quarter's risk value is compared with historical values ​​and industry benchmarks to identify risk data characteristics.

[0076] By marking risk data characteristics with risk data status, such as marking the debt ratio as deteriorating, structured risk analysis data is formed. Then, a standardized financial data risk analysis report is automatically generated by calling a preset plan template. This report will highlight the risk data status markings.

[0077] Environmental status awareness integrates existing mature data interfaces and rule engines to analyze the financial environment, and automatically obtains real-time publicly available macroeconomic indicators and industry average data through financial special data APIs.

[0078] Then, based on the financial planning amount, financial data risk value, and the year-on-year growth rate of Company A's main business revenue obtained from the financial special data API, when at least two of the above three indicators show unfavorable signals at the same time, such as the financial planning amount declining for three consecutive months, the financial data risk value showing an upward trend, and the year-on-year growth rate of Company A's main business revenue turning from positive to negative, the current financial environment is judged to be in a period of loss.

[0079] Finally, based on the financial environment status obtained from environmental status perception, the financial data risk threshold is adjusted through dynamic threshold optimization. Data from all historical periods marked as "loss periods" are retrieved from the existing financial database, and through retrospective analysis, it is analyzed which specific debt-to-equity ratio thresholds can generate the most effective early warnings during historical periods.

[0080] For example, in the past three loss periods, the accuracy of the warnings was highest and the false alarm rate was lowest when the threshold was set at 65%. Therefore, this threshold represents the optimized result of maximizing implicit rewards obtained through retrospective analysis.

[0081] To ensure the stability of threshold updates, the final financial data risk threshold can be determined by using an exponentially weighted moving average. The updated threshold is then weighted and merged with the previously used old threshold, with the old threshold having a weight of 0.6 and the new threshold having a weight of 0.4. This weighted calculation yields the financial data risk threshold.

[0082] The above are embodiments of the method proposed in this application. Based on the same inventive concept, embodiments of this application also provide a financial data risk analysis device based on reinforcement learning, the structure of which is as follows: Figure 5 As shown.

[0083] Figure 5 This is a schematic diagram of the internal structure of a financial data risk analysis device based on reinforcement learning, provided as an embodiment of this application. Figure 5 As shown, the device includes:

[0084] At least one processor 501;

[0085] And a memory 502 that is communicatively connected to at least one processor;

[0086] The memory 502 stores instructions executable by at least one processor, which are executed by at least one processor 501 to enable at least one processor 501 to:

[0087] Acquire financial data and perform time-series processing on the data to determine the financial time-series vector. Extract features from the financial time-series vector by dividing it into time periods to obtain the financial feature vector. Based on the financial feature vector, determine the financial planning risk value through financial planning risk analysis of the learning agent. Based on the financial planning risk value, obtain the trained reinforcement learning agent through parameterized learning updated by stochastic gradient descent. Input the financial time-series vector into the reinforcement learning agent to determine the risk value function, and based on the risk value function, determine the financial data risk analysis report through planning feature analysis.

[0088] Some embodiments of this application provide corresponding to Figure 1 A non-volatile computer storage medium for financial data risk analysis based on reinforcement learning, storing computer-executable instructions, wherein the computer-executable instructions are configured as follows:

[0089] Acquire financial data and perform time-series processing on the data to determine the financial time-series vector. Extract features from the financial time-series vector by dividing it into time periods to obtain the financial feature vector. Based on the financial feature vector, determine the financial planning risk value through financial planning risk analysis of the learning agent. Based on the financial planning risk value, obtain the trained reinforcement learning agent through parameterized learning updated by stochastic gradient descent. Input the financial time-series vector into the reinforcement learning agent to determine the risk value function, and based on the risk value function, determine the financial data risk analysis report through planning feature analysis.

[0090] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for IoT devices and media are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0091] The systems, media, and methods provided in this application are one-to-one correspondences. Therefore, the systems and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the systems and media will not be repeated here.

[0092] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0093] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0094] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0095] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0096] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0097] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0098] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0099] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0100] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A financial data risk analysis method based on reinforcement learning, characterized in that, The method includes: Acquire financial-related data and perform time-series processing on the financial-related data to determine the financial time-series vector; The financial time series vector is divided into time periods for feature extraction to obtain a financial feature vector; Based on the aforementioned financial feature vector, the financial planning risk value is determined through the financial planning risk analysis of the learning agent. The risk value of the financial plan is subjected to learning parameter generalization processing to determine the risk function pair, and based on the risk function pair, a set of learning parameters is obtained through feature basis design; Based on the set of learning parameters, a trained reinforcement learning agent is obtained through parameterized learning updated by stochastic gradient descent. The financial time series vector is input into the reinforcement learning agent to determine the risk value function, and based on the risk value function, a financial data risk analysis report is determined through planning feature analysis. Based on the aforementioned financial feature vector, the financial planning risk value is determined through financial planning risk analysis of the learning agent, specifically including: The financial plan corresponding to the financial feature vector is executed through a pre-set learning agent to obtain the current feature risk value; Based on the current characteristic risk value, execute the financial plan after a predetermined time period and determine the updated characteristic risk value; Based on the current characteristic risk value and the updated characteristic risk value, the financial plan risk value is determined through optimal risk value judgment.

2. The financial data risk analysis method based on reinforcement learning according to claim 1, characterized in that, The financial data is processed by concatenating indicators over time to determine the financial time-series vector, specifically including: The financial data is preprocessed to obtain normalized financial data; wherein, the data preprocessing includes: data deduplication, outlier correction, and data normalization; The normalized financial data is divided into data segments to determine the types of financial data; wherein, the types of financial data include: static financial data and dynamic financial data; The static financial data is sorted using fixed indicators to obtain a first financial time-series vector; The financial dynamic data is spliced ​​together with multi-period indicators in a time series to obtain a second financial time series vector; The financial time series vector is determined based on the first financial time series vector and the second financial time series vector.

3. The financial data risk analysis method based on reinforcement learning according to claim 1, characterized in that, The financial time series vector is divided into time periods for feature extraction to obtain a financial feature vector, specifically including: Data correlation analysis is performed on the financial feature vectors to obtain financial data correlation indicators; wherein, the data correlation analysis includes: correlation analysis and importance analysis; Based on the financial data correlation indicators, feature extraction is performed on the financial feature vector to determine the feature vector to be divided; The feature vector to be divided is divided into periodic intervals to obtain the financial feature vector.

4. The financial data risk analysis method based on reinforcement learning according to claim 1, characterized in that, Based on the learned parameter set, a trained reinforcement learning agent is obtained through parameterized learning updated by stochastic gradient descent, specifically including: The learned parameter set is subjected to stochastic gradient descent to determine the updated parameter set; Based on the updated parameter set, train the current learning agent until the model converges to obtain the trained reinforcement learning agent.

5. The financial data risk analysis method based on reinforcement learning according to claim 1, characterized in that, Based on the aforementioned risk value function, a financial data risk analysis report is determined through plan characteristic analysis, specifically including: The risk value function is time interval aligned to determine the risk value within the same time interval; The risk values ​​within the same time interval are compared to obtain the risk data characteristics corresponding to the risk values; Based on the aforementioned risk data characteristics, risk analysis data is determined through risk data status marking; Based on the risk analysis data, a financial data risk analysis report is generated using a planning template.

6. The financial data risk analysis method based on reinforcement learning according to claim 1, characterized in that, After determining the financial data risk analysis report based on the aforementioned risk value function and through planning characteristic analysis, the method further includes: The financial data risk analysis report is used to assess the financial environment status in order to determine the financial risk environment status. Based on the aforementioned financial risk environment, a financial data risk threshold is obtained through dynamic threshold optimization. Update the financial data risk analysis report based on the aforementioned financial data risk threshold.

7. The financial data risk analysis method based on reinforcement learning according to claim 6, characterized in that, Based on the aforementioned financial risk environment, financial data risk thresholds are obtained through dynamic threshold optimization, specifically including: The risk threshold is optimized by reinforcement learning on the aforementioned financial risk environment state to obtain the threshold reward function; Based on the threshold reward function, the financial data risk threshold is obtained through exponential weighted moving average processing.

8. A financial data risk analysis device based on reinforcement learning, characterized in that, The device includes: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to: Acquire financial-related data and perform time-series processing on the financial-related data to determine the financial time-series vector; The financial time series vector is divided into time periods for feature extraction to obtain a financial feature vector; Based on the aforementioned financial feature vector, the financial planning risk value is determined through the financial planning risk analysis of the learning agent. The risk value of the financial plan is subjected to learning parameter generalization processing to determine the risk function pair, and based on the risk function pair, a set of learning parameters is obtained through feature basis design; Based on the set of learning parameters, a trained reinforcement learning agent is obtained through parameterized learning updated by stochastic gradient descent. The financial time series vector is input into the reinforcement learning agent to determine the risk value function, and based on the risk value function, a financial data risk analysis report is determined through planning feature analysis. Based on the aforementioned financial feature vector, the financial planning risk value is determined through financial planning risk analysis of the learning agent, specifically including: The financial plan corresponding to the financial feature vector is executed through a pre-set learning agent to obtain the current feature risk value; Based on the current characteristic risk value, execute the financial plan after a predetermined time period and determine the updated characteristic risk value; Based on the current characteristic risk value and the updated characteristic risk value, the financial plan risk value is determined through optimal risk value judgment.

9. A non-volatile computer storage medium for financial data risk analysis based on reinforcement learning, storing computer-executable instructions, characterized in that, The computer-executable instructions are set as follows: Acquire financial-related data and perform time-series processing on the financial-related data to determine the financial time-series vector; The financial time series vector is divided into time periods for feature extraction to obtain a financial feature vector; Based on the aforementioned financial feature vector, the financial planning risk value is determined through the financial planning risk analysis of the learning agent. The risk value of the financial plan is subjected to learning parameter generalization processing to determine the risk function pair, and based on the risk function pair, a set of learning parameters is obtained through feature basis design; Based on the set of learning parameters, a trained reinforcement learning agent is obtained through parameterized learning updated by stochastic gradient descent. The financial time series vector is input into the reinforcement learning agent to determine the risk value function, and based on the risk value function, a financial data risk analysis report is determined through planning feature analysis. Based on the aforementioned financial feature vector, the financial planning risk value is determined through financial planning risk analysis of the learning agent, specifically including: The financial plan corresponding to the financial feature vector is executed through a pre-set learning agent to obtain the current feature risk value; Based on the current characteristic risk value, execute the financial plan after a predetermined time period and determine the updated characteristic risk value; Based on the current characteristic risk value and the updated characteristic risk value, the financial plan risk value is determined through optimal risk value judgment.

Images