Abnormal data identification method and device based on error correction model, equipment and medium

Abstract

The invention relates to the field of data analysis and digital finance, and discloses an abnormal data identification method and device based on an error correction model, equipment and a medium. Determining data index characteristics corresponding to the data indexes; generating external data features based on the external data and a preset BERT model; and determining target abnormal data according to a preset error correction model, the data index features and the external data features. Through the above mode, potential abnormal data can be predicted and identified more accurately by analyzing the relevance between the data indexes, more extensive information sources can be introduced by using the external data and the BERT-like model, the dimension of data analysis is enriched, the requirement of manual auditing is reduced through the automatic preset error correction model, and the accuracy of data analysis is improved. And the efficiency of identifying abnormal data in the field of digital finance is improved.

Description

Technical Field

[0001] The present application relates to the fields of data analysis and digital finance, and in particular to an abnormal data identification method, device, equipment and medium based on an error correction model. Background Art

[0002] In the current digital financial field, financial statements are an important reflection of the financial status of enterprises, and are also an important basis for investors, financial institutions and regulators to judge the health of enterprises. However, due to the complex financial data processing and recording process, there are some errors and anomalies in financial statements, such as repeated calculations, omissions, data entry errors, etc. These anomalies may lead to deviations in the judgment of the financial status of enterprises, thereby affecting investment decisions and risk control. It is very important to identify and correct financial statement anomalies. Existing financial statement anomaly identification technologies mainly include rule-based methods, statistical methods, deep learning-based methods, etc. Although these methods have certain application effects, they also have some limitations, such as the narrow scope of application of rule methods, the high requirements of statistical methods on data quality, and the weak interpretability of deep learning methods. Therefore, how to improve the efficiency of identifying abnormal data in the field of digital finance has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The present application provides an abnormal data identification method, device, equipment and medium based on an error correction model to improve the efficiency of identifying abnormal data in the field of digital finance.

[0004] In a first aspect, the present application provides a method for identifying abnormal data based on an error correction model, the method comprising:

[0005] Based on the correlation between the data indicators and the preset estimation model, determine the data indicator characteristics corresponding to each of the data indicators;

[0006] Generate external data features based on external data and a preset BERT-like model;

[0007] The target abnormal data is determined according to the preset error correction model, the data indicator characteristics and the external data characteristics.

[0008] In a second aspect, the present application also provides an abnormal data identification device based on an error correction model, the device comprising:

[0009] A data indicator feature determination module, used to determine the data indicator feature corresponding to each of the data indicators based on the correlation relationship between the data indicators and a preset estimation model;

[0010] An external data feature generation module is used to generate external data features based on external data and a preset BERT-like model;

[0011] The target abnormal data determination module is used to determine the target abnormal data according to a preset error correction model, the data indicator characteristics and the external data characteristics.

[0012] In a third aspect, the present application also provides a computer device, comprising a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the above-mentioned abnormal data identification method based on the error correction model when executing the computer program.

[0013] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the abnormal data identification method based on the error correction model as described above.

[0014] The present application discloses an abnormal data identification method, device, equipment and medium based on an error correction model. The abnormal data identification method based on the error correction model includes determining the data indicator characteristics corresponding to each of the data indicators based on the correlation relationship between the data indicators and the preset estimation model; generating external data characteristics based on external data and a preset BERT-like model; and determining target abnormal data according to the preset error correction model, the data indicator characteristics and the external data characteristics. In the above manner, the present application can more accurately predict and identify potential abnormal data by analyzing the correlation between data indicators. By using external data and BERT-like models, a wider range of information sources can be introduced to enrich the dimensions of data analysis. Through the automated preset error correction model, the need for manual review can be reduced, thereby improving the efficiency of identifying abnormal data in the field of digital finance. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0016] Figure 1 is a schematic flow chart of an abnormal data identification method based on an error correction model provided in the first embodiment of the present application;

[0017] Figure 2 is a schematic flow chart of an abnormal data identification method based on an error correction model provided in the second embodiment of the present application;

[0018] Figure 3is a schematic flow chart of an abnormal data identification method based on an error correction model provided in the third embodiment of the present application;

[0019] Figure 4 A schematic block diagram of an abnormal data identification device based on an error correction model provided in an embodiment of the present application;

[0020] Figure 5 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0022] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0023] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0024] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0025] The embodiments of the present application provide a method, device, equipment and medium for identifying abnormal data based on an error correction model. Among them, the abnormal data identification method based on the error correction model can be applied to a server. By analyzing the correlation between data indicators, potential abnormal data can be more accurately predicted and identified. By using external data and BERT-like models, a wider range of information sources can be introduced to enrich the dimensions of data analysis. Through the automated preset error correction model, the need for manual review is reduced, and the efficiency of identifying abnormal data in the field of digital finance is improved. Among them, the server can be an independent server or a server cluster.

[0026] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0027] See also Figure 1 , Figure 1 This is a schematic flow chart of an abnormal data identification method based on an error correction model provided by the first embodiment of the present application. The abnormal data identification method based on the error correction model can be applied to the server, and can more accurately predict and identify potential abnormal data by analyzing the correlation between data indicators. By using external data and BERT-like models, a wider range of information sources can be introduced to enrich the dimensions of data analysis. Through the automated preset error correction model, the need for manual review is reduced, and the efficiency of identifying abnormal data in the field of digital finance is improved.

[0028] like Figure 1 As shown, the abnormal data identification method based on the error correction model specifically includes steps S10 to S30.

[0029] Step S10: determining data indicator features corresponding to each data indicator based on the correlation relationship between the data indicators and a preset estimation model;

[0030] In one embodiment, the correlation between data indicators is not always direct, and sometimes multiple methods may be needed to determine the relationship between them. At the same time, it is necessary to select appropriate indicators and methods to study the correlation according to the specific situation.

[0031] This embodiment adopts the existing DeepFM-like model to mine the correlation between indicators, takes the financial statement indicators as the input of the DeepFM-like model, and learns the low-order and high-order interactions between different indicators, which helps to capture the complex relationship between different features and improve the accuracy of recognition.

[0032] Step S20: Generate external data features based on external data and a preset BERT-like model;

[0033] In a specific embodiment, when this embodiment is applied to financial statements, in addition to mining the correlation between financial indicators (data indicators), introducing external unstructured information such as related announcements and stocks into the model can enrich the representation of the vector, and use the external information as the input of a BERT (Bidirectional Encoder Representations from Transformers, a pre-trained language representation model) to output a text vector representation.

[0034] Step S30: determining target abnormal data according to a preset error correction model, the data indicator characteristics and the external data characteristics.

[0035] In one embodiment, the grammatical error correction model is applied to the task of identifying anomalies in financial statements. Currently, the grammatical error correction task is mainly implemented using a Seq2Seq framework similar to a machine translation task. Specifically, the input erroneous sentence is the source sentence, and the output correct sentence is the target sentence. This embodiment draws on this idea, compares the value of each indicator on the financial statement to a sentence, and uses normal indicators and abnormal indicators to identify the location of the abnormal indicator and automatically correct it through the error correction model. Therefore, in the model, this embodiment splices the text vector representation learned from external information with the various indicator values ​​of the financial statement as a sequence input to the Seq2Seq model, which can give the corresponding abnormal items.

[0036] For example, this embodiment applies syntax error correction to data indicator anomaly detection, integrates structured financial information with unstructured external information, and then uses the error correction model to provide the corresponding abnormal items in the financial statements. The specific process is divided into three parts:

[0037] First, a DeepFM-like method is used to identify low-order and high-order relationships between different data indicators, which can be financial indicators;

[0038] Secondly, in addition to data indicators, external information such as announcements and stock prices are introduced to jointly identify negative events in announcements, abnormal fluctuations in stock prices, and other information, and integrate them into the model;

[0039] Finally, the information learned above is used as a sequence and input into the error correction model to give the abnormal items in the corresponding financial statements.

[0040] This embodiment can be applied to the digital management scenario of financial business. Financial business and corporate financial statements may deviate from the real data due to input errors, repeated calculations, etc., which seriously affects the accuracy of corporate data records. For financial business such as health insurance, property insurance, life insurance data and corporate financial statements, the method of abnormality identification based on error correction technology in this embodiment can effectively reduce repeated calculations, omissions, and data entry errors in the data, thereby improving the accuracy of the data and better serving the needs of business development.

[0041] The present embodiment discloses an abnormal data identification method, device, equipment and medium based on an error correction model. The abnormal data identification method based on the error correction model includes determining the data indicator characteristics corresponding to each of the data indicators based on the correlation relationship between the data indicators and the preset estimation model; generating external data characteristics based on external data and a preset BERT-like model; and determining target abnormal data according to the preset error correction model, the data indicator characteristics and the external data characteristics. In the above manner, the present application can more accurately predict and identify potential abnormal data by analyzing the correlation between data indicators. By using external data and BERT-like models, a wider range of information sources can be introduced to enrich the dimensions of data analysis. Through the automated preset error correction model, the need for manual review can be reduced, thereby improving the efficiency of identifying abnormal data in the field of digital finance.

[0042] based on Figure 1 In the embodiment shown, in this embodiment, before step S10, the following steps are included:

[0043] Identifying an indicator value, an indicator ratio, and a time series of each of the data indicators, and determining an indicator characteristic of each of the data indicators based on the indicator value, the indicator ratio, and the time series;

[0044] Based on the preset correlation evaluation model and the indicator characteristics, a correlation relationship between the data indicators is established.

[0045] In one embodiment, when this embodiment is applied to a digital financial scenario, data containing numerical values ​​of financial indicators, such as revenue, cost, profit, etc., are collected to ensure the accuracy of the data and handle missing values ​​and outliers. Meaningful financial ratios, such as debt-to-asset ratio, current ratio, profit margin, etc., are determined and calculated based on the collected numerical data.

[0046] This embodiment ensures that the data is arranged in chronological order, such as by month, quarter or year, and analyzes the trend, seasonality, periodicity and other characteristics of the time series data. Identify the numerical range, distribution characteristics, etc. of each indicator. Analyze the stability and change trend of the ratio, determine the stationarity and autocorrelation of the time series, use the trained model to analyze the correlation strength and direction between different indicators, and build a correlation network between indicators based on the correlation analysis results.

[0047] The present embodiment discloses a method, device, equipment and medium for identifying abnormal data based on an error correction model. The method for identifying abnormal data based on an error correction model includes identifying the indicator value, indicator ratio and time series of each of the data indicators, and determining the indicator characteristics of each of the data indicators based on the indicator value, the indicator ratio and the time series; establishing the correlation relationship between each of the data indicators based on a preset correlation evaluation model and the indicator characteristics; determining the data indicator characteristics corresponding to each of the data indicators based on the correlation relationship between the data indicators and the preset estimation model; generating external data characteristics based on external data and a preset BERT-like model; determining target abnormal data according to the preset error correction model, the data indicator characteristics and the external data characteristics. Through the above-mentioned method, the present application better identifies and manages potential financial risks through correlation analysis by deeply understanding the intrinsic connection between different financial indicators. Time series analysis is used to predict future trends and provide a basis for strategic planning. The automated analysis process improves the efficiency of financial analysis, thereby improving the efficiency of identifying abnormal data in the field of digital finance.

[0048] In a more preferred embodiment, step S10 includes:

[0049] Identifying the high-order relationship and the low-order relationship between different data indicators through the DeepFM-like model;

[0050] Determining a feature weight of each of the data indicators according to the high-order relationship and the low-order relationship;

[0051] Based on the feature weight, the data indicator feature is determined.

[0052] In one embodiment, a DeepFM-like model is a machine learning model that combines a factorization machine (FM) and a deep neural network (DNN), which is mainly used to process large-scale data sets and capture complex interactions between features. This model is particularly suitable for recommendation systems, advertising click-through rate prediction, financial analysis and other fields. The DeepFM-like model includes a factorization machine (FM) part and a deep neural network (DNN) part. The factorization machine (FM) component is used to capture low-order interactions between features, especially binary interactions. The deep neural network (DNN) component is used to capture high-order interactions between features.

[0053] Clean the data, handle missing values ​​and outliers, and standardize or normalize the data to eliminate the dimensional effects between different indicators. Analyze the weights of the FM and DNN parts, identify which features contribute more to the model prediction, and determine the importance and characteristics of each data indicator based on the feature weights. Indicators with high feature weights may have a greater impact on the prediction results.

[0054] The present embodiment discloses a method, device, equipment and medium for identifying abnormal data based on an error correction model. The method for identifying abnormal data based on an error correction model includes identifying the high-order relationship and the low-order relationship between different data indicators through the DeepFM-like model; determining the feature weight of each data indicator based on the high-order relationship and the low-order relationship; determining the data indicator feature based on the feature weight; generating external data features based on external data and a preset BERT-like model; determining target abnormal data based on the preset error correction model, the data indicator feature and the external data feature. In the above manner, the present application can more accurately predict financial indicators by identifying high-order and low-order relationships, clarify which features have the greatest impact on the model's prediction results, and optimize the financial decision-making process based on feature weights. By identifying key features, better manage and reduce financial risks, and the feature weights learned by the model provide an intuitive understanding of the relationship between financial indicators, improving the efficiency of identifying abnormal data in the field of digital finance.

[0055] In a more preferred embodiment, identifying the high-order relationship and the low-order relationship between different data indicators through the DeepFM-like model includes:

[0056] Determine the linear relationship and the nonlinear relationship of each of the data indicators through the FM component, and determine the low-order relationship based on the linear relationship and the nonlinear relationship;

[0057] Each of the data indicators is nonlinearly transformed through the DNN component to determine the high-order relationship.

[0058] In one embodiment, a model structure is defined, including a factorization machine (FM) part and a deep neural network (DNN) part. The FM part is used to identify low-order relationships, which is usually achieved through factorization with fewer parameters; the DNN part is used to identify high-order relationships, which is usually achieved through a multi-layer perceptron (MLP).

[0059] The present embodiment discloses a method, device, equipment and medium for identifying abnormal data based on an error correction model. The method for identifying abnormal data based on an error correction model includes determining the linear relationship and nonlinear relationship of each of the data indicators through the FM component, and determining the low-order relationship based on the linear relationship and the nonlinear relationship; performing nonlinear transformation on each of the data indicators through the DNN component to determine the high-order relationship; determining the feature weight of each of the data indicators according to the high-order relationship and the low-order relationship; determining the data indicator feature based on the feature weight; generating external data features based on external data and a preset BERT-like model; determining target abnormal data based on the preset error correction model, the data indicator feature and the external data feature. Through the above-mentioned method, the present application can more accurately predict and identify potential abnormal data by analyzing the correlation between data indicators. By using external data and a BERT-like model, a wider range of information sources can be introduced to enrich the dimension of data analysis. Through an automated preset error correction model, the need for manual review can be reduced, thereby improving the efficiency of identifying abnormal data in the field of digital finance.

[0060] See also Figure 2 , Figure 2 This is a schematic flow chart of an abnormal data identification method based on an error correction model provided in the second embodiment of the present application. The abnormal data identification method based on the error correction model can be applied to a server to provide an in-depth semantic understanding of text data through a BERT model. The fused text vector provides a rich feature representation, which helps to improve the performance of the model, thereby improving the efficiency of identifying abnormal data in the field of digital finance.

[0061] based on Figure 1 The embodiment shown, this embodiment Figure 2 As shown, step S20 includes steps S201 to S203.

[0062] Step S201, converting the external data into data text, and extracting key information from the data text;

[0063] Step S202: Encode the key information through the BERT-like model to generate a key information text vector;

[0064] Step S203: performing vector fusion on the key information text vectors corresponding to the external data to generate the external data features.

[0065] In one embodiment, the external data is converted into text form, which may include describing numerical data, converting structured data into readable text, etc., extracting key information from the data text by using text analysis technology (such as keyword extraction, summary generation, etc.), and encoding the extracted key information using a pre-trained BERT-like model to generate a text vector. The key information text vector corresponding to each external data is fused to generate a comprehensive external data feature, and the fused vector can be used as a feature of the external data for further analysis or model training.

[0066] Specifically, external data sources (such as market reports, press releases, etc.) are converted into plain text format, and text processing techniques (such as NLP (Natural Language Processing) tools) are used to identify and extract key information in the text, such as entities, keywords, phrases, etc. A pre-trained BERT-like model is loaded, the extracted key information is input into the BERT model for encoding, and features are extracted from the output of the BERT model, such as the hidden state of the last layer, the output of the [CLS] tag, etc. For each external data point, there may be multiple key information text vectors. The fused vector constitutes the feature representation of the external data, and the generated external data features are used to train the prediction model or classification model.

[0067] The present embodiment discloses an abnormal data identification method, device, equipment and medium based on an error correction model. The abnormal data identification method based on the error correction model includes determining the data indicator features corresponding to each of the data indicators based on the correlation relationship between the data indicators and the preset estimation model; converting the external data into data text, and extracting the key information in the data text; encoding the key information through the BERT-like model to generate a key information text vector; performing vector fusion on the key information text vectors corresponding to each of the external data to generate the external data features; determining the target abnormal data according to the preset error correction model, the data indicator features and the external data features. In the above manner, the BERT model can provide an in-depth semantic understanding of the text data, and the fused text vector provides a rich feature representation, which helps to improve the performance of the model, thereby improving the efficiency of identifying abnormal data in the field of digital finance.

[0068] See also Figure 3 , Figure 3 This is a schematic flow chart of an abnormal data identification method based on an error correction model provided in the third embodiment of the present application. The abnormal data identification method based on the error correction model can be applied to the server, integrates the characteristics of data indicators and external data, provides a more comprehensive data analysis perspective, improves the accuracy of anomaly detection through the error correction model, and improves the efficiency of identifying abnormal data in the field of digital finance.

[0069] based on Figure 1 The embodiment shown, this embodiment Figure 3 As shown, step S30 includes step S301 to step S302.

[0070] Step S301: Fusing the data indicator features and the external data features to generate a fused feature sequence;

[0071] Step S302: identifying abnormal fields in the fused feature sequence through the preset error correction model, and determining the target abnormal data according to the abnormal fields.

[0072] In one embodiment, the data indicator features and external data features are concatenated at each data point to form a fused feature sequence, and different types of features are weighted to reflect their importance or influence in the model. For each data point, a sequence containing all fused features is constructed.

[0073] The fused feature sequence is input into the error correction model, which analyzes the sequence and identifies possible abnormal fields that deviate significantly from the normal pattern. Based on the location and content of the abnormal fields, the target abnormal data is determined, and these abnormal data are further reviewed or corrected. In addition, the recognition results of the model can be verified to ensure their accuracy and reliability.

[0074] The present embodiment discloses an abnormal data identification method, device, equipment and medium based on an error correction model, wherein the abnormal data identification method based on the error correction model includes determining the data indicator features corresponding to each of the data indicators based on the correlation relationship between the data indicators and the preset estimation model; generating external data features based on external data and a preset BERT-like model; fusing the data indicator features with the external data features to generate a fused feature sequence; identifying the abnormal fields in the fused feature sequence through the preset error correction model, and determining the target abnormal data based on the abnormal fields. In the above manner, the present application integrates the features of data indicators and external data, provides a more comprehensive data analysis perspective, improves the accuracy of anomaly detection through the error correction model, and improves the efficiency of identifying abnormal data in the field of digital finance.

[0075] Based on any of the above embodiments, in this embodiment, before step S30, the following steps are included:

[0076] Obtain a set of historical data indicators, and use the set of historical data indicators as a training set and a validation set;

[0077] Determining historical abnormal data in the historical data indicator set through the training set, the validation set and the initial error correction model;

[0078] The initial error correction model is trained based on the Seq2Seq method and the historical abnormal data to generate the preset error correction model.

[0079] In one embodiment, the historical data indicator set is divided into a training set and a validation set, and a suitable initial error correction model is selected, which can be a simple statistical model or a pre-trained deep learning model, and the initial error correction model is trained using the training set. The model performance is evaluated using the validation set, and the model parameters and structure are adjusted. Based on the Seq2Seq method, the initial error correction model is further trained using the determined historical abnormal data. After training with the Seq2Seq method, a preset error correction model is generated, which can identify and correct new abnormal data.

[0080] The present embodiment discloses a method, device, equipment and medium for identifying abnormal data based on an error correction model. The abnormal data identification method based on the error correction model includes determining the data indicator features corresponding to each of the data indicators based on the correlation relationship between the data indicators and the preset estimation model; generating external data features based on external data and a preset BERT-like model; obtaining a set of historical data indicators, and using the set of historical data indicators as a training set and a verification set; determining the historical abnormal data in the set of historical data indicators through the training set, the verification set and the initial error correction model; training the initial error correction model based on the Seq2Seq method and the historical abnormal data to generate the preset error correction model; determining the target abnormal data according to the preset error correction model, the data indicator features and the external data features. Through the above-mentioned method, the present application can more accurately predict and identify potential abnormal data by analyzing the correlation between data indicators. By using external data and BERT-like models, a wider range of information sources can be introduced to enrich the dimensions of data analysis. Through the automated preset error correction model, the need for manual review is reduced, and the efficiency of identifying abnormal data in the field of digital finance is improved.

[0081] See also Figure 4 , Figure 4 The embodiment of the present application provides a schematic block diagram of an abnormal data identification device based on an error correction model, and the abnormal data identification device based on the error correction model is used to execute the aforementioned abnormal data identification method based on the error correction model. The abnormal data identification device based on the error correction model can be configured on a server.

[0082] like Figure 4 As shown, the abnormal data identification device 400 based on the error correction model includes:

[0083] The data indicator feature determination module 410 is used to determine the data indicator feature corresponding to each of the data indicators based on the correlation relationship between the data indicators and the preset estimation model;

[0084] An external data feature generation module 420, for generating external data features based on external data and a preset BERT-like model;

[0085] The target abnormal data determination module 430 is used to determine the target abnormal data according to a preset error correction model, the data indicator characteristics and the external data characteristics.

[0086] Furthermore, the abnormal data identification device 400 based on the error correction model further includes:

[0087] An indicator characteristic determination module, used to identify an indicator value, an indicator ratio and a time series of each of the data indicators, and determine an indicator characteristic of each of the data indicators based on the indicator value, the indicator ratio and the time series;

[0088] The association relationship establishment module is used to establish the association relationship between each of the data indicators based on a preset association evaluation model and the indicator characteristics.

[0089] Furthermore, the data indicator feature determination module 410 includes:

[0090] A relationship identification unit, used for identifying the high-order relationship and the low-order relationship between different data indicators through the DeepFM-like model;

[0091] A feature weight determination unit, used to determine the feature weight of each of the data indicators according to the high-order relationship and the low-order relationship;

[0092] A data indicator feature determination unit is used to determine the data indicator feature based on the feature weight.

[0093] Furthermore, the relationship identification unit includes:

[0094] A low-order relationship determination subunit, used to determine the linear relationship and the nonlinear relationship of each of the data indicators through the FM component, and determine the low-order relationship based on the linear relationship and the nonlinear relationship;

[0095] The high-order relationship determination subunit is used to perform nonlinear transformation on each of the data indicators through the DNN component to determine the high-order relationship.

[0096] Furthermore, the external data feature generation module 420 includes:

[0097] A key information extraction unit, used for converting the external data into data text and extracting key information from the data text;

[0098] A key information text vector generating unit, used for encoding the key information through the BERT-like model to generate a key information text vector;

[0099] The external data feature generating unit is used to perform vector fusion on each of the key information text vectors corresponding to each of the external data to generate the external data feature.

[0100] Furthermore, the target abnormal data determination module 430 includes:

[0101] A fusion feature sequence generating unit, used for fusing the data indicator feature with the external data feature to generate a fusion feature sequence;

[0102] The target abnormal data determining unit is used to identify the abnormal fields in the fused feature sequence through the preset error correction model, and determine the target abnormal data according to the abnormal fields.

[0103] Furthermore, the abnormal data identification device 400 based on the error correction model further includes:

[0104] A historical data indicator set acquisition module is used to acquire a historical data indicator set and use the historical data indicator set as a training set and a verification set;

[0105] A historical abnormal data determination module, used to determine the historical abnormal data in the historical data indicator set through the training set, the verification set and the initial error correction model;

[0106] The preset error correction model generation module is used to train the initial error correction model based on the Seq2Seq method and the historical abnormal data to generate the preset error correction model.

[0107] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device and each module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0108] The above-mentioned device can be implemented in the form of a computer program. Figure 5 Runs on the computer device shown.

[0109] See also Figure 5 , Figure 5 1 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device may be a server.

[0110] See also Figure 5The computer device includes a processor, a memory and a network interface connected through a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0111] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any abnormal data identification method based on the error correction model.

[0112] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0113] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any abnormal data identification method based on the error correction model.

[0114] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0115] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0116] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0117] Based on the correlation between the data indicators and the preset estimation model, determine the data indicator characteristics corresponding to each of the data indicators;

[0118] Generate external data features based on external data and a preset BERT-like model;

[0119] The target abnormal data is determined according to the preset error correction model, the data indicator characteristics and the external data characteristics.

[0120] In one embodiment, based on the correlation between the data indicators and the preset estimation model, before determining the data indicator features corresponding to the data indicators, it is used to achieve:

[0121] Identifying an indicator value, an indicator ratio, and a time series of each of the data indicators, and determining an indicator characteristic of each of the data indicators based on the indicator value, the indicator ratio, and the time series;

[0122] Based on the preset correlation evaluation model and the indicator characteristics, a correlation relationship between the data indicators is established.

[0123] In one embodiment, based on the correlation between the data indicators and the preset estimation model, the data indicator features corresponding to the data indicators are determined to achieve:

[0124] Identifying the high-order relationship and the low-order relationship between different data indicators through the DeepFM-like model;

[0125] Determining a feature weight of each of the data indicators according to the high-order relationship and the low-order relationship;

[0126] Based on the feature weight, the data indicator feature is determined.

[0127] In one embodiment, the high-order relationship and the low-order relationship between different data indicators are identified by the DeepFM-like model to achieve:

[0128] Determine the linear relationship and the nonlinear relationship of each of the data indicators through the FM component, and determine the low-order relationship based on the linear relationship and the nonlinear relationship;

[0129] Each of the data indicators is nonlinearly transformed through the DNN component to determine the high-order relationship.

[0130] In one embodiment, external data features are generated based on external data and a preset BERT-like model to achieve:

[0131] Convert the external data into data text, and extract key information from the data text;

[0132] Encoding the key information through the BERT-like model to generate a key information text vector;

[0133] The key information text vectors corresponding to the external data are fused to generate the external data features.

[0134] In one embodiment, target abnormal data is determined according to a preset error correction model, the data indicator characteristics and the external data characteristics, so as to achieve:

[0135] Fusion of the data indicator features and the external data features to generate a fusion feature sequence;

[0136] The abnormal fields in the fused feature sequence are identified by the preset error correction model, and the target abnormal data are determined according to the abnormal fields.

[0137] In one embodiment, before determining the target abnormal data according to the preset error correction model, the data indicator characteristics and the external data characteristics, it is used to achieve:

[0138] Obtain a set of historical data indicators, and use the set of historical data indicators as a training set and a validation set;

[0139] Determining historical abnormal data in the historical data indicator set through the training set, the validation set and the initial error correction model;

[0140] The initial error correction model is trained based on the Seq2Seq method and the historical abnormal data to generate the preset error correction model.

[0141] A computer-readable storage medium is also provided in an embodiment of the present application, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and the processor executes the program instructions to implement any one of the abnormal data identification methods based on the error correction model provided in the embodiments of the present application.

[0142] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (Smart Medi a Card, SMC), a secure digital (Secure Digital, SD) card, a flash memory card (Flash Card), etc., equipped on the computer device.

[0143] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for identifying abnormal data based on an error correction model, characterized in that: include: Based on the correlation between the data indicators and the preset estimation model, determine the data indicator characteristics corresponding to each of the data indicators; Generate external data features based on external data and a preset BERT-like model; The target abnormal data is determined according to the preset error correction model, the data indicator characteristics and the external data characteristics.

2. The abnormal data identification method based on the error correction model according to claim 1 is characterized in that: Before determining the data indicator features corresponding to each of the data indicators based on the correlation relationship between the data indicators and the preset estimation model, the method includes: Identifying an indicator value, an indicator ratio, and a time series of each of the data indicators, and determining an indicator characteristic of each of the data indicators based on the indicator value, the indicator ratio, and the time series; Based on the preset correlation evaluation model and the indicator characteristics, a correlation relationship between the data indicators is established.

3. The abnormal data identification method based on the error correction model according to claim 2 is characterized in that: The association relationship includes a high-order relationship and a low-order relationship, the preset estimation model includes a DeepFM-like model, and the data indicator features corresponding to each of the data indicators are determined based on the association relationship between the data indicators and the preset estimation model, including: Identifying the high-order relationship and the low-order relationship between different data indicators through the DeepFM-like model; Determining a feature weight of each of the data indicators according to the high-order relationship and the low-order relationship; Based on the feature weight, the data indicator feature is determined.

4. The abnormal data identification method based on the error correction model according to claim 3 is characterized in that: The DeepFM-like model includes a factorization machine FM component and a deep neural network DNN component, and identifying the high-order relationship and the low-order relationship between different data indicators through the DeepFM-like model includes: Determine the linear relationship and the nonlinear relationship of each of the data indicators through the FM component, and determine the low-order relationship based on the linear relationship and the nonlinear relationship; Each of the data indicators is nonlinearly transformed through the DNN component to determine the high-order relationship.

5. The abnormal data identification method based on the error correction model according to claim 1 is characterized in that: The generating of external data features based on external data and a preset BERT-like model includes: Convert the external data into data text, and extract key information from the data text; Encoding the key information through the BERT-like model to generate a key information text vector; The key information text vectors corresponding to the external data are fused to generate the external data features.

6. The abnormal data identification method based on the error correction model according to claim 1 is characterized in that: The determining target abnormal data according to the preset error correction model, the data indicator characteristics and the external data characteristics includes: Fusion of the data indicator features and the external data features to generate a fusion feature sequence; The abnormal fields in the fused feature sequence are identified by the preset error correction model, and the target abnormal data are determined according to the abnormal fields.

7. The abnormal data identification method based on the error correction model according to any one of claims 1 to 6, characterized in that: Before determining the target abnormal data according to the preset error correction model, the data indicator characteristics and the external data characteristics, the method includes: Obtain a set of historical data indicators, and use the set of historical data indicators as a training set and a validation set; Determining historical abnormal data in the historical data indicator set through the training set, the validation set and the initial error correction model; The initial error correction model is trained based on the Seq2Seq method and the historical abnormal data to generate the preset error correction model.

8. An abnormal data identification device based on an error correction model, characterized in that: include: A data indicator feature determination module, used to determine the data indicator feature corresponding to each of the data indicators based on the correlation relationship between the data indicators and a preset estimation model; An external data feature generation module is used to generate external data features based on external data and a preset BERT-like model; The target abnormal data determination module is used to determine the target abnormal data according to a preset error correction model, the data indicator characteristics and the external data characteristics.

9. A computer device, characterized in that: The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the abnormal data identification method based on the error correction model as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the abnormal data identification method based on the error correction model as described in any one of claims 1 to 7.

Images