A data anomaly identification method, device, equipment and storage medium thereof

Abstract

The embodiment of the application belongs to the technical field of artificial intelligence and financial technology, is applied to a claim risk prediction business, and relates to a data anomaly identification method, device and equipment and a storage medium thereof, comprising: obtaining test data to be identified; performing time sequence processing and feature extraction on the test data to obtain a test feature set; performing update processing on the test feature set; inputting the updated test feature set into a trained data anomaly identification model; and predicting whether the test data is abnormal data according to the trained data anomaly identification model. The long short-term memory network is introduced to solve the problem that the related business data of traditional vehicle claims involves multiple links and is not conducive to time sequence training. Meanwhile, the gradient descent algorithm is introduced to screen out the model hyperparameter combination with the fastest cost function decreasing speed, so as to ensure the training convergence speed and the prediction speed, quickly predict whether the claim data is abnormal or not, and facilitate intelligent analysis of the claim data.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and financial technology, and is applied to claims risk prediction business, particularly to a data anomaly identification method, device, equipment and its storage medium. Background Technology

[0002] With the development of the computer industry, traditional financial businesses are gradually transforming towards fintech businesses, especially in insurance claims, where auto insurance is a common area of ​​business. Auto insurance has the largest number of claims and the highest payouts in the insurance industry. Clearly, risk control is paramount in insurance claims. Whether it's pre-insurance underwriting rules, in-claims risk control models, or post-claims corporate profiling, all are designed to better address risk leakage.

[0003] In existing auto insurance risk control, most deep learning models used are RNN (Recurrent Neural Network) models. RNN models excel at handling continuous features, where the output of one business process is the input of the next, and each link is tightly connected, with the previous result directly influencing the next. This makes them suitable for processing sequential business processes. However, for insurance claims, the various stages are relatively independent and cannot be connected, or the output of one stage is not the input of the next. Furthermore, the time-series data for claims is excessive and long-lasting, leading to severe gradient decay. Continuing to use RNN models in this case would severely impact the model's predictive performance and reduce accuracy. Summary of the Invention

[0004] The purpose of this application is to propose a data anomaly identification method, apparatus, device and its storage medium to solve the problem that in the prior art, the use of RNN models in claims business prediction will seriously affect the model prediction effect and reduce the prediction accuracy.

[0005] To address the aforementioned technical problems, this application provides a data anomaly identification method, employing the following technical solution:

[0006] A method for identifying data anomalies includes the following steps:

[0007] Obtain the test data to be identified;

[0008] The test data is subjected to time-series processing and feature extraction to obtain a test feature set;

[0009] The test feature set is subjected to feature engineering to obtain derived features;

[0010] The derived features are incorporated into the test feature set to update the test feature set;

[0011] The updated test feature set is input into the trained data anomaly detection model, and the trained data anomaly detection model is used to predict whether the test data is anomaly data.

[0012] Furthermore, the step of performing time-series processing and feature extraction on the test data to obtain a test feature set specifically includes:

[0013] The test data is organized into time series data according to a preset arrangement rule;

[0014] Based on a preset feature keyword table, feature data is extracted from the time series data to obtain a test feature set;

[0015] The step of extracting feature data from the time series data according to a preset feature keyword table to obtain a test feature set specifically includes:

[0016] Determine whether all feature data extracted from the feature keyword table are target extraction results;

[0017] If there are non-target extraction results, the non-target extraction results are filled with missing values, and the filled data is used as the corresponding target extraction results.

[0018] All target extraction results are obtained and added to a preset ordered set according to the time series data to generate a test feature set.

[0019] Furthermore, before performing the step of inputting the updated test feature set into the trained data anomaly detection model, the method further includes:

[0020] Obtain full sample data, wherein the full sample data includes positive sample data and negative sample data, the positive sample data is abnormal case closure data in the claims, and the negative sample data is normal case closure data in the claims;

[0021] According to the preset sampling method, the positive sample data is resampled to obtain the resampling result, and new positive sample data is obtained based on the resampling result;

[0022] The negative sample data and the new positive sample data are input together into the pre-built data anomaly detection model for model training to obtain the pre-trained data anomaly detection model. The data anomaly detection model is jointly constructed by a long short-term memory network and a gradient descent algorithm.

[0023] Input the full sample data into the pre-trained data anomaly identification model and obtain the model output results;

[0024] The output of the pre-trained data anomaly identification model is verified based on the annotation results of the full sample data and the model output results.

[0025] If the verification fails, the hyperparameters of the pre-built data anomaly identification model are tuned and iteratively trained until the model loss value meets the preset loss threshold. If the verification is successful, the iterative training stops and a successfully trained data anomaly identification model is obtained.

[0026] Furthermore, the step of resampling the positive sample data according to a preset sampling method, obtaining the resampling result, and obtaining new positive sample data based on the resampling result specifically includes:

[0027] Each data point in the positive sample data is upsampled according to a pre-set sampling frequency to obtain the resampling result;

[0028] The resampling results are merged into the positive sample data to form new positive sample data;

[0029] The step of inputting the negative sample data and the new positive sample data into the pre-constructed data anomaly detection model for training to obtain the pre-trained data anomaly detection model specifically includes:

[0030] The negative sample data and the new positive sample data are input together into the pre-built data anomaly identification model for model training;

[0031] According to the preset filtering rules, target recognition models are selected and used as the pre-trained data anomaly recognition models.

[0032] Furthermore, the step of selecting target recognition models according to preset filtering rules and using the target recognition models as the pre-trained data anomaly recognition models specifically includes:

[0033] Initialize the model hyperparameters, where the model parameters refer to configuration parameters outside the model, including learning rate, number of iterations, batch size, and the number of hidden layers in the long short-term memory network;

[0034] Randomly combine model hyperparameters to obtain all model hyperparameter combinations, and then set all the model hyperparameter combinations to the pre-built data anomaly identification model respectively;

[0035] The gradient descent algorithm is used to calculate the rate of decrease of the cost function corresponding to each combination of hyperparameters of the model during the model training process.

[0036] The model hyperparameter combination with the fastest cost function decrease rate is selected as the target hyperparameter combination.

[0037] Obtain the data anomaly identification model corresponding to the target hyperparameter combination as the target model, and use the target identification model as the pre-trained data anomaly identification model.

[0038] Furthermore, the step of verifying the output of the pre-trained data anomaly detection model based on the annotation results of the full sample data and the model output results specifically includes:

[0039] Based on the model output, count the number of positive and negative sample data output by the model;

[0040] Based on the annotation results of the full sample data, the number of actual positive and negative sample data in the full sample data is counted.

[0041] The number of positive and negative sample data output by the model and the actual number of positive and negative sample data in the full sample data are input as method parameters into the preset loss function to obtain the model loss value.

[0042] The model loss value is compared with a preset loss threshold.

[0043] If the model loss value is less than the preset loss threshold, the verification is successful;

[0044] If the model loss value is not less than the preset loss threshold, the verification fails.

[0045] Furthermore, the step of predicting whether the test data is anomalous based on the trained data anomaly identification model specifically includes:

[0046] Obtain the output of the trained data anomaly detection model;

[0047] If the output shows that the test data is positive sample data, then the test data is abnormal data;

[0048] If the output shows that the test data is negative sample data, then the test data is normal data.

[0049] To address the aforementioned technical problems, this application also provides a data anomaly identification device, which employs the following technical solution:

[0050] A data anomaly detection device, comprising:

[0051] The test data acquisition module is used to acquire the test data to be identified.

[0052] The feature extraction module is used to perform time-series processing and feature extraction on the test data to obtain a test feature set;

[0053] The feature engineering module is used to perform feature engineering processing on the test feature set to obtain derived features;

[0054] The feature update module is used to incorporate the derived features into the test feature set to update the test feature set;

[0055] The model recognition module is used to input the updated test feature set into the trained data anomaly recognition model, and predict whether the test data is anomalous based on the trained data anomaly recognition model.

[0056] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:

[0057] A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the data anomaly identification method described above.

[0058] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:

[0059] A computer-readable storage medium storing computer-readable instructions, which, when executed by a processor, implement the steps of the data anomaly identification method described above.

[0060] Compared with the prior art, the embodiments of this application have the following main advantages:

[0061] The data anomaly identification method described in this application involves: acquiring test data to be identified; performing time-series processing and feature extraction on the test data to obtain a test feature set; performing feature engineering on the test feature set to obtain derived features; incorporating the derived features into the test feature set to update the test feature set; inputting the updated test feature set into a trained data anomaly identification model; and predicting whether the test data is anomalous based on the trained data anomaly identification model. The method considers introducing a Long Short-Term Memory (LSTM) network to address the problem that using a conventional RNN to process time-series data across multiple stages during prediction can lead to a significant decrease in model prediction performance. Simultaneously, it combines an LSTM neural network to predict the correspondence between feature data and positive / negative output nodes in each stage of the auto insurance claims process. A gradient descent algorithm is introduced into the LSTM neural network to select the model hyperparameter combination with the fastest cost function reduction rate, further ensuring the model's training convergence speed and prediction speed. This tightly connects each stage of the claims process, quickly predicting whether claims data is abnormal, facilitating intelligent anomaly analysis of claims data. Attached Figure Description

[0062] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 This is an exemplary system architecture diagram to which this application can be applied;

[0064] Figure 2 This is a flowchart of an embodiment of the data anomaly identification method according to this application;

[0065] Figure 3 yes Figure 2 A flowchart of a specific embodiment of step 202 shown;

[0066] Figure 4 This is a flowchart of a specific embodiment of training a pre-built data anomaly identification model according to the data anomaly identification method described in the embodiments of this application;

[0067] Figure 5 yes Figure 4 A flowchart of a specific embodiment of step 402 shown;

[0068] Figure 6 yes Figure 4 A flowchart of a specific embodiment of step 403 shown;

[0069] Figure 7 yes Figure 6 A flowchart of a specific embodiment of step 602 shown;

[0070] Figure 8 yes Figure 4 A flowchart of a specific embodiment of step 405 shown;

[0071] Figure 9 This is a schematic diagram of a structure of an embodiment of the data anomaly identification device according to this application;

[0072] Figure 10 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0073] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0074] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0075] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0076] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0077] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0078] Terminal devices 101, 102, and 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc.

[0079] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on terminal devices 101, 102, and 103.

[0080] It should be noted that the data anomaly identification method provided in this application embodiment is generally executed by the server, and correspondingly, the data anomaly identification device is generally set in the server.

[0081] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0082] Continue to refer to Figure 2 A flowchart of an embodiment of the data anomaly identification method according to this application is shown. The data anomaly identification method includes the following steps:

[0083] Step 201: Obtain the test data to be identified.

[0084] In this embodiment, the test data to be identified includes all audit data related to claims processing after a car insurance accident occurs, including data from the reporting stage, the investigation stage, the accident liability determination stage, and the policy audit stage.

[0085] In this embodiment, the test data to be identified refers to a series of time-series data used for fraud identification. For example, the data from the reporting stage, the investigation stage, the accident liability determination stage, and the policy review stage have a certain time sequence in the entire claims event.

[0086] Step 202: Perform time-series processing and feature extraction on the test data to obtain a test feature set.

[0087] Continue to refer to Figure 3 , Figure 3 yes Figure 2 A flowchart of a specific embodiment of step 202 shown includes:

[0088] Step 301: Organize the test data into time series data according to the preset arrangement rules;

[0089] In this embodiment, the test data is organized into time series data according to a preset arrangement rule. That is, the test data is organized into time series data according to the time sequence of data in the reporting stage, investigation stage, accident liability determination stage, and policy review stage in the entire claims business process.

[0090] The time-series data refers to data arranged in chronological order. Accordingly, in the entire claims process, the data in the reporting stage, the investigation stage, the accident liability determination stage, and the policy review stage have a certain sequential order. Therefore, the test data is organized into time-series data by arranging the data in the reporting stage, the investigation stage, the accident liability determination stage, and the policy review stage in chronological order.

[0091] Step 302: Extract feature data from the time series data according to the preset feature keyword table to obtain the test feature set;

[0092] Specifically, feature keywords are pre-defined based on the entire claims process, specifically for data from the reporting stage, investigation stage, liability determination stage, and policy review stage. A feature keyword table is then constructed based on the corresponding feature keywords for each of these stages. The feature keywords can be determined using batches of historical claims data as empirical data, employing word frequency calculation to identify the specific feature keywords for each stage.

[0093] The step of extracting feature data from the time series data according to a preset feature keyword table to obtain a test feature set specifically includes:

[0094] Step 3021: Determine whether all feature data extracted from the feature keyword table are target extraction results;

[0095] Step 3022: If there are non-target extraction results, fill in the missing values ​​of the non-target extraction results and use the filled data as the corresponding target extraction results.

[0096] Specifically, if the feature data corresponding to a keyword cannot be extracted or the extraction data is incorrect, it means that the feature data corresponding to the keyword is not the target extraction result. In this case, missing value imputation is performed on the non-target extraction result, and the imputed data is used as the corresponding target extraction result. That is, when performing feature extraction, some feature data cannot be extracted and there are missing values. At this time, the mode data corresponding to the target feature data in the historical test data can be obtained, and the missing values ​​can be imputed with the mode data. The mode data refers to the data content with the highest frequency corresponding to the target feature data in the historical claims data.

[0097] Step 3023: Obtain all target extraction results and add all target extraction results into a preset ordered set according to the arrangement order of the time series data to generate a test feature set.

[0098] In this embodiment, by adding all feature data into a preset ordered set according to the arrangement order of the time series data, a test feature set is generated. This eliminates the need to input the entire text to be identified during testing, thereby reducing the amount of input data and removing most non-feature data, thus saving more computing resources.

[0099] Step 203: Perform feature engineering on the test feature set to obtain derived features.

[0100] Specifically, the feature engineering process performed on the test feature set to obtain derived features includes: creating new features based on existing features in the test feature set. For example, if the test feature set includes the reporting time and the time of the incident, new feature data such as the interval between the reporting time and the time of the incident can be created based on these two times. Another example is if the test feature set includes the date of the incident, and new feature data such as whether the date of the incident is a holiday can be created based on this date. Yet another example is if the test feature set includes the location of the incident, and new feature data such as whether the location of the incident is a bar can be created based on this location. Finally, if the test feature set includes the time of the incident, and new feature data such as whether the time of the incident is at night can be created based on this time.

[0101] Specifically, different derivation rules can be used to derive features from the test feature set to obtain derived features. In addition, for numerical feature data, multinomial derivation can also be used to obtain derived features. Feature derivation methods include, but are not limited to, univariate feature derivation, bivariate feature derivation, and multivariate feature derivation, where univariate, bivariate, and multivariate refer to the number of variables on which the feature data is derived. Alternatively, a text feature derivation method can be used, which includes analyzing the semantic information of the feature data in the test feature set and performing feature derivation based on the analyzed semantic information.

[0102] Step 204: Incorporate the derived features into the test feature set to update the test feature set.

[0103] By incorporating the derived features into the test feature set to update the test feature set, the data in the test feature set is further enriched, and the dimensionality of the feature data is increased. By increasing the dimensionality of the feature data, the accuracy of model recognition is further improved.

[0104] Step 205: Input the updated test feature set into the trained data anomaly detection model, and predict whether the test data is anomaly data based on the trained data anomaly detection model.

[0105] In this embodiment, before performing the step of inputting the updated test feature set into the trained data anomaly identification model, the method further includes: training the pre-built data anomaly identification model.

[0106] Continue to refer to Figure 4 , Figure 4 The flowchart shown illustrates a specific embodiment of training a pre-built data anomaly detection model using the data anomaly detection method according to embodiments of this application, including:

[0107] Step 401: Obtain full sample data, wherein the full sample data includes positive sample data and negative sample data, the positive sample data is abnormal case closure data in the claims, and the negative sample data is normal case closure data in the claims.

[0108] Specifically, obtaining the full sample data refers to obtaining all closed claims data in the auto insurance accident claims system. The full sample data includes positive and negative sample data that have been marked as true or false. The positive sample data consists of abnormally closed claims data, and the negative sample data consists of normally closed claims data. In other words, all closed claims data in the auto insurance accident claims system includes data that was normally closed through the claims process and data that was abnormally closed due to fraudulent claims. The negative sample data consists of data that was normally closed through the claims process, and the positive sample data consists of data that was abnormally closed due to fraudulent claims.

[0109] Step 402: Resample the positive sample data according to the preset sampling method, obtain the resampling result, and obtain new positive sample data based on the resampling result;

[0110] Since the data of all closed claims in the auto insurance accident claims system often includes a large number of cases that were closed normally through the claims process, possibly accounting for 98% of the total data, while the data of abnormally closed cases that were identified as fraudulent claims may only account for about 1%, the ratio of positive and negative sample data is extremely unbalanced. Therefore, it is considered to resample the positive sample data to increase the amount of positive sample data and make the ratio of positive and negative sample data more balanced.

[0111] Continue to refer to Figure 5 , Figure 5 yes Figure 4 A flowchart of a specific embodiment of step 402 shown includes:

[0112] Step 501: Upsample each data point in the positive sample data according to a preset sampling frequency to obtain the resampling result;

[0113] In this embodiment, upsampling means increasing the sampling frequency, collecting data from low-dimensional data to high-dimensional data, which is to obtain dense data from sparse data in higher dimensions.

[0114] The resampling process includes resampling the positive sample data using the resample function in the Pandas data analysis tool. Specifically, the Pandas library is a free and open-source third-party Python library and a Python data analysis tool. For time series data, since the amount of positive sample data is small, it can be regarded as having missing data. By setting the sampling frequency as the input parameter of the resample function, the resample function is used to upsample and enrich the amount of positive sample data.

[0115] Step 502: The resampling results are merged into the positive sample data to form new positive sample data.

[0116] Step 403: Input the negative sample data and the new positive sample data into the pre-built data anomaly detection model, train the model, and obtain the pre-trained data anomaly detection model. The data anomaly detection model is jointly constructed by a long short-term memory network and a gradient descent algorithm.

[0117] The Long Short-Term Memory (LSTM) network can be a recurrent LSTM neural network. Since the claims process includes different stages such as reporting, investigation, liability determination, and policy review, training involves data from multiple stages. Introducing an LSTM network addresses the issue that using a standard RNN would significantly reduce predictive performance when dealing with time-series data across multiple stages. Furthermore, a gradient descent algorithm is incorporated into the LSTM network to facilitate calculations of the LSTM network's processing results during model training, allowing for the determination of the model's cost function's rate of change.

[0118] Continue to refer to Figure 6 , Figure 6 yes Figure 4 A flowchart of a specific embodiment of step 403 shown includes:

[0119] Step 601: Input the negative sample data and the new positive sample data into the pre-built data anomaly identification model for model training;

[0120] In this embodiment, before performing the step of inputting the negative sample data and the new positive sample data into the pre-built data anomaly identification model for model training, the method further includes: performing time-series processing and feature extraction on the negative sample data and the new positive sample data to obtain a negative sample feature set and a positive sample feature set, respectively; performing feature engineering processing on the negative sample feature set and the positive sample feature set to obtain derived features; and incorporating the derived features into the corresponding sample feature set to update the negative sample feature set and the positive sample feature set.

[0121] Specifically, the timing processing, feature extraction, and feature engineering methods used during testing are consistent with those used during training.

[0122] Step 602: Select target recognition models according to preset screening rules, and use the target recognition models as the pre-trained data anomaly recognition models.

[0123] Continue to refer to Figure 7 , Figure 7 yes Figure 6 A flowchart of a specific embodiment of step 602 shown includes:

[0124] Step 701: Initialize model hyperparameters, wherein the model parameters refer to configuration parameters outside the model, including learning rate, number of iterations, batch size and the number of hidden layers of the long short-term memory network;

[0125] Step 702: Randomly combine model hyperparameters to obtain all model hyperparameter combinations, and set all model hyperparameter combinations to the pre-built data anomaly identification model respectively;

[0126] Step 703: Calculate the rate of decrease of the cost function corresponding to each combination of hyperparameters of the model during the model training process according to the gradient descent algorithm;

[0127] Step 704: Select the model hyperparameter combination with the fastest cost function reduction rate as the target hyperparameter combination;

[0128] Step 705: Obtain the data anomaly identification model corresponding to the target hyperparameter combination as the target model, and use the target identification model as the pre-trained data anomaly identification model.

[0129] The gradient descent algorithm is used to select the model hyperparameter combination with the fastest cost function reduction rate as the target hyperparameter combination, thereby determining the target model and further ensuring the training convergence speed and prediction speed of the model.

[0130] Step 404: Input the full sample data into the pre-trained data anomaly identification model and obtain the model output results;

[0131] Step 405: Based on the annotation results of the full sample data and the model output results, perform output verification on the pre-trained data anomaly identification model;

[0132] Continue to refer to Figure 8 , Figure 8 yes Figure 4 A flowchart of a specific embodiment of step 405 shown includes:

[0133] Step 801: Based on the model output results, count the number of positive and negative sample data output by the model;

[0134] Step 802: Based on the annotation results of the full sample data, count the actual number of positive and negative sample data in the full sample data;

[0135] Step 803: Input the number of positive and negative sample data output by the model and the actual number of positive and negative sample data in the full sample data as method parameters into the preset loss function to obtain the model loss value;

[0136] Step 804: Compare the model loss value with a preset loss threshold.

[0137] Step 805: If the model loss value is less than the preset loss threshold, the verification is successful;

[0138] Step 806: If the model loss value is not less than the preset loss threshold, the verification fails.

[0139] In this embodiment, when the loss function is called and executed, it can calculate the verification sample, that is, the proportion of the number of unidentified samples in the full sample data, and it can also calculate the recognition degree of positive and negative sample data. The recognition degree of positive and negative sample data is obtained by proportional calculation based on the number of positive and negative sample data output by the model and the number of positive and negative sample data in the full sample data.

[0140] Step 406: If the verification fails, the hyperparameters of the pre-built data anomaly identification model are tuned and iteratively trained until the model loss value meets the preset loss threshold. If the verification is successful, the iterative training is stopped, and a successfully trained data anomaly identification model is obtained.

[0141] In this embodiment, the purpose of training the pre-built data anomaly identification model is mainly to combine the LSTM neural network to predict the correspondence between the feature data of each link in the auto insurance claims business and the positive and negative output nodes.

[0142] In this embodiment, the step of predicting whether the test data is abnormal data based on the trained data anomaly identification model specifically includes: obtaining the output result of the trained data anomaly identification model; if the output result shows that the test data is positive sample data, then the test data is abnormal data; if the output result shows that the test data is negative sample data, then the test data is normal data.

[0143] This application obtains test data to be identified; performs time-series processing and feature extraction on the test data to obtain a test feature set; performs feature engineering on the test feature set to obtain derived features; incorporates the derived features into the test feature set to update the test feature set; inputs the updated test feature set into a trained data anomaly identification model, and predicts whether the test data is abnormal based on the trained data anomaly identification model. It considers introducing a Long Short-Term Memory (LSTM) network to address the problem that using a regular RNN to process time-series data across multiple stages during prediction can lead to a significant decrease in model prediction performance. Simultaneously, it combines an LSTM neural network to predict the correspondence between feature data and positive / negative output nodes in each stage of the auto insurance claims process. A gradient descent algorithm is introduced into the LSTM neural network to select the model hyperparameter combination with the fastest cost function reduction speed, further ensuring the model's training convergence speed and prediction speed. This tightly connects each stage of the claims process, quickly predicting whether there are anomalies in the claims data, facilitating intelligent anomaly analysis of the claims data.

[0144] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0145] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0146] In this embodiment, test data to be identified is acquired; time-series processing and feature extraction are performed on the test data to obtain a test feature set; feature engineering is performed on the test feature set to obtain derived features; the derived features are incorporated into the test feature set to update the test feature set; the updated test feature set is input into a trained data anomaly identification model, and the trained data anomaly identification model predicts whether the test data is anomalous. A Long Short-Term Memory (LSTM) network is introduced to address the problem that using a conventional RNN to process time-series data across multiple stages during prediction can lead to a significant decrease in model prediction performance. Furthermore, a gradient descent algorithm is introduced into the LSM network to select the model hyperparameter combination with the fastest cost function reduction rate, further ensuring the model's training convergence speed and prediction speed.

[0147] Further reference Figure 9 As a response to the above Figure 2 To implement the method shown, this application provides an embodiment of a data anomaly identification device, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0148] like Figure 9 As shown, the data anomaly identification device 900 described in this embodiment includes: a test data acquisition module 901, a feature extraction module 902, a feature engineering module 903, a feature update module 904, and a model recognition module 905. Wherein:

[0149] The test data acquisition module 901 is used to acquire the test data to be identified;

[0150] Feature extraction module 902 is used to perform time-series processing and feature extraction on the test data to obtain a test feature set;

[0151] Feature engineering module 903 is used to perform feature engineering processing on the test feature set to obtain derived features;

[0152] The feature update module 904 is used to incorporate the derived features into the test feature set in order to update the test feature set;

[0153] The model recognition module 905 is used to input the updated test feature set into the trained data anomaly recognition model, and predict whether the test data is abnormal based on the trained data anomaly recognition model.

[0154] This application obtains test data to be identified; performs time-series processing and feature extraction on the test data to obtain a test feature set; performs feature engineering on the test feature set to obtain derived features; incorporates the derived features into the test feature set to update the test feature set; inputs the updated test feature set into a trained data anomaly identification model, and predicts whether the test data is abnormal based on the trained data anomaly identification model. It considers introducing a Long Short-Term Memory (LSTM) network to address the problem that using a regular RNN to process time-series data across multiple stages during prediction can lead to a significant decrease in model prediction performance. Simultaneously, it combines an LSTM neural network to predict the correspondence between feature data and positive / negative output nodes in each stage of the auto insurance claims process. A gradient descent algorithm is introduced into the LSTM neural network to select the model hyperparameter combination with the fastest cost function reduction speed, further ensuring the model's training convergence speed and prediction speed. This tightly connects each stage of the claims process, quickly predicting whether there are anomalies in the claims data, facilitating intelligent anomaly analysis of the claims data.

[0155] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0156] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0157] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 10 , Figure 10 This is a basic structural block diagram of the computer device in this embodiment.

[0158] The computer device 10 includes a memory 10a, a processor 10b, and a network interface 10c that are interconnected via a system bus. It should be noted that only the computer device 10 with components 10a-10c is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0159] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0160] The memory 10a includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10a may be an internal storage unit of the computer device 10, such as the hard disk or memory of the computer device 10. In other embodiments, the memory 10a may also be an external storage device of the computer device 10, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. Of course, the memory 10a may include both internal storage units and external storage devices of the computer device 10. In this embodiment, the memory 10a is typically used to store the operating system and various application software installed on the computer device 10, such as computer-readable instructions for data anomaly detection methods. In addition, the memory 10a can also be used to temporarily store various types of data that have been output or will be output.

[0161] In some embodiments, the processor 10b may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 10b is typically used to control the overall operation of the computer device 10. In this embodiment, the processor 10b is used to execute computer-readable instructions stored in the memory 10a or to process data, for example, to execute computer-readable instructions for the data anomaly detection method.

[0162] The network interface 10c may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 10 and other electronic devices.

[0163] The computer device proposed in this embodiment belongs to the fields of artificial intelligence and financial technology, and is applied to claims risk prediction business. This application obtains test data to be identified; performs time-series processing and feature extraction on the test data to obtain a test feature set; performs feature engineering on the test feature set to obtain derived features; incorporates the derived features into the test feature set to update the test feature set; inputs the updated test feature set into a trained data anomaly identification model, and predicts whether the test data is abnormal based on the trained data anomaly identification model. Consideration is given to introducing a Long Short-Term Memory (LSTM) network to address the problem that using a regular RNN to process time-series data involves multiple stages during prediction, which would lead to a significant decrease in model prediction performance. Simultaneously, a gradient descent algorithm is introduced into the LTM network to select the model hyperparameter combination with the fastest cost function reduction speed, further ensuring the model's training convergence speed and prediction speed.

[0164] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by a processor to cause the processor to perform the steps of the data anomaly identification method described above.

[0165] The computer-readable storage medium proposed in this embodiment belongs to the fields of artificial intelligence and financial technology, and is applied in claims risk prediction business. This application obtains test data to be identified; performs time-series processing and feature extraction on the test data to obtain a test feature set; performs feature engineering on the test feature set to obtain derived features; incorporates the derived features into the test feature set to update the test feature set; inputs the updated test feature set into a trained data anomaly identification model, and predicts whether the test data is abnormal based on the trained data anomaly identification model. Considering the introduction of a Long Short-Term Memory (LSTM) network to address the problem that using a regular RNN to process time-series data involves multiple stages during prediction can lead to a significant decrease in model prediction performance. Simultaneously, a gradient descent algorithm is introduced into the LTM network to select the model hyperparameter combination with the fastest cost function reduction speed, further ensuring the model's training convergence speed and prediction speed.

[0166] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0167] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A method for identifying data anomalies, characterized in that, Includes the following steps: Acquire test data to be identified, wherein the test data to be identified includes all audit data related to claims business after a car insurance accident occurs, including data from the reporting stage, the investigation stage, the accident liability determination stage, and the policy review stage; The test data is subjected to time-series processing and feature extraction to obtain a test feature set. The time-series processing of the test data means that the test data is organized into time-series data according to the time sequence of the data in the reporting stage, the investigation stage, the accident liability determination stage, and the policy review stage in the entire claims business process. The test feature set is subjected to feature engineering to obtain derived features; The derived features are incorporated into the test feature set to update the test feature set; The updated test feature set is input into the trained data anomaly detection model, and the trained data anomaly detection model is used to predict whether the test data is anomaly data. The method further includes, before performing the step of inputting the updated test feature set into the trained data anomaly detection model: Obtain full sample data, wherein the full sample data includes positive sample data and negative sample data, the positive sample data is abnormal case closure data in the claims, and the negative sample data is normal case closure data in the claims; According to the preset sampling method, the positive sample data is resampled to obtain the resampling result, and new positive sample data is obtained based on the resampling result; The negative sample data and the new positive sample data are input together into the pre-built data anomaly detection model for model training to obtain the pre-trained data anomaly detection model. The data anomaly detection model is jointly constructed by a long short-term memory network and a gradient descent algorithm. Input the full sample data into the pre-trained data anomaly identification model and obtain the model output results; The output of the pre-trained data anomaly identification model is verified based on the annotation results of the full sample data and the model output results. If the verification fails, the hyperparameters of the pre-built data anomaly identification model are tuned and iteratively trained until the model loss value meets the preset loss threshold. If the verification is successful, the iterative training stops and a successfully trained data anomaly identification model is obtained.

2. The data anomaly identification method according to claim 1, characterized in that, The step of performing time-series processing and feature extraction on the test data to obtain a test feature set specifically includes: The test data is organized into time series data according to a preset arrangement rule; Based on a preset feature keyword table, feature data is extracted from the time series data to obtain a test feature set; The step of extracting feature data from the time series data according to a preset feature keyword table to obtain a test feature set specifically includes: Determine whether all feature data extracted from the feature keyword table are target extraction results; If there are non-target extraction results, the non-target extraction results are filled with missing values, and the filled data is used as the corresponding target extraction results. All target extraction results are obtained and added to a preset ordered set according to the time series data to generate a test feature set.

3. The data anomaly identification method according to claim 1, characterized in that, The steps of resampling the positive sample data according to a preset sampling method, obtaining the resampling result, and obtaining new positive sample data based on the resampling result specifically include: Each data point in the positive sample data is upsampled according to a pre-set sampling frequency to obtain the resampling result; The resampling results are merged into the positive sample data to form new positive sample data; The step of inputting the negative sample data and the new positive sample data into the pre-constructed data anomaly detection model for training to obtain the pre-trained data anomaly detection model specifically includes: The negative sample data and the new positive sample data are input together into the pre-built data anomaly identification model for model training; According to the preset filtering rules, target recognition models are selected and used as the pre-trained data anomaly recognition models.

4. The data anomaly identification method according to claim 3, characterized in that, The step of selecting target recognition models according to preset filtering rules and using the target recognition models as the pre-trained data anomaly recognition models specifically includes: Initialize the model hyperparameters, where the model parameters refer to configuration parameters outside the model, including learning rate, number of iterations, batch size, and the number of hidden layers in the long short-term memory network; Randomly combine model hyperparameters to obtain all model hyperparameter combinations, and then set all the model hyperparameter combinations to the pre-built data anomaly identification model respectively; The gradient descent algorithm is used to calculate the rate of decrease of the cost function corresponding to each combination of hyperparameters of the model during the model training process. The model hyperparameter combination with the fastest cost function decrease rate is selected as the target hyperparameter combination. Obtain the data anomaly identification model corresponding to the target hyperparameter combination as the target model, and use the target identification model as the pre-trained data anomaly identification model.

5. The data anomaly identification method according to claim 1, characterized in that, The step of verifying the output of the pre-trained data anomaly detection model based on the annotation results of the full sample data and the model output results specifically includes: Based on the model output, count the number of positive and negative sample data output by the model; Based on the annotation results of the full sample data, the number of actual positive and negative sample data in the full sample data is counted. The number of positive and negative sample data output by the model and the actual number of positive and negative sample data in the full sample data are input as method parameters into the preset loss function to obtain the model loss value. The model loss value is compared with a preset loss threshold. If the model loss value is less than the preset loss threshold, the verification is successful; If the model loss value is not less than the preset loss threshold, the verification fails.

6. The data anomaly identification method according to claim 1, characterized in that, The step of predicting whether the test data is anomalous based on the trained data anomaly detection model specifically includes: Obtain the output of the trained data anomaly detection model; If the output shows that the test data is positive sample data, then the test data is abnormal data; If the output shows that the test data is negative sample data, then the test data is normal data.

7. A data anomaly identification device, characterized in that, The data anomaly identification device implements the steps of the data anomaly identification method as described in any one of claims 1 to 6, and the data anomaly identification device comprises: The test data acquisition module is used to acquire test data to be identified, wherein the test data to be identified includes all audit data related to claims business after a car insurance accident occurs, including data from the reporting stage, the investigation stage, the accident liability determination stage, and the policy review stage. The feature extraction module is used to perform time-series processing and feature extraction on the test data to obtain a test feature set. The time-series processing of the test data means organizing the test data into time-series data according to the time sequence of data in the reporting stage, investigation stage, accident liability determination stage, and policy review stage in the entire claims business process. The feature engineering module is used to perform feature engineering processing on the test feature set to obtain derived features; The feature update module is used to incorporate the derived features into the test feature set to update the test feature set; The model recognition module is used to input the updated test feature set into the trained data anomaly recognition model, and predict whether the test data is anomalous based on the trained data anomaly recognition model.

8. A computer device comprising a memory and a processor, the memory storing computer-readable instructions, wherein the processor, when executing the computer-readable instructions, implements the steps of the data anomaly identification method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the data anomaly identification method as described in any one of claims 1 to 6.

Images