A method for identifying auto insurance fraud based on extracted text factor enhancement

Abstract

The present invention discloses a method for identifying auto insurance fraud based on enhanced extracted text factors. In the early stage of text extraction, the problems of missing and incorrect original data are solved by measuring text similarity, and the knowledge of auto insurance scenarios is used to construct a keyword vocabulary for describing vehicle accidents, thereby improving the accuracy of auto insurance scenario text segmentation. The present invention integrates part-of-speech syntactic information and designs a joint framework for pre-trained language models. Accident trigger words are extracted using pre-trained language model knowledge, and the syntactic relationship weights are learned in combination with the attention mechanism, and the extraction results are optimized through part-of-speech filtering. The present invention also proposes and designs an accident cause translation template to help extract the cause of the accident, which can effectively reduce the transmission error in the middle of the model. Finally, the extracted text factors are integrated, the discrete structured text is encoded using the Transformer encoder, and an integrated learning model is used to predict whether auto insurance fraud exists.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, and in particular relates to a method for identifying auto insurance fraud based on enhanced extracted text factors. Background Art

[0002] Traditional auto insurance fraud detection methods primarily focus on manually annotated structured data and are unable to effectively process rich natural language information. Furthermore, natural language descriptions contain a significant amount of redundant noise, which can confuse models and prevent them from capturing important information. Therefore, extracting structured key factors from auto insurance descriptions to help models identify auto insurance fraud is crucial.

[0003] Extracting text factors is a natural language processing task. Traditional text extraction methods include template matching and recurrent neural networks, but these methods have drawbacks and cannot effectively capture the semantics of text, resulting in very poor extraction accuracy. Most state-of-the-art text extraction models are based on pretrained language models, leveraging their semantic understanding to aid extraction. The process generally involves encoding the text into word-unit representations using a pretrained language model. A corresponding word-unit classification module is then designed to predict the type of each word, and finally, the content to be extracted is determined based on the type. Pretrained language models not only acquire extensive semantic knowledge from the original training corpus but also incorporate contextual information into the extraction process. However, existing pretrained models still have drawbacks. First, their large number of parameters and size make them unsuitable for processing semi-structured text. Second, existing pretrained language models are not fully adapted to downstream tasks, such as text extraction. This leads to error propagation between models and prevents the full performance of pretrained language models.

[0004] Traditional prediction models rely solely on simple one-hot encoding of discrete data, which directly loses all semantic information. Others use word2vec encoding, which encodes individual words and fails to capture contextual information. Furthermore, during the prediction phase, single prediction models are highly sensitive to data, lack robustness, and suffer from poor prediction performance. Summary of the Invention

[0005] The present invention aims to address the shortcomings of existing techniques in extracting structured key factors from auto insurance descriptive text and provide a method for identifying auto insurance fraud based on the enhancement of extracted text factors. This method designs a pre-trained language model framework that integrates part-of-speech and syntactic information. To address the problem of error propagation between the pre-trained language model and downstream tasks, a template for translating accident causes is proposed. The extracted text factors are integrated, and the discrete structured text is encoded using a Transformer encoder. Finally, an ensemble learning model is employed to predict auto insurance fraud.

[0006] The object of the present invention is achieved through the following technical solution: a method for identifying auto insurance fraud based on text factor extraction enhancement, comprising the following steps:

[0007] 1) Extract structured address information from the vehicle accident address text;

[0008] 2) Construct a vocabulary of vehicle accident description keywords to assist in the segmentation of vehicle accident descriptions and accident investigation descriptions;

[0009] 3) Use the pre-trained language model to extract accident trigger words from vehicle accident descriptions, and combine part-of-speech analysis and syntactic analysis to obtain the accident object;

[0010] 4) Design an accident cause translation template and use it with a pre-trained language model to extract the cause of the accident from the accident investigation description text;

[0011] 5) Use the pre-trained language model to perform text classification on the accident investigation description text to obtain the accident result classification;

[0012] 6) Integrate the structured data extracted from the text and construct a text encoder to encode the structured text;

[0013] 7) Use ensemble learning models to learn and identify auto insurance fraud.

[0014] Furthermore, in step 1):

[0015] Write the corresponding regular matching template and use it to extract structured address information from the natural language description text of the vehicle accident address; handle the format inconsistency problem uniformly and use similar text replacement method for errors and default texts; the text similarity calculation is the edit distance metric, for text T a 、T b The recursive formula for the edit distance between is as follows:

[0016]

[0017] Furthermore, in step 2):

[0018] The method for constructing a vehicle accident description keyword vocabulary first requires constructing 2-gram, 3-gram, and 4-gram vocabulary lists of vehicle accident descriptions and accident investigation description texts, and sorting them by word frequency to obtain the final vehicle accident description keyword vocabulary list;

[0019] The text of vehicle accident descriptions and accident investigation descriptions is segmented based on the special high-frequency words related to the vehicle accident field in the vehicle accident description keyword vocabulary. The segmentation process includes two cases:

[0020] a) If the word-meta obtained by word segmentation belongs to a keyword in the vocabulary, search and determine whether the word formed by combining the word-meta with surrounding word-meta is also a keyword, and use the combined word-meta as a new word-meta;

[0021] b) If the word lemma obtained by word segmentation does not belong to the keywords in the vocabulary, the word lemma obtained by word segmentation is treated as a separate word lemma.

[0022] Furthermore, in step 3):

[0023] On the basis of the Chinese pre-trained language model BERT-Base Chinese, multiple classifiers are added to form a trigger word extractor; for the input word segmentation result X = {x1, x2, ..., x n}, x i′ Indicates the corresponding word index, and the trigger word extractor processing flow is:

[0024] H=BERT Chinese (X)

[0025] O=softmax(tanh(HW hidden +b hidden )W+b)

[0026] Among them, BERT Chinese (·) represents the BERT-Base Chinese pre-trained language model, H is the intermediate variable output by BERT-Base Chinese, W is the learnable matrix, b is the error, and finally the multi-classifier output O combined with the multi-layer perceptron is used as the predicted probability output;

[0027] The Stanford Core NLP integrated framework is used to annotate text parts of speech and perform syntactic analysis to obtain the syntactic relationship between the word units and trigger words in the text, and to construct the relationship weight corresponding to the syntactic relationship:

[0028] S r =[S r,1 ,S r,2 ,…,S r,m ]

[0029] Among them, Sr,i″ represents the learnable relation weight corresponding to the syntactic relation;

[0030] According to the intermediate variable H output by the trigger word extractor above, the attention score between the keyword and other words is obtained:

[0031]

[0032] Among them, H i′ The intermediate variable representing the i′th word;

[0033] Using the relationship weight and trigger word attention, the correlation between the trigger word and other tokens is calculated:

[0034] S=S r,i″ attention (trigger word, current word)

[0035] Among them, S r,i″ The learnable relation weight represents the syntactic relationship between the trigger word and the current word, and attention(·) represents the attention score between the trigger word and the current word. The product of the two is the correlation degree, and they are arranged in order.

[0036] Finally, based on the word-part-of-speech features, the words with abnormal parts of speech are filtered out, and the words with the highest correlation with the trigger word are taken as the final accident objects;

[0037] The accident trigger word extraction and accident object extraction are divided into two independent submodules, so the two modules need to be optimized separately; for the input text X = {x1, x2, ..., x n The training loss function of the accident trigger word extraction module is designed as follows:

[0038]

[0039] in, is the output label probability distribution of the i′th word unit of the accident trigger word extraction module, It means The corresponding actual label probability distribution.

[0040] Furthermore, in step 4):

[0041] A template for translating accident causes was designed, and the template was used to convert the named entity recognition task into input suitable for the pre-trained model. Meaningless punctuation marks were removed, and the [CLS] placeholder in the model input indicated the beginning of a sentence, while the [SEP] placeholder indicated the sentence separation and end. Model training was similar to the text extraction process in step 3), and the translated text was directly used as input to the pre-trained language model. It should be noted that the model input also requires accident cause translation during the model prediction stage. The difference is that the accident cause to be predicted is replaced by the placeholder [MASK], and the output corresponding to [MASK] is directly predicted as the prediction result.

[0042] Furthermore, in step 5), the accident result classification is obtained using text classification:

[0043] a) First, the original text needs to be cleaned and redundant text without practical meaning needs to be deleted;

[0044] b) Obtain text representation using the [CLS] tag from the Chinese pre-trained language model;

[0045] c) Multilayer Perceptron classifies text, and its probability distribution p y as follows:

[0046] p y =softmax(MLP(H [CLS] ))

[0047] The category with the highest probability is taken as the final accident result classification.

[0048] Further, in step 6):

[0049] Before learning the auto insurance fraud behavior model, it is also necessary to integrate the structured data extracted from the text, such as address information, accident trigger words, accident objects, accident causes, accident results, accident road types, and vehicle repair shop grades. In addition, information that may affect the final recognition performance is extracted from the text, including escape information, vehicle damage, and personal injuries. Different encoding methods are adopted for the different characteristics of these structured data. Among them, the province and city information in the address, accident results, accident road types, vehicle repair shop grades, escape information, vehicle damage, and personal injuries are used as discrete type data, and the accident trigger words, accident objects, and accident causes are used as text data. The type data are all encoded using one-hot encoding, and the text data are encoded using a Transformer encoder based on the self-attention mechanism. The Transformer encoder is composed of multiple transformer components stacked together, and the word unit x i′ The corresponding encoding method is as follows:

[0050]

[0051]

[0052] in, Represents word x i′ At the output of the L-th layer transformer, represents the Transformer encoder input, word2vec(·) and position(·) represent the word x i′ word embedding and position embedding.

[0053] Further, in step 7):

[0054] The essence of the ensemble learning model is that multiple seed models vote to select the final prediction result. We selected six seed models: CatBoost, LightGBMLarge, LightGBMXT, LightGBM, XGBoost, and NeuralNetMXNet, and designed a set of learnable voting weights:

[0055] y p =λ1O CatBoost +λ2O LightGBM +λ3O LightGBMLarge + λ4O LightGBMXT +λ5O XGBoost +λ6O NeuralNetMXNet

[0056] The model loss calculation function is as follows:

[0057]

[0058] The beneficial effects of the present invention are:

[0059] (1) Using text similarity can effectively solve the problems of missing and incorrect data;

[0060] (2) The vehicle accident description keyword vocabulary constructed from knowledge in the vehicle accident field can significantly improve the accuracy of word segmentation, which lays a solid foundation for subsequent extraction tasks. Compared with simple word segmentation results, many special words related to vehicle insurance can be accurately segmented;

[0061] (3) The pre-trained language model combined with the framework model that integrates part-of-speech syntactic information can accurately extract accident trigger words and accident objects, with an accuracy rate of 97% and 86.5% respectively. In particular, the accuracy of extraction of long sentences is improved by more than 7.4%.

[0062] (4) The event translation template greatly reduces the error transmission between models, and the unit test accuracy of extracting accident causes reaches 83%, which is a significant performance improvement compared to traditional named entity recognition models;

[0063] (5) Transformer encoding can convert discrete data into vectors that are easy for machines to understand and process, and retains relevant domain knowledge during the encoding learning process;

[0064] (6) The ensemble learning model has higher accuracy in predicting auto insurance fraud than a single simple model;

[0065] (7) The ablation experiment results show that the above text extraction factors can significantly improve the prediction accuracy and help the prediction model to more accurately identify fraudulent behavior with an accuracy rate of over 87%. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 This is a flow chart of the method for identifying auto insurance fraud based on the enhanced extraction of text factors of the present invention;

[0067] Figure 2 Schematic diagram of accident trigger word extraction and accident object extraction;

[0068] Figure 3 Schematic diagram of the sample template for translating the cause of the accident. DETAILED DESCRIPTION

[0069] The present invention will be further described below with reference to implementation examples and accompanying drawings.

[0070] The present invention provides a method for identifying auto insurance fraud based on enhanced extraction of text factors. Key factors are extracted from auto insurance-related description texts, and the extracted text factors are used to enhance the model's ability to identify auto insurance fraud. Auto insurance-related description texts can be mainly divided into three categories: the address where the vehicle accident occurred, the vehicle accident description text, and the accident investigation description text. In addition, the relevant texts also contain structured data such as address information, accident trigger words, accident objects, accident causes, accident results, accident road types, vehicle repair shops, escape information, vehicle damage, and personal injuries. First, the text factor extraction of the vehicle accident address adopts a template matching method, and solves the problem of missing and incorrect original data by measuring text similarity. The factor extraction methods of the vehicle accident description text and the accident investigation description text are both based on pre-trained language models. At the beginning, the knowledge in the auto insurance field is used to construct a vehicle accident description keyword vocabulary. The difference between the two approaches lies in the integration of part-of-speech syntactic information during the extraction of vehicle accident description text factors. A pre-trained language model incorporating this information is designed to jointly extract accident trigger words and accident object factors. Furthermore, an accident cause translation template is proposed and designed to aid in extracting accident cause factors during the extraction of accident investigation description text factors. This approach effectively reduces the transmission error that often occurs within the model. Finally, the extracted text factors are integrated, the discrete structured text is encoded using a Transformer encoder, and an ensemble learning model is employed to predict auto insurance fraud.

[0071] like Figure 1 As shown, the present invention mainly includes the following steps:

[0072] 1) Extract structured address information from the natural language description text of the vehicle accident address.

[0073] The natural language description of the vehicle accident address is semi-structured data, for example, "No. ****, ** Road, Nanjing (City), Jiangning District (District / County / Town / Village), Jiangsu (Province)"; therefore, the present invention directly uses a template matching method to extract structured address information. The specific steps are: first, introduce a regular expression module, then write a corresponding regular matching template. Using the template, extract structured address information from the natural language description of the vehicle accident address; for example, {Province: "Jiangsu", City: "Nanjing", District / County / Town / Village: "Jiangning District", Detailed Location: "No. ****, ** Road"}.

[0074] It is worth noting that the original text has format inconsistencies or even errors. The present invention uniformly handles format inconsistencies and uses similar texts to replace errors and default texts. The text similarity calculation uses the edit distance metric. For text T a 、T b The recursive formula for the edit distance between is as follows:

[0075]

[0076] Among them, i and j represent T a 、T b The corresponding text sequence number, Represents text T a and T b Edit distance between.

[0077] 2) Construct a keyword vocabulary for vehicle accident descriptions to assist in the word segmentation of vehicle accident descriptions and accident investigation descriptions.

[0078] Method for constructing a keyword vocabulary for vehicle accident descriptions: First, construct 2-gram, 3-gram, and 4-gram vocabulary lists for vehicle accident descriptions and accident investigation descriptions, sort them by word frequency, and then manually screen them to obtain the final vehicle accident description keyword vocabulary list. Based on the high-frequency words related to vehicle accidents in the vehicle accident description keyword vocabulary list, we perform word segmentation on the natural language text of vehicle accident descriptions and accident investigation descriptions to improve word segmentation accuracy.

[0079] The following two situations may occur in the common word segmentation process:

[0080] 2.1) If the token obtained by word segmentation belongs to the keyword in the vehicle accident description keyword vocabulary, the surrounding tokens are searched to determine whether the word formed by the token and the surrounding tokens is also a keyword. If so, the combined token is used as a new segmentation token, which indicates that the combined token has more semantic integrity and is more suitable for processing as a separate token. If not, the token is processed as a separate token. For example, in the text "While driving, the target vehicle was scratched by an electric motorcycle.", "electric" and "motorcycle" are often divided into two separate tokens, but "electric motorcycle" as a whole has more semantic integrity as a token. This optimization method can effectively improve the accuracy of word segmentation.

[0081] 2.2) If the token obtained by word segmentation does not belong to a keyword in the vehicle accident description keyword vocabulary, the token obtained by word segmentation is treated as a separate token.

[0082] 3) If Figure 2 As shown in the figure, the pre-trained language model is used to extract accident trigger words in the vehicle accident description, and the accident object is obtained by combining part-of-speech analysis and syntactic analysis.

[0083] The trigger word extractor aims to predict whether a word in a text triggers an event. Based on the Chinese pre-trained language model BERT-Base Chinese, this paper adds multiple classifiers to form a trigger word extractor.

[0084] For the input word segmentation result X={x1,x2,…,x n}, x i′ Indicates the corresponding word index. The trigger word extractor processing flow is:

[0085] H=BERT Chinese (X)

[0086] O=softmax(tanh(HW hidden +b hidden )W+b)

[0087] Among them, BERT Chinese (·) represents the BERT-Base Chinese pre-trained language model, H is the intermediate variable output by BERT-Base Chinese; W is the learnable matrix, b is the error, W hidden 、b hidden It represents the matrix parameters and error parameters that can be learned in the final output layer of the model. Tanh and softmax are activation functions. O is the output of the multi-classifier combined with the multi-layer perceptron as the predicted probability output.

[0088] The trigger words in the text can be extracted based on the t-word prediction category. For example, in the text "While driving, the target car was scratched by an electric motorcycle.", the word "scratching" has the highest prediction probability in O and is extracted as the keyword that triggers the event.

[0089] This paper adopts the Stanford Core NLP integrated framework to mark the text part of speech and syntactic analysis, and obtains the syntactic relationship between each word in the text and the trigger word. i” , and constructed the relation weight S corresponding to the syntactic relation r :

[0090] S r =[S r,1 ,S r,2 ,…,S r,m ]

[0091] Among them, S r Represents the learnable relation weights of all syntactic relations, S r,i″ It represents the syntactic relation r i” The corresponding learnable relation weights, m represents the number of syntactic relations, and the subscript i″ is used to distinguish different syntactic relations.

[0092] Based on the intermediate variable H output by the trigger word extractor, we can obtain the attention score between the keyword and other word units. The specific operation is as follows:

[0093]

[0094] Among them, H i′ Represents the intermediate variable of the i′th word, H j′ Indicates non-keyword x in X i′ An intermediate variable of another word; H k Used for summation.

[0095] Using the relationship weight and trigger word attention, the correlation between the trigger word and other tokens can be calculated:

[0096] S=S r,i″ attention (trigger word, current word)

[0097] Among them, S r,i″ Indicates the syntactic relationship between the trigger word and the current word r i” The learnable relationship weight, attention(·) represents the attention function between the trigger word and the current word. The product of the two is the correlation degree, and they are arranged in order.

[0098] Finally, the present invention filters out word-grams with abnormal parts of speech according to their part-of-speech features, and takes the word-gram with the highest correlation with the trigger word as the final accident object.

[0099] Accident trigger word extraction and accident object extraction are divided into two independent submodules, so the two modules need to be optimized separately. n The training loss function of the accident trigger word extraction module is designed as follows:

[0100]

[0101] in, is the output label probability distribution of the i′th word unit of the accident trigger word extraction module, It means The corresponding actual label probability distribution (one-hot type), p1 represents the probability distribution similarity between the output label of the accident trigger word extraction module and the actual label, which is used to measure the prediction accuracy.

[0102] The training loss function of the accident object extraction module is the same as that of the accident trigger word extraction module:

[0103]

[0104] in, is the output label probability distribution of the i′th word in the accident object extraction module, It means The corresponding actual label probability distribution, p2 represents the similarity of the probability distribution between the output label of the accident object extraction module and the actual label, which is used to measure the prediction accuracy.

[0105] 4) Design an accident cause translation template and combine it with a pre-trained language model to extract the accident cause from the accident investigation description text.

[0106] Extracting accident causes from accident investigation descriptions is essentially a named entity recognition (NER) task within natural language processing. Unlike traditional NER tasks, this paper avoids the use of a pre-trained model and downstream task debugging. Instead, it designs an accident cause translation template. This template transforms the NER task into input suitable for the pre-trained model, eliminating the impact of errors introduced during the model adaptation process.

[0107] The specific translation template for the cause of the accident is as follows: the [CLS] placeholder indicates the beginning of a sentence, and the two [SEP] placeholders indicate the sentence separation and end; the text between [CLS] and the first [SEP] is included (meaningless punctuation is removed), and the cause of the accident token is included between the first [SEP] and the second [SEP]. For example:

[0108] [CLS] De-punctuation text [SEP] The accident is due to Accident cause word Lead to [SEP]

[0109] like Figure 3 For example, the text "The target vehicle changed lanes and hit a straight-moving vehicle." and "The target vehicle accidentally rolled into a ditch, causing serious damage." are labeled with "lane change" and "accidentally" as the corresponding accident causes. Ultimately, the accident cause translation template converts the corresponding pre-trained model input into: "[CLS] The target vehicle changed lanes and hit a straight-moving vehicle [SEP] The accident was caused by lane change [SEP]" and "[CLS] The target vehicle accidentally rolled into a ditch, causing serious damage [SEP] The accident was caused by carelessness [SEP]."

[0110] The training of the accident cause extraction model is similar to the text extraction process in step 3). The text translated by the accident cause translation template is directly used as the input of the pre-trained language model to extract the accident cause words in the text.

[0111] 4.1) Use the translation template to translate the cause of the accident into a pre-trained language model input in a special format.

[0112] It should be noted that the input in the model prediction phase also needs to be translated by the accident cause translation template. The difference is that the accident cause to be predicted is replaced by the placeholder [MASK], and the output corresponding to [MASK] is directly predicted. For example:

[0113] [CLS] De-punctuation text [SEP] The accident was caused by [MASK] [SEP]

[0114] The accident cause translation template minimizes the error transmission between the language model and the named entity recognition task, retains the text semantic information, and can effectively improve the accuracy of named entity recognition.

[0115] 4.2) The pre-trained language model obtains the text semantic representation vector through the multi-head self-attention mechanism and text word order, and obtains the [MASK] semantic representation through the fine-tuning module.

[0116] 4.3) Calculate the probability distribution of the cause of the accident through the model’s final [MASK] semantic representation output.

[0117] 5) Use the pre-trained language model to perform text classification on the accident investigation description text to obtain the accident result classification.

[0118] The accident investigation description text roughly describes the final result of the accident. The present invention organizes and analyzes the accident results and divides the accident results into six categories according to their characteristics: "hitting a moving object", "hitting a fixed object", "vehicle collision", "damage to driving", "damage to driving", and "damage to parking".

[0119] The present invention uses text classification to obtain accident result classification, and its specific operations are as follows:

[0120] 5.1) First, the original accident investigation description text needs to be cleaned and redundant text with no practical significance needs to be deleted;

[0121] 5.2) Use the [CLS] tag in the Chinese pre-trained language model to obtain the text representation H [CLS] ;

[0122] 5.3) Multilayer Perceptron for text H [CLS] For classification, its probability distribution p y as follows:

[0123] p y =softmax(MLP(H [CLS] ))

[0124] Here, softmax is the activation function, and MLP stands for multi-layer perceptron. The category with the highest probability is the final accident outcome classification.

[0125] 6) Integrate the structured data extracted from the text and construct a text encoder to encode the structured text.

[0126] Before learning the auto insurance fraud behavior model, it is necessary to integrate the structured data extracted from the text in the above steps, including address information, accident trigger words, accident subject, accident cause, accident outcome, accident road type, and vehicle repair shop rating. Furthermore, the present invention extracts information from the text that may affect final recognition performance, including escape information, vehicle damage, and personal injury information. The accident road type, vehicle repair shop, escape information, vehicle damage, and personal injury information are extracted using a template matching method.

[0127] This paper employs different encoding methods to address the different characteristics of these structured data. Specifically, the province and city information in the address, accident outcome, escape information, vehicle damage, and personal injury status are treated as discrete type data, while the accident trigger, accident object, and accident cause are treated as text data. The type data is encoded using one-hot encoding, while the text data is encoded using a Transformer encoder based on a self-attention mechanism.

[0128] The Transformer encoder is composed of multiple transformer components stacked together, and the word x i′ The corresponding encoding method is as follows:

[0129]

[0130]

[0131] in, Represents word x i′ At the output of the L-th layer transformer, represents the Transformer encoder input, word2vec(·) and position(·) represent the word x i′ word embedding and position embedding.

[0132] 7) Based on the characteristics and complexity of the data, an ensemble learning model is used to learn and identify auto insurance fraud.

[0133] The essence of an ensemble learning model is that multiple sub-models vote to select the final prediction result. The sub-models have extremely fast response speeds and high accuracy, and the ensemble learning model can achieve higher prediction accuracy, stronger robustness, and greater flexibility. This embodiment of the present invention selects six sub-models: CatBoost, LightGBMLarge, LightGBMXT, LightGBM, XGBoost, and NeuralNetMXNet, and designs a set of learnable voting weights λ1 to λ6:

[0134] y p =λ1O CatBoost +λ2O LightGBM +λ3O LightGBMLarge + λ4O LightGBMXT +λ5O XGBoost +λ6O NeuralNetMXNet

[0135] Among them, O CatBoost , O LightGBM , O LightGBMLarge , O LightGBMXT , O XGBoost , O NeuralNetMXNet is the predicted probability output of each sub-model. p is the predicted probability distribution.

[0136] The loss calculation function of the integrated learning model is as follows:

[0137]

[0138] Where N represents the training data set, |N| represents the number of samples of the training data, which is used to calculate the average loss value; y p,i″′ is the output label probability distribution of the i′′th training sample, y i″′ Then it means y p,i″′ The corresponding actual label probability distribution (one-hot type), p represents the similarity of the probability distribution between the output label and the actual label, which is used to measure the prediction accuracy.

[0139] According to experimental results, the present invention breaks through the limitations of traditional auto insurance fraud identification methods, extracts important information from relevant description texts, reduces the interference of redundant information and noise on prediction tasks, and converts the extracted factors into more effective expressions to enhance the performance of auto insurance fraud identification. Among them, the constructed vehicle accident description keyword vocabulary can effectively improve the accuracy of word segmentation and help subsequent steps to obtain better text semantic integrity; innovatively proposes a method of combining accident trigger words with part-of-speech analysis and syntactic analysis, and using the attention mechanism to extract accident objects, which can effectively improve the accuracy of extracting accident objects; the designed accident cause translation template breaks through the shackles between traditional language models and downstream tasks, and will maximize the advantages of pre-trained language models; the constructed text encoder can effectively convert text into information that is easy for computers to process, effectively improving information utilization. The results of various comparative experiments and unit tests show that the extracted text factors can significantly enhance the performance of auto insurance fraud identification.

[0140] The above embodiments are only preferred implementations of the present invention. It should be pointed out that for those skilled in the art, several improvements and equivalent substitutions can be made without departing from the principles of the present invention. These technical solutions after improvements and equivalent substitutions to the claims of the present invention all fall within the scope of protection of the present invention.

Claims

1. A method for identifying auto insurance fraud based on text factor extraction, characterized by: The steps include: 1) Extract structured address information from the vehicle accident address text; 2) Construct a vocabulary of vehicle accident description keywords to assist in the segmentation of vehicle accident descriptions and accident investigation descriptions; 3) Use the pre-trained language model to extract accident trigger words from vehicle accident descriptions, and combine part-of-speech analysis and syntactic analysis to obtain the accident object; 4) Design an accident cause translation template and use it with a pre-trained language model to extract the cause of the accident from the accident investigation description text; 5) Use the pre-trained language model to perform text classification on the accident investigation description text to obtain the accident result classification; 6) Integrate the structured data extracted from the text and construct a text encoder to encode the structured text; 7) Use ensemble learning models to learn and identify auto insurance fraud.

2. The method for identifying auto insurance fraud based on text factor extraction enhancement according to claim 1 is characterized in that: In step 1): Write the corresponding regular matching template and use it to extract structured address information from the natural language description text of the vehicle accident address; handle the format inconsistency problem uniformly and use similar text replacement method for errors and default texts; the text similarity calculation is the edit distance metric, for text T a 、T b The recursive formula for the edit distance between is as follows:

3. The method for identifying auto insurance fraud based on text factor extraction enhancement according to claim 1 is characterized in that: In step 2): The method for constructing a vehicle accident description keyword vocabulary first requires constructing 2-gram, 3-gram, and 4-gram vocabulary lists of vehicle accident descriptions and accident investigation description texts, and sorting them by word frequency to obtain the final vehicle accident description keyword vocabulary list; The text of vehicle accident descriptions and accident investigation descriptions is segmented based on the special high-frequency words related to the vehicle accident field in the vehicle accident description keyword vocabulary. The segmentation process includes two cases: 2.1) If the segmented word is a keyword in the vocabulary, search and determine whether the word formed by combining the segmented word with surrounding word elements is also a keyword, and use the combined word element as the new segmented word element; 2.2) If the word gram obtained by word segmentation does not belong to the keywords in the vocabulary, the word gram obtained by word segmentation will be treated as a separate word gram.

4. The method for identifying auto insurance fraud based on text factor extraction enhancement according to claim 1 is characterized in that: In step 3): On the basis of the Chinese pre-trained language model BERT-Base Chinese, multiple classifiers are added to form a trigger word extractor; for the input word segmentation result X = {x1, x2, ..., x n }, x i′ Indicates the corresponding word index, and the trigger word extractor processing flow is: H=BERT Chinese (X) O=softmax(tanh(HW hidden +b hidden )W+b) Among them, BERT Chinese (·) represents the BERT-Base Chinese pre-trained language model, H is the intermediate variable output by BERT-BaseChinese, W is the learnable matrix, b is the error, and finally the multi-classifier output O combined with the multi-layer perceptron is used as the predicted probability output; The Stanford Core NLP integrated framework is used to annotate text parts of speech and perform syntactic analysis to obtain the syntactic relationship between the word units and trigger words in the text, and to construct the relationship weight corresponding to the syntactic relationship: S r =[S r，1 ,S r，2 ,…,S r，m ] Among them, S ri″ represents the learnable relation weight corresponding to the syntactic relation; According to the intermediate variable H output by the trigger word extractor above, the attention score between the keyword and other words is obtained: Among them, H i′ The intermediate variable representing the i′th word; Using the relationship weight and trigger word attention, the correlation between the trigger word and other tokens is calculated: S=S r，i″ attention (trigger word, current word) Among them, S r，i″ The learnable relation weight represents the syntactic relationship between the trigger word and the current word, and attention(·) represents the attention score between the trigger word and the current word. The product of the two is the correlation degree, and they are arranged in order. Finally, based on the word-part-of-speech features, the words with abnormal parts of speech are filtered out, and the words with the highest correlation with the trigger word are taken as the final accident objects; The accident trigger word extraction and accident object extraction are divided into two independent submodules, so the two modules need to be optimized separately; for the input text X = {x1, x2, ..., x n The training loss function of the accident trigger word extraction module is designed as follows: in, is the output label probability distribution of the i′th word unit of the accident trigger word extraction module, It means The corresponding actual label probability distribution.

5. The method for identifying auto insurance fraud based on text factor extraction enhancement according to claim 1 is characterized in that: In step 4): A template for translating accident causes was designed, and the template was used to convert the named entity recognition task into input suitable for the pre-trained model. Meaningless punctuation marks were removed, and the [CLS] placeholder in the model input indicated the beginning of a sentence, while the [SEP] placeholder indicated the sentence separation and end. Model training was similar to the text extraction process in step 3), and the translated text was directly used as input to the pre-trained language model. It should be noted that the model input also requires accident cause translation during the model prediction stage. The difference is that the accident cause to be predicted is replaced by the placeholder [MASK], and the output corresponding to [MASK] is directly predicted as the prediction result.

6. The method for identifying auto insurance fraud based on text factor extraction enhancement according to claim 1 is characterized in that: In step 5), the accident result classification is obtained using text classification: 5.1) First, the original text needs to be cleaned and redundant text without practical meaning needs to be deleted; 5.2) Obtain text representation using the [CLS] tag from the Chinese pre-trained language model; 5.3) Multilayer Perceptron classifies text, and its probability distribution p y as follows: p y =softmax(MLP(H [CLS] )) The category with the highest probability is taken as the final accident result classification.

7. The method for identifying auto insurance fraud based on text factor extraction enhancement according to claim 1 is characterized in that: In step 6): Before learning the auto insurance fraud behavior model, it is also necessary to integrate the structured data extracted from the text, such as address information, accident trigger words, accident objects, accident causes, accident results, accident road types, and vehicle repair shop grades. In addition, information that may affect the final recognition performance is extracted from the text, including escape information, vehicle damage, and personal injuries. Different encoding methods are adopted for the different characteristics of these structured data. Among them, the province and city information in the address, accident results, accident road types, vehicle repair shop grades, escape information, vehicle damage, and personal injuries are used as discrete type data, and the accident trigger words, accident objects, and accident causes are used as text data. The type data are all encoded using one-hot encoding, and the text data are encoded using a Transformer encoder based on the self-attention mechanism. The Transformer encoder is composed of multiple transformer components stacked together, and the word unit x i′ The corresponding encoding method is as follows: in, Represents word x i′ At the output of the L-th layer transformer, represents the Transformer encoder input, word2vec(·) and position(·) represent the word x respectively. i′ Word embedding and position embedding.

8. The method for identifying auto insurance fraud based on text factor extraction enhancement according to claim 1 is characterized in that: In step 7): The essence of the ensemble learning model is that multiple seed models vote to select the final prediction result. We selected six seed models: CatBoost, LightGBMLarge, LightGBMXT, LightGBM, XGBoost, and NeuralNetMXNet, and designed a set of learnable voting weights: y p =λ1O CatBoost +λ2O LightGBM +λ3O LightGBMLarge +λ4O LightGBMXT +λ5O XGBoost +λ6O NeuralNetMXNet The model loss calculation function is as follows:

Images