An insurance data analysis method and system based on multimodal pre-trained large model

Abstract

The present invention discloses an insurance data analysis method and system based on a multimodal pre-trained large model, which relates to the field of insurance data analysis. The key points of the technical solution include: collecting comprehensive insurance data, performing data quality enhancement processing on the comprehensive insurance data, obtaining a quality enhancement data set, performing cross-modal integration processing on the quality enhancement data set, and obtaining a correlation integration chart; constructing a multi-task driven model, and adaptively updating the model, inputting the correlation integration chart into the pre-trained multi-task driven model, and generating an insurance analysis report; displaying the insurance analysis report on a computer terminal page, and interacting with the user interface, so as to achieve high-quality analysis and processing of insurance data and adaptive updating of the model, and provide high-quality and high-efficiency services.

Description

Technical Field

[0001] The present invention relates to the field of insurance data analysis, and more specifically, to an insurance data analysis method and system based on a multimodal pre-trained large model. Background Art

[0002] Analyzing insurance data based on multimodal pre-trained large models (e.g., models that combine multimodal information such as text, images, and speech) is a cutting-edge method, but there may be problems in practical applications, mainly including:

[0003] Insurance data usually includes multiple modalities such as text (policy, claim application), image (car accident photos, medical images), and voice (customer phone recordings). These data may come from different systems or channels, with inconsistent formats, and are difficult to integrate directly. The information expression methods between different modalities vary greatly. For example, text is discrete, images are continuous, and voice is sequential. How to align and associate the information of these modalities may be technically difficult.

[0004] Insurance data (such as environmental data and user behavior data in risk assessment) may become invalid over time, and pre-trained large models may not be updated in a timely manner. In addition, multimodal large models are constantly updated to adapt to new data and new scenarios, resulting in high training costs, and updates can easily lead to unstable model performance.

[0005] Therefore, based on the above problems, it is necessary to design insurance data analysis based on multimodal pre-trained large models. Summary of the invention

[0006] In view of the problems existing in the prior art, the purpose of the present invention is to provide an insurance data analysis method and system based on a multimodal pre-trained large model.

[0007] To achieve the above object, the present invention provides the following technical solution: The insurance data analysis method based on the multimodal pre-trained large model includes:

[0008] Step S1: Collect comprehensive insurance data, perform data enhancement processing on the comprehensive insurance data to obtain an enhanced data set, perform cross-modal integration processing on the enhanced data set, and obtain a related integration chart;

[0009] Step S2: construct a multi-task driving model, adaptively update the model, input the associated integration graph into the pre-trained multi-task driving model, and generate an insurance analysis report;

[0010] Step S3: Display the insurance analysis report on the computer page and interact with the user interface.

[0011] Preferably, the comprehensive insurance data includes text data, image data and voice data;

[0012] Text data includes policy information, claims application materials, customer information, customer service records, contract terms and anti-fraud data;

[0013] Image data includes photos of accident scenes, photos of damaged items, medical images, and identification documents and tickets;

[0014] Voice data includes customer service call recordings and voice messages;

[0015] Text data is stored in a standardized format, image data is formatted in a common format and converted to a uniform resolution, and voice data is converted to a standard audio format.

[0016] Preferably, the method of performing data enhancement processing on the comprehensive insurance data to obtain an enhanced data set, performing cross-modal integration processing on the enhanced data set, and obtaining a related integration chart includes:

[0017] For text data, remove irrelevant characters and stop words, use dictionary-natural language processing technology to segment text data, and obtain the feature word set by calculating the weight priority value of each word;

[0018] For image data, identify each picture, and segment the picture into main area pictures through adaptive segmentation to obtain the main picture block set;

[0019] For voice data, use speech recognition technology to convert it into text, obtain keywords, and form a keyword set;

[0020] Collect feature word sets, main image block sets and keyword sets to form quality-enhanced data sets;

[0021] The feature words, main area maps and keywords in the feature word set, main image block set and keyword set are simulated into nodes respectively, the correlation value between the nodes is calculated, and the correlation threshold is set. When the correlation value is greater than or equal to the correlation threshold, there is an edge between the nodes, and the correlation value is used as the weight of the edge;

[0022] For feature words and keywords, word vectors are obtained through Word2Vec or BERT, and the cosine similarity between word vectors is calculated and used as the relevance value. For feature words and main area graphs, keywords and main area graphs, they are transformed into words and images, and the deep feature vectors of the images are obtained through pre-trained convolutional neural networks. The cosine similarity between the deep feature vectors and word vectors is calculated and used as the relevance value;

[0023] Set up a weighted graph G = (V, E), where V is the node set and E is the edge set, and fill in the nodes and edges to obtain the associated integrated graph.

[0024] Preferably, the method of segmenting text data using dictionary-natural language processing technology and obtaining a feature word set by calculating the weight priority value of each word includes:

[0025] The definition dictionary is represented by a triple, that is, the dictionary ,in, Indicates the first The words in the dictionary are pre-set based on insurance professional terms, proper nouns and keywords in the field of anti-fraud. Indicates The priority of the words, Indicates The part-of-speech tags of the words, Representation dictionary The total number of words in An index of words;

[0026] Use natural language processing technology to segment text data according to the words in the dictionary and divide the text data into words;

[0027] The weight priority value of each word is calculated through the hybrid TF-IDF algorithm, and a boundary threshold is set. The words with a weight priority value greater than or equal to the boundary threshold are recorded as feature words, and all feature words are collected to form a feature word set.

[0028] Preferably, the weight priority value of each word is calculated by a hybrid TF-IDF algorithm, and the specific method includes:

[0029] Set the weight priority for each word ,in, Expressing words The weight priority value, Expressing words In the documentation The word frequency after sublinear scaling in , Expressing words The smoothed inverse document frequency, Expressing words The part-of-speech weight, Expressing words In the documentation relevance to the context;

[0030] and, ,in, Expressing words In the documentation The frequency of words in ;

[0031] and, ,in, Represents the total number of documents in the text data, Indicates that it contains words The number of documents;

[0032] For words In the documentation The relevance to the context, setting the words For the target word, set is the context of the target word, in the document Randomly select a category of context and calculate the word The mutual information with the context of the category is calculated by traversing all categories and accumulated, which is recorded as words In the documentation Relevance to context ;

[0033] In the documentation Randomly select a context of a category and calculate the word The formula for the mutual information with the context of this category is ,in, Expressing words and Mutual information of category context, Expressing words and The joint probability of categories appearing at the same time, Expressing words The marginal probability of occurrence, express The marginal probability of a class occurring.

[0034] Preferably, the method of segmenting the picture into main area pictures by adaptive segmentation and obtaining the main picture block set comprises:

[0035] For each image, denoising, normalization and enhancement preprocessing are used to convert all images into a formatted representation of uniform size;

[0036] For identity documents and tickets, they are converted into text and processed through image recognition technology;

[0037] For photos of accident scenes, photos of damaged items, and medical images, edge detection algorithms are used to perform preliminary segmentation on the images to obtain preliminary images;

[0038] For the preliminary image, splitting is performed in each region according to the edge features, and the regions with connected edge features are merged to obtain the final image, which is recorded as the main region. All the main regions are collected to obtain the main tile set.

[0039] Preferably, the method of converting it into text using speech recognition technology, obtaining keywords, and forming a keyword set includes:

[0040] Speech recognition technology includes HMM-GMM technology, deep learning technology or end-to-end speech recognition technology;

[0041] After converting it into text using speech recognition technology, keywords are extracted using the same processing method as text data to form a keyword set;

[0042] Among them, in the dictionary that defines the voice data, the words in the dictionary are pre-set based on specific business terms, common customer expressions, numbers and dates, frequently asked questions, emotional words, and regional and personalized words.

[0043] Preferably, the method of constructing a multi-task driving model, adaptively updating the model, inputting the associated integration chart into the pre-trained multi-task driving model, and generating an insurance analysis report includes:

[0044] The framework of the multi-task driven model is based on neural network learning and LSTM technology, including input layer, hidden layer and output layer. The input of the input layer is set as an associated integration chart, and the output of the output layer is an insurance analysis report. The output layer has three output heads in parallel, namely type prediction head, liability identification head and claim amount prediction head.

[0045] The type prediction head is used to predict the category of insurance claims. The output is the category label of the insurance claim. The activation function is Softmax, and the loss function is set to the cross entropy loss function.

[0046] The responsibility identification head is used to predict the party responsible for the accident. The output is the label of the responsible party. The activation function is Softmax, and the loss function is set to multi-class cross entropy.

[0047] The claim amount prediction head is used to predict continuous values. The output is the claim amount, and the loss function is set to mean square error.

[0048] Set the total loss function of the model to the sum of the loss functions of the three output heads, and the optimization goal is to minimize the total loss function, using Adam or SGD optimizer;

[0049] The model is continuously updated using an adaptive hybrid approach;

[0050] Use historical data to build a training set to pre-train the model until the optimization target is reached, obtain the trained multi-task driven model, input the associated integration chart into the trained multi-task driven model, and obtain the insurance analysis report.

[0051] Preferably, the method of continuously updating the model using the adaptive hybrid method comprises:

[0052] Extract the loss functions of the three output heads and set the update threshold. When the value of the loss function is greater than the update threshold, identify the output head corresponding to the loss function, mark the corresponding hidden layer and output layer, freeze the hidden layer and output layer of other output heads, and form a new model.

[0053] Build a new training set using the most recent data, and use the new training set to retrain the new model until the optimization goal is reached and the loss function value is less than the update threshold, and then obtain the trained new model.

[0054] Unfreeze the frozen output head to obtain an adaptive and continuously updated model.

[0055] An insurance data analysis system based on a multimodal pre-trained large model, comprising:

[0056] Data collection and integration module: used to collect comprehensive insurance data, perform data enhancement and cross-modal integration processing on the comprehensive insurance data, and obtain related integration charts;

[0057] Model fitting and updating module: used to build and adaptively update the multi-task driving model, input the associated integration graph into the pre-trained multi-task driving model, and generate an insurance analysis report;

[0058] Visualization module: used to arrange the insurance analysis report on the computer page and interact with the user interface.

[0059] Compared with the prior art, the present invention has the following beneficial effects:

[0060] By integrating text, image and voice data, we can understand insurance events from a more comprehensive perspective and improve the accuracy and completeness of analysis. Using hybrid TF-IDF algorithms, image recognition, voice recognition and other technologies, we extract feature words, main area maps and keywords from text, image and voice data, and establish correlations, so as to effectively extract key information from the data. We accurately extract the features of text, image and voice data, and achieve accurate correlation analysis by calculating the correlations between nodes.

[0061] By integrating multimodal data, the model can better understand insurance events, reduce information loss caused by a single data source, and improve the accuracy of analysis results. Improved data analysis efficiency: Through data enhancement and feature extraction, the utilization efficiency of raw data is improved, and the impact of noise and irrelevant information is reduced.

[0062] This solution builds a model that can handle multiple tasks and continuously learn through a multi-task driven model and an adaptive update mechanism, which improves the accuracy and generalization ability of the model. It uses standardized data formats, pre-trained models, and adaptive update mechanisms to improve the efficiency of data processing and the scalability of the model. The adaptive update mechanism enables the model to be continuously optimized and adapted to new data and analysis tasks, improving the long-term effectiveness of the model. The automated feature extraction and model building process improves the efficiency of data analysis and reduces labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 The present invention proposes a structural schematic diagram of an insurance data analysis system based on a multimodal pre-trained large model;

[0064] Figure 2 Schematic diagram of the method applied to the insurance data analysis system based on the multimodal pre-trained large model in the present invention. DETAILED DESCRIPTION

[0065] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0066] Embodiment 1, refer to Figure 1 and Figure 2 , Example 1 further illustrates an insurance data analysis system based on a multimodal pre-trained large model proposed by the present invention.

[0067] Analyzing insurance data based on multimodal pre-trained large models (e.g., models that combine multimodal information such as text, images, and speech) is a cutting-edge method, but there may be problems in practical applications, mainly including:

[0068] A. Uneven data quality and difficulty in integrating multimodal data;

[0069] a1. Heterogeneity of data sources: Insurance data usually includes multiple modalities such as text (policy, claim application), image (car accident photos, medical images), voice (customer phone recordings), etc. These data may come from different systems or channels, with inconsistent formats, and are difficult to integrate directly.

[0070] a2. Difficulty in data cleaning: Insurance data may be missing, incomplete, noisy (such as blurry photos or handwritten text that is difficult to recognize), etc., which may affect the training and analysis of multimodal models.

[0071] a3. Modal imbalance: In the field of automobile insurance, data of different modalities may contribute unevenly to specific tasks. Some modalities (such as text data) may dominate, while image or voice data is less, or images and voice are dominant, causing the model's performance to be biased towards one modality.

[0072] a4. Difficulty of cross-modal alignment: The ways of expressing information between different modalities vary greatly. For example, text is discrete, images are continuous, and speech is sequential. There may be technical difficulties in aligning and associating the information of these modalities.

[0073] B. Dynamic and timeliness issues;

[0074] b1. Data timeliness: Insurance data (such as environmental data and user behavior data in risk assessment) may become invalid over time, and pre-trained large models may not be updated in a timely manner. In addition, multimodal large models are constantly updated to adapt to new data and new scenarios, resulting in high training costs, and updates can easily lead to unstable model performance.

[0075] Step S1: Collect comprehensive insurance data, perform data enhancement processing on the comprehensive insurance data to obtain an enhanced data set, perform cross-modal integration processing on the enhanced data set, and obtain a related integration chart;

[0076] Data collection and integration module: used to collect comprehensive insurance data, perform data enhancement and cross-modal integration processing on the comprehensive insurance data, and obtain related integration charts;

[0077] Comprehensive insurance data includes text data, image data and voice data;

[0078] Text data includes policy information, claims application materials, customer information, customer service records, contract terms and anti-fraud data;

[0079] Policy information includes key information such as the policyholder, insured, insurance type, insurance amount, premium, insurance period, etc.; claim application materials include accident description, loss report, medical record description and claim history, etc.; customer information includes basic information (name, ID number, contact information), occupation, address, etc.; customer service records are communication records between customers and insurance companies, including emails, chat records, phone records, etc.; contracts and terms include text data such as insurance terms and exemptions; anti-fraud data includes text records describing suspicious behavior or the claims process.

[0080] Image data includes photos of accident scenes, photos of damaged items, medical images, and identification documents and tickets;

[0081] Photos of accident scenes, such as car accident scenes, property damage scenes, etc.; photos of damaged items, such as damaged vehicles, houses, properties, etc.; medical images, such as X-rays, CT scans, MRI images, etc. (for medical insurance); identification documents and bills, such as ID photos, invoices, receipts, medical lists, etc.

[0082] Voice data includes customer service call recordings and voice messages;

[0083] Customer service telephone recordings include recordings of customers submitting claims, inquiring about insurance policies, complaining, etc.; voice messages include records of voice communications between insurance agents or claims adjusters and customers.

[0084] For the above data categories, data collection methods can be divided into the following categories:

[0085] The collection of text data includes customer input (obtaining text data such as policies, applications, descriptions, etc. submitted by customers through online forms, emails, mobile applications, etc. when customers purchase insurance or make claims), business system extraction (exporting structured policies, claims records, etc. from the insurance company's core business systems (such as CRM, ERP systems)) and OCR technology (using optical character recognition (OCR) technology to digitize scanned text data such as policies and receipts).

[0086] Methods for collecting image data include customer upload (customers upload photos of the accident scene, receipts, etc. through mobile applications, websites or emails), collection by claims adjusters (photos or videos taken by claims adjusters at the accident scene), provision by medical institutions (medical images (such as X-rays) provided by hospitals or clinics) and automated equipment (for example, in the field of auto insurance, images automatically captured by vehicle cameras or driving recorders when an accident occurs).

[0087] Methods for collecting voice data include call center recordings (recordings of calls made by customers to the insurance company's customer service hotline), voice interaction systems (using voice recognition technology (such as IVR systems, interactive voice response) to automatically collect voice information submitted by customers, such as claims applications or question consultations), voice input in mobile applications (customers submit voice descriptions of accident situations or claims requirements through the insurance company's mobile applications), and voice communication records between insurance agents and customers (recordings of telephone communications between agents and customers, especially during claims negotiations or insurance recommendations).

[0088] Store text data in a standardized format (such as JSON or CSV), image data in a common format (such as JPEG or PNG) and convert it to a uniform resolution, and voice data into a standard audio format (such as WAV or MP3).

[0089] A method for enhancing the quality of comprehensive insurance data to obtain an enhanced dataset and performing cross-modal integration processing on the enhanced dataset to obtain an associated integration chart includes:

[0090] For text data, remove irrelevant characters (such as special symbols, HTML tags, etc.) and stop words (such as "de", "shi", etc.), use dictionary-natural language processing technology to tokenize the text data, and obtain a set of feature words by calculating the priority value of the proportion of each word;

[0091] The insurance field is a professional field, and a custom professional dictionary is a collection of vocabulary in a specific field, which is used to enhance the capabilities of text processing tools so that they can more accurately identify and segment domain-related terms.

[0092] For image data, identify each picture, segment the picture into image blocks through adaptive segmentation, and use YOLO technology to extract the main area map to obtain a set of main image blocks;

[0093] For voice data, use speech recognition technology to convert it into text, obtain keywords, and form a set of keywords;

[0094] Collect the set of feature words, the set of main image blocks, and the set of keywords to form an enhanced dataset;

[0095] Simulate the feature words, main area maps, and keywords in the set of feature words, the set of main image blocks, and the set of keywords into nodes respectively, calculate the association value between the nodes, and set an association threshold. When the association value is greater than or equal to the association threshold, there is an edge between the nodes, and the association value is used as the weight of the edge;

[0096] By comparing the metadata of the image (such as file name, timestamp) with the policy number and accident description in the text, find the matching text and image pairs. For the feature words and keywords, obtain word vectors through Word2Vec or BERT, calculate the cosine similarity between the word vectors, and use it as the association value. For the feature words and main area maps, and keywords and main area maps, convert them into words and images, obtain the deep feature vector of the image through a pre-trained convolutional neural network, calculate the cosine similarity between the deep feature vector and the word vector, and use it as the association value;

[0097] Set a weighted graph G=(V, E), where V is the set of nodes and E is the set of edges, and fill in the nodes and edges to obtain an associated integration chart.

[0098] The method for using dictionary-natural language processing technology to tokenize text data and obtain a set of feature words by calculating the priority value of the proportion of each word includes:

[0099] Define the dictionary represented by triples, that is, the dictionary ,in, Indicates the first The words in the dictionary are pre-set based on insurance professional terms, proper nouns and keywords in the field of anti-fraud. Indicates The priority of each word can be manually marked or the TF-IDF value can be calculated to assign a weight to each word in the dictionary to accurately reflect the importance of the term. It is used to determine the relative importance of the terms when processing text. Indicates The part-of-speech tag of a word is used to identify the grammatical category of the word. For example, in natural language processing, common part-of-speech tags include noun ("n"), proper noun ("nz"), verb ("v"), adjective ("a"), etc. Representation dictionary The total number of words in An index of words;

[0100] Insurance professional terms include keywords in insurance contract clauses, claims process, and policy information, such as: policy number, insurance amount, deductible and compensation ratio; proper nouns include insurance company name, product name, customer common phrases, such as: XX Insurance Company, critical illness insurance, additional accidental injury insurance, etc.; anti-fraud field keywords are keywords or phrases used to describe suspicious behaviors, such as: false invoices, duplicate claims, excessive treatment and falsified medical records.

[0101] For example, the dictionary ,in, Indicates that the priority of the word "insurance policy number" is 7 and the part of speech is noun ("n"); The word " " has a priority of 9 and a part of speech of noun ("nz"); The word " " has a priority of 10 and a part of speech of noun ("nz").

[0102] By analyzing newly added text data, the dictionary is dynamically updated to add high-frequency but not yet covered field terms.

[0103] Use natural language processing technology (such as Chinese word segmentation tool Jieba or BERT tokenizer) to segment text data according to the words in the dictionary and divide the text data into words;

[0104] During the process of word segmentation of text using natural language processing technology, a dictionary can be used to control the word segmentation results to align them with the domain requirements. For example, the phrase "insurance amount" can be regarded as a whole instead of being segmented into "insurance" and "amount". It can also give priority to identifying high-frequency professional terms. The core goal of word segmentation is to maximize the segmentation accuracy of the text. Suppose the input text is: {The customer submitted a critical illness insurance policy with the policy number 123456 and the insurance amount is 500,000}, the word segmentation task can be formalized as , ,……], where and are the words after word segmentation. After introducing the dictionary, the word segmentation algorithm will give priority to referring to the dictionary when splitting each word to ensure that the words in the dictionary are recognized first. For example, if the dictionary contains "critical illness insurance", it will no longer be segmented into "critical illness" and "insurance".

[0105] Calculate the priority value of each word through a hybrid TF-IDF algorithm, set a threshold, mark the words with a priority value greater than or equal to the threshold as feature words, and collect all the feature words to form a feature word set.

[0106] Calculate the priority value of each word through a hybrid TF-IDF algorithm. The specific methods include:

[0107] Set the priority value of each word , where represents the priority value of the word , represents the word frequency of the word after sublinear scaling in the document . The word frequency means the number of times the word appears in the document. represents the smoothed inverse document frequency of the word . represents the part-of-speech weight of the word . represents the relevance of the word to the context in the document ;

[0108] This weight in the part-of-speech weight can be manually assigned different values according to the part of speech of the word (such as noun, verb, adjective, etc.) to reflect the importance of different parts of speech in semantic understanding.

[0109] And, , where represents the word frequency of the word in the document ;

[0110] By using logarithmic scaling for high-frequency words, their excessive impact on the final result is reduced, while maintaining a certain degree of distinction and avoiding the problem of all high-frequency words being over-compressed. By using the converted formula, the importance of words can be more accurately evaluated, thereby improving the performance and accuracy of text processing tasks.

[0111] and, ,in, Represents the total number of documents in the text data, Indicates that it contains words The number of documents;

[0112] By adding 1 to the numerator and denominator respectively, we ensure that the IDF value will not be 0, making the IDF value more stable and avoiding the occurrence of extreme values. By using this smoothed IDF formula, the importance of words can be more accurately evaluated, thereby improving the performance and accuracy of text processing tasks.

[0113] For words In the documentation The relevance to the context, setting the words For the target word, set is the context of the target word (which can be a word, phrase, or other feature) in the document Randomly select a context of one category (assuming there are 3 different contexts in the document, then the context category is 3), and calculate the word The mutual information with the context of the category is calculated by traversing all categories and accumulated, which is recorded as words In the documentation Relevance to context ;

[0114] In the documentation Randomly select a context of a category and calculate the word The formula for the mutual information with the context of this category is ,in, Expressing words and Mutual information of category context, Expressing words and The joint probability of categories appearing at the same time, that is, words and The proportion of the number of times a category appears at the same time in the total number of samples, Expressing words The marginal probability of occurrence, express The marginal probability of a category appearing is the marginal probability of a word. or The proportion of the number of times a category appears in the total number of samples.

[0115] By combining smoothed IDF, sublinear TF scaling, part-of-speech weights, and the relevance of words in specific contexts, we improve the traditional TF-IDF model to make it more adaptable to different application scenarios and better reflect the importance of words in text data.

[0116] The image is segmented into main area images by adaptive segmentation, and the method of obtaining the main tile set includes:

[0117] For each image, denoising, normalization and enhancement preprocessing are used to convert all images into a formatted representation of uniform size;

[0118] Image denoising can use Gaussian filtering or median filtering to remove noise; image normalization is to normalize the image to a fixed size (such as 224x224 pixels); data enhancement includes rotation, scaling, cropping, flipping and other data enhancement methods to increase robustness.

[0119] For identity documents and tickets, they are converted into text and processed through image recognition technology;

[0120] For photos of accident scenes, photos of damaged items, and medical images, edge detection algorithms (such as Canny edge detection) are used to perform preliminary segmentation on the images to obtain preliminary images, which have a larger area.

[0121] For the preliminary image, each region is split according to edge features (such as thickness, direction, etc.), and regions with connected edge features are merged to reduce unnecessary details and improve the accuracy of segmentation. After splitting and merging, the final segmentation result is output to obtain the final image, which is recorded as the main area map. All main areas are collected to obtain the main tile set.

[0122] Methods for converting speech recognition technology into text and obtaining keywords to form a keyword set include:

[0123] Speech recognition technology includes HMM-GMM technology, deep learning technology or end-to-end speech recognition technology;

[0124] HMM-GMM technology is a combination model based on HMM and GMM. HMM is a hidden Markov model used to model the statistical characteristics of speech signals. GMM is used to model the probability distribution of speech signals and is often used in combination with HMM to form a model.

[0125] Speech recognition based on deep learning technology includes CNN (commonly used to extract speech features in speech recognition), RNN and its variants (such as LSTM and GRU, used to model time series data, suitable for sequence-to-sequence conversion in speech recognition) or Transformer models.

[0126] The end-to-end speech recognition system directly converts speech signals into text without the need for intermediate speech feature extraction steps, such as CTC and Attention Mechanism.

[0127] After converting it into text using speech recognition technology, keywords are extracted using the same processing method as text data to form a keyword set;

[0128] Among them, in the dictionary that defines the voice data, the words in the dictionary are pre-set based on specific business terms, common customer expressions, numbers and dates, frequently asked questions, emotional words, and regional and personalized words.

[0129] Specific business terms refer to the fact that these recordings and messages involve insurance claims, policy consultation, complaints and other businesses, so they need to contain a large number of specific insurance industry terms. These terms have fixed meanings in the insurance industry, such as: claims, policies, indemnity, complaints and claim forms.

[0130] Common customer expressions are some common expressions that customers may use on the phone. These expressions need to be included in the dictionary, such as: Hello, Please tell me, or I would like to wait.

[0131] Numbers and dates refer to numbers and dates that customers may mention in speech, which also need to be included in the dictionary. For example: one (1), two (2), three (3), etc., date format such as: October 1, 2023 (2023-10-01), time format such as: 3:00 p.m. (3PM).

[0132] Common questions and expressions are some common questions or expressions that customers may ask during the call, which need to be included in the dictionary, such as: where is the insurance policy, what is the claims process, when does the policy expire, or where is my claim form, etc.

[0133] When customers make complaints or express dissatisfaction, they may use some emotional words, which also need to be included in the dictionary, such as: dissatisfaction, anger, and disappointment.

[0134] Regional and personalized vocabulary refers to the fact that according to the regional and personalized characteristics of the customer, some specific words or expressions may be used, which also need to be included in the dictionary.

[0135] Step S2: construct a multi-task driving model, adaptively update the model, input the associated integration graph into the pre-trained multi-task driving model, and generate an insurance analysis report;

[0136] Model fitting and updating module: used to build and adaptively update the multi-task driving model, input the associated integration graph into the pre-trained multi-task driving model, and generate an insurance analysis report;

[0137] The method of constructing a multi-task driving model, adaptively updating the model, inputting the associated integration graph into the pre-trained multi-task driving model, and generating an insurance analysis report includes:

[0138] The framework of the multi-task driven model is based on neural network learning and LSTM technology, including input layer, hidden layer and output layer. The input of the input layer is set as an associated integration chart, and the output of the output layer is an insurance analysis report. The output layer has three output heads in parallel, namely type prediction head, liability identification head and claim amount prediction head.

[0139] The type prediction head is used to predict the category of insurance claims (such as auto insurance, medical insurance, property insurance, etc.). The output is the category label of the insurance claim. The activation function is Softmax, and the loss function is set to the cross entropy loss function.

[0140] The responsibility identification head is used to predict the responsible party for the accident (such as the driver of the front car or the driver of the rear car). The output is the label of the responsible party. The activation function is Softmax, and the loss function is set to multi-class cross entropy.

[0141] The claim amount prediction head is used to predict continuous values. The output is the claim amount, and the loss function is set to mean square error.

[0142] Set the total loss function of the model to the sum of the loss functions of the three output heads, and the optimization goal is to minimize the total loss function, using Adam or SGD optimizer;

[0143] The model is continuously updated using an adaptive hybrid approach; making it always timely and reliable.

[0144] Use historical data to build a training set to pre-train the model until the optimization target is reached, obtain the trained multi-task driven model, input the associated integration chart into the trained multi-task driven model, and obtain the insurance analysis report.

[0145] Methods for continuously updating models using adaptive hybrid methods include:

[0146] Extract the loss functions of the three output heads and set the update threshold. When the value of the loss function is greater than the update threshold, identify the output head corresponding to the loss function, mark the corresponding hidden layer and output layer, freeze the hidden layer and output layer of other output heads, and form a new model.

[0147] Build a new training set using the most recent data, and use the new training set to retrain the new model until the optimization goal is reached and the loss function value is less than the update threshold, and then obtain the trained new model.

[0148] Unfreeze the frozen output head to obtain an adaptive and continuously updated model.

[0149] Step S3: Displaying the insurance analysis report on a computer terminal page and interacting with the user interface;

[0150] Visualization module: used to arrange the insurance analysis report on the computer page and interact with the user interface;

[0151] Users can click, query, download and perform other interactive operations on the insurance analysis report through the computer page (computer display screen, where the computer refers to a smart device, which can be a mobile phone or tablet).

[0152] This method is based on a multimodal pre-trained large model and integrates text, image and voice data to analyze insurance data. The specific steps include data collection and quality enhancement, construction and adaptive update of a multi-task driven model, and generation and interactive display of insurance analysis reports.

[0153] Example 2, refer to Figure 1 and Figure 2 , Example 2 further illustrates an insurance data analysis system based on a multimodal pre-trained large model proposed by the present invention.

[0154] The insurance data analysis method based on a multimodal pre-trained large model comprises the following steps:

[0155] Step S1: Collect comprehensive insurance data, perform data enhancement processing on the comprehensive insurance data to obtain an enhanced data set, perform cross-modal integration processing on the enhanced data set, and obtain a related integration chart;

[0156] Step S2: construct a multi-task driving model, adaptively update the model, input the associated integration graph into the pre-trained multi-task driving model, and generate an insurance analysis report;

[0157] Step S3: Display the insurance analysis report on the computer page and interact with the user interface.

[0158] The insurance data analysis system based on a multimodal pre-trained large model is applied to an insurance data analysis method based on a multimodal pre-trained large model, including:

[0159] Data collection and integration module: used to collect comprehensive insurance data, perform data enhancement and cross-modal integration processing on the comprehensive insurance data, and obtain related integration charts;

[0160] Model fitting and updating module: used to build and adaptively update the multi-task driving model, input the associated integration graph into the pre-trained multi-task driving model, and generate an insurance analysis report;

[0161] Visualization module: used to arrange the insurance analysis report on the computer page and interact with the user interface;

[0162] The modules are connected to each other via wired and / or wireless means.

[0163] In addition, according to the implementation of the present application, the process described in the accompanying drawings of an insurance data analysis system based on a multimodal pre-trained large model can be implemented as a computer software program. For example, the present application provides a non-transitory machine-readable storage medium, which stores machine-readable instructions, and the machine-readable instructions can be executed by a processor to execute instructions corresponding to the method steps provided in the present application. Of course, the architecture shown in the accompanying drawings of an insurance data analysis system based on a multimodal pre-trained large model is only exemplary. When implementing different devices, adaptive selection or adjustment can be performed according to actual needs.

[0164] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters and thresholds in the formula are set by technicians in this field according to actual conditions.

[0165] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technical users in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. An insurance data analysis method based on a multimodal pre-trained large model, characterized in that: The insurance data analysis method based on the multimodal pre-trained large model includes: Step S1: Collect comprehensive insurance data, perform data enhancement processing on the comprehensive insurance data to obtain an enhanced data set, perform cross-modal integration processing on the enhanced data set, and obtain a related integration chart; Comprehensive insurance data includes text data. For text data, the dictionary-natural language processing technology is used to segment the text data, and the weight priority value of each word is calculated by the hybrid TF-IDF algorithm. The threshold is set, and the words with a weight priority value greater than or equal to the threshold are recorded as feature words; Step S2: construct a multi-task driving model, adaptively update the model, input the associated integration graph into the pre-trained multi-task driving model, and generate an insurance analysis report; The framework of the multi-task driven model is based on neural network learning and LSTM technology, including input layer, hidden layer and output layer. The input of the input layer is set as an associated integration chart, and the output of the output layer is an insurance analysis report. The output layer has three output heads in parallel, namely type prediction head, liability identification head and claim amount prediction head. Extract the loss functions of the three output heads and set the update threshold. When the value of the loss function is greater than the update threshold, identify the output head corresponding to the loss function, mark the corresponding hidden layer and output layer, freeze the hidden layer and output layer of other output heads, and form a new model. Unfreeze the frozen output head to obtain an adaptive and continuously updated model; Step S3: Display the insurance analysis report on the computer page and interact with the user interface.

2. The insurance data analysis method based on multimodal pre-trained large model according to claim 1 is characterized in that: The comprehensive insurance data includes text data, image data and voice data; Text data includes policy information, claims application materials, customer information, customer service records, contract terms and anti-fraud data; Image data includes photos of accident scenes, photos of damaged items, medical images, and identification documents and tickets; Voice data includes customer service call recordings and voice messages; Text data is stored in a standardized format, image data is formatted in a common format and converted to a uniform resolution, and voice data is converted to a standard audio format.

3. The insurance data analysis method based on multimodal pre-trained large model according to claim 2 is characterized in that: The method of performing data enhancement processing on the comprehensive insurance data to obtain an enhanced data set, performing cross-modal integration processing on the enhanced data set, and obtaining a related integration chart includes: For text data, remove irrelevant characters and stop words, use dictionary-natural language processing technology to segment text data, and obtain the feature word set by calculating the weight priority value of each word; For image data, identify each picture, and segment the picture into main area pictures through adaptive segmentation to obtain the main picture block set; For voice data, use speech recognition technology to convert it into text, obtain keywords, and form a keyword set; Collect feature word sets, main image block sets and keyword sets to form quality-enhanced data sets; The feature words, main area maps and keywords in the feature word set, main image block set and keyword set are simulated into nodes respectively, the correlation value between the nodes is calculated, and the correlation threshold is set. When the correlation value is greater than or equal to the correlation threshold, there is an edge between the nodes, and the correlation value is used as the weight of the edge; For feature words and keywords, word vectors are obtained through Word2Vec or BERT, and the cosine similarity between word vectors is calculated and used as the relevance value. For feature words and main area graphs, keywords and main area graphs, they are transformed into words and images, and the deep feature vectors of the images are obtained through pre-trained convolutional neural networks. The cosine similarity between the deep feature vectors and word vectors is calculated and used as the relevance value; Set up a weighted graph G = (V, E), where V is the node set and E is the edge set, and fill in the nodes and edges to obtain the associated integrated graph.

4. The insurance data analysis method based on multimodal pre-trained large model according to claim 3 is characterized in that: The method of using the dictionary-natural language processing technology to segment text data and obtaining a feature word set by calculating the weight priority value of each word includes: The definition dictionary is represented by a triple, that is, the dictionary ,in, Indicates the first The words in the dictionary are pre-set based on insurance professional terms, proper nouns and keywords in the field of anti-fraud. Indicates The priority of the words, Indicates The part-of-speech tags of the words, Representation dictionary The total number of words in An index of words; Use natural language processing technology to segment text data according to the words in the dictionary and divide the text data into words; The weight priority value of each word is calculated through the hybrid TF-IDF algorithm, and a boundary threshold is set. The words with a weight priority value greater than or equal to the boundary threshold are recorded as feature words, and all feature words are collected to form a feature word set.

5. The insurance data analysis method based on multimodal pre-trained large model according to claim 4 is characterized in that: The weight priority value of each word is calculated by the hybrid TF-IDF algorithm, and the specific method includes: Set the weight priority for each word ,in, Expressing words The weight priority value, Expressing words In the documentation The word frequency after sublinear scaling in Expressing words The smoothed inverse document frequency, Expressing words The part-of-speech weight, Expressing words In the documentation relevance to the context; and, ,in, Expressing words In the documentation The frequency of words in ; and, ,in, Represents the total number of documents in the text data, Indicates that it contains words The number of documents; For words In the documentation The relevance of the words to the context For the target word, set is the context of the target word, in the document Randomly select a category of context and calculate the word The mutual information with the context of the category is calculated by traversing all categories and accumulated, which is recorded as words In the documentation Relevance to context ; In the documentation Randomly select a context of a category and calculate the word The formula for the mutual information with the context of this category is ,in, Expressing words and Mutual information of category context, Expressing words and The joint probability of categories appearing at the same time, Expressing words The marginal probability of occurrence, express The marginal probability of a class occurring.

6. The insurance data analysis method based on multimodal pre-trained large model according to claim 5 is characterized in that: The method of dividing the picture into main area pictures by adaptive segmentation and obtaining the main picture block set includes: For each image, denoising, normalization and enhancement preprocessing are used to convert all images into a formatted representation of uniform size; For identity documents and tickets, they are converted into text and processed through image recognition technology; For photos of accident scenes, photos of damaged items, and medical images, edge detection algorithms are used to perform preliminary segmentation on the images to obtain preliminary images; For the preliminary image, splitting is performed in each region according to the edge features, and the regions with connected edge features are merged to obtain the final image, which is recorded as the main region. All the main regions are collected to obtain the main tile set.

7. The insurance data analysis method based on multimodal pre-trained large model according to claim 6 is characterized in that: The method of converting it into text using speech recognition technology, obtaining keywords, and forming a keyword set includes: Speech recognition technology includes HMM-GMM technology, deep learning technology or end-to-end speech recognition technology; After converting it into text using speech recognition technology, keywords are extracted using the same processing method as text data to form a keyword set; Among them, in the dictionary that defines the voice data, the words in the dictionary are pre-set based on specific business terms, common customer expressions, numbers and dates, frequently asked questions, emotional words, and regional and personalized words.

8. The insurance data analysis method based on multimodal pre-trained large model according to claim 7 is characterized in that: The method of constructing a multi-task driving model, adaptively updating the model, inputting the associated integration graph into the pre-trained multi-task driving model, and generating an insurance analysis report also includes: The type prediction head is used to predict the category of insurance claims. The output is the category label of the insurance claim. The activation function is Softmax, and the loss function is set to the cross entropy loss function. The responsibility identification head is used to predict the party responsible for the accident. The output is the label of the responsible party. The activation function is Softmax, and the loss function is set to multi-class cross entropy. The claim amount prediction head is used to predict continuous values. The output is the claim amount, and the loss function is set to mean square error. Set the total loss function of the model to the sum of the loss functions of the three output heads, and the optimization goal is to minimize the total loss function, using Adam or SGD optimizer; The model is continuously updated using an adaptive hybrid approach; Use historical data to build a training set to pre-train the model until the optimization target is reached, obtain the trained multi-task driven model, input the associated integration chart into the trained multi-task driven model, and obtain the insurance analysis report.

9. The insurance data analysis method based on multimodal pre-trained large model according to claim 8 is characterized in that: The method of continuously updating the model using the adaptive hybrid method also includes: A new training set is constructed using recent data, and the new model is retrained using the new training set until the optimization goal is reached and the loss function value is less than the update threshold, thus obtaining a trained new model.

10. An insurance data analysis system based on a multimodal pre-trained large model, applied to an insurance data analysis method based on a multimodal pre-trained large model as described in any one of claims 1 to 9, characterized in that: The insurance data analysis system based on the multimodal pre-trained large model includes: Data collection and integration module: used to collect comprehensive insurance data, perform data enhancement and cross-modal integration processing on the comprehensive insurance data, and obtain related integration charts; Model fitting and updating module: used to build and adaptively update the multi-task driving model, input the associated integration graph into the pre-trained multi-task driving model, and generate an insurance analysis report; Visualization module: used to arrange the insurance analysis report on the computer page and interact with the user interface.

Images