Intelligent agent method for vehicle insurance special case review based on privacy protection heterogeneous data fusion

Abstract

This invention relates to the fields of data security and privacy protection technology, and in particular provides a method for intelligent agent review of special auto insurance cases based on privacy-protected heterogeneous data fusion. The method includes: performing privacy-protected preprocessing on the acquired heterogeneous data to obtain privacy-protected heterogeneous data; mapping AI-assisted listening results, AI-assisted viewing results, structured case fields, review rules, and historical review logs to a unified review fact table, and calculating a cross-source consistency score based on the unified review fact table; calculating the data source credibility weight and fusing review features from different sources according to the data source credibility weight to generate a comprehensive case review representation; performing review judgment through an AI-assisted review module, performing pre-submission verification before submitting the review results, and completing automatic submission and audit traceability. This method improves the efficiency of special auto insurance case review processing, enhances the security and interpretability of automatic review, and improves the feasibility of heterogeneous data fusion processing.

Description

Technical Field

[0001] This invention relates to the fields of data security and privacy protection technology, and in particular to a method for intelligent agent review of special cases in auto insurance based on privacy-preserving heterogeneous data fusion. Background Technology

[0002] The review process for special auto insurance cases typically involves processing heterogeneous data from multiple sources, including call recordings, SMS or notification texts, paper documents, case fields from business systems, review rules, and historical review logs. Traditional review methods usually rely on manual listening to recordings, reading materials, verifying system fields, and manually submitting review conclusions. For a single case, manual listening and submission often take several minutes; when the daily case volume is large, repetitive listening, material verification, and system operations consume a significant amount of professional manpower.

[0003] Existing automation solutions typically handle only a single step, such as simply converting speech to text or performing OCR recognition on paper documents. They lack unified modeling of the call intent, the content of text messages or documents, system structured fields, and review rules. Such solutions struggle to determine whether a customer's true needs during a call align with those in a text message or paper document, to verify consistency between document fields and business system fields, and to automatically generate and submit review conclusions while meeting review rules.

[0004] Meanwhile, the data from special auto insurance case reviews contains a large amount of sensitive personal and business information, such as customer names, mobile phone numbers, ID card numbers, license plate numbers, case numbers, policy numbers, addresses, bank accounts, voiceprints from recorded conversations, SMS content, review comments, and processing logs. Using raw data directly during training, inference, submission, or quality inspection could easily lead to privacy breaches and compliance risks. This is especially true when multiple branch offices, review centers, or third-party service providers collaborate on model training and process optimization, making direct sharing of raw data difficult. Summary of the Invention

[0005] In view of this, the present invention provides a method for reviewing special cases of auto insurance based on privacy-preserving heterogeneous data fusion, in order to solve the problems of low efficiency, low value of repetitive labor, and high cost of resources in manual review.

[0006] In a first aspect, the present invention provides a method for intelligent agent review of special cases in auto insurance based on privacy-preserving heterogeneous data fusion, the method comprising:

[0007] Step 1: Obtain heterogeneous data of special car insurance cases for review, and perform privacy protection preprocessing on the heterogeneous data to obtain privacy-protected heterogeneous data; Step 2: The AI-assisted listening module performs normalization, text transcription, speaker separation, intent recognition, and fact extraction on the call recording data to obtain the AI-assisted listening results; Step 3: Use the AI-assisted review module to perform OCR recognition, layout parsing, field extraction, material integrity verification, approval form layout verification, signature or seal area detection, and material credibility calculation on SMS text, SMS screenshots, paper material images, and video attachments to obtain the AI-assisted review results. Step 4: Map the AI-assisted listening results, AI-assisted viewing results, structured case fields, review rules, and historical review logs to a unified review fact table, and calculate the cross-source consistency score based on the unified review fact table; Step 5: Calculate the data source credibility weight and integrate the review characteristics of different sources based on the data source credibility weight to generate a comprehensive case review characterization; Step 6: Perform review and judgment through the AI-assisted review module, perform pre-submission verification before submitting the review results, and complete automatic submission and audit traceability.

[0008] Optionally, the heterogeneous data in step 1 includes one or more of the following: call recordings, SMS text or SMS screenshots, images of paper materials, structured case data from business systems, review rule data, and historical review logs. To model heterogeneous review data uniformly, for any auto insurance special case review case, a heterogeneous review data object is constructed using the case number claim_id as the primary key. Its expression is: ; in, This represents the set of review data for the i-th case; This indicates call recording data; This refers to the text message or a screenshot of the text message. Represents images of paper materials; This represents the structured case field in the business system; Indicates the review rules; This indicates the historical review log; right A unified audit field dictionary is established for the data, mapping the fields of [Customer Request], [Reason for Processing], [Customer Confirmation Result], [Material Status], [Case Number], [Policy Number], [Audit Conclusion], and [Automatic Submission Allowed] to standard fields, forming a unified audit fact table with the following expression: ; Each audit fact Represented as: ; in, For fact field names, For field values, For source data, To identify confidence levels, For fact timestamps, For evidence location information, In a desensitized state, This is a hash digest of the field.

[0009] Optionally, the privacy protection preprocessing in step 1 includes sensitive information identification and desensitization, privacy protection, AIGC content authentication, anti-replay attack detection, and prompt word injection interception; privacy protection processing is performed before data entry, model training, model inference, and review submission; for text, optical character recognition (OCR) results, and speech transcription results, rule matching and named entity recognition models are used to identify sensitive fields. Let the input text sequence be: ; The BERT-CRF named entity recognition model outputs the following sensitive entity label sequence: ; The conditional probability of CRF is: ; in, A represents the label transition matrix, and E represents the label emission score output by the coding model. Perform the following privacy protection processing on the identified sensitive fields: a. Text-sensitive fields are encrypted using mask replacement, tokenization, or format preservation encryption; b. Sensitive fields in images of paper materials or screenshots of text messages are blurred, obscured, or partially encrypted; c. The original call recordings are stored in encrypted form, and the de-identified transcribed text is used first during the reasoning process; d. Structured business fields employ field-level encryption and role-based access control; e. Model input, output, and review logs should retain only necessary anonymized summaries and evidence indexes; Based on privacy protection, AI-native security detection and protection processing are performed, including AIGC authentication, anti-replay attack detection, and prompt word injection interception; for SMS screenshots, paper material images or video attachments, frequency domain features, noise distribution features, compression trace features, and local alteration features are extracted; for call recordings, acoustic playback features, synthesized speech features, and voiceprint consistency features are extracted. The comprehensive security risk score is defined as follows: ; in, This indicates the overall safety risk score of the input materials; This indicates the probability that an image, text message screenshot, material, or voice message was generated by AIGC or deepfake. This indicates the probability that the call recording is susceptible to replay attacks or synthetic speech attacks. This indicates the probability that the text input contains prompt words or malicious commands. This indicates the probability that the material has been partially altered, defaced, or spliced. to This represents the weighting parameter for each security risk item; if Greater than the safety threshold If this occurs, the safety circuit breaker will be triggered, stopping the automatic review and submission and transferring the case to manual review. In a multi-institution joint training scenario, federated learning is adopted; in the t-th round of training, the local model parameters of the k-th institution are... The sample size is If the total number of samples is N, then the central aggregation parameter is: ; in, These are the aggregated global model parameters. For the local model parameters uploaded by the kth client, Let be the number of local samples for the k-th client. The total number of samples from all participating clients; In training tasks that enhance privacy budget control, differential privacy stochastic gradient descent is employed; for single-sample gradients... To perform cropping, the expression is: ; in, For the j-th sample, client, or parameter group, the original gradient is... Let C be the gradient after clipping, and C be the clipping threshold. Let L be the 2-norm of the gradient; Next, Gaussian noise is added to the batch gradient, and its expression is: ; Where B is the batch size. This represents the noise figure.

[0010] Optionally, step 2 includes: The AI-assisted listening module is used to deeply understand call recordings and analyze the intentions of both parties in the conversation. Its processing flow includes audio preprocessing, speech recognition, speaker separation, dialogue segmentation, intent recognition, fact extraction, and call credibility calculation; firstly, the call recording is processed... Preprocessing is performed to obtain the processed audio. : ; in, This represents a speech preprocessing function used to perform preprocessing operations on the input call recording, such as noise reduction, silent segment processing, and volume normalization. Then, a speech recognition model is used to generate timestamped transcribed text. : ; in, This represents an automatic speech recognition function used to convert pre-processed call recordings into speech-to-text with timestamp information. The speaker segment set is obtained by using a speaker separation model: ; in, This indicates the speaker role corresponding to the m-th dialogue segment, including customer, agent, surveyor, or reviewer; This represents the start timestamp of the m-th dialogue segment, used to identify the start time position of the current dialogue segment in the call recording; This represents the end timestamp of the m-th dialogue segment, used to identify the end time position of the current dialogue segment in the call recording; This represents the speech-to-text content corresponding to the m-th dialogue segment; Each dialogue segment is input into the intent recognition model, and the segment text vector is denoted as . The probability of intent is: ; in, Indicates the category of the dialogue intent to be identified; This represents the m-th dialogue segment. The semantic vector obtained after encoding by the text encoding model; The weight matrix represents the intent classification model; This represents the bias vector of the intent classification model; softmax represents the normalization exponential function, used to convert the scores of each intent category into a probability distribution. This represents the semantic vector of the m-th dialogue segment. Under the given conditions, the probability that the current segment belongs to the current intent category; Intent categories include confirmation, denial, withdrawal, request for supplementary materials, consent to processing, incomplete information, dispute, complaint, and invalid call; the category with the highest probability is taken as the main intent of the current dialogue segment. ; in, This represents the category parameter used to determine the maximum value of the objective function; This indicates that the maximum probability value among all candidate intent categories is selected; The highest probability value is then used as the confidence level for intent recognition of that segment: ; in, This represents the maximum predicted probability value of the m-th dialogue segment across all candidate intent categories, used to characterize the confidence level of the intent recognition result for the current dialogue segment; Next, extract the facts of the call: ; in, This represents the set of factual features extracted from the call recording; claim_no represents the case number, extracted from the case number, report number, or business transaction number in the speech-to-text; policy_no represents the policy number, extracted from the policy number or insurance contract number; customer_intent represents the customer's main claim or intention; confirm_result represents the customer's confirmation status of the review matter or processing result; material_status represents the status of the materials mentioned by the customer or agent during the call; special_note represents special prompts or abnormal information identified during the call; asr_conf represents the average transcription confidence score output by the speech recognition model; speaker_role represents the speaker role identification result, including customer, agent, surveyor, or reviewer. The credibility score for a call is defined as follows: ; in, Score the credibility of the call data source; For speech recognition confidence, Separate confidence levels for the speaker. To determine the confidence level for intent identification, For the integrity of critical information, To determine the degree of factual conflict within the same call, to These are preset weights or weight parameters obtained through training with historical audit samples. , , , and All values ​​are normalized to the [0,1] interval; Normalize to the [0,1] interval using a truncation function or sigmoid function for subsequent data source credibility fusion.

[0011] Optionally, step 3 includes: The AI-assisted viewing module is used to understand unstructured information from SMS text, SMS screenshots, images of paper materials, and video attachments; it performs OCR recognition, layout parsing, field extraction, and material integrity verification on SMS text, SMS screenshots, and images of paper materials. For screenshots of text messages or images of printed materials, an OCR model is used to output a set of text blocks: ; Where L represents the number of text blocks identified; This represents the identified text of the l-th text block; This represents the coordinates of the l-th text block in the material image, including the x-coordinate of the top-left corner, the y-coordinate of the top-left corner, the width of the text box, and the height of the text box; This represents the OCR recognition confidence score of the l-th text block, with values ​​normalized to the [0,1] interval; For tables, signatures, seals, and date areas, the layout parsing model is used to output layout elements: ; Where Q represents the number of page elements identified; This indicates the type of the q-th page element, including title, body text, table, seal, signature, date, amount, and number; This represents the region coordinates of the q-th page element in the material image; This represents the confidence level of the q-th page element. Material facts are obtained based on entity extraction models and field mapping dictionaries: ; in, This represents the set of factual features extracted from SMS text, SMS screenshots, images of paper materials, or video attachments; claim_no represents the case number identified in the material; policy_no represents the policy number identified in the material; customer_name represents the customer name or anonymized customer identifier identified in the material; notice_time represents the time field in the SMS, notification, or material; material_type represents the material type; process_reason represents the processing reason or application reason recorded in the material; document_result represents the processing conclusion or customer confirmation result expressed in the material; signature_flag represents whether a signature or seal was detected; seal_flag represents whether a seal was detected; and ocr_conf represents the overall OCR confidence level of the material. The material integrity score is: ; in, Indicates the applicable law based on the i-th case. The required material field set obtained Indicates the review rules; Indicates from The actual set of material fields identified in the data. Represents images of paper materials; Indicates the number of fields that have met the requirements; This is used to avoid division by zero errors when the required field set is empty; The value range is [0, 1], and the larger the value, the more intact the material. Material credibility score is defined as: ; in, Indicates the overall OCR recognition confidence level of the material; The confidence level of the page layout analysis is represented by the number of page elements. We get the weighted average. Indicates the material integrity score; Indicates the time-sensitivity of materials. This indicates the risk of material tampering. Indicates the degree of field missingness, by Or the proportion of missing key fields is indicated. to This indicates preset weights or weights obtained through training based on historical review samples. , , , , and All are normalized to the [0,1] interval. Normalize to the [0,1] interval using a truncation function or a sigmoid function.

[0012] Optionally, step 4 includes: The consistency between the customer's requests during the call and those in the materials is as follows: ; in, This indicates that the AI-assisted listening module identifies the customer's intent from the call recording. This indicates the customer intent or processing request identified by the AI-assisted module from SMS text, SMS screenshots, or paper materials. `exact_match` represents the matching function that checks whether the standardized intent tags match. This refers to descriptive segments in the call transcript that are relevant to the customer's request. This represents descriptive segments in the text material related to customer needs; Emb represents the text vectorization encoding function; and cos represents the cosine similarity. This represents the weighting coefficient between exact tag matching and semantic similarity, with a value range of [0,1]. The call confirmation result is consistent with the system case status as follows: ; in, This indicates the customer confirmation status extracted from the call recording. This indicates the case status or processing status in the business system. "Match" represents the status mapping matching function. When the customer confirms that the status and the system status meet the preset business correspondence, a higher score is taken; otherwise, a lower score is taken. The key fields such as case number and policy number are consistent as follows: ; Where K represents the number of critical fields that need to be validated. This represents the value of the k-th key field identified from the material or text message. This represents the k-th field value in the business system, and `exact_match` represents the exact matching function after field standardization. The value range is [0,1], and the larger the value, the more consistent the material field is with the system field; The completeness of materials and consistency with the audit rules are as follows: ; in, Representing images from paper materials The set of submitted material fields or material types identified in the data. Indicates the review rules The required material fields or set of material types; The consistency of similar historical cases is as follows: ; in, This represents the review representation vector of the current case. Let the vector represent the review process for the j-th historical case. This means selecting the historical case with the highest similarity to the current case from the set of similar historical cases, where cos represents the cosine similarity. Combining the consistency scores .

[0013] Optionally, step 5 includes: Calculate the data source credibility for call recordings, text messages or screenshots, paper materials, structured case fields, review rules, and historical logs respectively; The original credibility score of the s-th data source is: ; in, Score the data quality. To score for completeness, To identify confidence levels for the model, To ensure consistency with other data sources, To mitigate the risk of tampering, The missing rate, to This represents the learnable weight parameters; the credibility of each data source is normalized to obtain the credibility-normalized weights: ; Map features from different sources to a unified vector space: ; in, Represents the set of factual features of the call; Represents the set of factual characteristics of the material; This indicates the structured case characteristics of the business system; Indicates the characteristics of the review rules; This indicates characteristics of historical review logs or similar historical cases. , , , and These represent linear mapping matrices from different data sources; , , , and These represent the corresponding bias vectors; The weighted fusion vector is: ; Employing attention mechanisms to learn the interaction relationships between different data sources: ; in, The expression represents the attention fusion result; Q represents the query matrix, obtained by linear transformation of the data source vector; K represents the key matrix, obtained by linear transformation of the data source vector; V represents the value matrix, obtained by linear transformation of the data source vector. K represents the transpose of K; d represents the vector dimension, used to scale the dot product result; softmax is used to obtain the interaction weights between different data sources; through the attention mechanism, the association between call facts, material facts, structured fields and review rules is learned; The final comprehensive review of the case is characterized as follows: .

[0014] Optionally, step 6 includes: The AI-assisted review module automatically completes review judgment, generates review opinions, and plans submission actions based on the comprehensive review characteristics of the case, review rules, and process status. First, the strong rule engine determines whether there are situations requiring manual review or where automatic submission is prohibited: ; Where, rule_flag represents the rule determination result output by the strong rule engine; RuleEngine represents the rule engine function; This represents the unified audit fact table for the i-th case; This represents the set of review rules applicable to the i-th case; In AI-assisted review, cancellation and reopening links are constructed based on the type of business action. The cancellation link is used to determine whether the customer has a clear, genuine and sufficient intention to cancel or withdraw the case. The reopening link is used to determine whether the supplementary materials meet the compliance elements of the approval form, signature, seal, date and system status required for reopening. The confidence level of the intention to cancel is defined as: ; in, This represents the confidence level of the intention to cancel the i-th case; denoted as S-type activation function; s represents the data source number involved in the cancellation intent judgment, and the data sources include call recordings, SMS texts, case notes, screenshots, and historical review logs; alpha_s represents the credibility normalization weight of the s-th data source; This represents the content of the evidence in the s-th data source; This represents the cancellation intent feature or intent score extracted from the data source; This indicates a bias term used to determine the intent to cancel. The compliance determination for reissuing vouchers is as follows: ; in, This indicates the compliance determination result for reopening vouchers; Indicates the validity status of the approval form; This indicates the validity of the signature or seal; `rule_flag = allow` means that the strong rule engine determines that the current case status and business rules allow reopening; if... If the value is 0, then the corresponding missing feedback is generated; Then, a hierarchical risk scoring model based on cross-source consistency and data source credibility is used to calculate the probability of case anomalies; the hierarchical risk scoring model combines cross-source consistency vector, data source credibility, rule hit characteristics, structured case characteristics, and historical review characteristics into a risk feature vector: ; in, This represents the risk score input feature vector for the i-th case; concat represents the vector concatenation operation. Represents a cross-source consistency vector; This indicates the credibility score of the call; Indicates the credibility score of the material; Indicates the rule's hit characteristics; This indicates the structured case characteristics of the business system; Indicates historical review characteristics; This indicates a comprehensive review of the case. The risk scoring model outputs the probability of case anomalies based on X_risk_i: ; in, The risk scoring model uses LightGBM, taking cross-source consistency features, data source credibility features, rule hit features, and historical review features as inputs, and using historical manual review conclusions, quality inspection results, or reasons for return as supervision labels for training. The value range is [0,1]; To reduce probability bias caused by different branch offices, different processing types, or different time batches, an order-preserving regression probability calibration function based on branch office and processing type grouping is used. Perform calibration: ; in, This indicates the probability of an anomaly in the case after calibration. This indicates the branch company to which the case belongs. and case handling type The order-preserving regression calibration function is obtained by training the model output probability and the actual quality inspection label in the historical audit samples, and maintains the monotonic relationship between the input probability and the calibrated probability. Risk score: ; in, This represents the risk score for the i-th case; round represents the rounding function. The value ranges from 0 to 100, with higher values ​​indicating that the case requires more manual review or special investigation. The review agent determines the status based on the current review status. A sequence of actions can be generated using a set of actions A and rule constraints: ; in, This represents the review action selected by the review agent in the t-th process state; 'a' represents the candidate action. Indicates the current process status Comprehensive Case Review Characteristics and review rules The strategy probability or action score for choosing action a under given conditions; This indicates that the action with the highest probability or score has been selected. The action set includes reading case information, verifying call facts, verifying material facts, executing rule verification, generating review opinions, pre-submission verification, calling business interfaces to submit results, and transferring to manual review; Combining rule_flag, Contribution of risk factors and The review conclusion was: ; in, 'Decision' represents the final review decision for the i-th case; 'Decision' represents the decision fusion function. The action_sequence represents the set of risk factor contributions, obtained from SHAP values, feature importance, or rule hit weights; action_sequence represents a sequence of multiple risk factors. The sequence of review actions; This includes one or more of the following: automatic approval, automatic cancellation or case closure, manual review, special verification, supplementary materials, and prohibition of submission; when When setting safety circuit breaker conditions: ; in, Indicates whether the i-th case triggers the circuit breaker; This indicates the overall security risk score; Indicates the safety risk threshold. > This indicates that the overall security risk score is greater than the security risk threshold; This indicates that the confidence level of the intention to cancel is in the fuzzy range; Indicates cross-source consistency value, Indicates the consistency threshold. This indicates that cross-source consistency is less than the consistency threshold; Indicates the confidence level of the signature or seal test; Indicates the signature detection threshold. This indicates that the confidence level for signature or seal detection is less than the signature detection threshold; when If the submission fails, the automatic submission will be stopped and the submission will be transferred to a human reviewer.

[0015] Optionally, in step 6, a pre-submission verification is performed before submitting the review result, including field integrity verification, case status verification, permission verification, duplicate submission verification, and de-identification status verification; after the verification is passed, the review conclusion is submitted through the business system API or process automation. The audit explanation is generated jointly by rule hits, cross-source conflict items, and model contributions; the feature contribution is calculated using SHAP values, then the contribution of the j-th risk factor is: ; in, This represents the contribution of the j-th risk factor to the output of the risk scoring model; This represents the SHAP interpretation value calculated for the j-th feature; This represents a risk scoring model; This represents the risk score input feature vector for the i-th case; Used to filter the risk factors that contribute the most and map them to the audit explanation text; At the same time, a log of the review process for each case is recorded: ; Wherein, claim_id represents the case number; model_version represents the model version; rule_version represents the rule version; and evidence_hash represents the hash summary of key evidence materials or the audit fact table. Indicates the review decision; action_sequence indicates the sequence of review actions; submit_result indicates the submission result from the business system; time_stamp indicates the timestamp of the log record. Calculate hash digests for key log fields: ; Used to verify whether logs have been tampered with, and to support subsequent quality inspection, traceability and compliance audit; Manual review results and quality inspection results are used as feedback samples and enter the training data pool. For data that is allowed to be returned, only the de-identified fact fields, model features, review conclusions, and quality inspection labels are returned. For multi-institutional scenarios, federated learning is used to return model parameters in order to avoid the original customer data from flowing across institutions.

[0016] The technical solution provided by this invention includes the following steps: First, acquiring heterogeneous data from special auto insurance case review cases; second, performing privacy-protected preprocessing on the heterogeneous data to obtain privacy-protected heterogeneous data; third, using an AI-assisted listening module to perform normalization, text transcription, speaker separation, intent recognition, and review fact extraction on the call recording data to obtain AI-assisted listening results; fourth, using an AI-assisted viewing module to perform OCR recognition, layout parsing, field extraction, material integrity verification, approval form layout verification, signature or seal area detection, and material credibility calculation on SMS text, SMS screenshots, paper material images, and video attachments to obtain AI-assisted viewing results; and fifth, mapping the AI-assisted listening results, AI-assisted viewing results, structured case fields, review rules, and historical review logs into a unified review fact table. This method calculates cross-source consistency scores based on a unified audit fact table; calculates data source credibility weights and integrates audit features from different sources according to these weights to generate a comprehensive case audit representation; and executes audit judgments through an AI-assisted audit module, performing pre-submission verification before submitting the audit results, and completing automatic submission and audit traceability. By linking AI-assisted listening, AI-assisted viewing, and AI-assisted auditing to form a closed-loop process from understanding unstructured information, cross-source verification, risk assessment to audit submission, this method automates repetitive hearings, material verification, and system submission operations. Under the conditions of meeting audit rules and privacy protection requirements, it improves the efficiency of special auto insurance case audit processing, enhances the security and interpretability of automatic audits, and improves the feasibility of heterogeneous data fusion processing. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart of the intelligent agent method for reviewing special cases of auto insurance based on privacy-preserving heterogeneous data fusion provided in an embodiment of the present invention; Figure 2 A flowchart of another method for reviewing special cases of auto insurance based on privacy-preserving heterogeneous data fusion provided in an embodiment of the present invention; Figure 3 The diagram illustrates the technical effectiveness of the embodiments of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” used in the embodiments of this invention are also intended to include the plural forms unless the context clearly indicates otherwise.

[0021] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0022] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0023] This invention provides a method for intelligent agents to review special cases in auto insurance based on privacy-preserving heterogeneous data fusion, such as... Figure 1 and Figure 2 As shown, the method includes: Step 1: Obtain heterogeneous data of special car insurance cases for review, and perform privacy protection preprocessing on the heterogeneous data to obtain privacy-protected heterogeneous data.

[0024] In this embodiment of the invention, the heterogeneous data in step 1 includes one or more of the following: call recordings, SMS text or SMS screenshots, images of paper materials, structured case data from business systems, review rule data, and historical review logs. To model heterogeneous review data uniformly, for any auto insurance special case review case, a heterogeneous review data object is constructed using the case number claim_id as the primary key. Its expression is: ; in, This represents the set of review data for the i-th case; This indicates call recording data; This refers to the text message or a screenshot of the text message. Represents images of paper materials; This represents the structured case field in the business system; Indicates the review rules; This indicates the historical review log; right A unified audit field dictionary is established for the data, mapping the fields of [Customer Request], [Reason for Processing], [Customer Confirmation Result], [Material Status], [Case Number], [Policy Number], [Audit Conclusion], and [Automatic Submission Allowed] to standard fields, forming a unified audit fact table with the following expression: ; Each audit fact Represented as: ; in, For fact field names, For field values, For source data, To identify confidence levels, For fact timestamps, For evidence location information, In a desensitized state, This is a hash digest of the field.

[0025] In this embodiment of the invention, the privacy protection preprocessing in step 1 includes sensitive information identification and desensitization, privacy protection, AIGC content authentication, anti-replay attack detection, and prompt word injection interception; privacy protection processing is performed before data entry, model training, model inference, and review submission; for text, optical character recognition (OCR) results, and speech transcription results, rule matching and named entity recognition models are used to identify sensitive fields; Let the input text sequence be: ; The BERT-CRF named entity recognition model outputs the following sensitive entity label sequence: ; The conditional probability of CRF is: ; in, A represents the label transition matrix, and E represents the label emission score output by the coding model. In this embodiment of the invention, the sensitive entity categories include PERSON, PHONE, ID_CARD, PLATE_NO, POLICY_NO, CLAIM_NO, ADDRESS, BANK_CARD, and VOICE_ID.

[0026] Perform the following privacy protection processing on the identified sensitive fields: a. Text-sensitive fields are encrypted using mask replacement, tokenization, or format preservation encryption; b. Sensitive fields in images of paper materials or screenshots of text messages are blurred, obscured, or partially encrypted; c. The original call recordings are stored in encrypted form, and the de-identified transcribed text is used first during the reasoning process; d. Structured business fields employ field-level encryption and role-based access control; e. Model input, output, and review logs should retain only necessary anonymized summaries and evidence indexes; Based on privacy protection, AI-native security detection and protection processing are performed, including AIGC authentication, anti-replay attack detection, and prompt word injection interception; for SMS screenshots, paper material images or video attachments, frequency domain features, noise distribution features, compression trace features, and local alteration features are extracted; for call recordings, acoustic playback features, synthesized speech features, and voiceprint consistency features are extracted. The comprehensive security risk score is defined as follows: ; in, This indicates the overall safety risk score of the input materials; This indicates the probability that an image, text message screenshot, material, or voice message was generated by AIGC or deepfake. This indicates the probability that the call recording is susceptible to replay attacks or synthetic speech attacks. This indicates the probability that the text input contains prompt words or malicious commands. This indicates the probability that the material has been partially altered, defaced, or spliced. to This represents the weighting parameter for each security risk item; if Greater than the safety threshold If this occurs, the safety circuit breaker will be triggered, stopping the automatic review and submission and transferring the case to manual review. In a multi-institution joint training scenario, federated learning is adopted; in the t-th round of training, the local model parameters of the k-th institution are... The sample size is If the total number of samples is N, then the central aggregation parameter is: ; in, These are the aggregated global model parameters. For the local model parameters uploaded by the kth client, Let be the number of local samples for the k-th client. The total number of samples from all participating clients; In training tasks that enhance privacy budget control, differential privacy stochastic gradient descent is employed; for single-sample gradients... To perform cropping, the expression is: ; in, For the j-th sample, client, or parameter group, the original gradient is... Let C be the gradient after clipping, and C be the clipping threshold. Let L be the 2-norm of the gradient; Next, Gaussian noise is added to the batch gradient, and its expression is: ; Where B is the batch size. This represents the noise figure.

[0027] Step 2: The AI-assisted listening module performs normalization, text transcription, speaker separation, intent recognition, and fact extraction on the call recording data to obtain the AI-assisted listening results.

[0028] In this embodiment of the invention, step 2 includes: The AI-assisted listening module is used to deeply understand call recordings and analyze the intentions of both parties in the conversation. Its processing flow includes audio preprocessing, speech recognition, speaker separation, dialogue segmentation, intent recognition, fact extraction, and call credibility calculation; firstly, the call recording is processed... Preprocessing is performed to obtain the processed audio. : ; in, This represents a speech preprocessing function used to perform preprocessing operations on the input call recording, such as noise reduction, silent segment processing, and volume normalization. Then, a speech recognition model is used to generate timestamped transcribed text. : ; in, This represents an automatic speech recognition function used to convert pre-processed call recordings into speech-to-text with timestamp information. The speaker segment set is obtained by using a speaker separation model: ; in, This indicates the speaker role corresponding to the m-th dialogue segment, including customer, agent, surveyor, or reviewer; This represents the start timestamp of the m-th dialogue segment, used to identify the start time position of the current dialogue segment in the call recording; This represents the end timestamp of the m-th dialogue segment, used to identify the end time position of the current dialogue segment in the call recording; This represents the speech-to-text content corresponding to the m-th dialogue segment; Each dialogue segment is input into the intent recognition model, and the segment text vector is denoted as . The probability of intent is: ; in, Indicates the category of the dialogue intent to be identified; This represents the m-th dialogue segment. The semantic vector obtained after encoding by the text encoding model; The weight matrix represents the intent classification model; This represents the bias vector of the intent classification model; softmax represents the normalization exponential function, used to convert the scores of each intent category into a probability distribution. This represents the semantic vector of the m-th dialogue segment. Under the given conditions, the probability that the current segment belongs to the current intent category; Intent categories include confirmation, denial, withdrawal, request for supplementary materials, consent to processing, incomplete information, dispute, complaint, and invalid call; the category with the highest probability is taken as the main intent of the current dialogue segment. ; in, This represents the category parameter used to determine the maximum value of the objective function; This indicates that the maximum probability value among all candidate intent categories is selected; The highest probability value is then used as the confidence level for intent recognition of that segment: ; in, This represents the maximum predicted probability value of the m-th dialogue segment across all candidate intent categories, used to characterize the confidence level of the intent recognition result for the current dialogue segment; Next, extract the facts of the call: ; in, This represents the set of factual features extracted from the call recording; claim_no represents the case number, extracted from the case number, report number, or business transaction number in the speech-to-text transcript; policy_no represents the policy number, extracted from the policy number or insurance contract number; customer_intent represents the customer's main request or intention, such as agreeing to the processing, requesting withdrawal of the case, requesting supplementary materials, raising a dispute, or requesting manual processing; confirm_result represents the customer's confirmation status of the review matters or processing results, such as confirmed, not confirmed, explicitly denied, or incomplete confirmation; material_status represents the status of the materials mentioned by the customer or agent in the call, such as submitted, not submitted, needing supplementation, unclear materials, or inconsistent materials; special_note represents special prompts or abnormal information identified in the call, such as customer complaints, risk disputes, contradictory information, duplicate calls, or needing manual review; asr_conf represents the average transcription confidence score output by the speech recognition model; speaker_role represents the speaker role identification result, including customer, agent, surveyor, or reviewer. In the specific implementation, Each fact field can also include field-level confidence levels, start and end times of the evidence segment, and original text evidence. For example, customer_intent is represented as: ; Wherein, value represents the value of the customer intent field; conf represents the extraction confidence level of the field; start_time and end_time represent the start and end times of the corresponding voice evidence segment in the call recording; evidence_text represents the corresponding speech-to-text transcript or the anonymized speech-to-text transcript.

[0029] The credibility score for a call is defined as follows: ; in, Score the credibility of the call data source; For speech recognition confidence, Separate confidence levels for the speaker. To determine the confidence level for intent identification, For the integrity of critical information, To determine the degree of factual conflict within the same call, to These are preset weights or weight parameters obtained through training with historical audit samples. , , , and All values ​​are normalized to the [0,1] interval; Normalize to the [0,1] interval using a truncation function or sigmoid function for subsequent data source credibility fusion.

[0030] Step 3: Use the AI-assisted review module to perform OCR recognition, layout parsing, field extraction, material integrity verification, approval form layout verification, signature or seal area detection, and material credibility calculation on SMS text, SMS screenshots, paper material images, and video attachments to obtain the AI-assisted review results.

[0031] In this embodiment of the invention, step 3 includes: The AI-assisted viewing module is used to understand unstructured information from SMS text, SMS screenshots, images of paper materials, and video attachments; it performs OCR recognition, layout parsing, field extraction, and material integrity verification on SMS text, SMS screenshots, and images of paper materials. For screenshots of text messages or images of printed materials, an OCR model is used to output a set of text blocks: ; Where L represents the number of text blocks identified; This represents the identified text of the l-th text block; This represents the coordinates of the l-th text block in the material image, including the x-coordinate of the top-left corner, the y-coordinate of the top-left corner, the width of the text box, and the height of the text box; This represents the OCR recognition confidence score of the l-th text block, with values ​​normalized to the [0,1] interval; For tables, signatures, seals, and date areas, the layout parsing model is used to output layout elements: ; Where Q represents the number of page elements identified; This indicates the type of the q-th page element, including title, body text, table, seal, signature, date, amount, and number; This represents the region coordinates of the q-th page element in the material image; This represents the recognition confidence level of the q-th page element; for strong verification elements such as signatures, seals, and dates, the system can... and Determine whether the corresponding elements exist, whether their positions are reasonable, and whether they meet the requirements of the review rules.

[0032] Material facts are obtained based on entity extraction models and field mapping dictionaries: ; in, This represents the set of factual features extracted from SMS text, SMS screenshots, images of paper materials, or video attachments; claim_no represents the case number identified in the material; policy_no represents the policy number identified in the material; customer_name represents the customer name or anonymized customer identifier identified in the material; notice_time represents the time field in the SMS, notification, or material; material_type represents the material type; process_reason represents the processing reason or application reason recorded in the material; document_result represents the processing conclusion or customer confirmation result expressed in the material; signature_flag represents whether a signature or seal was detected; seal_flag represents whether a seal was detected; and ocr_conf represents the overall OCR confidence level of the material. The material integrity score is: ; in, Indicates the applicable law based on the i-th case. The required material field set obtained Indicates the review rules; Indicates from The actual set of material fields identified in the data. Represents images of paper materials; Indicates the number of fields that have met the requirements; This is used to avoid division by zero errors when the required field set is empty; The value range is [0, 1], and the larger the value, the more intact the material. Material credibility score is defined as: ; in, Indicates the overall OCR recognition confidence level of the material; The confidence level of the page layout analysis is represented by the number of page elements. We get the weighted average. Indicates the material integrity score; Indicates the time-sensitivity of materials. This indicates the risk of material tampering. Indicates the degree of field missingness, by Or the proportion of missing key fields is indicated. to This indicates preset weights or weights obtained through training based on historical review samples. , , , , and All are normalized to the [0,1] interval. Normalize to the [0,1] interval using a truncation function or a sigmoid function.

[0033] Step 4: Map the AI-assisted listening results, AI-assisted viewing results, structured case fields, audit rules, and historical audit logs to a unified audit fact table, and calculate the cross-source consistency score based on the unified audit fact table.

[0034] In this embodiment of the invention, step 4 includes: The consistency between the customer's requests during the call and those in the materials is as follows: ; in, This indicates that the AI-assisted listening module identifies the customer's intent from the call recording. This indicates the customer intent or processing request identified by the AI-assisted module from SMS text, SMS screenshots, or paper materials. `exact_match` represents the matching function that checks whether the standardized intent tags match. This refers to descriptive segments in the call transcript that are relevant to the customer's request. This represents descriptive segments in the text material related to customer needs; Emb represents the text vectorization encoding function; and cos represents the cosine similarity. This represents the weighting coefficient between exact tag matching and semantic similarity, with a value range of [0,1]. The call confirmation result is consistent with the system case status as follows: ; in, This indicates the customer confirmation status extracted from the call recording. This indicates the case status or processing status in the business system. "Match" represents the status mapping matching function. When the customer confirms that the status and the system status meet the preset business correspondence, a higher score is taken; otherwise, a lower score is taken. The key fields such as case number and policy number are consistent as follows: ; Where K represents the number of critical fields that need to be validated. This represents the value of the k-th key field identified from the material or text message. This represents the k-th field value in the business system, and `exact_match` represents the exact matching function after field standardization. The value range is [0,1], and the larger the value, the more consistent the material field is with the system field; The completeness of materials and consistency with the audit rules are as follows: ; in, Representing images from paper materials The set of submitted material fields or material types identified in the data. Indicates the review rules The required material fields or set of material types; The consistency of similar historical cases is as follows: ; in, This represents the review representation vector of the current case. Let the vector represent the review process for the j-th historical case. This means selecting the historical case with the highest similarity to the current case from the set of similar historical cases, where cos represents the cosine similarity. Combining the consistency scores .

[0035] Step 5: Calculate the data source credibility weight and integrate the review features of different sources based on the data source credibility weight to generate a comprehensive case review characterization.

[0036] In this embodiment of the invention, step 5 includes: Calculate the data source credibility for call recordings, text messages or screenshots, paper materials, structured case fields, review rules, and historical logs respectively; The original credibility score of the s-th data source is: ; in, Score the data quality. To score for completeness, To identify confidence levels for the model, To ensure consistency with other data sources, To mitigate the risk of tampering, The missing rate, to This represents the learnable weight parameters; the credibility of each data source is normalized to obtain the credibility-normalized weights: ; Map features from different sources to a unified vector space: ; in, Represents the set of factual features of the call; Represents the set of factual characteristics of the material; This indicates the structured case characteristics of the business system; Indicates the characteristics of the review rules; This indicates characteristics of historical review logs or similar historical cases. , , , and These represent linear mapping matrices from different data sources; , , , and These represent the corresponding bias vectors; The weighted fusion vector is: ; Employing attention mechanisms to learn the interaction relationships between different data sources: ; in, The expression represents the attention fusion result; Q represents the query matrix, obtained by linear transformation of the data source vector; K represents the key matrix, obtained by linear transformation of the data source vector; V represents the value matrix, obtained by linear transformation of the data source vector. K represents the transpose of K; d represents the vector dimension, used to scale the dot product result; softmax is used to obtain the interaction weights between different data sources; through the attention mechanism, the association between call facts, material facts, structured fields and review rules is learned; The final comprehensive review of the case is characterized as follows: .

[0037] Step 6: Perform review and judgment through the AI-assisted review module, perform pre-submission verification before submitting the review results, and complete automatic submission and audit traceability.

[0038] In this embodiment of the invention, step 6 includes: The AI-assisted review module automatically completes review judgment, review opinion generation, and submission action planning (rule verification, risk assessment, and review decision-making) based on the comprehensive review characteristics of the case, review rules, and process status. First, the strong rule engine determines whether there are situations requiring manual review or where automatic submission is prohibited: ; Where, rule_flag represents the rule determination result output by the strong rule engine; RuleEngine represents the rule engine function; This represents the unified audit fact table for the i-th case; This represents the set of review rules applicable to the i-th case; In AI-assisted review, cancellation and reopening links are constructed based on the type of business action. The cancellation link is used to determine whether the customer has a clear, genuine and sufficient intention to cancel or withdraw the case. The reopening link is used to determine whether the supplementary materials meet the compliance elements of the approval form, signature, seal, date and system status required for reopening. The confidence level of the intention to cancel is defined as: ; in, This represents the confidence level of the intention to cancel the i-th case; denoted as S-type activation function; s represents the data source number involved in the cancellation intent judgment, and the data sources include call recordings, SMS texts, case notes, screenshots, and historical review logs; alpha_s represents the credibility normalization weight of the s-th data source; This represents the content of the evidence in the s-th data source; This represents the cancellation intent feature or intent score extracted from the data source; This indicates a bias term used to determine the intent to cancel. The compliance determination for reissuing vouchers is as follows: ; in, This indicates the compliance determination result for reopening vouchers; Indicates the validity status of the approval form; This indicates the validity of the signature or seal; `rule_flag = allow` means that the strong rule engine determines that the current case status and business rules allow reopening; if... If the value is 0, a corresponding missing feedback will be generated, such as non-standard approval form, no signature detected, missing seal, invalid date, or case status that does not allow reopening; Then, a hierarchical risk scoring model based on cross-source consistency and data source credibility is used to calculate the probability of case anomalies. This hierarchical risk scoring model does not make judgments directly based on data from a single source, but rather combines cross-source consistency vectors, data source credibility, rule hit characteristics, structured case characteristics, and historical review characteristics into a risk feature vector. ; in, This represents the risk score input feature vector for the i-th case; concat represents the vector concatenation operation. Represents a cross-source consistency vector; This indicates the credibility score of the call; Indicates the credibility score of the material; Indicates the rule's hit characteristics; This indicates the structured case characteristics of the business system; Indicates historical review characteristics; This indicates a comprehensive review of the case. The risk scoring model outputs the probability of case anomalies based on X_risk_i: ; in, The risk scoring model uses LightGBM, taking cross-source consistency features, data source credibility features, rule hit features, and historical review features as inputs, and using historical manual review conclusions, quality inspection results, or reasons for return as supervision labels for training. The value range is [0,1]; To reduce probability bias caused by different branch offices, different processing types, or different time batches, an order-preserving regression probability calibration function based on branch office and processing type grouping is used. Perform calibration: ; in, This indicates the probability of an anomaly in the case after calibration. This indicates the branch company to which the case belongs. and case handling type The order-preserving regression calibration function is obtained by training the model output probability and the actual quality inspection label in the historical audit samples, and maintains the monotonic relationship between the input probability and the calibrated probability. Risk score: ; in, This represents the risk score for the i-th case; round represents the rounding function. The value ranges from 0 to 100, with higher values ​​indicating that the case requires more manual review or special investigation. The review agent determines the status based on the current review status. A sequence of actions can be generated using a set of actions A and rule constraints: ; in, This represents the review action selected by the review agent in the t-th process state; 'a' represents the candidate action. Indicates the current process status Comprehensive Case Review Characteristics and review rules The strategy probability or action score for choosing action a under given conditions; This indicates that the action with the highest probability or score has been selected. The action set includes reading case information, verifying call facts, verifying material facts, executing rule verification, generating review opinions, pre-submission verification, calling business interfaces to submit results, and transferring to manual review; Combining rule_flag, Contribution of risk factors and The review conclusion was: ; in, 'Decision' represents the final review decision for the i-th case; 'Decision' represents the decision fusion function. The action_sequence represents the set of risk factor contributions, obtained from SHAP values, feature importance, or rule hit weights; action_sequence represents a sequence of multiple risk factors. The sequence of review actions; This includes one or more of the following: automatic approval, automatic cancellation or closure, manual review, special verification, supplementary materials, and prohibition of submission; by combining strong rule gating, credibility-weighted risk scoring, and action decision-making, it is possible to avoid a single machine learning model directly controlling the review submission, thereby improving the security and interpretability of automatic review.

[0039] when When setting safety circuit breaker conditions: ; in, Indicates whether the i-th case triggers the circuit breaker; This indicates the overall security risk score; Indicates the safety risk threshold. > This indicates that the overall security risk score is greater than the security risk threshold; This indicates that the confidence level of the intention to cancel is in the fuzzy range; Indicates cross-source consistency value, Indicates the consistency threshold. This indicates that cross-source consistency is less than the consistency threshold; Indicates the confidence level of the signature or seal test; Indicates the signature detection threshold. This indicates that the confidence level for signature or seal detection is less than the signature detection threshold; when If the submission fails, the automatic submission will be stopped and the submission will be transferred to a human reviewer.

[0040] In this embodiment of the invention, step 6 performs pre-submission verification before submitting the review results, including field integrity verification, case status verification, permission verification, duplicate submission verification, and de-identification status verification; after the verification is passed, the review conclusion is submitted through the business system API or process automation. The audit explanation is generated jointly by rule hits, cross-source conflict items, and model contributions; the feature contribution is calculated using SHAP values, then the contribution of the j-th risk factor is: ; in, This represents the contribution of the j-th risk factor to the output of the risk scoring model; This represents the SHAP interpretation value calculated for the j-th feature; This represents a risk scoring model; This represents the risk score input feature vector for the i-th case; Used to filter the risk factors that contribute the most and map them to the audit explanation text; At the same time, a log of the review process for each case is recorded: ; Wherein, claim_id represents the case number; model_version represents the model version; rule_version represents the rule version; and evidence_hash represents the hash summary of key evidence materials or the audit fact table. Indicates the review decision; action_sequence indicates the sequence of review actions; submit_result indicates the submission result from the business system; time_stamp indicates the timestamp of the log record. Calculate hash digests for key log fields: ; Used to verify whether logs have been tampered with, and to support subsequent quality inspection, traceability and compliance audit; Manual review results and quality inspection results are used as feedback samples and enter the training data pool. For data that is allowed to be returned, only the de-identified fact fields, model features, review conclusions, and quality inspection labels are returned. For multi-institutional scenarios, federated learning is used to return model parameters in order to avoid the original customer data from flowing across institutions.

[0041] In the special case review scenario of auto insurance, real case samples containing call recordings, SMS screenshots, images of paper materials, structured fields of business systems, and historical review records are selected for verification. The review agent can automatically identify key information such as case number, complainant, insured, policyholder, compensation amount, relevant personnel, whether there is expense liability insurance and cancellation of insurance, and output the analysis process and review conclusion of the application case.

[0042] In embodiments of the present invention, such as Figure 3 As shown, the system intelligently reviews special auto insurance cases. The system automatically identifies key information such as the claim number, the claimant, the claimant's phone number, the insured, the insured's phone number, outstanding amount, personal details (whether there are injuries), type of insurance coverage (whether there is third-party liability insurance or out-of-pocket medical expense insurance), and the type of insurance to be cancelled. It also verifies the customer's intention by combining this information with the recorded follow-up call. Analysis shows that the follow-up call recording matches the case contact person information, both the policyholder and the insured clearly expressed their agreement to cancel the case, and the recording content does not significantly conflict with the case information in the system. Ultimately, the system generates a review conclusion of "Information verified, can be processed according to procedure," meaning the review is approved and the case can be cancelled.

[0043] According to the trial operation statistics, the system handles an average of 2,600 cases per day, with a quality inspection accuracy rate of 100%. It has been promoted to 31 branches, with a job replacement rate of 100%, significantly improving the processing efficiency, consistency of conclusions, and traceability of special auto insurance cases.

[0044] This invention involves technologies such as speech recognition, dialogue intent recognition, OCR text recognition, layout parsing, entity extraction, cross-source consistency verification, data source credibility fusion, rule engine, risk scoring, review action planning, automated process submission, log auditing, federated learning, and differential privacy. The system constructs a closed-loop optimization mechanism based on review result feedback, manual review results, quality inspection results, and historical audit logs. For data that is allowed to be recirculated, only the anonymized review fact fields, model features, review conclusions, and quality inspection labels are recirculated and used as incremental training samples in the training data pool. The system continuously updates and optimizes the risk scoring model, rule engine parameters, cross-source consistency weights, and data source credibility weights based on feedback samples, forming an intelligent closed loop of review—feedback—learning—optimization. In multi-institutional collaborative scenarios, federated learning is used to recirculate model parameters, enabling continuous enhancement of model capabilities while avoiding the cross-institutional transfer of original customer data, thus balancing review effectiveness and privacy protection requirements.

[0045] The technical solution provided by this invention includes the following steps: First, acquiring heterogeneous data from special auto insurance case review cases; second, performing privacy-protected preprocessing on the heterogeneous data to obtain privacy-protected heterogeneous data; third, using an AI-assisted listening module to perform normalization, text transcription, speaker separation, intent recognition, and review fact extraction on the call recording data to obtain AI-assisted listening results; fourth, using an AI-assisted viewing module to perform OCR recognition, layout parsing, field extraction, material integrity verification, approval form layout verification, signature or seal area detection, and material credibility calculation on SMS text, SMS screenshots, paper material images, and video attachments to obtain AI-assisted viewing results; and fifth, mapping the AI-assisted listening results, AI-assisted viewing results, structured case fields, review rules, and historical review logs into a unified review fact table. This method calculates cross-source consistency scores based on a unified audit fact table; calculates data source credibility weights and integrates audit features from different sources according to these weights to generate a comprehensive case audit representation; and executes audit judgments through an AI-assisted audit module, performing pre-submission verification before submitting the audit results, and completing automatic submission and audit traceability. By linking AI-assisted listening, AI-assisted viewing, and AI-assisted auditing to form a closed-loop process from understanding unstructured information, cross-source verification, risk assessment to audit submission, this method automates repetitive hearings, material verification, and system submission operations. Under the conditions of meeting audit rules and privacy protection requirements, it improves the efficiency of special auto insurance case audit processing, enhances the security and interpretability of automatic audits, and improves the feasibility of heterogeneous data fusion processing.

[0046] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for intelligent agent review of special auto insurance cases based on privacy-preserving heterogeneous data fusion, characterized in that, The method includes: Step 1: Obtain heterogeneous data of special car insurance cases for review, and perform privacy protection preprocessing on the heterogeneous data to obtain privacy-protected heterogeneous data; Step 2: The AI-assisted listening module performs normalization, text transcription, speaker separation, intent recognition, and fact extraction on the call recording data to obtain the AI-assisted listening results; Step 3: Use the AI-assisted review module to perform OCR recognition, layout parsing, field extraction, material integrity verification, approval form layout verification, signature or seal area detection, and material credibility calculation on SMS text, SMS screenshots, paper material images, and video attachments to obtain the AI-assisted review results. Step 4: Map the AI-assisted listening results, AI-assisted viewing results, structured case fields, review rules, and historical review logs to a unified review fact table, and calculate the cross-source consistency score based on the unified review fact table; Step 5: Calculate the data source credibility weight and integrate the review characteristics of different sources based on the data source credibility weight to generate a comprehensive case review characterization; Step 6: Perform review and judgment through the AI-assisted review module, perform pre-submission verification before submitting the review results, and complete automatic submission and audit traceability.

2. The method according to claim 1, characterized in that, The heterogeneous data in step 1 includes one or more of the following: call recordings, SMS text or SMS screenshots, images of paper materials, structured case data from business systems, review rule data, and historical review logs. To model heterogeneous review data uniformly, for any auto insurance special case review case, a heterogeneous review data object is constructed using the case number claim_id as the primary key. Its expression is: ； in, This represents the set of review data for the i-th case; This indicates call recording data; This refers to the text message or a screenshot of the text message. Represents images of paper materials; This represents the structured case field in the business system; Indicates the review rules; This indicates the historical review log; right A unified audit field dictionary is established for the data, mapping the fields of [Customer Request], [Reason for Processing], [Customer Confirmation Result], [Material Status], [Case Number], [Policy Number], [Audit Conclusion], and [Automatic Submission Allowed] to standard fields, forming a unified audit fact table with the following expression: ； Each audit fact Represented as: ； in, For fact field names, For field values, For source data, To identify confidence levels, For fact timestamps, For evidence location information, In a desensitized state, This is a hash digest of the field.

3. The method according to claim 2, characterized in that, The privacy protection preprocessing in step 1 includes sensitive information identification and desensitization, privacy protection, AIGC content authentication, anti-replay attack detection, and prompt word injection interception; privacy protection processing is performed before data entry, model training, model inference, and review submission; for text, optical character recognition (OCR) results, and speech transcription results, rule matching and named entity recognition models are used to identify sensitive fields; Let the input text sequence be: ； The BERT-CRF named entity recognition model outputs the following sensitive entity label sequence: ； The conditional probability of CRF is: ； in, A represents the label transition matrix, and E represents the label emission score output by the coding model. Perform the following privacy protection processing on the identified sensitive fields: a. Text-sensitive fields are encrypted using mask replacement, tokenization, or format preservation encryption; b. Sensitive fields in images of paper materials or screenshots of text messages are blurred, obscured, or partially encrypted; c. The original call recordings are stored in encrypted form, and the de-identified transcribed text is used first during the reasoning process; d. Structured business fields employ field-level encryption and role-based access control; e. Model input, output, and review logs should retain only necessary anonymized summaries and evidence indexes; Based on privacy protection, AI-native security detection and protection processing are performed, including AIGC authentication, anti-replay attack detection, and prompt word injection interception; for SMS screenshots, paper material images or video attachments, frequency domain features, noise distribution features, compression trace features, and local alteration features are extracted; for call recordings, acoustic playback features, synthesized speech features, and voiceprint consistency features are extracted. The comprehensive security risk score is defined as follows: ； in, This indicates the overall safety risk score of the input materials; This indicates the probability that an image, text message screenshot, material, or voice message was generated by AIGC or deepfake. This indicates the probability that the call recording is susceptible to replay attacks or synthetic speech attacks. This indicates the probability that the text input contains prompt words or malicious commands. This indicates the probability that the material has been partially altered, defaced, or spliced. to This represents the weighting parameter for each security risk item; if Greater than the safety threshold If this occurs, the safety circuit breaker will be triggered, stopping the automatic review and submission and transferring the case to manual review. In a multi-institution joint training scenario, federated learning is adopted; in the t-th round of training, the local model parameters of the k-th institution are... The sample size is If the total number of samples is N, then the central aggregation parameter is: ； in, These are the aggregated global model parameters. For the local model parameters uploaded by the kth client, Let be the number of local samples for the k-th client. The total number of samples from all participating clients; In training tasks that enhance privacy budget control, differential privacy stochastic gradient descent is employed; for single-sample gradients... To perform cropping, the expression is: ； in, For the j-th sample, client, or parameter group, the original gradient is... Let C be the gradient after clipping, and C be the clipping threshold. Let L be the 2-norm of the gradient; Next, Gaussian noise is added to the batch gradient, and its expression is: ； Where B is the batch size. This represents the noise figure.

4. The method according to claim 3, characterized in that, Step 2 includes: The AI-assisted listening module is used to deeply understand call recordings and analyze the intentions of both parties in the conversation. Its processing flow includes audio preprocessing, speech recognition, speaker separation, dialogue segmentation, intent recognition, fact extraction, and call credibility calculation; firstly, the call recording is processed... Preprocessing is performed to obtain the processed audio. : ； in, This represents a speech preprocessing function used to perform preprocessing operations on the input call recording, such as noise reduction, silent segment processing, and volume normalization. Then, a speech recognition model is used to generate timestamped transcribed text. : ； in, This represents an automatic speech recognition function used to convert pre-processed call recordings into speech-to-text with timestamp information. The speaker segment set is obtained by using a speaker separation model: ； in, This indicates the speaker role corresponding to the m-th dialogue segment, including customer, agent, surveyor, or reviewer; This represents the start timestamp of the m-th dialogue segment, used to identify the start time position of the current dialogue segment in the call recording; This represents the end timestamp of the m-th dialogue segment, used to identify the end time position of the current dialogue segment in the call recording; This represents the speech-to-text content corresponding to the m-th dialogue segment; Each dialogue segment is input into the intent recognition model, and the segment text vector is denoted as . The probability of intent is: ； in, Indicates the category of the dialogue intent to be identified; This represents the m-th dialogue segment. The semantic vector obtained after encoding by the text encoding model; The weight matrix represents the intent classification model; This represents the bias vector of the intent classification model; softmax represents the normalization exponential function, used to convert the scores of each intent category into a probability distribution. This represents the semantic vector of the m-th dialogue segment. Under the given conditions, the probability that the current segment belongs to the current intent category; Intent categories include confirmation, denial, withdrawal, request for supplementary materials, consent to processing, incomplete information, dispute, complaint, and invalid call; the category with the highest probability is taken as the main intent of the current dialogue segment. ； in, This represents the category parameter used to determine the maximum value of the objective function; This indicates that the maximum probability value among all candidate intent categories is selected; The highest probability value is then used as the confidence level for intent recognition of that segment: ； in, This represents the maximum predicted probability value of the m-th dialogue segment across all candidate intent categories, used to characterize the confidence level of the intent recognition result for the current dialogue segment; Next, extract the facts of the call: ； in, This represents the set of factual features extracted from the call recording; claim_no represents the case number, extracted from the case number, report number, or business transaction number in the speech-to-text; policy_no represents the policy number, extracted from the policy number or insurance contract number; customer_intent represents the customer's main claim or intention; confirm_result represents the customer's confirmation status of the review matter or processing result; material_status represents the status of the materials mentioned by the customer or agent during the call; special_note represents special prompts or abnormal information identified during the call; asr_conf represents the average transcription confidence score output by the speech recognition model; speaker_role represents the speaker role identification result, including customer, agent, surveyor, or reviewer. The credibility score for a call is defined as follows: ； in, Score the credibility of the call data source; For speech recognition confidence, Separate confidence levels for the speaker. To determine the confidence level for intent identification, For the integrity of critical information, To determine the degree of factual conflict within the same call, to These are preset weights or weight parameters obtained through training with historical audit samples. , , , and All values ​​are normalized to the [0,1] interval; Normalize to the [0,1] interval using a truncation function or sigmoid function for subsequent data source credibility fusion.

5. The method according to claim 4, characterized in that, Step 3 includes: The AI-assisted viewing module is used to understand unstructured information from SMS text, SMS screenshots, images of paper materials, and video attachments; it performs OCR recognition, layout parsing, field extraction, and material integrity verification on SMS text, SMS screenshots, and images of paper materials. For screenshots of text messages or images of printed materials, an OCR model is used to output a set of text blocks: ； Where L represents the number of text blocks identified; This represents the identified text of the l-th text block; This represents the coordinates of the l-th text block in the material image, including the x-coordinate of the top-left corner, the y-coordinate of the top-left corner, the width of the text box, and the height of the text box; This represents the OCR recognition confidence score of the l-th text block, with values ​​normalized to the [0,1] interval; For tables, signatures, seals, and date areas, the layout parsing model is used to output layout elements: ； Where Q represents the number of page elements identified; This indicates the type of the q-th page element, including title, body text, table, seal, signature, date, amount, and number; This represents the region coordinates of the q-th page element in the material image; This represents the confidence level of the q-th page element. Material facts are obtained based on entity extraction models and field mapping dictionaries: ； in, This represents the set of factual features extracted from SMS text, SMS screenshots, images of paper materials, or video attachments; claim_no represents the case number identified in the material; policy_no represents the policy number identified in the material; customer_name represents the customer name or anonymized customer identifier identified in the material; notice_time represents the time field in the SMS, notification, or material; material_type represents the material type; process_reason represents the processing reason or application reason recorded in the material; document_result represents the processing conclusion or customer confirmation result expressed in the material; signature_flag represents whether a signature or seal was detected; seal_flag represents whether a seal was detected; and ocr_conf represents the overall OCR confidence level of the material. The material integrity score is: ； in, Indicates the applicable law based on the i-th case. The required material field set obtained Indicates the review rules; Indicates from The actual set of material fields identified in the data. Represents images of paper materials; Indicates the number of fields that have met the requirements; This is used to avoid division by zero errors when the required field set is empty; The value range is [0, 1], and the larger the value, the more intact the material. Material credibility score is defined as: ； in, Indicates the overall OCR recognition confidence level of the material; The confidence level of the page layout analysis is represented by the number of page elements. We get the weighted average. Indicates the material integrity score; Indicates the time-sensitivity of materials. This indicates the risk of material tampering. Indicates the degree of field missingness, by Or the proportion of missing key fields is indicated. to This indicates preset weights or weights obtained through training based on historical review samples. , , , , and All are normalized to the [0,1] interval. Normalize to the [0,1] interval using a truncation function or a sigmoid function.

6. The method according to claim 5, characterized in that, Step 4 includes: The consistency between the customer's requests during the call and those in the materials is as follows: ； in, This indicates that the AI-assisted listening module identifies the customer's intent from the call recording. This indicates the customer intent or processing request identified by the AI-assisted module from SMS text, SMS screenshots, or paper materials. `exact_match` represents the matching function that checks whether the standardized intent tags match. This refers to descriptive segments in the call transcript that are relevant to the customer's request. This represents descriptive segments in the text material related to customer needs; Emb represents the text vectorization encoding function; and cos represents the cosine similarity. This represents the weighting coefficient between exact tag matching and semantic similarity, with a value range of [0,1]. The call confirmation result is consistent with the system case status as follows: ； in, This indicates the customer confirmation status extracted from the call recording. This indicates the case status or processing status in the business system. "Match" represents the status mapping matching function. When the customer confirms that the status and the system status meet the preset business correspondence, a higher score is taken; otherwise, a lower score is taken. The key fields such as case number and policy number are consistent as follows: ； Where K represents the number of critical fields that need to be validated. This represents the value of the k-th key field identified from the material or text message. This represents the k-th field value in the business system, and `exact_match` represents the exact matching function after field standardization. The value range is [0,1], and the larger the value, the more consistent the material field is with the system field; The completeness of materials and consistency with the audit rules are as follows: ； in, Representing images from paper materials The set of submitted material fields or material types identified in the data. Indicates the review rules The required material fields or set of material types; The consistency of similar historical cases is as follows: ； in, This represents the review representation vector of the current case. Let the vector represent the review process for the j-th historical case. This means selecting the historical case with the highest similarity to the current case from the set of similar historical cases, where cos represents the cosine similarity. Combining the consistency scores .

7. The method according to claim 6, characterized in that, Step 5 includes: Calculate the data source credibility for call recordings, text messages or screenshots, paper materials, structured case fields, review rules, and historical logs respectively; The original credibility score of the s-th data source is: ； in, Score the data quality. To score for completeness, To identify confidence levels for the model, To ensure consistency with other data sources, To mitigate the risk of tampering, The missing rate, to This represents the learnable weight parameters; the credibility of each data source is normalized to obtain the credibility-normalized weights: ； Map features from different sources to a unified vector space: ； in, Represents the set of factual features of the call; Represents the set of factual characteristics of the material; This indicates the structured case characteristics of the business system; Indicates the characteristics of the review rules; This indicates characteristics of historical review logs or similar historical cases. , , , and These represent linear mapping matrices from different data sources; , , , and These represent the corresponding bias vectors; The weighted fusion vector is: ； Employing attention mechanisms to learn the interaction relationships between different data sources: ； in, The expression represents the attention fusion result; Q represents the query matrix, obtained by linear transformation of the data source vector; K represents the key matrix, obtained by linear transformation of the data source vector; V represents the value matrix, obtained by linear transformation of the data source vector. K represents the transpose of K; d represents the vector dimension, used to scale the dot product result; softmax is used to obtain the interaction weights between different data sources; through the attention mechanism, the association between call facts, material facts, structured fields and review rules is learned; The final comprehensive review of the case is characterized as follows: 。 8. The method according to claim 7, characterized in that, Step 6 includes: The AI-assisted review module automatically completes review judgment, generates review opinions, and plans submission actions based on the comprehensive review characteristics of the case, review rules, and process status. First, the strong rule engine determines whether there are situations requiring manual review or where automatic submission is prohibited: ； Where, rule_flag represents the rule determination result output by the strong rule engine; RuleEngine represents the rule engine function; This represents the unified audit fact table for the i-th case; This represents the set of review rules applicable to the i-th case; In AI-assisted review, cancellation and reopening links are constructed based on the type of business action. The cancellation link is used to determine whether the customer has a clear, genuine and sufficient intention to cancel or withdraw the case. The reopening link is used to determine whether the supplementary materials meet the compliance elements of the approval form, signature, seal, date and system status required for reopening. The confidence level of the intention to cancel is defined as: ; in, This represents the confidence level of the intention to cancel the i-th case; denoted as S-type activation function; s represents the data source number involved in the cancellation intent judgment, and the data sources include call recordings, SMS texts, case notes, screenshots, and historical review logs; alpha_s represents the credibility normalization weight of the s-th data source; This represents the content of the evidence in the s-th data source; This represents the cancellation intent feature or intent score extracted from the data source; This indicates a bias term used to determine the intent to cancel. The compliance determination for reissuing vouchers is as follows: ； in, This indicates the compliance determination result for reopening vouchers; Indicates the validity status of the approval form; This indicates the validity of the signature or seal; `rule_flag = allow` means that the strong rule engine determines that the current case status and business rules allow reopening; if... If the value is 0, then the corresponding missing feedback is generated; Then, a hierarchical risk scoring model based on cross-source consistency and data source credibility is used to calculate the probability of case anomalies; the hierarchical risk scoring model combines cross-source consistency vector, data source credibility, rule hit characteristics, structured case characteristics, and historical review characteristics into a risk feature vector: ； in, This represents the risk score input feature vector for the i-th case; concat represents the vector concatenation operation. Represents a cross-source consistency vector; This indicates the credibility score of the call; Indicates the credibility score of the material; Indicates the rule's hit characteristics; This indicates the structured case characteristics of the business system; Indicates historical review characteristics; This indicates a comprehensive review of the case. The risk scoring model outputs the probability of case anomalies based on X_risk_i: ； in, The risk scoring model uses LightGBM, taking cross-source consistency features, data source credibility features, rule hit features, and historical review features as inputs, and using historical manual review conclusions, quality inspection results, or reasons for return as supervision labels for training. The value range is [0,1]; To reduce probability bias caused by different branch offices, different processing types, or different time batches, an order-preserving regression probability calibration function based on branch office and processing type grouping is used. Perform calibration: ； in, This indicates the probability of an anomaly in the case after calibration. This indicates the branch company to which the case belongs. and case handling type The order-preserving regression calibration function is obtained by training the model output probability and the actual quality inspection label in the historical audit samples, and maintains the monotonic relationship between the input probability and the calibrated probability. Risk score: ； in, This represents the risk score for the i-th case; round represents the rounding function. The value ranges from 0 to 100, with higher values ​​indicating that the case requires more manual review or special investigation. The review agent determines the review status based on the current review status. A sequence of actions can be generated using a set of actions A and rule constraints: ； in, This represents the review action selected by the review agent in the t-th process state; 'a' represents the candidate action. Indicates the current process status Comprehensive Case Review Characteristics and review rules The strategy probability or action score for choosing action a under given conditions; This indicates that the action with the highest probability or score has been selected. The action set includes reading case information, verifying call facts, verifying material facts, executing rule verification, generating review opinions, pre-submission verification, calling business interfaces to submit results, and transferring to manual review; Combining rule_flag, Risk factor contribution and The review conclusion was: ； in, 'Decision' represents the final review decision for the i-th case; 'Decision' represents the decision fusion function. The action_sequence represents the set of risk factor contributions, obtained from SHAP values, feature importance, or rule hit weights; action_sequence represents a sequence of multiple risk factors. The sequence of review actions; This includes one or more of the following: automatic approval, automatic cancellation or case closure, manual review, special verification, supplementary materials, and prohibition of submission; when When setting safety circuit breaker conditions: ; in, Indicates whether the i-th case triggers the circuit breaker; This indicates the overall security risk score; Indicates the safety risk threshold. > This indicates that the overall security risk score is greater than the security risk threshold; This indicates that the confidence level of the intention to cancel is in the fuzzy range; Indicates cross-source consistency value, Indicates the consistency threshold. This indicates that cross-source consistency is less than the consistency threshold; Indicates the confidence level of the signature or seal test; Indicates the signature detection threshold. This indicates that the confidence level for signature or seal detection is less than the signature detection threshold; when If the submission fails, the automatic submission will be stopped and the submission will be transferred to a human reviewer.

9. The method according to claim 8, characterized in that, In step 6, a pre-submission verification is performed before submitting the review results, including field integrity verification, case status verification, permission verification, duplicate submission verification, and de-identification status verification. After verification, the review conclusion is submitted via business system API or process automation. The audit explanation is generated jointly by rule hits, cross-source conflict items, and model contributions; the feature contribution is calculated using SHAP values, then the contribution of the j-th risk factor is: ； in, This represents the contribution of the j-th risk factor to the output of the risk scoring model; This represents the SHAP interpretation value calculated for the j-th feature; This represents a risk scoring model; This represents the risk score input feature vector for the i-th case; Used to filter the risk factors that contribute the most and map them to the audit explanation text; At the same time, a log of the review process for each case is recorded: ； Wherein, claim_id represents the case number; model_version represents the model version; rule_version represents the rule version; and evidence_hash represents the hash summary of key evidence materials or the audit fact table. Indicates the review decision; action_sequence indicates the sequence of review actions; submit_result indicates the submission result from the business system; time_stamp indicates the timestamp of the log record. Calculate hash digests for key log fields: ; Used to verify whether logs have been tampered with, and to support subsequent quality inspection, traceability and compliance audit; Manual review results and quality inspection results are used as feedback samples and enter the training data pool. For data that is allowed to be returned, only the de-identified fact fields, model features, review conclusions, and quality inspection labels are returned. For multi-institutional scenarios, federated learning is used to return model parameters in order to avoid the original customer data from flowing across institutions.

Images