Risk case judgment system and method based on multiple modes

Abstract

The invention relates to a multi-mode-based risk case judgment system and method, and belongs to the technical field of artificial intelligence and machine learning. A structured Transform (SAINT) model is adopted to encode structured fields of a case, and field interaction and case association are captured by utilizing an attention mechanism between columns and samples of the structured fields; meanwhile, a pre-training language model (RoBERTa) is adopted to carry out deep semantic coding on a text field; and fusing the two types of feature vectors through a multi-modal feature fusion module, and inputting the fused feature vectors into a prediction module to obtain a risk score. According to the method, deep complementation and automatic feature learning of cross-modal information are realized, the recognition capability of the model for high-risk cases can be remarkably improved in a scene with an extremely low risk case proportion, the PR-AUC is improved by more than 35% and the recall rate is improved by about 50% through experimental verification, and meanwhile, the artificial feature engineering cost and the model maintenance difficulty are greatly reduced.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and machine learning technology, and in particular to a risk case determination system and method based on multimodality. Background Technology

[0002] In insurance, finance, and risk management, accurate and efficient assessment of risk cases (such as fraud, abnormal claims, and irregularities) is of significant business value. Traditional risk assessment relies primarily on manual experience review or rule-based systems, but these methods are inefficient, have limited coverage, and struggle to adapt to complex and ever-changing risk patterns. With the development of machine learning technology, gradient boosting tree models, such as XGBoost, are widely used in risk prediction for structured data. These models can model fields such as numerical values ​​and categories in tables, automating risk screening to some extent and improving processing efficiency.

[0003] However, in real-world business scenarios, risk assessment often relies not only on structured tabular data (such as amount, time, location, and category tags) but also on a large amount of unstructured text information, such as case review notes, user instructions, warning records, and communication logs. This text data contains rich contextual semantics, logical relationships, and risk implications, which have a crucial impact on the final risk assessment. Existing mainstream technologies, such as XGBoost, while adept at handling structured features, have the following significant limitations when dealing with such multimodal, high-dimensional, and strongly imbalanced risk assessment tasks: 1. Traditional models struggle to directly understand and process natural language text. They typically rely on manual keyword extraction and shallow text representation methods such as TF-IDF, which prevents the effective capture of deeper semantic information, sentiment, and logical contradictions in the text, resulting in the omission of numerous risk clues.

[0004] 2. Business data contains a large number of high cardinality category features (such as region, ID, etc.) and multi-label fields. Traditional methods usually use one-hot encoding, which can easily lead to feature dimension explosion and data sparsity, thereby affecting the stability of model training and generalization performance.

[0005] 3. Risky cases account for a very small percentage in real-world scenarios (e.g., less than 1%). This extremely unbalanced data distribution makes traditional models prone to bias towards the majority class (normal cases), resulting in a low recognition rate (recall rate) for the minority class (risky cases).

[0006] 4. The occurrence and development of cases often follow a temporal sequence and procedural logic (such as a series of events including reporting, submission, review, and compensation). Traditional models struggle to directly model the dependencies between such time-series behaviors and events. Furthermore, cases may be potentially linked through shared attributes (such as the same phone number, device, or address), but traditional models typically assume independent samples and cannot capture such association patterns.

[0007] 5. The model's performance is highly dependent on manual feature engineering, including field cleaning, combination, transformation, and time difference calculation. This process is costly, has poor maintainability, and struggles to cover all potential risk patterns, thus limiting the model's expressive and adaptive capabilities.

[0008] Therefore, existing technical solutions suffer from inherent limitations in handling complex risk assessment tasks that integrate structured data and unstructured text, including limited recognition capabilities, low automation, and poor scalability. There is an urgent need for an intelligent assessment system and method that can deeply integrate multimodal information, automatically learn features and relationships, and maintain high performance under extreme imbalance conditions, in order to improve the accuracy, efficiency, and business adaptability of risk identification. Summary of the Invention

[0009] This invention designs a risk case determination system and method based on multimodality. The technical problem it solves is that existing technical solutions have inherent defects such as limited recognition ability, low degree of automation, and poor scalability when dealing with complex risk determination tasks that integrate structured data and unstructured text.

[0010] To solve the aforementioned technical problems, the present invention adopts the following solution: A multimodal risk case determination system, characterized in that it includes: The data preprocessing module is used to acquire structured data and unstructured text data of the target case, and to standardize the structured data and the unstructured text data to obtain structured feature sequences and text feature sequences. The structured data modeling module, built on the structured Transformer (SAINT) model, is used to receive the structured feature sequence, perform feature learning through inter-column self-attention mechanism and inter-sample attention mechanism, and output structured feature vector; The text data modeling module, built on the pre-trained language model RoBERTa, is used to receive the text feature sequence, perform deep semantic encoding, and output the text semantic vector. A multimodal feature fusion module is used to fuse the structured feature vector with the text semantic vector to obtain a multimodal fused feature vector; The risk prediction module is used to calculate the model's AUC and F1 score on the data based on the multimodal fusion feature vector.

[0011] AUC (Area Under the Curve), in machine learning classification tasks, usually refers specifically to the area under the ROC curve or the area under the PR curve (PR-AUC). It is an evaluation metric used to measure the overall performance of a classification model across all possible thresholds. The AUC value is a scalar between 0 and 1; a larger value indicates better overall ranking ability or classification performance of the model.

[0012] F1 is the harmonic mean of precision and recall. It is an evaluation metric used to comprehensively measure the accuracy and coverage of a model's predictions at a given classification threshold. The F1 value is a scalar between 0 and 1. F1 is calculated on a dataset by comparing the model's predictions for all samples in that dataset with the true labels. It is not the model's output for a single sample.

[0013] Preferably, the column self-attention mechanism in the structured data modeling module is used to model the interaction relationship between feature fields within the same case sample, and the sample attention mechanism is used to model the association relationship between different case samples.

[0014] Preferably, the multimodal feature fusion module performs fusion in any of the following ways: vector concatenation, weighted summation, or fusion based on cross-attention.

[0015] Preferably, the risk prediction module includes a multilayer perceptron, and the output layer of the multilayer perceptron uses a sigmoid activation function to output the predicted probability that the target case is a risky case.

[0016] Preferably, it also includes a model training module, which uses Focal Loss as the loss function and combines it with a category weighting or sampling strategy to jointly train the structured data modeling module, text data modeling module, multimodal feature fusion module and risk prediction module to address the imbalance between the number of risky cases and normal cases.

[0017] Preferably, the system is applied in scenarios such as insurance fraud detection, financial risk control, or compliance auditing.

[0018] A method for determining risk cases based on multimodal approaches, characterized by the following steps: Obtain structured and unstructured text data of the target case, and preprocess them to obtain structured feature sequences and text feature sequences; The structured feature sequence is input into a structured Transformer (SAINT) model and encoded through inter-column self-attention and inter-sample attention mechanisms to obtain a structured feature vector. The text feature sequence is input into a model based on the pre-trained language model RoBERTa for semantic encoding to obtain a text semantic vector; By fusing the structured feature vector with the text semantic vector, a multimodal fusion feature vector is obtained; Based on the multimodal fusion feature vector, AUC and F1 scores are generated to help determine whether the target case is risky.

[0019] Preferably, the fusion step specifically involves concatenating, weighting, or interacting with the text semantic vector using the structured feature vector and the text semantic vector.

[0020] Preferably, the step of generating a risk score includes: inputting the multimodal fusion feature vector into a classifier to obtain the predicted probability that the target case is a risky case; and outputting a risk determination conclusion based on the comparison result between a preset threshold and the predicted probability.

[0021] Preferably, prior to the acquisition step, a model training step is also included. The training step uses historical case data containing risk labels and performs end-to-end training on the SAINT-based model, the RoBERTa-based model, and the fusion prediction model with the goal of optimizing the area under the precision-recall curve.

[0022] The multimodal risk case determination system and method have the following beneficial effects: (1) This invention significantly improves the comprehensive identification and ranking capabilities in imbalanced scenarios. By using the SAINT model for deep modeling of field interactions and sample associations in structured data, and the RoBERTa model for accurate capture of deep semantics in text, combined with cross-modal fusion, it achieves a more comprehensive and efficient utilization of risk information. Compared with traditional models (such as XGBoost) that can only handle a single modality or rely on shallow features, this solution is particularly effective in identifying minority risk patterns in highly imbalanced business scenarios where the proportion of risk cases is extremely low. This results in substantial improvements in key indicators such as recall, area under the precision-recall curve (PR-AUC), and overall ranking capability (AUC), effectively alleviating the "minority class submersion" problem caused by data imbalance in traditional models and achieving better risk screening performance.

[0023] (2) This invention establishes a connection channel between structured data and text semantics through a multimodal feature fusion module, realizing deep fusion and complementarity between different modal information. When one modality of data has noise, missing data, or ambiguous expression, such as incomplete table fields or brief text descriptions, information from another modality can provide effective compensation and verification. This mechanism significantly improves the model's tolerance to data defects and its stable performance in different business scenarios, enhancing the model's generalization ability.

[0024] (3) This invention significantly reduces the reliance on complex manual feature engineering. Traditional risk control models heavily rely on expert experience for tedious feature construction and rule definition. This invention utilizes SAINT's attention mechanism and RoBERTa's pre-trained knowledge to achieve end-to-end automatic feature learning from raw heterogeneous data to final risk assessment. This avoids the subjectivity, limitations, and high maintenance costs of manual feature engineering, and significantly reduces the technical threshold and time cost of model construction, iteration, and optimization.

[0025] (4) Compared to opaque "black box" models in the prediction process, this invention provides rich model insights. By analyzing the attention weights of SAINT, the contribution of different structured features and their interactions to the prediction can be understood; by visualizing the attention distribution of RoBERTa, key semantic units in the text that influence decision-making can be located. In addition, the model can provide decision-making references based on the learned patterns. These functions provide reviewers with clear decision-making support, which helps in risk tracing, strategy optimization, and the establishment of a trustworthy human-machine collaborative workflow.

[0026] (5) The SAINT, RoBERTa, and fusion module architecture adopted in this invention features modularity and decoupling. When the business requires the addition of new data types (such as image information, speech-to-text transcription) or new structured fields, there is no need to reconstruct the system core; it can be flexibly extended by integrating the corresponding encoding modules. This enables the technical framework to adapt to future business development and changes, and has the long-term potential to support the construction of a wider range of multimodal intelligent risk control platforms. Attached Figure Description

[0027] Figure 1 : A schematic diagram of the module connections of the multimodal risk case determination system of the present invention; Figure 2 : A flowchart of the multimodal risk case determination method of the present invention. Detailed Implementation

[0028] The following is combined with Figure 1 and Figure 2 The present invention will be further described as follows: Example 1: A Multimodal Risk Case Determination System The system provided in this embodiment addresses the challenge of coexisting structured data and unstructured text, and extremely unbalanced risk cases (e.g., accounting for only 1%) in real-world business scenarios such as insurance risk control and fraud detection. It constructs an end-to-end intelligent judgment platform. Figure 1 As shown, the system specifically includes the following modules connected in sequence: 1. Data preprocessing and feature extraction module This module serves as the system entry point, responsible for standardizing and vectorizing the raw heterogeneous data to provide high-quality input for subsequent deep learning models. Its processing flow is divided into two paths: Structured data pathway: For the massive fields in the table, perform the following operations: Fill missing values ​​(e.g., median filling) and Z-score normalization for numerical fields (e.g., amount, duration); embed and encode high cardinality categorical features (e.g., region, province, city, district, type label) and map them into low-dimensional dense vectors, completely avoiding the feature dimension expansion and information dilution problems caused by traditional one-hot encoding; parse multi-label fields to generate effective multi-value representations; convert time fields into timestamps and derive time-series features such as time intervals, days of the week, and time periods, laying the foundation for subsequent modeling of time series behavior.

[0029] Text data path: For unstructured text such as audit notes, warning fields, and case descriptions, perform text cleaning (remove irrelevant characters and unify format), word segmentation, and use the vocabulary of the pre-trained RoBERTa model to tokenize and generate fixed-length token ID sequences (e.g., 128 characters long). Text that is too long is truncated, and text that is too short is padded.

[0030] 2. Structured Data Modeling Module (SAINT) This module is a structured Transformer (SAINT) model designed specifically for tabular data. Its core innovation lies in the dual-path attention mechanism, which aims to systematically solve the shortcomings of traditional models (such as XGBoost) in high-dimensional sparsity, diverse categories, and imbalanced sample scenarios.

[0031] Intra-sample attention: This mechanism learns the complex non-linear interactions between feature fields within a single sample, such as automatically discovering implicit relationships between "claim amount" and "insurance duration," and between "accident type" and "repair shop location." It effectively captures field combinations that traditional models rely on manually constructed features, avoiding tedious feature engineering.

[0032] Inter-sample Attention: This mechanism models the potential associations between samples from different cases. By calculating the similarity between samples, the feature representations of samples with similar risk patterns (such as the same suspicious phone number or similar behavioral sequences) reinforce each other. This mechanism is particularly suitable for extremely imbalanced data distributions in fraud detection, enabling scarce high-risk samples to obtain stronger feature signals through mutual association, thereby significantly improving the model's ability to identify minority classes (recall rate).

[0033] 3. Text Data Modeling Module (RoBERTa) This module employs the powerful pre-trained language model RoBERTa to perform deep semantic encoding on text information, thereby addressing the problem that traditional models cannot understand natural language.

[0034] The model utilizes its deep Transformer architecture to understand the contextual semantics, logical relationships, and tone of texts such as review comments and user instructions. Compared to shallow TF-IDF or keyword matching, RoBERTa can accurately identify implicit risk semantics such as "insufficient materials require supplementation," "contradictory explanations," and "unclear situation requiring verification."

[0035] In practice, several parameters at the bottom of RoBERTa can be frozen, with only the top layer fine-tuned, to efficiently adapt to the specific context of risk control while preserving general language knowledge. After encoding, the hidden state corresponding to the [CLS] marker is taken as the semantic summary vector of the entire text.

[0036] 4. Multimodal feature fusion module This module is responsible for unifying and fusing deep features from different modalities to form a comprehensive case representation, thereby solving the problem of isolated cross-modal information.

[0037] The fusion method can be flexibly selected according to business needs and computing resources: the splicing method is lightweight and efficient, suitable for large-scale real-time scenarios; weighted fusion allows the model to dynamically learn the importance weights of structured and text features; the cross-attention mechanism can achieve more refined intermodal interactions, such as allowing text semantics to query related structured events, or vice versa, thereby deeply modeling the explanatory and corroborative relationship between "text description" and "tabular data".

[0038] 5. AUC and F1 scoring and prediction module The fused multimodal feature vector is input into a prediction head (e.g., a 2-layer MLP), and finally outputs a continuous value between 0 and 1 through the Sigmoid activation function, which serves as the risk probability of the case.

[0039] The score can be directly used for business decisions: set a risk threshold (e.g., 0.8), and cases that exceed the threshold will be automatically marked as "high risk" and pushed to the manual review queue for priority processing; the score can also be used for case risk ranking to achieve a closed loop of machine screening and manual review, thereby significantly reducing the amount of manual review.

[0040] 6. Model Training Strategy To ensure optimal model performance on complex imbalanced data, the following targeted strategies are employed during the training phase: The loss function uses Focal Loss to reduce the impact of a large number of easily classifiable normal samples on gradient updates, forcing the model to focus on risky samples that are difficult to distinguish.

[0041] Sampling and weighting are combined using a class-weighted loss function or undersampling of the majority class during data loading to further alleviate class imbalance.

[0042] Optimization and fine-tuning utilize the AdamW optimizer, coupled with cosine learning rate decay with warmup. SAINT undergoes full training, while RoBERTa employs a hierarchical, progressive fine-tuning strategy to stabilize the training process and prevent overfitting.

[0043] 7. Model Deployment and Service Module During the inference deployment phase, the system is encapsulated as a high-efficiency service pipeline. New case data flows through the same preprocessing pipeline as in the training phase, and is then fed into the loaded SAINT and RoBERTa models for forward inference. After calculation by the fusion and prediction modules, risk scores are output to the business system in real time, achieving automated and intelligent initial risk screening.

[0044] 8. Performance Evaluation System To comprehensively evaluate the model's performance, this system employs a set of evaluation metrics for imbalanced classification tasks, including: AUC (measuring overall ranking ability), Area Under the Precision-Recall Curve (PR-AUC) (a core metric, focusing on minority classes), Recall (measuring the ability to capture high-risk cases), F1-score, and a decimal plot. The decimal plot provides a visual indication of whether high-risk cases are effectively concentrated in high-score segments by the model, ensuring the model has practical business value.

[0045] Example 2: A Multimodal Risk Case Determination Method like Figure 2 As shown, based on the same inventive concept, this embodiment provides a method flow corresponding to the system, characterized by including the following steps: S101: Data Acquisition and Preprocessing.

[0046] Obtain structured tabular data and related text data for cases to be judged.

[0047] S102: Deep encoding of structured features.

[0048] The preprocessed structured data is input into the trained SAINT model, and its inter-column and inter-sample attention mechanisms are used to generate structured feature vectors that contain internal interactions and sample associations.

[0049] S103: Text semantic deep encoding.

[0050] The preprocessed text data is input into the finely tuned RoBERTa model to extract the semantic vectors corresponding to the [CLS] tags, thereby capturing the deep risk semantics in the text.

[0051] S104: Multimodal feature fusion.

[0052] The feature vectors output from steps S102 and S103 are fused (e.g., concatenated) to obtain a unified multimodal case representation.

[0053] S105: Risk Prediction and Output.

[0054] The fused features are input into the prediction head to calculate the risk probability, and the judgment result (such as risk level, score and warning signal) is output according to the preset threshold.

[0055] The present invention has been described above by way of example with reference to the accompanying drawings. Obviously, the implementation of the present invention is not limited to the above-described manner. Any improvements made using the inventive concept and technical solution of the present invention, or the direct application of the inventive concept and technical solution of the present invention to other occasions without modification, are all within the protection scope of the present invention.

Claims

1. A risk case determination system based on multimodal methods, characterized in that, include: The data preprocessing module is used to acquire structured data and unstructured text data of the target case, and to standardize the structured data and the unstructured text data to obtain structured feature sequences and text feature sequences. The structured data modeling module, built on the structured Transformer (SAINT) model, is used to receive the structured feature sequence, perform feature learning through inter-column self-attention mechanism and inter-sample attention mechanism, and output structured feature vector; The text data modeling module, built on the pre-trained language model RoBERTa, is used to receive the text feature sequence, perform deep semantic encoding, and output the text semantic vector. A multimodal feature fusion module is used to fuse the structured feature vector with the text semantic vector to obtain a multimodal fused feature vector; The risk prediction module is used to calculate the risk score of the target case based on the multimodal fusion feature vector.

2. The risk case determination system based on multimodal analysis according to claim 1, characterized in that: The column self-attention mechanism in the structured data modeling module is used to model the interaction relationship between feature fields within the same case sample, and the sample attention mechanism is used to model the association relationship between different case samples.

3. The risk case determination system based on multimodal analysis according to claim 1 or 2, characterized in that: The multimodal feature fusion module performs fusion in any of the following ways: vector concatenation, weighted summation, or fusion based on cross-attention.

4. The risk case determination system based on multimodal analysis according to claim 1, characterized in that: The risk prediction module includes a multilayer perceptron, and the output layer of the multilayer perceptron uses a sigmoid activation function to determine whether the target case is a risky case.

5. The risk case determination system based on multimodal analysis according to claim 1, characterized in that: It also includes a model training module, which uses Focal Loss as the loss function and combines it with a category weighting or sampling strategy to jointly train the structured data modeling module, text data modeling module, multimodal feature fusion module and risk prediction module to address the imbalance between the number of risky cases and normal cases.

6. The risk case determination system based on multimodal analysis according to claim 1, characterized in that, The system is applied in scenarios such as insurance fraud detection, financial risk control, or compliance auditing.

7. A method for determining risk cases based on multimodal approaches, characterized in that, Includes the following steps: Obtain structured and unstructured text data of the target case, and preprocess them to obtain structured feature sequences and text feature sequences; The structured feature sequence is input into the structured Transformer (SAINT) model and encoded through inter-column self-attention and inter-sample attention mechanisms to obtain the structured feature vector; The text feature sequence is input into a model based on the pre-trained language model RoBERTa for semantic encoding to obtain a text semantic vector; By fusing the structured feature vector with the text semantic vector, a multimodal fusion feature vector is obtained; Based on the multimodal fusion feature vector, AUC and F1 scores are generated to help determine whether the target case is risky.

8. The risk case determination method based on multimodal analysis according to claim 7, characterized in that: The fusion step specifically involves concatenating, weighting, or interacting with the text semantic vector using the structured feature vector and the text semantic vector.

9. The risk case determination method based on multimodality according to claim 7, characterized in that: The steps for generating a risk score include: inputting the multimodal fusion feature vector into a classifier to obtain the predicted probability that the target case is a risky case; and outputting a risk determination conclusion based on the comparison result between a preset threshold and the predicted probability.

10. The risk case determination method based on multimodality according to claim 7, characterized in that: Prior to the acquisition step, there is also a model training step, which uses historical case data containing risk labels and performs end-to-end training on the SAINT-based model, the RoBERTa-based model, and the fusion prediction model with the goal of optimizing the area under the precision-recall curve.

Images