A health risk assessment method and system based on multi-modal physical examination report analysis

Abstract

The application discloses a health risk assessment method and system based on multi-modal physical examination report analysis, and the method comprises the following steps: S1, collecting physical examination report data in different data formats; S2, using different analysis methods to analyze the physical examination report data in different data formats; S3, comprehensively assessing health risks, using a multi-task learning framework to simultaneously predict health risks of a user in multiple dimensions; S4, analyzing the suitability of an insurance product according to the health risk assessment result, and simultaneously converting the health risk assessment result into premium influencing factors; and S5, outputting a final underwriting suggestion. The application can simultaneously understand and process various modal information such as text description, structured table and examination image in a physical examination report. Through a self-adaptive multi-task learning health risk assessment framework, a multi-task learning architecture is used to simultaneously predict the results of multiple risk dimensions in a unified model.

Description

Technical Field

[0001] This invention belongs to the field of health management and insurance risk control technology. Specifically, it relates to a health risk assessment method and system based on multimodal physical examination report analysis. Background Technology

[0002] With increasing health awareness, regular physical examinations have become an important part of health management. Medical examination institutions generate a large number of reports daily, containing information such as users' various physiological indicators, examination results, and doctors' recommendations. Meanwhile, the insurance industry is also paying increasing attention to the health status of insured individuals. In the area of ​​combining health management and insurance risk control, the core challenge lies in how to accurately and quickly extract key health information from physical examination reports and transform it into quantifiable risk assessment results for determining insurance benefits.

[0003] Existing technical solutions: Option 1: Traditional Medical Examination Report Parsing Method Based on Rule Matching: Traditional medical examination report parsing methods mainly use rule matching technology. A large number of keywords and regular expression templates are predefined. When the system receives the medical examination report text, it identifies key fields in the report through string matching.

[0004] Option 2: Template-based Insurance Underwriting Assistance System: Most existing insurance underwriting assistance systems operate using template-based methods. Insurance companies pre-design underwriting questionnaire templates and select applicable questionnaire content from a template library based on factors such as the insured's basic information, insurance products, and insured amount.

[0005] Option 3: Health Risk Prediction Model Based on Unimodal Data: In recent years, with the development of machine learning technology, some health risk prediction systems have begun to adopt data-driven methods. These systems typically build risk prediction models based on users' historical physical examination data and using classification algorithms.

[0006] The drawback of existing technologies is that the data formats of physical examination reports are diverse, making it difficult to match and identify them with precise rules. User health risks are multi-dimensional, making it impossible to achieve a rapid and comprehensive health risk assessment through a single model. Summary of the Invention

[0007] To address the challenges of diverse data formats in existing health risk assessment reports, which make precise matching and identification difficult, and the multi-dimensional nature of user health risks, making rapid and comprehensive health risk assessment impossible with a single model, this invention provides a health risk assessment method and system based on multimodal physical examination report parsing.

[0008] To achieve the above-mentioned technical objectives, the technical solution adopted by the present invention is as follows: A health risk assessment method based on multimodal physical examination report parsing, comprising the following steps: S1, used to collect physical examination report data in different data formats; S2. Different parsing methods are used to parse physical examination reports for different data formats; S3. Conduct a comprehensive health risk assessment, using a multi-task learning framework to predict users' health risks across multiple dimensions. S4. Conduct a policy product suitability analysis based on the health risk assessment results, and convert them into factors affecting premiums. S5. Output the final underwriting recommendation.

[0009] Furthermore, the data formats for physical examination reports include structured electronic physical examination reports, PDF format scan reports, and image format image reports; The methods for parsing physical examination reports include text parsing, table parsing, image parsing, deep fusion of multimodal information, and structured data output.

[0010] Furthermore, the detailed steps for parsing the medical examination report text include: Format recognition: A format classification model based on convolutional neural networks (CNN) is used to identify the format type of the input report; Text region detection: The Faster R-CNN object detection network was used to locate text regions in the report; Character recognition: An end-to-end CRNN (Convolutional Recurrent Neural Network) + CTC (Connection Temporal Classification) model is used for character recognition; Semantic understanding: The BERT pre-trained language model is used to perform deep semantic understanding on the identified text.

[0011] Furthermore, the detailed steps for parsing the medical examination report form include: Table detection: The table area in the detection report was detected using the YOLOv5 object detection network; Table structure recognition: The TreeDecoder model is used to recognize the row and column structure of the table; Cell content extraction: Perform OCR recognition and content understanding on the content of each cell.

[0012] Furthermore, the detailed steps for image analysis in a medical examination report include: Image preprocessing: Standardize medical images, including size normalization, grayscale equalization, and contrast enhancement; Feature extraction: The EfficientNet-B4 convolutional neural network was used to extract deep features from medical images; Image classification: A classification network is used for diagnostic analysis of medical images.

[0013] Furthermore, the deep fusion steps of the multimodal information after parsing the medical examination report include: Modality coding: unifying the parsing results of different modalities into semantic vectors; Cross-modal attention: Design a cross-modal attention mechanism to achieve information interaction between different modalities; Feature fusion: A gating fusion mechanism is used to integrate features from three modalities (text modality, table modality, and image modality).

[0014] Furthermore, structured data output: The multimodal fusion results are converted into a structured physical examination data format to facilitate subsequent health risk assessment. The output data includes: personal information fields (name, gender, age, examination date, etc.), indicator data fields (indicator name, raw value, standard value, standard unit, reference range, abnormality markers, etc.), and image conclusion fields (image type, examination conclusion, abnormality description, etc.). The structured data is output in JSON format for integration with the comprehensive health risk assessment.

[0015] Furthermore, the detailed steps for the comprehensive health risk assessment in step S3 are as follows: Detailed steps for conducting a comprehensive health risk assessment in step S3: S301. Risk Factor Extraction: Extract various factors related to health risks, including structured indicator factors, textual description factors, image feature factors, and time-series change factors. S302. Multi-task definition and label construction: Define the set of prediction targets; S303, Model Framework Design: Predictions of different health dimensions at different time points are treated as independent sub-tasks. Ridge regression combined with ℓ2,1 norm penalty and time smoothing constraint is used to capture the inherent correlation between tasks. S304. Construction of Joint Loss Function and Weight Balancing: The total loss function for multiple tasks is an adaptive weighted form based on Bayesian uncertainty, balancing the shared and specific feature weights of multi-task learning. S305, Model Training and Regularization Strategy: Use the Adam optimizer for end-to-end training and introduce a time smoothing regularization term that combines multi-task sharing to avoid overfitting of multi-task ridge regression on time series data. S306, The model outputs the risk probability / score for each dimension.

[0016] Further, the detailed steps of step S4 are as follows: Product compatibility analysis involves analyzing the product characteristics of the policy to be evaluated: insurance type, coverage period, sum insured level, and premium pricing. Risk-rate mapping transforms health risk assessment results into factors influencing insurance premiums.

[0017] Furthermore, the underwriting recommendation generation includes outputting the underwriting conclusion, coverage conditions, decision basis, and alternative solutions.

[0018] A health risk assessment system based on multimodal physical examination report parsing includes a physical examination report acquisition unit, a multimodal parsing unit, a health risk assessment unit, an underwriting decision support unit, and a result output unit; The physical examination report collection unit interfaces with the physical examination institution's system to collect physical examination report data in different data formats, including structured electronic physical examination reports, PDF scan reports, and image reports.

[0019] The multimodal parsing unit is used to parse medical examination reports using different parsing methods for different data formats; The health risk assessment unit conducts a comprehensive health risk assessment, employing a multi-task learning framework to predict users' health risks across multiple dimensions. The underwriting decision support unit analyzes the suitability of health risk assessment results for insurance products and converts them into factors affecting premiums. The results output unit outputs the final underwriting recommendation. The underwriting recommendation includes the underwriting conclusion, coverage conditions, decision basis, and alternative solutions.

[0020] Compared with the prior art, the present invention has the following advantages: It can simultaneously understand and process multiple modalities of information in medical examination reports, including text descriptions, structured tables, and examination images. A cross-modal attention mechanism was designed to enable parsing models from different modalities to reference and verify each other.

[0021] The health risk assessment framework employs an adaptive multi-task learning architecture, simultaneously predicting outcomes across multiple risk dimensions within a unified model. Through multi-task learning, the model can share high-level semantic features across different tasks.

[0022] A risk decision-making mechanism with enhanced interpretability is proposed, enabling the risk assessment results output by the model to be understood and reviewed by humans. Several interpretability techniques are employed, including feature importance analysis, decision path visualization, and case similarity retrieval. Attached Figure Description

[0023] Figure 1 This is an overall flowchart of a health risk assessment method based on multimodal physical examination report parsing in an embodiment of the present invention. Detailed Implementation

[0024] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to embodiments and accompanying drawings. The content mentioned in the embodiments is not intended to limit the present invention.

[0025] like Figure 1 As shown, this embodiment provides a health risk assessment method based on multimodal physical examination report parsing, including the following steps: S1, used to collect physical examination report data in different data formats; S2. Different parsing methods are used to parse physical examination reports for different data formats; S3. Conduct a comprehensive health risk assessment, using a multi-task learning framework to predict users' health risks across multiple dimensions. S4. Conduct a policy product suitability analysis based on the health risk assessment results, and convert them into factors affecting premiums. S5. Output the final underwriting recommendation.

[0026] The data formats for physical examination reports include structured electronic physical examination reports, PDF scan reports, and image reports. The methods for parsing physical examination reports include text parsing, table parsing, image parsing, deep fusion of multimodal information, and structured data output.

[0027] Detailed steps for parsing a medical examination report include: Format recognition: A format classification model based on convolutional neural networks (CNN) is used to identify the format type of the input report.

[0028] The model first preprocesses the input document, including grayscale conversion, adaptive binarization, and noise removal. Then, the preprocessed image is input into a CNN classification network, which outputs a formatted class probability distribution. The CNN network uses ResNet50 as its backbone and contains 50 layers of deep residual networks, effectively extracting multi-scale image features. The classification network uses cross-entropy loss as its loss function.

[0029] Where k is the category index, and y is the index for traversing all categories. k For true labels (one-hot encoded), the position corresponding to the true category is 1, and the rest are 0. k The probability of the k-th class predicted by the model satisfies The format recognition accuracy rate reaches 98.5%.

[0030] Text region detection: The Faster R-CNN object detection network was used to locate text regions in the report.

[0031] Faster R-CNN consists of two parts: a Region Proposal Network (RPN) and a Detection Network. The RPN generates candidate region proposals on the feature map using a sliding window, and the quality of the proposals is filtered by an IoU (Intersection over Union) threshold. The Detection Network classifies the proposed regions and performs bounding box regression. The loss function of the Detection Network is:

[0032] Among them, L cls For classification loss, determine the category of the object within the candidate box, L bbox For bounding box regression loss, the predicted box coordinates are corrected using L1, Smooth L1, and GloU losses, with λ being a balancing weight coefficient used to adjust the proportion of classification loss to bounding box regression loss. The mean accuracy (mAP) for text region detection reaches 96.7%.

[0033] Character recognition: An end-to-end CRNN (Convolutional Recurrent Neural Network) + CTC (Connection Temporal Classification) model is used for character recognition.

[0034] The CRNN network structure consists of three parts: a feature extraction layer, a sequence modeling layer, and a decoding layer. The feature extraction layer uses a VGG16 convolutional neural network to extract spatial feature sequences from the input image. The sequence modeling layer uses a bidirectional LSTM (Long Short-Term Memory) network to model the contextual dependencies of the character sequences. The decoding layer uses a CTC decoder to handle the problem of inconsistent lengths between the input and output sequences. The CTC loss function is defined as:

[0035] Where X is the temporal feature sequence of the input image, Y is the true label sequence (target output character / label), and p(Y|X) is the conditional probability, which is the joint probability of the model outputting the corresponding label sequence Y given input X.

[0036] The formula for character recognition accuracy is: Accuracy = (TP + TN) / (TP + TN + FP + FN) Wherein, TP represents a true positive: the character recognition result is correct, and the target character is accurately identified; TN represents a true negative: a non-target character / interference content is judged as non-target, and the judgment is correct; FP represents a false positive: a non-target character / interference content is misjudged as the target character (false detection); FN represents a false negative: the target character is missed / misrecognized (missed detection); the denominator TP + TN + FP + FN = total number of samples. The accuracy rate for printed text recognition reaches 99.2%, and for handwritten text, it reaches 95.8%.

[0037] Semantic understanding: The BERT pre-trained language model is used to perform deep semantic understanding on the identified text.

[0038] The BERT model employs a Transformer architecture, incorporating a multi-head self-attention mechanism and a multi-layer feedforward neural network. The input to BERT is represented as the sum of three embeddings: E = E token + E position + E segment E token For word embedding, E position For position embedding, E segment Embedded for paragraph type. The formula for calculating multi-head self-attention is:

[0039] Where Q, K, and V are the query, key, and value matrices, respectively, and d k is the dimension of the key vector.

[0040] Semantic understanding tasks include named entity recognition and relation extraction. The F1 score for entity recognition reaches 92.7%, calculated as: F1 = 2·(Precision·Recall) / (Precision + Recall), where Precision = TP / (TP+FP) and Recall = TP / (TP + FN).

[0041] Detailed steps for interpreting a medical examination report form include: Table detection: The table area in the detection report is detected using the YOLOv5 object detection network.

[0042] YOLOv5 uses CSPDarknet as its backbone network, comprising three main components: the backbone, the neck, and the head. The backbone extracts multi-scale features, the neck uses PANet (Path Aggregation Network) for feature pyramid fusion, and the head performs object detection. The precision and recall for table detection reach 97.3% and 96.8%, respectively.

[0043] Table structure recognition: The TreeDecoder model is used to recognize the row and column structure of the table.

[0044] This model models the table structure recognition problem as a tree structure prediction problem. The model first detects the bounding boxes of all cells in the table, and then constructs the complete table structure tree by predicting parent-child relationships. Cell coordinate prediction uses bounding box regression, and the regression loss function is Smooth L1 Loss.

[0045] Among them, box i Predict the bounding box coordinates for the model (i represents the bounding box coordinate components, such as x, y, w, h), box* i Let g(·) be the coordinates of the true bounding box, and g(·) be the Smooth L1 single-component loss function. This involves summing all coordinate dimensions of the bounding box. The F1 score for table structure recognition reaches 95.2%.

[0046] Cell Content Extraction: OCR recognition and content understanding are performed on the content of each cell. A CRNN+CTC model is used to recognize the text within the cell, and a table-based question-answering model is used to locate the numerical content. The table-based question-answering model is based on the BERT architecture; the input is the table context and the question text, and the output is the coordinates of the answer's position in the table. The model uses a PointerNetwork to decode the answer's position and outputs the probability distribution of the start and end positions. The cell content extraction accuracy reaches 97.8%.

[0047] Detailed steps for analyzing images in a medical examination report include: Image preprocessing: Standardize the medical images, including size normalization, grayscale equalization, and contrast enhancement.

[0048] Size normalization scales the image to a fixed resolution (e.g., 224×224 pixels) using bilinear interpolation. Gray-level equalization employs histogram equalization to enhance image contrast. The contrast enhancement formula is:

[0049] Among them, I original I represents the original image pixel values. enhanced To enhance the pixel value, α is the contrast coefficient (gain) and β is the bias coefficient.

[0050] Feature extraction: The EfficientNet-B4 convolutional neural network was used to extract deep features from medical images.

[0051] EfficientNet-B4 employs a compound scaling method, simultaneously adjusting the network's depth, width, and resolution. The network's feature extraction part comprises multiple modules, each containing depthwise separable convolutions, batch normalization, and the Swish activation function. The feature extraction layer outputs a global feature vector with 1792 dimensions. The average feature extraction time is 120ms per image.

[0052] Image classification: A classification network is used for diagnostic analysis of medical images.

[0053] A classification network is used for diagnostic analysis of medical images. The classification network uses a global average pooling layer to pool feature maps into feature vectors, and then outputs the classification results through a fully connected layer. The classification loss function uses weighted cross-entropy loss to handle the class imbalance problem.

[0054] Where N is the total number of samples, C is the total number of categories, and W C y is the specific weight coefficient for class C. i,c P is the One-Hot encoded label for the i-th sample, with a value of 1 if the sample belongs to class C, and 0 otherwise. i,c The model outputs the predicted probability that the i-th sample belongs to class C after Softmax activation. For common examination image types (such as electrocardiogram and chest X-ray), the classification accuracy reaches 91.5%.

[0055] The deep fusion steps of multimodal information after parsing the medical examination report include: Modal coding: unifying the parsing results of different modalities into semantic vectors.

[0056] The text modality uses a BERT encoder to output a text semantic vector (768-dimensional), the table modality uses a TableBERT encoder to output a table semantic vector (768-dimensional), and the image modality uses an EfficientNet encoder to output an image feature vector (1792-dimensional). All modality vectors are mapped to a unified dimension (512-dimensional) through a projection layer. The calculation formula for the projection layer is as follows:

[0057] Among them, h orig The original feature vector / matrix is ​​the feature output from the network's preprocessor; W is the projection weight matrix, a learnable parameter responsible for dimensionality transformation and feature mapping; b is the bias vector, a learnable parameter; h prig These are new features after projection.

[0058] Cross-modal attention: Design cross-modal attention mechanisms to achieve information interaction between different modalities.

[0059] Cross-modal attention calculates the degree of attention given by the text modality to the table modality and the image modality. The formula for calculating the attention weights is:

[0060] Among them, h texti h is the feature vector of the i-th token in the text, with dimensions [1, d]; modalj W is the j-th feature vector of another modality (image / table), with dimensions [1, d]. attThis is a learnable attention transformation matrix, with dimensions d × d, used for cross-modal feature adaptation; d is the feature hiding dimension. This is the scaling factor; For modal feature transpose, dimension [d, 1]; a i,j Let be the attention weight of the i-th position of the text to the j-th position of the modality; Softmax is the normalization along the j-th dimension, fixing the sum of all weights of each text token to 1. Through the cross-modal attention mechanism, parsing models of different modalities can refer to and verify each other, improving the accuracy and consistency of parsing.

[0061] Feature fusion: A gating fusion mechanism is used to integrate features from three modalities (text modality, table modality, and image modality).

[0062] The fusion calculation formula is as follows: Among them, F text F table F image Features are categorized into three modalities: text, tables, and images; [F text F table F image ] represents the vector concatenation operation; W g σ is the gating weight matrix; σ is the sigmoid activation function (outputting gating weights in the range of [0,1]), and ⊙ is the element-wise multiplication operation (Hadamard product). The gating mechanism automatically learns the importance weights of different modal features, achieving adaptive multimodal feature fusion. The fused feature vector has a dimension of 512 and is used for downstream health risk assessment tasks.

[0063] Structured Data Output: The multimodal fusion results are converted into a structured physical examination data format to facilitate subsequent health risk assessment. Output data includes: personal information fields (name, gender, age, examination date, etc.), indicator data fields (indicator name, raw value, standard value, standard unit, reference range, abnormality markers, etc.), and image conclusion fields (image type, examination conclusion, abnormality description, etc.). The structured data is output in JSON format for integration with comprehensive health risk assessment.

[0064] Detailed steps for conducting a comprehensive health risk assessment in step S3: S301. Risk Factor Extraction: Extract various factors related to health risks, including structured indicator factors, textual description factors, image feature factors, and time-series change factors. S302. Multi-task definition and label construction: Clearly define the prediction target set Y={Y1,Y2,…,Yk}, for example, Y1 is the probability of diabetes incidence (regression / binary classification), Y2 is the risk of hypertension complications (binary classification), and Y3 is the cardiac function classification (multi-class classification); divide health risks into dimensions such as: cardiovascular disease risk, metabolic disease risk, liver disease risk, kidney disease risk, thyroid disease risk, and malignant tumor risk; ensure that there is a pathophysiological correlation between tasks to support knowledge transfer.

[0065] S303, Model Framework Design: Shared Bottom: The input layer extracts general health features (such as age, BMI, inflammatory markers, and metabolic syndrome features) through several fully connected / Transformer encoding layers; Gating Mechanism Layer: Adopting an MMoE structure, multiple expert sub-networks are set up. Each task dynamically weights and combines expert outputs through an independent Gate network to mitigate negative transfer; Task-Specific Layer: For predictions of different health dimensions at different time points, as independent sub-tasks, ridge regression combined with ℓ2,1 norm penalties and temporal smoothing constraints is used to capture the intrinsic correlation between tasks; Output Layer: The output of the shared layer is connected to the independent fully connected layers of each task, outputting the risk score or probability of the corresponding dimension; The entire prediction model uses the multi-task ridge regression algorithm for prediction, and its calculation formula is as follows: Where X t Let Y be the feature matrix of the t-th task. t For the corresponding risk label, w t Let W be the task weight vector. 2,1 It is a ℓ2,1 norm, used for filtering shared features across tasks. The global regularization parameter (ridge parameter) takes a value ≥ 0 and controls the overall weight contraction strength. The larger the value, the closer the weight is to 0. The L2 norm of a vector, which is the square root of the sum of the squares of its elements, is used to measure the length of the residuals or weights in a single task. W is a p×n dimensional multi-task weight matrix, with each column corresponding to the regression coefficient vector of a subtask. The multi-task ridge regression prediction model, by sharing feature matrices and global regularization, can fully explore the common information between different prediction tasks, and has better generalization performance in scenarios with limited sample size.

[0066] S304. Construction of Joint Loss Function and Weight Balancing: The total loss function for multiple tasks is calculated using an adaptive weighted form based on Bayesian uncertainty. in, The final total loss value of the multi-task ridge regression is the core optimization objective of model training; n is the total number of sub-tasks in the joint training; such as sub-tasks like chronic disease risk and complication risk in multi-dimensional health risk prediction scenarios. The basic ridge regression loss for the i-th single task is... ; the sum of squared residuals corresponding to a single task. The trainable uncertainty parameter for the i-th subtask is automatically learned by the model during training and the task weights are dynamically adjusted. The Frobenius norm of the matrix is ​​calculated by taking the square root of the sum of the squares of all elements in the matrix, which is used to measure the overall size of the weight matrix. This formula can automatically learn the optimal weights for each task without the need for manual setting of weight coefficients. It balances the shared and exclusive feature weights in multi-task learning, and avoids the dominant task from masking the gradients of secondary tasks.

[0067] S305, Model Training and Regularization Strategy: End-to-end training is performed using the Adam optimizer, and a time-smoothing regularization term is introduced to prevent overfitting; this is combined with the calculation formula for the time-smoothing regularization term shared across multiple tasks: in, is the global L2 regularization hyperparameter of traditional ridge regression, which controls the overall shrinkage strength of single-task weights; K is the total number of subtasks in multi-task ridge regression, such as different chronic disease prediction subtasks under multi-dimensional health risk prediction. This is the time dimension weight matrix corresponding to the k-th subtask, with each column corresponding to the weight vector of a time step; The Frobenius norm of the matrix is ​​used to calculate the square root of the sum of squares of all elements in the matrix, which is used to measure the overall complexity of the single-task weight matrix. The hyperparameter for time smoothing regularization, with a value ≥ 0, controls the penalty intensity for changes in the weights of the time dimension. The larger the value, the smoother the weight changes over time. T is the total number of time steps in the time series, such as the total number of years in a health risk prediction scenario divided by years. This is the task weight vector corresponding to the t-th time step, with the same dimension as the number of input features; The task weight vector corresponding to the (t-1)th time step is calculated by difference with the weight of the current time step.

[0068] This regularization term effectively avoids overfitting in multi-task ridge regression on time-series data, preventing the model from fitting random noise in the time dimension and making the time trend of the prediction results more consistent with business logic. λ can be determined simultaneously through ridge trace plots combined with cross-validation. t and λ w The optimal value is determined to ensure that the model achieves the best balance between fitting accuracy and smoothness.

[0069] S306, The model outputs the risk probability / score for each dimension.

[0070] Detailed steps of step S4: Product compatibility analysis involves analyzing the product characteristics of the policy to be evaluated: insurance type, coverage period, sum insured level, and premium pricing. Risk-rate mapping transforms health risk assessment results into factors influencing insurance premiums.

[0071] The underwriting recommendation generation includes outputting the underwriting conclusion, coverage conditions, decision basis, and alternative solutions.

[0072] A health risk assessment system based on multimodal physical examination report parsing includes a physical examination report acquisition unit, a multimodal parsing unit, a health risk assessment unit, an underwriting decision support unit, and a result output unit; The physical examination report collection unit interfaces with the physical examination institution's system to collect physical examination report data in different data formats, including structured electronic physical examination reports, PDF scan reports, and image reports.

[0073] The multimodal parsing unit is used to parse medical examination reports using different parsing methods for different data formats; The health risk assessment unit conducts a comprehensive health risk assessment, employing a multi-task learning framework to predict users' health risks across multiple dimensions. The underwriting decision support unit analyzes the suitability of health risk assessment results for insurance products and converts them into factors affecting premiums. The results output unit outputs the final underwriting recommendation. The underwriting recommendation includes the underwriting conclusion, coverage conditions, decision basis, and alternative solutions.

[0074] Compared with the prior art, the present invention has the following advantages: It can simultaneously understand and process multiple modalities of information in medical examination reports, including text descriptions, structured tables, and examination images. A cross-modal attention mechanism was designed to enable parsing models from different modalities to reference and verify each other.

[0075] The health risk assessment framework employs an adaptive multi-task learning architecture, simultaneously predicting outcomes across multiple risk dimensions within a unified model. Through multi-task learning, the model can share high-level semantic features across different tasks.

[0076] A risk decision-making mechanism with enhanced interpretability is proposed, enabling the risk assessment results output by the model to be understood and reviewed by humans. Several interpretability techniques are employed, including feature importance analysis, decision path visualization, and case similarity retrieval.

[0077] The above provides a detailed description of a health risk assessment method and system based on multimodal physical examination report analysis provided in this application. The specific embodiments are described only to aid in understanding the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A health risk assessment method based on multimodal physical examination report parsing, characterized in that, Including the following steps: S1, used to collect physical examination report data in different data formats; S2. Different parsing methods are used to parse physical examination reports for different data formats; S3. Conduct a comprehensive health risk assessment, using a multi-task learning framework to predict users' health risks across multiple dimensions. S4. Conduct a policy product suitability analysis based on the health risk assessment results, and convert them into factors affecting premiums. S5. Output the final underwriting recommendation.

2. The health risk assessment method based on multimodal physical examination report parsing according to claim 1, characterized in that, The data formats for physical examination reports include structured electronic physical examination reports, PDF scan reports, and image reports. The methods for parsing physical examination reports include text parsing, table parsing, image parsing, deep fusion of multimodal information, and structured data output.

3. The health risk assessment method based on multimodal physical examination report parsing according to claim 2, characterized in that, Detailed steps for parsing a medical examination report include: Format recognition: A format classification model based on convolutional neural networks is used to identify the format type of the input report; Text region detection: The Faster R-CNN object detection network was used to locate text regions in the report; Character recognition: End-to-end convolutional recurrent neural networks and connection-based temporal classification models are used for character recognition; Semantic understanding: The BERT pre-trained language model is used to perform deep semantic understanding on the identified text.

4. The health risk assessment method based on multimodal physical examination report parsing according to claim 2, characterized in that, Detailed steps for interpreting a medical examination report form include: Table detection: The table area in the detection report was detected using the YOLOv5 object detection network; Table structure recognition: The TreeDecoder model is used to recognize the row and column structure of the table; Cell content extraction: Perform OCR recognition and content understanding on the content of each cell.

5. A health risk assessment method based on multimodal physical examination report parsing according to claim 2, characterized in that, Detailed steps for analyzing images in a medical examination report include: Image preprocessing: Standardize medical images, including size normalization, grayscale equalization, and contrast enhancement; Feature extraction: The EfficientNet-B4 convolutional neural network was used to extract deep features from medical images; Image classification: A classification network is used for diagnostic analysis of medical images.

6. A health risk assessment method based on multimodal physical examination report parsing according to claim 2, characterized in that, The deep fusion steps of multimodal information after parsing the medical examination report include: Modality coding: unifying the parsing results of different modalities into semantic vectors; Cross-modal attention: Design a cross-modal attention mechanism to achieve information interaction between different modalities; Feature fusion: A gating fusion mechanism is used to integrate features from three modalities.

7. A health risk assessment method based on multimodal physical examination report parsing according to claim 6, characterized in that, Detailed steps for conducting a comprehensive health risk assessment in step S3: S301. Risk Factor Extraction: Extract various factors related to health risks, including structured indicator factors, textual description factors, image feature factors, and time-series change factors. S302. Multi-task definition and label construction: Define the set of prediction targets; S303, Model Framework Design: Predictions of different health dimensions at different time points are treated as independent sub-tasks. Ridge regression combined with ℓ2,1 norm penalty and time smoothing constraint is used to capture the inherent correlation between tasks. S304. Construction of Joint Loss Function and Weight Balancing: The total loss function for multiple tasks is an adaptive weighted form based on Bayesian uncertainty, balancing the shared and specific feature weights of multi-task learning. S305, Model Training and Regularization Strategy: Use the Adam optimizer for end-to-end training and introduce a time smoothing regularization term that combines multi-task sharing to avoid overfitting of multi-task ridge regression on time series data. S306, The model outputs the risk probability / score for each dimension.

8. A health risk assessment method based on multimodal physical examination report parsing according to claim 7, characterized in that, Detailed steps of step S4: Product compatibility analysis involves analyzing the product characteristics of the policy to be evaluated: insurance type, coverage period, sum insured level, and premium pricing. Risk-rate mapping transforms health risk assessment results into factors influencing insurance premiums.

9. A health risk assessment system based on multimodal physical examination report parsing, characterized in that, It includes a physical examination report collection unit, a multimodal analysis unit, a health risk assessment unit, an underwriting decision support unit, and a result output unit; The physical examination report collection unit interfaces with the physical examination institution's system to collect physical examination report data in different data formats; The multimodal parsing unit is used to parse medical examination reports using different parsing methods for different data formats; The health risk assessment unit conducts a comprehensive health risk assessment, employing a multi-task learning framework to predict users' health risks across multiple dimensions. The underwriting decision support unit analyzes the suitability of health risk assessment results for insurance products and converts them into factors affecting premiums. The result output unit outputs the final underwriting recommendation.

Images