Group insurance policy processing method and device based on multi-modal learning, equipment and medium

Abstract

The invention relates to the technical field of data processing, and discloses a group insurance policy processing method and device based on multi-modal learning, equipment and a medium. According to the scheme, a first semantic embedding feature of a target field is extracted from a group insurance policy text and image through a multi-modal large model; the second semantic embedding feature is extracted from the system filling text and the target field in the group insurance policy image, and the text and image information is fully fused, so that the key information in the group insurance policy can be captured more comprehensively and accurately. The feature similarity of the first semantic embedding feature and the second semantic embedding feature is calculated through a cross-modal semantic alignment algorithm, so that the semantic difference is recognized more accurately. The difference detection result is generated according to the semantic similarity, the problem field in the group insurance policy is positioned, the auditing efficiency and accuracy are improved, and potential risks are avoided. According to the scheme, claim settlement disputes caused by information errors of group insurance policies in the financial field and the medical field can be reduced, and the insurance service efficiency and quality are improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a group insurance policy processing method, apparatus, device and storage medium based on multimodal learning. Background Art

[0002] In the financial, insurance, and healthcare sectors, processing group insurance policies presents numerous challenges, particularly when dealing with non-standardized text. Group insurance policies typically contain numerous important fields, such as special provisions, insurance limits, and liability limits, all of which have diverse and complex formats. Inconsistencies between the system-entered content and the policy text can lead to risk leakage, which is difficult to fully detect and address through manual quality inspection alone.

[0003] The existing related processing technologies have the following technical problems:

[0004] 1. Insufficient ability to process non-standardized data: The format and content of group insurance policies are extremely flexible. Custom fields such as special agreements or liability clauses are often expressed in natural language. Large-scale OCR model extraction can still result in errors or omissions, while rule-matching methods struggle to adapt to language diversity.

[0005] 2. Unable to process multimodal information: Many policy fields are presented not only in text form, but also include tables, pictures or other visual content. Existing technologies make it difficult to jointly compare this visual information with text.

[0006] 3. Insufficient understanding of contextual semantics: When comparing complex clauses (such as special agreements), existing technologies can only handle word-by-word or field-by-field matching, making it difficult to capture implicit changes in the clauses at the semantic level, such as whether the "liability limit" has been expanded or whether the "insurance amount" has exceeded the standard.

[0007] 4. High dependence on manual labor and low efficiency: To avoid risks caused by discrepancies, manual quality inspection remains an important means, but quality inspection is inefficient, costly, and inevitably prone to omissions.

[0008] The above technical problems lead to low accuracy of existing technologies in group insurance policy processing, which in turn leads to low efficiency and high risk of group insurance policy processing. Therefore, how to improve the accuracy of group insurance policy processing has become a technical problem that needs to be solved urgently. Summary of the Invention

[0009] The present invention provides a group insurance policy processing method, device, equipment and medium based on multimodal learning to solve the technical problem that the existing technology has low accuracy in group insurance policy processing, which leads to low efficiency and high risk of group insurance policy processing.

[0010] First, a group insurance policy processing method based on multimodal learning is provided, including:

[0011] Extracting semantic features corresponding to at least one target field from the group insurance policy text and the group insurance policy image based on the multimodal large model to obtain a first semantic embedding feature;

[0012] Extracting semantic features corresponding to at least one target field in the system filled-in text and the group insurance policy image based on the multimodal large model to obtain a second semantic embedding feature;

[0013] Calculating feature similarity between the first semantic embedding feature and the second semantic embedding feature based on a cross-modal semantic alignment algorithm to obtain semantic similarity of at least one target field;

[0014] Based on the semantic similarity of the at least one target field, a difference detection result of the group insurance policy is generated.

[0015] In a second aspect, a group insurance policy processing device based on multimodal learning is provided, comprising:

[0016] a first semantic feature extraction module, configured to extract, based on the multimodal large model, semantic features corresponding to at least one target field in the group insurance policy text and the group insurance policy image, to obtain a first semantic embedding feature;

[0017] A second semantic feature extraction module is configured to extract semantic features corresponding to at least one target field in the system filled-in text and the group insurance policy image based on the multimodal large model to obtain a second semantic embedding feature;

[0018] a similarity calculation module, configured to calculate the feature similarity between the first semantic embedding feature and the second semantic embedding feature based on a cross-modal semantic alignment algorithm, and obtain the semantic similarity of at least one target field;

[0019] The difference detection module is used to generate a difference detection result of the group insurance policy based on the semantic similarity of the at least one target field.

[0020] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned group insurance policy processing method based on multimodal learning are implemented.

[0021] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the above-mentioned group insurance policy processing method based on multimodal learning are implemented.

[0022] In the solution implemented by the aforementioned multimodal learning-based group insurance policy processing method, apparatus, computer equipment, and storage medium, a multimodal large model extracts semantic features of the target field from the group insurance policy text and image to obtain a first semantic embedding feature. This fully integrates text and image information, and compared to traditional single-text processing methods, it can more comprehensively and accurately capture key information in the group insurance policy, laying the foundation for subsequent accurate comparison. Semantic features of the target field in the system-entered text and group insurance policy image are extracted to obtain a second semantic embedding feature, enriching the semantic information required for comparison and ensuring the integrity of the comparison process. The feature similarity between the first and second semantic embedding features is calculated using a cross-modal semantic alignment algorithm, accurately quantifying the semantic similarity of the target field. This enables more precise identification of semantic differences and effectively reduces processing errors caused by semantic misunderstanding deviations. Difference detection results are generated based on semantic similarity, directly locating problematic fields in the group insurance policy, improving quality inspection efficiency and accuracy and avoiding potential risks. In the financial sector, this solution can reduce the risk of claims caused by errors in policy processing, improve business processing efficiency, and enhance customer satisfaction. In the medical field, it helps to accurately process medical insurance policies, reduce claims disputes caused by information errors, improve the efficiency and quality of medical services, and protect the rights and interests of patients. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0024] Figure 1 1 is a schematic diagram of an application environment of a group insurance policy processing method based on multimodal learning in one embodiment of the present invention;

[0025] Figure 2 A flowchart of a first embodiment of a group insurance policy processing method based on multimodal learning provided by an embodiment of the present invention;

[0026] Figure 3 1 is a schematic structural diagram of a group insurance policy processing device based on multimodal learning in one embodiment of the present invention;

[0027] Figure 4 is a structural diagram of a computer device in one embodiment of the present invention;

[0028] Figure 5 FIG. 2 is another structural diagram of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0030] The group insurance policy processing method based on multimodal learning provided by the embodiment of the present invention can be applied in Figure 1 In an application environment, the client communicates with the server through a network. After receiving the group insurance policy image uploaded by the client, the server can extract the semantic features of the group insurance policy text and the group insurance policy image corresponding to at least one target field based on the multimodal large model to obtain a first semantic embedding feature; extract the semantic features of the system-filled text and the group insurance policy image corresponding to at least one target field based on the multimodal large model to obtain a second semantic embedding feature; calculate the feature similarity of the first semantic embedding feature and the second semantic embedding feature based on a cross-modal semantic alignment algorithm to obtain the semantic similarity of at least one target field; and generate a difference detection result for the group insurance policy based on the semantic similarity of the at least one target field.

[0031] In the present invention, in order to address the technical problem of low accuracy in group insurance policy processing in the financial and medical fields, which leads to low efficiency and high risk in group insurance policy processing, a multimodal large model can be used to extract the semantic features of the target field from the group insurance policy text and image to obtain a first semantic embedding feature, which fully integrates the text and image information. Compared with the traditional single text processing method, it can capture the key information in the group insurance policy more comprehensively and accurately, laying the foundation for subsequent accurate comparison. Semantic features are extracted from the target field in the system-filled text and the group insurance policy image to obtain a second semantic embedding feature, which enriches the semantic information required for the comparison and ensures the integrity of the comparison process. The feature similarity of the first semantic embedding feature and the second semantic embedding feature is calculated through a cross-modal semantic alignment algorithm, and the semantic similarity of the target field is accurately quantified, so that semantic differences can be identified more accurately, effectively reducing processing errors caused by semantic understanding deviations. Difference detection results are generated based on semantic similarity, and problem fields in the group insurance policy are directly located, which improves quality inspection efficiency and accuracy and avoids potential risks. In the financial sector, this solution can reduce the risk of claims caused by errors in policy processing, improve business processing efficiency, and enhance customer satisfaction. In the medical field, it helps to accurately process medical insurance policies, reduce claims disputes caused by information errors, improve the efficiency and quality of medical services, and protect the rights and interests of patients.

[0032] The client can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices. The server can be implemented as an independent server or a server cluster consisting of multiple servers.

[0033] The present invention is described in detail below through specific examples.

[0034] See also Figure 2 As shown, Figure 2 A flowchart of a first embodiment of a group insurance policy processing method based on multimodal learning provided by an embodiment of the present invention includes the following steps:

[0035] S101: extracting semantic features corresponding to at least one target field in the group insurance policy text and the group insurance policy image based on the multimodal large model to obtain a first semantic embedding feature;

[0036] Large multimodal models can use BLIP-2 (Bootstrapping Language-Image Pretraining-2). BLIP-2 introduces a cross-modal encoder and decoder, jointly training vision and language models to improve the performance of multimodal tasks and enable cross-modal information flow. Through BLIP-2's dual-stream architecture, it fuses policy text (after OCR extraction) and visual content (charts or screenshots of specific fields) and maps them into a unified high-dimensional semantic space.

[0037] In one embodiment, before extracting semantic features corresponding to at least one target field in the group policy text and the group policy image based on the multimodal large model to obtain the first semantic embedding feature, the method further includes: obtaining the group policy image; extracting image text in the group policy image based on the natural language recognition large model to obtain the group policy text; and extracting image features corresponding to the target field in the group policy image based on the image encoder to obtain a visual embedding feature.

[0038] In one embodiment, before extracting semantic features, it is necessary to first obtain group policy images. Group policy images are policy documents in image form, typically obtained from scanned paper policies or electronic policies stored directly in image format. They contain rich visual information such as text, tables, and seals. These images can be in various formats, such as JPEG and PNG.

[0039] For example, the stored group insurance policy image files can be retrieved from the insurance company's imaging system, or the paper group insurance policy can be scanned and converted into image format using devices such as high-definition cameras and scanners to provide original image materials for subsequent processing.

[0040] In one embodiment, after obtaining the group insurance policy image, in order to facilitate semantic understanding and subsequent processing of the policy content, it is necessary to use a large natural language recognition model to extract text information from the image to obtain the group insurance policy text.

[0041] The large-scale natural language recognition model can be implemented using optical character recognition (OCR) technology combined with a language model. The OCR model is responsible for identifying text characters from images, while the language model performs semantic understanding and text structuring on the recognized text. During the extraction process, the model identifies text areas in the image, extracts and arranges the text in reading order, and forms editable group insurance policy text. This collected data is preprocessed, including noise removal, format correction, and image resizing and normalization to ensure it meets the requirements of subsequent model input.

[0042] Specifically, the group insurance policy image is fed into a large natural language recognition model. First, the OCR component detects and recognizes text in the image to obtain the text content. The language model then uses this recognized text for correction and semantic analysis to generate a structured group insurance policy document containing various policy information, such as the policy number, insured information, and insurance terms. This serves as the basis for extracting semantic features.

[0043] To leverage the visual information in the image, an image encoder is used to extract image features corresponding to target fields in the group insurance policy image and generate visual embeddings. This image encoder, such as a model based on a CNN or Transformer architecture, processes images corresponding to target fields in the policy (e.g., image regions for fields like name, address, insurance amount, and special agreements) and extracts visual embedding vectors. These vectors represent the semantic features of the image and convert the image's visual content into a numerical form that the model can process.

[0044] For example, a pre-trained image encoder, such as a convolutional neural network (CNN) such as ResNet and VGG, is used as the visual embedding model V. The group insurance policy image is input into the image encoder, and the model automatically extracts various features in the image, including low-level features such as edges, textures, and shapes, as well as higher-level semantic features.

[0045] For the area where the target field is located, the region proposal network (RPN) and other technologies can be used to first locate the approximate position of the target field, and then feature extraction is performed on these areas to generate a visual embedding feature vector related to the target field. This vector can represent information such as the visual appearance and spatial position of the target field in the image.

[0046] Specifically, the group insurance policy image is input into the image encoder, and the target detection or region positioning technology is used to determine the image region I of each target field (such as name, address, insurance amount, special agreement, etc.) in the group insurance policy image. l (k∈{name, address, insurance amount, special agreement}). These image regions are input into the visual embedding model V respectively. After the convolution layer and pooling layer, the image features are extracted to generate the visual embedding features corresponding to each field. Right now:

[0047]

[0048] Among them, V is the visual embedding model, I k is the image region of field k. These visual embedding features can represent the visual appearance and spatial location of the target field in the image.

[0049] Furthermore, based on the multimodal large model, the first text semantic feature corresponding to the target field in the group insurance policy text is extracted; based on the cross-modal semantic mapping mechanism, the first text semantic feature and the visual embedding feature are combined to generate the first semantic embedding feature.

[0050] In one embodiment, the text encoder component of a multimodal large model (e.g., a Transformer-based language model) is used to extract the first textual semantic features corresponding to the target field in the group insurance policy text. Then, to fuse the text and image information, the first textual semantic features are combined with the visual embedding features using a cross-modal semantic mapping mechanism to obtain the first semantic embedding features.

[0051] The purpose of the cross-modal semantic mapping mechanism is to map features from different modalities (text and image) into a unified semantic space, allowing them to be compared and analyzed within the same space. Specifically, the cross-modal semantic mapping mechanism considers the correlation and complementarity between text and image features, and fuses the first text semantic features with the visual embedding features through a specific mapping function or network structure.

[0052] Specifically, the text content of the target field in the group insurance policy is input into the text encoder of the multimodal large model. After internal model processing, a first text semantic feature vector is generated. Simultaneously, the previously extracted visual embedding features are fused with this first text semantic feature to generate a first semantic embedding feature that combines text semantics and image visual information. This first semantic embedding feature reflects the comprehensive semantic information of the target field in both text and image.

[0053] For example, the first text semantic feature and the visual embedding feature are combined through a cross-modal semantic mapping mechanism to obtain a first semantic embedding feature Right now:

[0054]

[0055] Among them, Q is the cross-modal semantic mapping mechanism, V(θ) represents the visual embedding features extracted by the visual embedding model based on parameter θ, and L(T k ) and represents the text T k The first text semantic feature is obtained after input into the language model for processing.

[0056] Through a cross-modal semantic mapping mechanism, textual and visual features are integrated to form a comprehensive semantic representation. The first semantic embedding feature not only incorporates the semantic information of the group insurance policy text but also incorporates the visual information of the group insurance policy image, thereby more accurately reflecting the overall semantics of the target field.

[0057] For example, when processing the "name" field in a group insurance policy, the first text semantic feature may capture the description of the name in the text, while the visual embedding feature captures the visual style of the name part in the image (such as font, position, etc.). Through the cross-modal semantic mapping mechanism, this information is combined to generate a first semantic embedding feature that can reflect both text content and visual features.

[0058] S102: Extracting semantic features corresponding to at least one target field in the system filled-in text and the group insurance policy image based on the multimodal large model to obtain a second semantic embedding feature;

[0059] In one embodiment, in the group insurance policy image quality inspection task, to verify the semantic consistency between the system-entered text and the corresponding target fields in the group insurance policy text and further optimize the quality inspection process, semantic features can be extracted from the system-entered text based on a multimodal large model and combined with visual embedding features to generate a second semantic embedding feature. The system-entered text is typically obtained from the business system.

[0060] Furthermore, based on the multimodal large model, the second text semantic feature corresponding to the target field in the text filled in by the system is extracted; based on the cross-modal semantic mapping mechanism, the second text semantic feature and the visual embedding feature are combined to generate the second semantic embedding feature.

[0061] In one embodiment, a pre-trained language model (such as BERT, RoBERTa, etc.) is used to process the target field text in the system-filled text. For the text content of each target field in the system-filled text, a second text semantic feature L(S) is generated through a semantic embedding model and text embedding operation. k ).

[0062] Then, a cross-modal semantic mapping mechanism is used to fuse the visual embedding features with the second text semantic features to generate the second semantic embedding features.

[0063]

[0064] Among them, Q is the cross-modal semantic mapping mechanism, V(θ) represents the visual embedding features extracted by the visual embedding model based on parameter θ, and L(S k ) and represents the text S k The first text semantic feature is obtained after input into the language model for processing.

[0065] The second semantic embedding feature not only contains the semantic information of the text, but also integrates the visual features of the image, thereby more comprehensively representing the comprehensive semantic information of the target field in the system-filled text and image. It can be used for subsequent quality inspection, comparison and other tasks to achieve consistency verification of the semantics of the corresponding target field in the system-filled text and the group insurance policy image, which helps to discover possible errors or differences.

[0066] S103: Calculating feature similarity between the first semantic embedding feature and the second semantic embedding feature based on a cross-modal semantic alignment algorithm to obtain semantic similarity of at least one target field;

[0067] The core of the cross-modal semantic alignment algorithm is to calculate the difference between semantic embedding features from different modalities (text and image) through a suitable similarity measurement method. The embodiment of the present application selects cosine similarity as the metric because it can effectively measure the directional similarity of two vectors in the semantic space, and the semantic embedding feature is represented in the form of a vector. Feature similarity calculation methods such as Euclidean distance and Manhattan distance can also be used for calculation.

[0068] Specifically, a cross-modal semantic mapping mechanism combines textual semantic features with visual embedding features, and cosine similarity is used to calculate field consistency. For each target field, the first and second semantic embedding features generated previously are obtained. Then, a cross-modal semantic alignment algorithm is applied to calculate the cosine similarity between these two features to obtain semantic similarity.

[0069] For example, for each target field k, the first semantic embedding feature of the group policy text is calculated Second semantic embedding feature of the text filled in by the system The cosine similarity Sim(k) between:

[0070]

[0071] Among them, Sim(k) represents the semantic similarity of the target field k, cos(·) represents the cosine similarity calculation function, Represents the first semantic embedding feature of the corresponding target field in the group insurance policy text, Represents the second semantic embedding feature of the corresponding target field in the system-filled text.

[0072] The calculated feature similarity is the semantic similarity of the corresponding target field, reflecting the degree of semantic consistency between the system-entered text and the target field in the group policy image. Semantic similarity typically ranges from -1 to 1, with values ​​closer to 1 indicating greater semantic similarity and values ​​closer to -1 indicating greater semantic difference.

[0073] S104: Generate a difference detection result of the group insurance policy based on the semantic similarity of the at least one target field.

[0074] In one embodiment, a semantic similarity threshold ω can be set. Difference detection results for group insurance policies are generated based on the comparison of the semantic similarity Sim(k) of each target field with the threshold ω. The difference detection results can be presented in the form of a report, including the similarity value of each target field, the consistency determination, and detailed information about the difference fields.

[0075] Specifically, based on business needs and historical data experience, set a reasonable similarity threshold ω. For example, you can set ω = 0.8, indicating that when the similarity is lower than 0.8, the target field is considered to be different. For each target field k, compare its semantic similarity Sim(k) with the threshold ω. If Sim(k) < ω, mark the field as a different field; otherwise, mark it as a consistent field.

[0076] Compare the results of all target fields and create a difference detection report. This report may include: basic information about the group policy (such as policy number and insured); detailed information about each target field, including field name, text content in the image, text content entered by the system, semantic similarity value, and consistency judgment results; statistical information about the difference fields, such as the number and percentage of difference fields; and the overall quality inspection conclusion, such as whether the quality inspection was passed and which fields need correction.

[0077] Furthermore, a semantic similarity threshold of the target field is obtained; based on a comparison result of the semantic similarity and the semantic similarity threshold, a marking state of the target field is determined; and based on the marking state and text information of the target field, the difference detection result is generated.

[0078] In one embodiment, the semantic similarity threshold is a preset value used to determine whether the semantic similarity of the target field is within an acceptable range. The setting of the semantic similarity threshold can be determined based on business needs, historical data experience, and risk sensitivity.

[0079] For example, the semantic similarity threshold of the target field can be set based on the specific business scenario and needs, combined with the analysis results of historical data. For example, in the insurance industry, a higher threshold (such as 0.9) can be set for key fields such as insurance amount and insured, while a lower threshold (such as 0.7) can be appropriately set for some non-key fields.

[0080] Specifically, the target field's semantic similarity is compared with a preset threshold to determine its flagging status. For each target field, its semantic similarity value is obtained and compared with the preset threshold. Based on the comparison result, the target field's status is flagged. For example, if the semantic similarity is below the threshold, the field is flagged as a "different field"; otherwise, it is flagged as a "consistent field."

[0081] The marking status may be represented in binary form (0 indicates consistency, 1 indicates difference) or other forms.

[0082] In one embodiment, the target field's differential content and potential risks are analyzed by combining the flag status and text information of the target field. Natural language processing techniques can be used to extract the most important semantic features from semantic embeddings and generate natural language risk warnings. Differential content analysis can include changes such as additions, deletions, and modifications to the text, as well as the impact of these changes on business logic and risks.

[0083] For example, for fields marked as differences, the differences between the group policy text and the system-filled text are compared to analyze the specific changes in the text. For example, by comparing the vocabulary, semantics, and structure of the text, it is determined whether the difference content is an addition, deletion, or modification of the text. Based on the difference content, combined with business rules and risk models, the potential risks that may be caused by the difference are evaluated. For example, in the insurance industry, the expansion of the scope of insurance liability may lead to additional compensation risks. The difference content and risk analysis results are compiled into a report, including the basic information of the group policy, the difference content of each target field, potential risks, and the overall quality inspection conclusions.

[0084] In one embodiment, the target fields may include simple fields such as name, address, and insurance amount, or complex fields such as special agreements. For complex fields, simply comparing the features of the fields themselves is not accurate; contextual information is needed to understand the implicit semantics of the complex fields.

[0085] Furthermore, when the target field is a complex field, based on the multimodal big model, the context information of the target field in the group insurance policy text is extracted to obtain a first context semantic feature; based on the multimodal big model, the context information of the target field in the system-filled text is extracted to obtain a second context semantic feature; based on the semantic similarity of the first context semantic feature and the second context semantic feature, a difference detection result of the group insurance policy is generated.

[0086] Specifically, for special agreements, main insurance and supplementary insurance clauses in group insurance policies, semantic parsing can be achieved through the query mechanism of BLIP-2 to capture implicit semantic changes between field contexts.

[0087] For special agreed original text T 特 and the system fills in the text S 特 , extract contextual semantic embedding features through BLIP-2:

[0088]

[0089] Among them, T 特 Indicates the special agreement text of the group insurance policy (such as special agreement, main insurance and supplementary insurance clauses, etc.), S 特 Special agreed text indicating the text filled in by the system. Indicates special agreed text T 特 The first contextual semantic feature of Represents the feature convention text S 特 The second context semantic feature of L(T) is represented by V(θ), which is a visual embedding model with parameter θ and is used to extract the visual features of the image. 特 ) means to convert the text T 特 The text language feature representation obtained after input into the language model for processing, L(S 特 ) means to convert the text S 特 The text language feature representation obtained after input into the language model for processing. Q represents a cross-modal semantic mapping mechanism or function, which is used to fuse and map visual features and text language features to generate a semantic embedding vector.

[0090] Calculate the semantic difference between the original text of the special agreement and the text filled in by the system:

[0091]

[0092] in, Represents the cosine similarity between two semantic embedding vectors.

[0093] d(T 特 , S 特 ) indicates the special agreement text of the group insurance policy text T特 Special agreed text S with system filled text 特 The semantic difference between .

[0094] When the semantic difference value d(T 特 , S 特 ) exceeds the set semantic difference threshold μ, an automatic prompt is generated and the clause is marked as a risky clause.

[0095] For example, for comparison and optimization of special agreement fields, assuming that the original text policy specifically stipulates T k "Insurance liability is limited to business activities in area A"; the content filled in by the system is S k : "Insurance liability includes business activities in all areas."

[0096] First, perform data input and preprocessing on the group insurance policy text and system filled text, extract the text field T 特 and S 特 . Contextual feature information of text fields is extracted through BLIP-2, and contextual semantic embedding features of the text fields in the group insurance policy text are generated respectively:

[0097]

[0098] And the contextual semantic embedding features of the text field in the text filled in the system:

[0099]

[0100] Calculate the difference value of the feature similarity of two contextual semantic embedding features:

[0101]

[0102] Assume that d(T 特 , S 特 ) = 0.85, and the semantic difference threshold is μ = 0.8, indicating that the similarity difference value exceeds the difference value threshold, marking the text field as a high-risk field. At this point, a difference detection result can be generated, which can include the difference content and potential analysis. For example, the difference content: the original policy limits liability to region A, but the system extends it to all regions; potential risk: the expanded insurance coverage may lead to additional claims risk.

[0103] Furthermore, when the target field is marked as a high-risk field, the difference content and potential risks of the target field are analyzed based on the text information of the target field; and a risk explanation formula for the target field is generated based on the difference content and potential risks of the target field.

[0104] In one embodiment, when the target field is marked as a high-risk field, its difference content and potential risks can be deeply analyzed based on the text information of the target field.

[0105] The BLIP-2 model captures the contextual semantics of the text and, by comparing the semantic embeddings of two texts, identifies key text segments or semantic elements that lead to semantic differences. For example, if the original text reads "Insurance liability is limited to business activities in Region A," and the system fills in the text with "Insurance liability includes business activities in all regions," the difference can be identified as the geographical scope of the insurance coverage.

[0106] Combine business rules and risk models to assess potential risks associated with these differences. In the example above, based on insurance knowledge, expanding the scope of insurance liability could result in additional claims risk, as the insurer would need to assume liability for operating activities in all regions, potentially exceeding the original risk assessment and pricing range.

[0107] Based on the difference content and potential risks of the target field, a risk explanation formula can be generated to clarify the differences and potential risks in the form of natural language.

[0108] For example, the automatically generated explanation formula can be expressed as:

[0109] Description (k) = Difference content (T k ,S k ) + potential risk (k)

[0110] Combine the differences and potential risks in the above formula into a detailed explanation report. The report will include a comparison of the specific content of the difference fields, a detailed description of the differences, and warnings about potential risks.

[0111] For example, if the semantic difference threshold is μ=0.8, then if we calculate d(T 特 , S 特 )=0.85, the special agreement clause is marked as a high-risk clause, and a prompt message is generated, indicating the difference content and potential risks, such as "Expansion of coverage in the special agreement = difference content (the original policy specifically stipulates that the liability scope is limited to area A, and the system fills in the extension to all areas) + potential risk (the expansion of the insurance liability scope may lead to additional compensation risk)".

[0112] As can be seen, in the above scheme, the semantic features of the target field are extracted from the group insurance policy text and image through a multimodal large model to obtain the first semantic embedding feature, which fully integrates the text and image information. Compared with the traditional single text processing method, it can more comprehensively and accurately capture the key information in the group insurance policy, laying the foundation for subsequent accurate comparison. The semantic features of the target field in the system-filled text and the group insurance policy image are extracted to obtain the second semantic embedding feature, which enriches the semantic information required for the comparison and ensures the integrity of the comparison process. The feature similarity of the first semantic embedding feature and the second semantic embedding feature is calculated through a cross-modal semantic alignment algorithm, and the semantic similarity of the target field is accurately quantified, so that semantic differences can be more accurately identified, effectively reducing processing errors caused by semantic understanding deviations. Difference detection results are generated based on semantic similarity, and problem fields in the group insurance policy are directly located, which improves the efficiency and accuracy of the review and avoids potential risks. In the financial sector, this solution can reduce the risk of claims caused by errors in policy processing, improve business processing efficiency, and enhance customer satisfaction. In the medical field, it helps to accurately process medical insurance policies, reduce claims disputes caused by information errors, improve the efficiency and quality of medical services, and protect the rights and interests of patients.

[0113] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0114] In one embodiment, a group insurance policy processing device based on multimodal learning is provided. The group insurance policy processing device based on multimodal learning corresponds one-to-one with the group insurance policy processing method based on multimodal learning in the above embodiment. Figure 3 As shown, the group insurance policy processing device based on multimodal learning includes: a first semantic feature extraction module 201, a second semantic feature extraction module 202, a similarity calculation module 203 and a difference detection module 204. The functional modules are described in detail as follows:

[0115] A first semantic feature extraction module 201 is configured to extract semantic features corresponding to at least one target field in the group insurance policy text and the group insurance policy image based on the multimodal large model to obtain a first semantic embedding feature;

[0116] A second semantic feature extraction module 202 is configured to extract semantic features corresponding to at least one target field in the system filled text and the group insurance policy image based on the multimodal large model to obtain a second semantic embedding feature;

[0117] A similarity calculation module 203 is configured to calculate the feature similarity between the first semantic embedding feature and the second semantic embedding feature based on a cross-modal semantic alignment algorithm to obtain the semantic similarity of at least one target field;

[0118] The difference detection module 204 is configured to generate a difference detection result for the group insurance policy based on the semantic similarity of the at least one target field.

[0119] In one embodiment, the multimodal learning-based group insurance policy processing device further includes an image processing module, including:

[0120] An image acquisition unit, used for acquiring a group insurance policy image;

[0121] An image text recognition unit, configured to extract the image text from the group insurance policy image based on a natural language recognition large model to obtain the group insurance policy text;

[0122] The visual feature extraction unit is used to extract the image features corresponding to the target field in the group insurance policy image based on the image encoder to obtain visual embedding features.

[0123] In one embodiment, the first semantic feature extraction module 201 includes:

[0124] a first text feature extraction unit, configured to extract a first text semantic feature corresponding to the target field in the group insurance policy text based on the multimodal large model;

[0125] A first semantic feature generating unit is configured to combine the first text semantic feature and the visual embedding feature based on a cross-modal semantic mapping mechanism to generate the first semantic embedding feature.

[0126] In one embodiment, the second semantic feature extraction module 202 includes:

[0127] A second text feature extraction unit is used to extract a second text semantic feature corresponding to the target field in the text filled in by the system based on the multimodal large model;

[0128] The second semantic feature generating unit is used to combine the second text semantic feature and the visual embedding feature based on a cross-modal semantic mapping mechanism to generate the second semantic embedding feature.

[0129] In one embodiment, the multimodal learning-based group insurance policy processing device further includes a contextual semantic analysis module, including:

[0130] a first contextual semantic feature obtaining unit configured to extract contextual information of the target field in the group insurance policy text based on a multimodal large model to obtain a first contextual semantic feature when the target field is a complex field;

[0131] A second contextual semantic feature obtaining unit is configured to extract contextual information of the target field in the system-filled text based on the multimodal large model to obtain a second contextual semantic feature;

[0132] A difference detection result generating unit is used to generate a difference detection result of the group insurance policy based on the semantic similarity between the first context semantic feature and the second context semantic feature.

[0133] In one embodiment, the difference detection module 204 includes:

[0134] A threshold acquisition unit, configured to acquire a semantic similarity threshold of the target field;

[0135] a marking state determining unit, configured to determine a marking state of the target field based on a comparison result of the semantic similarity and the semantic similarity threshold;

[0136] A difference detection result generating unit is configured to generate the difference detection result based on the marking state and the text information of the target field.

[0137] In one embodiment, the difference detection result generating unit includes:

[0138] a high-risk field analysis subunit, configured to analyze the difference content and potential risks of the target field based on the text information of the target field when the target field is marked as a high-risk field;

[0139] The explanation formula generation subunit is used to generate a risk explanation formula for the target field based on the difference content and potential risks of the target field.

[0140] The present invention provides a group insurance policy processing device based on multimodal learning. The semantic features of the target field are extracted from the group insurance policy text and image through a multimodal large model to obtain a first semantic embedding feature, which fully integrates the text and image information. Compared with the traditional single text processing method, it can capture the key information in the group insurance policy more comprehensively and accurately, laying the foundation for subsequent accurate comparison. The semantic features of the target field in the system-filled text and the group insurance policy image are extracted to obtain a second semantic embedding feature, which enriches the semantic information required for the comparison and ensures the integrity of the comparison process. The feature similarity of the first semantic embedding feature and the second semantic embedding feature is calculated through a cross-modal semantic alignment algorithm, and the semantic similarity of the target field is accurately quantified, so that semantic differences can be identified more accurately, effectively reducing processing errors caused by semantic understanding deviations. Difference detection results are generated based on semantic similarity, and problem fields in the group insurance policy are directly located, which improves the review efficiency and accuracy and avoids potential risks.

[0141] The specific limitations of the multimodal learning-based group insurance policy processing device can be found in the limitations of the multimodal learning-based group insurance policy processing method described above and will not be repeated here. Each module in the multimodal learning-based group insurance policy processing device can be implemented in whole or in part through software, hardware, or a combination thereof. Each of the modules can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each of the modules.

[0142] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a web application bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating web application, a computer program and a database. The internal memory provides an environment for the operation of the operating web application and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, it realizes the functions or steps on the service side of a group insurance policy processing method based on multimodal learning.

[0143] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, memory, network interface, display screen and input device connected via a web application bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores operating web applications and computer programs. The internal memory provides an environment for the operation of the operating web applications and computer programs in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it realizes the functions or steps on the client side of a group insurance policy processing method based on multimodal learning.

[0144] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are performed:

[0145] Extracting semantic features corresponding to at least one target field from the group insurance policy text and the group insurance policy image based on the multimodal large model to obtain a first semantic embedding feature;

[0146] Extracting semantic features corresponding to at least one target field in the system filled-in text and the group insurance policy image based on the multimodal large model to obtain a second semantic embedding feature;

[0147] Calculating feature similarity between the first semantic embedding feature and the second semantic embedding feature based on a cross-modal semantic alignment algorithm to obtain semantic similarity of at least one target field;

[0148] Based on the semantic similarity of the at least one target field, a difference detection result of the group insurance policy is generated.

[0149] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0150] Extracting semantic features corresponding to at least one target field from the group insurance policy text and the group insurance policy image based on the multimodal large model to obtain a first semantic embedding feature;

[0151] Extracting semantic features corresponding to at least one target field in the system filled-in text and the group insurance policy image based on the multimodal large model to obtain a second semantic embedding feature;

[0152] Calculating feature similarity between the first semantic embedding feature and the second semantic embedding feature based on a cross-modal semantic alignment algorithm to obtain semantic similarity of at least one target field;

[0153] Based on the semantic similarity of the at least one target field, a difference detection result of the group insurance policy is generated.

[0154] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can be found in the relevant descriptions of the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0155] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0156] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0157] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A group insurance policy processing method based on multimodal learning, characterized in that: The method comprises: Extracting semantic features corresponding to at least one target field from the group insurance policy text and the group insurance policy image based on the multimodal large model to obtain a first semantic embedding feature; Extracting semantic features corresponding to at least one target field in the system filled-in text and the group insurance policy image based on the multimodal large model to obtain a second semantic embedding feature; Calculating feature similarity between the first semantic embedding feature and the second semantic embedding feature based on a cross-modal semantic alignment algorithm to obtain semantic similarity of at least one target field; Based on the semantic similarity of the at least one target field, a difference detection result of the group insurance policy is generated.

2. The group insurance policy processing method based on multimodal learning according to claim 1 is characterized in that: Before extracting the semantic features corresponding to at least one target field from the group insurance policy text and the group insurance policy image based on the multimodal large model and obtaining the first semantic embedding feature, the method further includes: Get group policy images; Extracting image text from the group insurance policy image based on a natural language recognition large model to obtain the group insurance policy text; Based on the image encoder, image features corresponding to the target field in the group insurance policy image are extracted to obtain visual embedding features.

3. The group insurance policy processing method based on multimodal learning according to claim 2 is characterized in that: The extracting, based on the multimodal large model, semantic features corresponding to at least one target field in the group insurance policy text and the group insurance policy image to obtain a first semantic embedding feature includes: Extracting a first text semantic feature corresponding to the target field in the group insurance policy text based on the multimodal large model; Based on a cross-modal semantic mapping mechanism, the first text semantic feature and the visual embedding feature are combined to generate the first semantic embedding feature.

4. The group insurance policy processing method based on multimodal learning according to claim 2 is characterized in that: The method of extracting semantic features corresponding to at least one target field in the system-filled text and the group insurance policy image based on the multimodal large model to obtain a second semantic embedding feature includes: Extracting a second text semantic feature corresponding to the target field from the text filled in by the system based on the multimodal large model; Based on a cross-modal semantic mapping mechanism, the second text semantic feature and the visual embedding feature are combined to generate the second semantic embedding feature.

5. The group insurance policy processing method based on multimodal learning according to claim 1 is characterized in that: After calculating the feature similarity between the first semantic embedding feature and the second semantic embedding feature based on the cross-modal semantic alignment algorithm and obtaining the semantic similarity of at least one target field, the method further includes: When the target field is a complex field, context information of the target field in the group insurance policy text is extracted based on the multimodal large model to obtain a first contextual semantic feature; Extracting context information of the target field in the system-filled text based on the multimodal large model to obtain a second contextual semantic feature; Based on the semantic similarity between the first contextual semantic feature and the second contextual semantic feature, a difference detection result of the group insurance policy is generated.

6. The group insurance policy processing method based on multimodal learning according to claim 1 is characterized in that: Generating a difference detection result of the group insurance policy based on the semantic similarity of the at least one target field includes: Obtaining a semantic similarity threshold of the target field; Determining a marking state of the target field based on a comparison result of the semantic similarity and the semantic similarity threshold; The difference detection result is generated based on the mark status and the text information of the target field.

7. The group insurance policy processing method based on multimodal learning according to claim 6 is characterized in that: The generating the difference detection result based on the mark state and the text information of the target field includes: When the target field is marked as a high-risk field, analyzing the difference content and potential risks of the target field based on the text information of the target field; Based on the difference content and potential risks of the target field, a risk explanation formula for the target field is generated.

8. A group insurance policy processing device based on multimodal learning, characterized in that: The group insurance policy processing device based on multimodal learning includes: a first semantic feature extraction module, configured to extract, based on the multimodal large model, semantic features corresponding to at least one target field in the group insurance policy text and the group insurance policy image, to obtain a first semantic embedding feature; A second semantic feature extraction module is configured to extract semantic features corresponding to at least one target field in the system filled-in text and the group insurance policy image based on the multimodal large model to obtain a second semantic embedding feature; a similarity calculation module, configured to calculate the feature similarity between the first semantic embedding feature and the second semantic embedding feature based on a cross-modal semantic alignment algorithm, and obtain the semantic similarity of at least one target field; The difference detection module is used to generate a difference detection result of the group insurance policy based on the semantic similarity of the at least one target field.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the group insurance policy processing method based on multimodal learning as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the group insurance policy processing method based on multimodal learning as described in any one of claims 1 to 7 are implemented.

Images