A heterogeneous insurance policy image information extraction method, system, device and medium

Abstract

The application provides a heterogeneous insurance policy image information extraction method, system, device and medium. The method comprises the following steps: acquiring and preprocessing the insurance policy image; analyzing the global layout features of the image through a preset insurance policy visual classification model to obtain a layout version code; based on the layout version code, calling a matched information extraction model from a preset model library; determining a target information region on the insurance policy image through the information extraction model combined with the corresponding visual layout prior knowledge of the layout version code to obtain a corresponding target information category; performing optical character recognition on the target information region to obtain text content; mapping the target information category to a structured field and combining it with the text content to generate key-value pair data; and outputting the key-value pair data in a structured data format. The technical solution provided by the application realizes adaptive recognition of different layout policies and improves the extraction accuracy and efficiency of heterogeneous insurance policies.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method, system, device and medium for extracting information from heterogeneous insurance policy images. Background Technology

[0002] With the deepening of digital transformation in the insurance industry, the demand for electronic and structured policy information is growing. Automated information processing of massive historical and new policies has become a core requirement for improving business efficiency and achieving intelligent underwriting and claims settlement.

[0003] In existing technologies, information extraction from policy images is typically achieved through systems that combine general optical character recognition (OCR) with predefined rule templates. For example, the system first identifies all text in the image using a general OCR engine, and then uses manually configured keywords or layout rules (such as the "policyholder" field being located in a specific area of ​​the image) to locate and extract key information from the identified text. However, existing methods based on "general OCR + fixed rules" risk low accuracy in information extraction for heterogeneous policies with varying layouts. Summary of the Invention

[0004] In view of this, this application provides a method, system, device and medium for extracting heterogeneous insurance policy image information to solve the above problems.

[0005] Firstly, a method for extracting information from heterogeneous insurance policy images is provided, the method comprising:

[0006] Acquire the policy image to be processed and perform image preprocessing on the policy image;

[0007] The global layout features of the preprocessed policy image are analyzed by a preset policy visual classification model to obtain the layout version code corresponding to the preprocessed policy image.

[0008] Based on the layout version code, the information extraction model that matches the layout version code is called from the preset model library;

[0009] By combining the information extraction model with the visual layout prior knowledge corresponding to the layout version coding, multiple target information regions are identified on the policy image, and the target information category corresponding to each target information region is obtained.

[0010] Optical character recognition is performed on multiple target information regions to obtain text content, and the text content is normalized to obtain normalized text content.

[0011] The target information categories are mapped to predefined structured fields and combined with normalized text content to generate key-value pair data;

[0012] The key-value pair data is encapsulated into a preset structured data format for output.

[0013] The above technical solution, by sequentially introducing image preprocessing, layout classification, model selection, region detection, character recognition, text normalization, and structured output into the information extraction process, enables the system to first determine the layout version code based on the overall layout characteristics of the policy image. Based on this, it automatically calls the information extraction model matching the code, and then combines prior knowledge of visual layout to locate and identify key field regions. The recognition results are then uniformly formatted and output as structured data. This eliminates the need for manual template configuration in the information extraction process, enabling stable and accurate field extraction from heterogeneous policies from different insurance companies, with different formats, or significantly different layouts. This improves the system's accuracy and adaptability in multi-layout environments.

[0014] Optionally, the global layout features of the preprocessed policy image are analyzed using a preset policy visual classification model to obtain the layout version code corresponding to the preprocessed policy image, specifically including:

[0015] The preprocessed policy image is input into the policy visual classification model, which is a deep convolutional neural network model.

[0016] The policy image is forward propagated through multiple convolutional and pooling layers of the policy visual classification model to extract the global layout features of the policy image layer by layer and generate a high-dimensional feature map.

[0017] Expand all the feature values ​​contained in the high-dimensional feature map according to a preset arrangement order to generate a one-dimensional feature vector;

[0018] The one-dimensional feature vector is classified using a fully connected layer and a classification activation function in the policy visual classification model to output the version code.

[0019] The above technical solution utilizes a deep convolutional neural network to extract global layout features of policy images and generate layout version codes. The system can accurately identify policy types from different insurance companies, with different templates or layout structures, thus providing accurate layout recognition basis for the subsequent extraction stage and avoiding extraction deviation problems caused by misjudging layouts by general OCR.

[0020] Optionally, based on the layout version code, an information extraction model matching the layout version code is retrieved from a pre-defined model library, specifically including:

[0021] The model library is constructed as a model routing table, which is a mapping data structure used to store key-value pairs. The key-value pairs are used to establish a preset mapping relationship between the layout version code and the resource path of the information extraction model.

[0022] The obtained version code is used as the key to perform a query operation in the model routing table to obtain the resource path corresponding to the key;

[0023] Based on the resource path, access the physical storage medium to read the model file from the information extraction model;

[0024] The model file is loaded into the computational memory to invoke the information extraction model.

[0025] The above technical solution constructs a mapping table between the version code and the corresponding information extraction model, enabling the system to automatically select the matching extraction model after recognizing the policy version, without the need for manual specification of rule templates. This improves the accuracy and automation of model calls and reduces extraction errors caused by template mismatch between heterogeneous policies.

[0026] Optionally, by combining the information extraction model with prior knowledge of visual layout corresponding to the layout version encoding, multiple target information regions are identified on the policy image, and the target information categories corresponding to each target information region are obtained, specifically including:

[0027] The information extraction model is used to perform forward propagation calculation on the preprocessed policy image to generate multiple candidate bounding boxes. The candidate bounding boxes contain the predicted coordinates, the target information category, and the classification confidence corresponding to the target information category.

[0028] Each candidate bounding box is processed by a preset nonmaximum suppression algorithm to obtain multiple processed candidate bounding boxes;

[0029] Select target candidate bounding boxes with a classification confidence score higher than a preset threshold from each processed candidate bounding box, and use the predicted coordinates of the target candidate bounding boxes as the localization results of multiple target information regions, and record the target information category corresponding to the target candidate bounding boxes.

[0030] The above technical solution detects and locates multiple target information regions in an image by combining prior knowledge of the layout. It can automatically adjust the region positioning strategy according to different layout structures, so that even when faced with insurance policies with large differences in field layout, it can still accurately find key field regions such as "policy number" and "policyholder", thus improving the adaptability and accuracy of field positioning.

[0031] Optionally, a forward propagation calculation is performed on the preprocessed policy image using an information extraction model to generate multiple candidate bounding boxes, specifically including:

[0032] Through multiple sequentially connected computational stages in the information extraction model, a stage-by-stage forward propagation computation is performed on the preprocessed policy image.

[0033] During the forward propagation computation, output feature maps from multiple different computation stages are acquired;

[0034] The acquired output feature maps are combined to generate multi-scale feature maps;

[0035] The detection head network of the information extraction model performs calculations on multiple preset anchor boxes in the multi-scale feature map. For each anchor box in each anchor box, the corresponding coordinate offset, classification result related to the target information category, and classification confidence are output.

[0036] The coordinate offset is applied to the coordinates of each of the multiple preset anchor boxes, and the predicted coordinates of multiple candidate bounding boxes are calculated and generated.

[0037] The above technical solution, by employing multi-scale feature fusion and anchor box detection mechanisms in the information extraction model, enables the system to simultaneously detect field text regions of different sizes and positions, ensuring a high detection recall and positioning accuracy in various policy formats (such as A4 single page, fold-out, or mixed text and image layouts), thereby reducing missed detections.

[0038] Optionally, optical character recognition is performed on multiple target information regions to obtain text content, and the text content is normalized to obtain normalized text content, specifically including:

[0039] Optical character recognition is performed on each target information region to obtain the text content corresponding to each target information region.

[0040] The text content is searched using a preset regular expression, the target text part that conforms to the regular expression definition rules is identified and extracted, and the target text part is treated as non-standard format data.

[0041] Retrieve the conversion rule set corresponding to the layout version code from the preset conversion rule base;

[0042] By using the parsing and formatting rules in the transformation rule set, non-standard format data is parsed and reconstructed to generate standard format data, which is then used as normalized text content.

[0043] The above technical solution normalizes OCR recognition results by combining regular expressions and the corresponding conversion rule set for the layout. It can convert data in various formats such as date, amount, and ID number into a unified structure, thus ensuring the consistency and parsability of field content even when there are large differences in layout, and reducing errors in subsequent structured processing.

[0044] Optionally, the target information categories can be mapped to predefined structured fields and combined with normalized text content to generate key-value pair data, specifically including:

[0045] Retrieve the target field definition table corresponding to the layout version code from the structured field library;

[0046] Based on the target information category, all normalized text content and the corresponding target information region and classification confidence are grouped to form multiple candidate content groups, where each candidate content group corresponds to a target information category.

[0047] Traverse multiple candidate content groups, and based on the classification confidence or the preset layout spatial position rules corresponding to the target information area contained in each candidate content group, determine the target normalized text content from the candidate content group, and use the target normalized text content as the representative value associated with the target information category corresponding to the candidate content group.

[0048] Using the target information category associated with each representative value as the query condition, the corresponding predefined structured field is searched in the target field definition table, and the predefined structured field is used as the key and the representative value is used as the value to generate key-value pair data.

[0049] The above technical solution maps the identification results to predefined structured fields by using a target information category and confidence level filtering mechanism. This ensures that when multiple candidate results appear in the same field, the most credible representative value can be automatically selected. This enables accurate matching and standard output of field content in multi-format and multi-model environments, thereby improving the overall accuracy of heterogeneous policy information extraction.

[0050] Secondly, a heterogeneous insurance policy image information extraction system is provided, the system comprising:

[0051] The image acquisition module is configured to acquire the policy image to be processed and perform image preprocessing on the policy image;

[0052] The layout classification module is configured to analyze the global layout features of the preprocessed policy image through a preset policy visual classification model, and obtain the layout version code corresponding to the preprocessed policy image.

[0053] The model scheduling module is configured to call information extraction models that match the format version code from a preset model library based on the format version code.

[0054] The information extraction module is configured to use an information extraction model combined with prior knowledge of visual layout corresponding to layout version encoding to determine multiple target information regions on the policy image and obtain the target information category corresponding to each target information region.

[0055] The text recognition module is configured to perform optical character recognition on multiple target information regions to obtain text content, and to normalize the text content to obtain normalized text content.

[0056] The relation mapping module is configured to map target information categories to predefined structured fields and combine them with normalized text content to generate key-value pair data.

[0057] The data output module is configured to encapsulate key-value pairs into a preset structured data format for output.

[0058] Thirdly, an electronic device is provided, including a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform any of the methods described above.

[0059] Fourthly, a computer-readable storage medium is provided, which stores instructions that, when executed, perform the method described in any of the above.

[0060] In summary, implementing one or more technical solutions provided in this application has at least the following technical effects or advantages:

[0061] By introducing a multi-stage collaborative mechanism into the information extraction process, end-to-end automation and structured output of heterogeneous policy information extraction are achieved. This eliminates reliance on manual template configuration or fixed rules, fundamentally reducing the manual costs and maintenance complexity of policy information digitization. Furthermore, this method possesses excellent scalability and self-learning capabilities, enabling it to quickly adapt to new samples by updating the model library or transforming the rule set as new policy types or formats are introduced, thus improving the system's long-term availability and maintainability in complex insurance industry scenarios. Attached Figure Description

[0062] Figure 1 This is an exemplary system architecture diagram of a heterogeneous insurance policy image information extraction method or a heterogeneous insurance policy image information extraction system applied in this application;

[0063] Figure 2 This is a flowchart illustrating a method for extracting heterogeneous insurance policy image information disclosed in this application;

[0064] Figure 3 This is a schematic diagram of a heterogeneous insurance policy image information extraction system disclosed in this application;

[0065] Figure 4 This is a schematic diagram of the structure of an electronic device disclosed in this application.

[0066] Explanation of reference numerals in the attached figures: 100, System architecture; 101, First terminal device; 102, Second terminal device; 103, Third terminal device; 104, Network; 105, Server; 301, Image acquisition module; 302, Layout classification module; 303, Model scheduling module; 304, Information extraction module; 305, Text recognition module; 306, Relationship mapping module; 307, Data output module; 401, Processor; 402, Communication bus; 403, User interface; 404, Network interface; 405, Memory. Detailed Implementation

[0067] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0068] In the description of the embodiments of this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Rather, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0069] In the description of the embodiments of this application, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0070] Figure 1 A schematic diagram of an exemplary system architecture is shown, illustrating an embodiment of a heterogeneous policy image information extraction method or a heterogeneous policy image information extraction system to which this application can be applied.

[0071] like Figure 1As shown, the system architecture 100 may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0072] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as model training applications, video recognition applications, web browser applications, social platform software, etc.

[0073] Terminal devices 101, 102, and 103 can be either hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices with displays, including but not limited to smartphones, tablets, e-book readers, MP3 (Moving Picture Experts Group Audio Layer III) players, MP4 (Moving Picture Experts Group Audio Layer IV) players, laptops, and desktop computers, etc. When terminal devices 101, 102, and 103 are software, they can be installed in the aforementioned electronic devices. They can be implemented as multiple software programs or software modules (e.g., multiple software programs or software modules used to provide distributed services) or as a single software program or software module. No specific limitations are imposed here.

[0074] When terminals 101, 102, and 103 are hardware devices, video capture devices can also be installed on them. These video capture devices can be various devices capable of capturing video, such as cameras, sensors, etc. Users can use the video capture devices on terminals 101, 102, and 103 to capture video.

[0075] Server 105 can be a server that provides various services, such as a backend server for processing data displayed on terminal devices 101, 102, and 103. The backend server can analyze and process the received data and can feed back the processing results (such as recognition results) to the terminal devices.

[0076] It should be noted that a server can be either hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software modules (e.g., multiple software programs or software modules used to provide distributed services), or as a single software program or software module. No specific limitations are made here.

[0077] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included. In particular, if the target data does not need to be obtained remotely, the above system architecture may exclude the network and include only terminal devices or servers.

[0078] Figure 2 This is a flowchart illustrating a method for extracting heterogeneous policy image information according to an embodiment of this application. This method can be implemented using a computer program, a microcontroller, or run on a heterogeneous policy image information extraction system. The computer program can be integrated into an application or run as a standalone utility application. The specific steps of the heterogeneous policy image information extraction method are described in detail below.

[0079] S201: Obtain the policy image to be processed and perform image preprocessing on the policy image.

[0080] In the embodiments of this application, image preprocessing refers to a series of standardization and optimization operations performed on the original policy image before it is fed into a subsequent deep learning model (such as a layout classification model). The purpose is to eliminate noise, distortion, or uneven lighting generated during image acquisition, improve image quality, and make it meet the input requirements of subsequent model processing. For example, a policy photo taken at an angle with shadows can be corrected into a regular, black-and-white image.

[0081] Specifically, to execute this method, a policy image to be processed must first be obtained. This policy image can come from various input channels, such as image files generated by sales personnel using a high-resolution scanner to batch scan a library of paper policy documents (e.g., TIFF or PDF format), or policy photos (e.g., JPEG or PNG format) taken and uploaded by policyholders using applications on mobile devices (e.g., smartphones). Since the quality of these raw policy images often varies, and issues such as tilted shooting angles, uneven lighting, background noise, blurry printing, or physical stains may exist, these defects can severely interfere with the accuracy of subsequent steps. Therefore, image preprocessing of the policy image is necessary. The preprocessing flow may include one or more of the following sub-steps: analyzing the tilt angle of the image using a tilt correction algorithm (such as text line detection based on Hough transform) and performing a rotation operation to restore the layout and text in the image to a horizontal or vertical state; employing image denoising techniques (such as median filtering or Gaussian filtering) to smooth the image and remove random salt-and-pepper noise or scan spots; adjusting brightness and contrast, or performing adaptive threshold binarization to convert the complex grayscale image into a clear black-and-white binary image to sharpen text edges. After image preprocessing, the system outputs a clear, regular, and standardized policy image.

[0082] S202: Analyze the global layout features of the preprocessed policy image using a preset policy visual classification model to obtain the layout version code corresponding to the preprocessed policy image.

[0083] For example, the pre-defined policy visual classification model can be one or more trained classifiers. Upon receiving a pre-processed policy image, the model analyzes it as a whole, extracting key visual patterns (i.e., global layout features) that distinguish the layout type, and compares them with multiple known layouts stored in the model library. The model outputs an identifier that best matches the image layout, i.e., a layout version code, such as "Template_C_2024," to indicate the specific format version of the policy.

[0084] In one possible implementation, the global layout features of the preprocessed policy image are analyzed using a preset policy visual classification model to obtain a layout version code corresponding to the preprocessed policy image. Specifically, this includes: inputting the preprocessed policy image into the policy visual classification model, which is a deep convolutional neural network model; performing forward propagation calculations on the policy image through multiple convolutional and pooling layers of the policy visual classification model to extract the global layout features of the policy image layer by layer and generate a high-dimensional feature map; expanding all feature values ​​contained in the high-dimensional feature map according to a preset arrangement order to generate a one-dimensional feature vector; and performing classification calculations on the one-dimensional feature vector through the fully connected layer and classification activation function of the policy visual classification model to output the layout version code.

[0085] In this embodiment, the global layout feature refers to a macroscopic feature used to describe the overall structure and content arrangement of a policy image. It does not focus on specific text content (such as "Zhang San"), but rather on the relative position, size, and distribution patterns of different semantic elements on the page (such as titles, tables, seals, and signature fields). For example, the "Insured Information Table" of Company A's policy is always in the upper left corner of the page, while Company B's policy is in the middle. This stable structural difference, which can be used to distinguish different layouts, is a manifestation of the "global layout feature." The layout version code is a specific code or label (such as "Template_001" or "Version_A_2023") used to uniquely identify these different feature combinations.

[0086] Specifically, after obtaining the preprocessed policy image, the image is first resized to the standard input size required by a Deep Convolutional Neural Network (DCNN) model (e.g., scaled or padded to 224x224 pixels) and then input into the policy visual classification model. This DCNN model can specifically choose a well-known and mature image classification architecture in the field. For example, ResNet (e.g., ResNet-50), VGGNet (e.g., VGG-16), or EfficientNet can be used. A preferred implementation is that the model is not trained from scratch, but rather employs a transfer learning strategy. Specifically, the backbone network of the model (i.e., multiple convolutional and pooling layers) is initialized using weight parameters pre-trained on a large public dataset (e.g., ImageNet). During forward propagation computation, the pre-trained convolutional layers first act as a general feature extractor to extract shallow texture and deep structural features of the image layer by layer, while the pooling layers are responsible for reducing the dimensionality of the feature maps and providing translation invariance. Through this layer-by-layer extraction process, the model mathematically captures and encodes the global layout features of the policy image, generating a high-dimensional feature map at the end of the backbone network. All feature values ​​contained in this high-dimensional feature map are flattened, i.e., unfolded according to a preset order, to generate a one-dimensional feature vector. This one-dimensional feature vector is fed into the model's classification head, which typically consists of one or more fully connected layers. A classification activation function (e.g., the Softmax function) is used to perform classification calculations on the one-dimensional feature vector (which should be the output vector of the fully connected layers). Under the transfer learning strategy, this classification head is fine-tuned on the specific policy dataset of this application. This fine-tuning process enables the model to retain its general layout understanding capabilities while specifically learning how to distinguish between a limited number of policy layouts defined in this application (e.g., layouts A, B, and C). The Softmax function ultimately outputs a probability distribution vector, where each dimension corresponds to a preset layout category. The category label corresponding to the dimension with the highest probability value is output as the layout version code for the image.

[0087] S203: Based on the layout version encoding, call the information extraction model that matches the layout version encoding from the preset model library.

[0088] For example, after determining the format version code of the policy image (e.g., "Template_A"), using this code as an index, a specific information extraction model matching the "Template_A" format is located and prepared for invocation from a preset model library. Meanwhile, extraction models in the model library used to process other formats (such as "Template_B" or "Template_C") are not activated, thus ensuring that the subsequent step S204 can use the correct dedicated model for information extraction.

[0089] In one possible implementation, based on the layout version code, an information extraction model matching the layout version code is called from a preset model library. Specifically, this includes: constructing the model library as a model routing table, which is a mapping data structure for storing key-value pairs. These key-value pairs are used to establish a preset mapping relationship between the layout version code and the resource path of the information extraction model; using the obtained layout version code as the key, performing a query operation in the model routing table to obtain the resource path corresponding to the key; accessing the physical storage medium according to the resource path to read the model file of the information extraction model; and loading the model file into computing memory to call the information extraction model.

[0090] In this embodiment, the model routing table refers to a data structure that serves as an "address book" or "index" for managing and scheduling multiple information extraction models. It represents a one-to-one correspondence between a "format version code" and the physical storage location (i.e., resource path) of the dedicated "information extraction model" corresponding to that format. For example, the model routing table may store a record with the "key" "Template_A_2023" (representing a format version code) and the "value" being the string " / opt / models / extractor_A_v2.onnx" (representing the resource path of the model file on the server).

[0091] Specifically, after obtaining the layout version code, the system needs to invoke a matching, pre-defined information extraction model based on this code. To achieve this dynamic invocation, the pre-defined model library is technically implemented as a model routing table. This model routing table is an efficient mapping data structure, such as a hash table, a dictionary, or a configuration file (such as a JSON (JavaScript Object Notation) or XML (Extensible Markup Language) file) loaded into memory at system startup. This model routing table stores multiple key-value pairs, which are used to establish a pre-defined mapping relationship between the layout version code and the resource paths of the information extraction model. The system uses the obtained layout version code as the key to perform an efficient query operation (e.g., hash lookup) in this model routing table to obtain the resource path (i.e., the value) corresponding to the key (i.e., the layout version code). After obtaining the resource path (e.g., a string pointing to " / nfs / models / policy_vB.pth"), the system accesses the physical storage medium storing the model based on this path. This medium can be a local server's solid-state drive, hard disk drive, network attached storage (NAS), or object storage (S3), etc. The purpose of this access is to read the complete model file of the information extraction model. After the model file is read, the system loads it into computing memory (such as RAM), and the appropriate deep learning framework (such as PyTorch or TensorFlow) deserializes and instantiates it to prepare the information extraction model for use, putting it in a ready state.

[0092] S204: By combining the information extraction model with the visual layout prior knowledge corresponding to the layout version encoding, multiple target information regions are identified on the policy image, and the target information category corresponding to each target information region is obtained.

[0093] For example, after invoking an information extraction model specific to the "Template_A" layout, the model utilizes its prior knowledge of the visual layout specific to the "Template_A" layout learned during training (e.g., "Insured" is always in the top left corner, and "Policy Number" is always in the top right corner) to analyze the policy image. It precisely locates the coordinates of multiple target information regions such as "Insured" and "Policy Number" on the image and obtains the target information categories corresponding to these regions (e.g., labeling the [10, 50, 100, 70] region as the "Insured_Name" class).

[0094] In one possible implementation, multiple target information regions are identified on the policy image by combining an information extraction model with prior visual layout knowledge corresponding to the layout version encoding, and the target information category corresponding to each target information region is obtained. Specifically, this includes: performing forward propagation calculation on the preprocessed policy image using the information extraction model to generate multiple candidate bounding boxes, each candidate bounding box containing predicted coordinates, target information category, and classification confidence corresponding to the target information category; processing each candidate bounding box using a preset nonmaximum suppression algorithm to obtain multiple processed candidate bounding boxes; selecting target candidate bounding boxes with classification confidence higher than a preset threshold from each processed candidate bounding box, and using the predicted coordinates of the target candidate bounding boxes as the localization results of multiple target information regions, and recording the target information category corresponding to the target candidate bounding boxes.

[0095] In this embodiment, the target information category refers to a predefined semantic label used to represent the business meaning of a specific target information area on the policy image or the field type to which it belongs. It solves the semantic classification problem of "what is this area?" For example, the target information category of the area where "Zhang San" is located on the policy image may be defined as "Insured_Name" (insured's name), while the category of the area where "10,000 yuan" is located may be defined as "Insurance_Amount" (insurance amount). This category is the core basis for relation mapping in the subsequent step S206.

[0096] Specifically, after the information extraction model matching the specific layout version code is successfully invoked in step S203, the system takes the preprocessed policy image as input and performs forward propagation calculations on the policy image through this information extraction model. It should be noted that this information extraction model (e.g., an object detection model) incorporates prior visual layout knowledge corresponding to the layout version code. This is because it is specifically trained on a dataset containing only that specific layout, and its model weights have implicitly learned the inherent spatial layout rules of that specific layout (e.g., the "Insured Person" field is always in the upper left corner, and the "Policy Number" is always in the upper right corner). During the forward propagation calculation, the model's detection head generates multiple candidate bounding boxes on the image. The number of these candidate bounding boxes may be very large (e.g., hundreds or thousands), and there are a large number of overlapping predictions for the same target. Each candidate bounding box is a data structure containing three key pieces of information: predicted coordinates (e.g., [x_min, y_min, x_max, y_max], defining the region's location), target information category (the model's semantic prediction of the region's content, such as "Insured_Name"), and classification confidence (a probability value between 0 and 1, representing the model's confidence in its classification prediction). Because the original output candidate bounding boxes contain significant redundancy and overlap, the system processes each candidate bounding box using a pre-defined Non-Maximum Suppression (NMS) algorithm.

[0097] Furthermore, the NMS algorithm relies on a preset Intersection over Union (IoU) threshold during execution. This IoU threshold is a floating-point number between 0 and 1, used to define excessive overlap. This threshold can be set according to the desired level of suppression; for example, the IoU threshold can be preset to 0.5, 0.6, or 0.7.

[0098] Furthermore, the NMS algorithm iteratively selects the box with the highest confidence and removes (suppresses) other boxes with excessive overlap with it, thus obtaining multiple processed candidate bounding boxes. To ensure the quality of the final result, the system filters out boxes from each processed candidate bounding box that have a classification confidence higher than a preset threshold (e.g., the threshold can be set to 0.85 or 0.9). These filtered boxes are the target candidate bounding boxes. The system uses the predicted coordinates of these high-quality target candidate bounding boxes as the localization results of multiple target information regions and records the target information category corresponding to each target candidate bounding box. For example, the final output of this step may be a list containing [{region: [10, 50, 100, 70], category: "Insured_Name"}, {region: [11, 200, 100, 220], category: "Policy_Number"}].

[0099] In one possible implementation, a forward propagation calculation is performed on the preprocessed policy image using an information extraction model to generate multiple candidate bounding boxes. Specifically, this includes: performing a stage-by-stage forward propagation calculation on the preprocessed policy image through multiple sequentially connected calculation stages in the information extraction model; acquiring output feature maps from multiple different calculation stages during the forward propagation calculation; combining the acquired output feature maps to generate a multi-scale feature map; performing calculations on multiple preset anchor boxes in the multi-scale feature map using the detection head network of the information extraction model; and outputting the corresponding coordinate offset, classification result related to the target information category, and classification confidence for each anchor box within each anchor box; applying the coordinate offset to the coordinates of each of the multiple preset anchor boxes to calculate and generate the predicted coordinates of multiple candidate bounding boxes.

[0100] In this embodiment, a multi-scale feature map refers to a pyramid-shaped structure in a deep neural network that integrates feature information from different levels. Specifically, shallower layers of the network output feature maps have high resolution and strong positional information, making them suitable for detecting small targets (such as the "age" field), while deeper layers output feature maps have low resolution and strong semantic information, making them suitable for detecting large targets (such as the "insured information" table). By combining these output feature maps from different computational stages (e.g., through the lateral connections and top-down paths of a Feature Pyramid Network (FPN), the generated multi-scale feature map allows subsequent detection head networks to make predictions simultaneously at multiple scales, thereby significantly improving the model's ability to detect target information regions of different sizes on the policy.

[0101] Specifically, the preferred information extraction model in this application may adopt the architecture of a Single Shot Detector (SSD) or the "YOLO" (You Only Look Once) series of models. The preprocessed policy image is first fed into the model's backbone network (e.g., ResNet or VGG). The system performs stage-by-stage forward propagation computation on the preprocessed policy image through multiple sequentially connected computation stages in this backbone network (e.g., ResNet's C3 to C5 stages). During the forward propagation computation, the system acquires output feature maps from multiple different computation stages. To achieve multi-scale detection, the system combines the acquired output feature maps. For example, in the SSD architecture, this might refer to directly constructing the detection head in parallel on the output feature maps of multiple different stages of the backbone network (e.g., VGG-16's conv4_3, fc7, etc.). In architectures like YOLOv3 / v5, this typically refers to using an FPN structure, which fuses deep semantic features with shallow spatial features through top-down and lateral connections to generate multi-scale feature maps (e.g., P3, P4, P5). The detection head network of the information extraction model (in SSD or YOLO, this usually refers to a set of small convolutional layers appended to the ends of the feature maps at each scale) computes values ​​on multiple pre-defined anchor boxes within the multi-scale feature maps. The size and aspect ratio of these anchor boxes can be pre-defined based on the policy training dataset used in this application, using clustering algorithms such as k-means, to better match the statistical characteristics of target regions such as "policyholder" and "policy number" on the policy.

[0102] Furthermore, for each anchor box within each anchor box, the detection head network outputs three pieces of information in parallel: 1) the corresponding coordinate offset (Δx, Δy, Δw, Δh); 2) the classification result related to the target information category (e.g., a vector indicating the probability of it being in the category of "background," "policyholder," "policy number," etc.); and 3) the classification confidence (usually referring to the confidence that the target is included). The system applies the calculated coordinate offsets to the coordinates of each of the multiple preset anchor boxes, and uses a decoding function to calculate and generate the predicted coordinates of multiple candidate bounding boxes. These generated candidate bounding boxes may be very large in number (tens of thousands), but they already contain complete prediction information (location, category, confidence) for all potential targets and will serve as input to the NMS algorithm.

[0103] In another possible implementation, a forward propagation calculation is performed on the preprocessed policy image using an information extraction model to generate multiple candidate bounding boxes. Specifically, in this embodiment, the anchor-free detector refers to a target detection model architecture that does not rely on a preset anchor box template. Unlike anchor-based detectors (such as SSD or YOLO) that adjust anchor boxes by regressing coordinate offsets, the anchor-free detector represents a more direct localization paradigm, treating target detection as a pixel-by-pixel dense prediction task. For example, the model directly learns to predict the distance of a pixel to the four sides (left, top, right, and bottom) of the bounding box of the target it is responsible for, or directly predicts the key points of the target (such as the top left and bottom right corners).

[0104] Specifically, to enable those skilled in the art to implement this application, the "information extraction model" can also adopt an "anchor-free detector" architecture, such as the FCOS (Fully Convolutional One-Stage Object Detection) model. In this embodiment, the preprocessed policy image is also fed into the backbone network of the model (such as ResNet) and combined with FPN to generate multi-scale feature maps (e.g., P3 to P7). The detection head network of the information extraction model (in FCOS, the detection head network is also attached to the end of each scale feature map and shares weights across different scale levels) performs calculations at each spatial location (pixel) of the multi-scale feature map. For each location on the feature map, the detection head network no longer calculates the "coordinate offset" of the anchor box, but instead outputs the results of three branches in parallel. The classification result related to the target information category is a vector indicating whether the region where the location falls is "background" or a predefined target information category such as "policyholder" or "policy number," and outputs the corresponding classification confidence score. Bounding box regression: A four-dimensional vector (l, t, r, b), where each value represents the distance from the current position (pixel) to the left, top, right, and bottom edges of the target bounding box it is responsible for. Centrality: A scalar score from 0 to 1, used to evaluate the distance from the current position to the center of the target bounding box it is responsible for. The system applies this (l, t, r, b) distance vector to the coordinates of each position on the feature map. For example, to calculate the absolute coordinates (x_min) of the left edge of the target bounding box, the system obtains the horizontal coordinates (current_x) of the current position and subtracts the model-predicted distance to the left edge (l), i.e., x_min = current_x - l. Similarly, the complete coordinate calculation includes y_min = current_y - t, x_max = current_x + r, and y_max = current_y + b, thus calculating and generating the predicted coordinates of multiple candidate bounding boxes. Simultaneously, the system multiplies the centrality score by the classification confidence score, which serves as the final ranking criterion for the candidate bounding boxes. These generated candidate bounding boxes will serve as input to the NMS algorithm.

[0105] S205: Perform optical character recognition on multiple target information regions to obtain text content, and normalize the text content to obtain normalized text content.

[0106] For example, after identifying multiple target information areas such as "insured's name" and "effective date," optical character recognition is performed on these image area slices to obtain the original text content (e.g., "Zhang San" or "November 10, 2024"). To unify the format differences that may exist in different versions (e.g., "November 10, 2024" vs. "2024 / 11 / 10"), the system will normalize the identified text content and convert it into a preset standard format (e.g., uniformly "2024-11-10"), resulting in a normalized text content with a consistent format.

[0107] In one possible implementation, optical character recognition (OCR) is performed on multiple target information regions to obtain text content, and the text content is normalized to obtain normalized text content. Specifically, this includes: performing OCR on each target information region to obtain text content corresponding to each target information region; searching the text content using a preset regular expression to identify and extract target text portions that conform to the regular expression definition rules, and treating the target text portions as non-standard format data; retrieving a conversion rule set corresponding to the version encoding from a preset conversion rule base; and parsing and reconstructing the non-standard format data using the parsing and formatting rules in the conversion rule set to generate standard format data, and using the standard format data as normalized text content.

[0108] In this embodiment, normalization refers to the process of converting raw text content, which may have diverse formats and is obtained through optical character recognition, into a predefined, unified, and easily processed standard format. The core of this process is to address the problem of multiple textual expressions of the same business meaning (e.g., "November 10, 2024", "November 10, 2025", or "12-NOV-2024"). For example, a normalization process would uniformly convert all identified date strings (i.e., non-standard format data) into the "YYYY-MM-DD" format (i.e., standard format data), such as "2024-11-10". Parsing and formatting rules are used to represent the specific conversion logic from non-standard format data to standard format data, while regular expressions are often used to accurately extract these non-standard format data from the original text.

[0109] Specifically, after determining the coordinates of multiple target information regions in step S204, the system first performs OCR on the image slices corresponding to each target information region. This step can be executed by one or more OCR engines (such as Tesseract or commercial OCR services) to obtain the original text content corresponding to each target information region (for example, the OCR result of a certain region may be "Effective Date: November 10, 2024"). Since the original text content may contain noise that is not target information (such as "Effective Date"), the system then searches the text content through a preset regular expression, identifies and extracts the target text part that truly has value and conforms to the regular expression definition rules (for example, applying the regular expression "\d{4} year \d{1,2} month \d{1,2} day" can extract "November 10, 2024"), and uses this target text part as non-standard format data. Since the date or amount formats of different policy layouts (Layout A and Layout B) may be different, their normalization rules should also be different. Therefore, the system retrieves the conversion rule set corresponding to the obtained layout version code (such as "Template_A_2023") from the preset conversion rule library. The system applies the parsing and formatting rules in this conversion rule set to parse (for example, identify the corresponding numerical values of "year", "month", and "day") and reconstruct (for example, splice again in the format of "YYYY-MM-DD") the non-standard format data ("November 10, 2024") obtained in the previous step to generate the corresponding standard format data ("2024-11-10"). The system then uses this standard format data as the final output of this step, that is, the normalized text content.

[0110] S206: Map the target information category to a predefined structured field and combine it with the normalized text content to generate key-value pair data.

[0111] Exemplarily, this step performs the final assembly of the target information category and the normalized text content. For example, the system will receive the category "Insured_Name" and the text "Zhang San". Through the mapping and combination of this step, the category is converted into a predefined structured field (such as "insured_name"), and a key-value pair data that can be directly consumed by a downstream system is generated (such as {"insured_name": "Zhang San"}).

[0112] In one possible implementation, the target information category is mapped to a predefined structured field and combined with normalized text content to generate key-value pair data. Specifically, this includes: retrieving a target field definition table corresponding to the layout version code from a structured field library; grouping all normalized text content and corresponding target information regions with classification confidence scores based on the target information category to form multiple candidate content groups, where each candidate content group corresponds to a target information category; traversing multiple candidate content groups, determining the target normalized text content from the candidate content groups based on the classification confidence score or the preset layout spatial location rules corresponding to the target information region contained in each candidate content group, and using the target normalized text content as a representative value associated with the target information category corresponding to the candidate content group; using the target information category associated with each representative value as a query condition, searching for the corresponding predefined structured field in the target field definition table, and combining the predefined structured field as the key and the representative value as the value to generate key-value pair data.

[0113] In this embodiment, a predefined structured field refers to a standard "key" with fixed business meaning used in the final output data (e.g., JSON or XML files). It represents the final information unit that the system needs to extract. For example, regardless of whether the policy is printed with "Insured Person," "Applicant," or "Insured Person's Name," they may all be uniformly mapped to a predefined structured field named "insured_name" in the final output data. The target field definition table, on the other hand, is a lookup table used to store the mapping relationship between target information categories and this predefined structured field.

[0114] Specifically, this step is the final stage of data structuring. The system retrieves the target field definition table corresponding to the version code obtained in step S202 from a structured field library. The reason for needing a version code is that models with different versions (e.g., Chinese and English versions) may output different target information categories (e.g., "policyholder" and "Applicant"), but they ultimately need to be mapped to the same predefined structured field (e.g., "insured_name"). Based on the target information category, the system groups all normalized text content and its corresponding target information regions with classification confidence scores. The purpose of this step is to aggregate all predictions for the same category (e.g., the model may identify "policyholder" in two places) to form multiple candidate content groups, each corresponding to a unique target information category. The system iterates through multiple candidate content groups to resolve potential one-to-many conflicts. Based on the classification confidence score or the preset layout spatial location rule corresponding to the target information region contained in each candidate content group, a unique target normalized text content is determined from that candidate content group. For example, in a rule based on classification confidence, the system selects the text with the highest confidence in the group; in a rule based on layout spatial location, for example, for the category "policy number," the system might select the text whose spatial location is closest to the top of the image. This determined text serves as a representative value associated with the target information category corresponding to the candidate content group. The system uses this representative value to generate the final data. Using the target information category associated with each representative value (e.g., "Insured_Name") as the query condition, the system searches for the corresponding predefined structured field (e.g., finding "insured_name") in the target field definition table obtained in the first step. Using this predefined structured field as the final key and the representative value (e.g., "Zhang San") as the value, a key-value pair data (e.g., {"insured_name": "Zhang San"}) is generated. After the traversal is complete, the system will summarize all the key-value pairs generated (e.g., [{"insured_name": "Zhang San"}, {"policy_id": "123456"}]) for encapsulation in the subsequent step S207.

[0115] S207: Encapsulate key-value pairs into a preset structured data format and output it.

[0116] In this application embodiment, the preset structured data format refers to a data encapsulation specification with standard syntax and hierarchy that is defined before system implementation. It is used to represent the final organization form of all "key-value pair data" in this application, so that downstream systems (such as databases and business applications) can perform unified parsing and consumption. For example, the preset structured data format can be a JSON object, which organizes all scattered key-value pair data under a root node; or an XML document, in which each key-value pair data is encapsulated as an XML tag.

[0117] Specifically, after generating one or more key-value pairs in step S206 (e.g., discrete entries like {"insured_name":"Zhang San"} and {"policy_id":"123456"}), this step performs the final encapsulation operation. This operation aggregates the multiple key-value pairs output from step S206, which may be in a memory list, and formats them according to a preset structured data format.

[0118] In a preferred embodiment, the preset structured data format can be specified as JSON. In this case, the encapsulation step merges all key-value pairs into a single JSON object. For example, the system creates a root object containing status codes, messages, and a "policy_data" sub-object, while key-value pairs such as "insured_name" and "policy_id" are organized as attributes of the "policy_data" sub-object. In another embodiment, the preset structured data format can also be XML. The encapsulation step then generates an XML document structure accordingly, for example, creating a root tag named "PolicyExtract" and nesting sub-tags such as "StatusCode," "Message," and "PolicyData" within it, while key-value pairs such as "insured_name" and "policy_id" exist as child elements of the "PolicyData" tag. This complete data packet conforming to the preset structured data format and directly parsable by a machine (e.g., a downstream system) will be output, for example, returned as the response body of an API call, or written to a specified database field or file.

[0119] Figure 3 This is a schematic diagram of a heterogeneous insurance policy image information extraction system according to an embodiment of this application. This system can be implemented through software, hardware, or a combination of both, becoming all or part of the overall system. For example... Figure 3 As shown, the system includes:

[0120] Image acquisition module 301 is configured to acquire the policy image to be processed and perform image preprocessing on the policy image;

[0121] The layout classification module 302 is configured to analyze the global layout features of the preprocessed policy image through a preset policy visual classification model, and obtain the layout version code corresponding to the preprocessed policy image.

[0122] The model scheduling module 303 is configured to call information extraction models that match the format version code from a preset model library based on the format version code.

[0123] The information extraction module 304 is configured to determine multiple target information regions on the policy image by combining the information extraction model with the visual layout prior knowledge corresponding to the layout version code, and to obtain the target information category corresponding to each target information region.

[0124] The text recognition module 305 is configured to perform optical character recognition on multiple target information regions to obtain text content, and to normalize the text content to obtain normalized text content.

[0125] The relation mapping module 306 is configured to map target information categories to predefined structured fields and combine them with normalized text content to generate key-value pair data.

[0126] The data output module 307 is configured to encapsulate key-value pair data into a preset structured data format for output.

[0127] Based on the above embodiments, as an optional embodiment, the layout classification module 302 is specifically used for: inputting the preprocessed policy image into the policy visual classification model, which is a deep convolutional neural network model; performing forward propagation calculations on the policy image through multiple convolutional and pooling layers of the policy visual classification model to extract the global layout features of the policy image layer by layer and generate a high-dimensional feature map; expanding all feature values ​​contained in the high-dimensional feature map according to a preset arrangement order to generate a one-dimensional feature vector; and performing classification calculations on the one-dimensional feature vector through the fully connected layer and classification activation function of the policy visual classification model to output the layout version code.

[0128] Based on the above embodiments, as an optional embodiment, the model scheduling module 303 is specifically used to: construct the model library into a model routing table, the model routing table being a mapping data structure for storing key-value pairs, the key-value pairs being used to establish a preset mapping relationship between the version code and the resource path of the information extraction model; using the obtained version code as the key, performing a query operation in the model routing table to obtain the resource path corresponding to the key; accessing the physical storage medium according to the resource path to read the model file of the information extraction model; and loading the model file into the computing memory to call the information extraction model.

[0129] Based on the above embodiments, as an optional embodiment, the information extraction module 304 is specifically used to: perform forward propagation calculation on the preprocessed policy image through the information extraction model to generate multiple candidate bounding boxes, each candidate bounding box containing predicted coordinates, target information category, and classification confidence corresponding to the target information category; process each candidate bounding box through a preset non-maximum suppression algorithm to obtain multiple processed candidate bounding boxes; select target candidate bounding boxes with classification confidence higher than a preset threshold from each processed candidate bounding box, and use the predicted coordinates of the target candidate bounding boxes as the localization results of multiple target information regions, and record the target information category corresponding to the target candidate bounding boxes.

[0130] Based on the above embodiments, as an optional embodiment, the information extraction module 304 is specifically used to: perform stage-by-stage forward propagation calculation on the preprocessed policy image through multiple sequentially connected calculation stages in the information extraction model; during the forward propagation calculation process, acquire output feature maps of multiple different calculation stages; combine the acquired output feature maps to generate a multi-scale feature map; perform calculations on multiple preset anchor boxes in the multi-scale feature map through the detection head network of the information extraction model, and output the corresponding coordinate offset, classification result related to the target information category, and classification confidence for each anchor box in each anchor box; apply the coordinate offset to the coordinates of each of the multiple preset anchor boxes, calculate and generate the predicted coordinates of multiple candidate bounding boxes.

[0131] Based on the above embodiments, as an optional embodiment, the text recognition module 305 is specifically used for: performing optical character recognition on each target information region to obtain text content corresponding to each target information region; searching the text content using a preset regular expression to identify and extract the target text portion that conforms to the regular expression definition rules, and treating the target text portion as non-standard format data; retrieving the conversion rule set corresponding to the version encoding from a preset conversion rule library; and parsing and reconstructing the non-standard format data using the parsing and formatting rules in the conversion rule set to generate standard format data, and treating the standard format data as normalized text content.

[0132] Based on the above embodiments, as an optional embodiment, the relationship mapping module 306 is specifically used for: retrieving the target field definition table corresponding to the layout version code from the structured field library; grouping all normalized text content and corresponding target information regions and classification confidence scores according to the target information category to form multiple candidate content groups, wherein each candidate content group corresponds to a target information category; traversing multiple candidate content groups, determining the target normalized text content from the candidate content groups based on the classification confidence score or the preset layout spatial position rule corresponding to the target information region contained in each candidate content group, and using the target normalized text content as the representative value associated with the target information category corresponding to the candidate content group; using the target information category associated with each representative value as the query condition, searching for the corresponding predefined structured field in the target field definition table, and using the predefined structured field as the key and the representative value as the value to combine and generate key-value pair data.

[0133] It should be noted that the system provided in the above embodiments is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0134] This embodiment also discloses an electronic device, as shown in the reference. Figure 4 The electronic device may include: at least one processor 401, at least one communication bus 402, user interface 403, network interface 404, and at least one memory 405.

[0135] The communication bus 402 is used to enable communication between these components.

[0136] The user interface 403 may include a display screen and a camera. Optionally, the user interface 403 may also include a standard wired interface and a wireless interface.

[0137] The network interface 404 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0138] The processor 401 may include one or more processing cores. The processor 401 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 405, and by calling data stored in memory 405. Optionally, the processor 401 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 401 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor 401.

[0139] The memory 405 may include random access memory (RAM) or read-only memory. Optionally, the memory 405 may include a non-transitory computer-readable storage medium. The memory 405 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 405 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 405 may also be at least one storage device located remotely from the aforementioned processor 401. Figure 4 As shown, the memory 405, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program for a heterogeneous policy image information extraction method.

[0140] exist Figure 4In the electronic device shown, the user interface 403 is mainly used to provide an input interface for the user and to obtain the user input data; while the processor 401 can be used to call an application program stored in the memory 405 for extracting heterogeneous policy image information. When executed by one or more processors 401, the electronic device executes one or more methods as described in the above embodiments.

[0141] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0142] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0143] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings or direct couplings or communication connections may be through some service interfaces; indirect couplings or communication connections between apparatuses or units may be electrical or other forms.

[0144] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0145] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0146] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory 405 and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory 405 includes various media capable of storing program code, such as a USB flash drive, external hard drive, magnetic disk, or optical disk.

[0147] The foregoing description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of other embodiments of this disclosure upon considering the disclosure in this specification. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are considered exemplary only, and the scope of this application is defined by the claims.

Claims

1. A method for extracting information from heterogeneous insurance policy images, characterized in that, The method includes: Acquire the policy image to be processed and perform image preprocessing on the policy image; The global layout features of the preprocessed policy image are analyzed by a preset policy visual classification model to obtain the layout version code corresponding to the preprocessed policy image. Based on the layout version code, an information extraction model that matches the layout version code is called from a preset model library; By combining the information extraction model with the visual layout prior knowledge corresponding to the layout version encoding, multiple target information regions are determined on the policy image, and the target information category corresponding to each target information region is obtained. Optical character recognition is performed on the multiple target information regions to obtain text content, and the text content is normalized to obtain normalized text content. The target information category is mapped to a predefined structured field and combined with the normalized text content to generate key-value pair data; The key-value pair data is encapsulated into a preset structured data format and output, wherein... Based on the layout version code, an information extraction model matching the layout version code is retrieved from a preset model library, specifically including: The model library is constructed as a model routing table, which is a mapping data structure for storing key-value pairs. The key-value pairs are used to establish a preset mapping relationship between the version code and the resource path of the information extraction model. The obtained version code is used as a key to perform a query operation in the model routing table to obtain the resource path corresponding to the key; According to the resource path, access the physical storage medium to read the model file of the information extraction model; The model file is loaded into computing memory to invoke the information extraction model; By combining the information extraction model with the visual layout prior knowledge corresponding to the layout version encoding, multiple target information regions are determined on the policy image, and the target information category corresponding to each target information region is obtained, specifically including: The information extraction model is used to perform forward propagation calculation on the preprocessed policy image to generate multiple candidate bounding boxes. Each candidate bounding box contains predicted coordinates, the target information category, and the classification confidence corresponding to the target information category. The candidate bounding boxes are processed by a preset nonmaximum suppression algorithm to obtain multiple processed candidate bounding boxes; Target candidate bounding boxes with a classification confidence level higher than a preset threshold are selected from each of the processed candidate bounding boxes, and the predicted coordinates of the target candidate bounding boxes are used as the localization results of the multiple target information regions. The target information category corresponding to the target candidate bounding boxes is also recorded.

2. The method according to claim 1, characterized in that, The step of analyzing the global layout features of the preprocessed policy image using a preset policy visual classification model to obtain the layout version code corresponding to the preprocessed policy image specifically includes: The preprocessed policy image is input into the policy visual classification model, which is a deep convolutional neural network model. The policy image is forward propagated through multiple convolutional and pooling layers of the policy visual classification model to extract the global layout features of the policy image layer by layer and generate a high-dimensional feature map. The high-dimensional feature map is expanded according to a preset arrangement order to generate a one-dimensional feature vector; The one-dimensional feature vector is classified and calculated using the fully connected layer and classification activation function of the policy visual classification model to output the version code.

3. The method according to claim 1, characterized in that, The step of performing forward propagation calculations on the preprocessed policy image using the information extraction model to generate multiple candidate bounding boxes specifically includes: Through multiple sequentially connected computational stages in the information extraction model, a stage-by-stage forward propagation computation is performed on the preprocessed policy image. During the forward propagation calculation process, output feature maps of multiple different calculation stages are obtained; The acquired output feature maps are combined to generate a multi-scale feature map; The detection head network of the information extraction model performs calculations on multiple preset anchor boxes in the multi-scale feature map. For each anchor box in each anchor box, the corresponding coordinate offset, the classification result related to the target information category, and the classification confidence are output. The coordinate offset is applied to the coordinates of each of the multiple preset anchor boxes to calculate and generate the predicted coordinates of the multiple candidate bounding boxes.

4. The method according to claim 1, characterized in that, The step of performing optical character recognition on the multiple target information regions to obtain text content, and then normalizing the text content to obtain normalized text content, specifically includes: Optical character recognition is performed on each of the target information regions to obtain the text content corresponding to each of the target information regions. The text content is searched using a preset regular expression, and the target text portion that conforms to the regular expression definition rules is identified and extracted, and the target text portion is treated as non-standard format data. Retrieve the conversion rule set corresponding to the version code from the preset conversion rule base; The non-standard format data is parsed and reconstructed using the parsing and formatting rules in the conversion rule set to generate the standard format data, and the standard format data is used as the normalized text content.

5. The method according to claim 1, characterized in that, The step of mapping the target information category to a predefined structured field and combining it with the normalized text content to generate key-value pair data specifically includes: Retrieve the target field definition table corresponding to the layout version code from the structured field library; Based on the target information category, all the normalized text content and the corresponding target information region and classification confidence are grouped to form multiple candidate content groups, wherein each candidate content group corresponds to one target information category; Traverse the multiple candidate content groups, and based on the classification confidence or the preset layout spatial position rule corresponding to the target information region contained in each candidate content group, determine the target normalized text content from the candidate content group, and use the target normalized text content as a representative value associated with the target information category corresponding to the candidate content group. Using the target information category associated with each representative value as the query condition, the corresponding predefined structured field is searched in the target field definition table, and the predefined structured field is used as the key and the representative value is used as the value to generate the key-value pair data.

6. A heterogeneous insurance policy image information extraction system, characterized in that, The system includes: The image acquisition module is configured to acquire the policy image to be processed and perform image preprocessing on the policy image; The layout classification module is configured to analyze the global layout features of the preprocessed policy image through a preset policy visual classification model, and obtain the layout version code corresponding to the preprocessed policy image. The model scheduling module is configured to call information extraction models that match the version code from a preset model library, based on the version code. Based on the layout version code, an information extraction model matching the layout version code is retrieved from a preset model library, specifically including: The model library is constructed as a model routing table, which is a mapping data structure for storing key-value pairs. The key-value pairs are used to establish a preset mapping relationship between the version code and the resource path of the information extraction model. The obtained version code is used as a key to perform a query operation in the model routing table to obtain the resource path corresponding to the key; According to the resource path, access the physical storage medium to read the model file of the information extraction model; The model file is loaded into computing memory to invoke the information extraction model; The information extraction module is configured to use the information extraction model combined with prior visual layout knowledge corresponding to the layout version code to determine multiple target information regions on the policy image, and obtain the target information category corresponding to each target information region. By combining the information extraction model with the visual layout prior knowledge corresponding to the layout version encoding, multiple target information regions are determined on the policy image, and the target information category corresponding to each target information region is obtained, specifically including: The information extraction model is used to perform forward propagation calculation on the preprocessed policy image to generate multiple candidate bounding boxes. Each candidate bounding box contains predicted coordinates, the target information category, and the classification confidence corresponding to the target information category. The candidate bounding boxes are processed by a preset nonmaximum suppression algorithm to obtain multiple processed candidate bounding boxes; Target candidate bounding boxes with a classification confidence level higher than a preset threshold are selected from each of the processed candidate bounding boxes, and the predicted coordinates of the target candidate bounding boxes are used as the localization results of the multiple target information regions. The target information category corresponding to the target candidate bounding box is also recorded. The text recognition module is configured to perform optical character recognition on the multiple target information regions to obtain text content, and to normalize the text content to obtain normalized text content. The relationship mapping module is configured to map the target information category to a predefined structured field and combine it with the normalized text content to generate key-value pair data. The data output module is configured to encapsulate the key-value pair data into a preset structured data format for output.

7. An electronic device, characterized in that, The device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions. The user interface and the network interface are both used to communicate with other devices. The processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method as described in any one of claims 1-5.

Images