An image classification method, device and equipment for claim settlement materials and a medium

Abstract

This invention relates to the field of artificial intelligence and discloses a method, apparatus, device, and medium for image classification of claims materials. The method includes: acquiring image data of claims materials to be classified; performing initial image classification on the claims material image data to obtain initial image classification results; performing OCR recognition and initial text classification on the claims material image data to obtain initial text classification results; and, based on a preset classification integration strategy, directly outputting the final image classification result based on the consistency between the initial image classification result and the initial text classification result, or updating the classification results by calling different preset classifiers before outputting the final image classification result. This invention can be applied to business scenarios such as fintech and healthcare. By constructing a dual-dimensional initial classification system of images and text and combining it with a multi-classifier integration strategy, and dynamically scheduling classifiers based on classification consistency to complete result verification and updates, it effectively improves the accuracy and generalization ability of claims material image classification.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology and can be applied to business areas such as fintech and healthcare. In particular, it relates to an image classification method, apparatus, device, and medium for claims materials. Background Technology

[0002] With the full arrival of the era of smart claims processing, the digital transformation of claims business in the insurance industry continues to deepen, resulting in an explosive growth in claims material image data. These claims material image data sources are generally characterized by diverse types and dynamically changing scenarios. For example, in the medical claims field, images of claims materials issued by different hospitals, third-party testing institutions, and other entities typically cover various types such as medical invoices, prescriptions, test reports, diagnostic reports, and expense lists. The format and specifications of these documents vary significantly between different institutions, and the document styles dynamically iterate with adjustments to medical policies and system upgrades. Similarly, in the property insurance claims field, image materials such as accident scene photos, repair lists, invoices, and police accident reports include both standardized printed text and complex images with handwritten notes and snapshots, showing significant differences in material types and content formats across different scenarios.

[0003] Currently, digital claims processing typically relies on deep learning-based image classification technologies (such as ResNet, VGGNet, and Vit) combined with OCR (Optical Character Recognition) to extract information and automatically classify claims materials. While deep learning has made significant progress in image recognition, the diverse types of claims materials and the dynamic nature of the scenes make existing classification schemes ill-suited to meet varied recognition needs. This leads to classification errors, reduces accuracy, and ultimately impacts reimbursement efficiency. Summary of the Invention

[0004] The main objective of this invention is to provide a method, apparatus, device, and medium for image classification of claims materials, aiming to solve the technical problem of inaccurate classification of claims reimbursement materials in the prior art, so as to improve the accuracy and generalization ability of image classification of claims reimbursement materials.

[0005] The technical solution of the present invention is as follows: The first aspect of this invention provides an image classification method for claims materials, comprising: Obtain image data of the claims materials to be classified; Extract image features from the image data of the claims materials and perform initial image classification to obtain the initial image classification result; The image data of the claim materials is subjected to OCR recognition to obtain OCR-recognized text; Initial text classification is performed based on keyword group matching of the OCR-recognized text to obtain initial text classification results; Based on a preset classification integration strategy, the final image classification result is directly output according to the consistency between the initial image classification result and the initial text classification result, or the final image classification result is output after calling different preset classifiers to update the initial image classification result and / or the initial text classification result.

[0006] A second aspect of the present invention provides an image classification device for claims materials, comprising: The acquisition module is used to acquire image data of the claims materials to be classified; The initial image classification module is used to extract image features from the image data of the claims materials and perform initial image classification to obtain the initial image classification result. The OCR recognition module is used to perform OCR recognition on the image data of the claim materials to obtain OCR-recognized text. The initial text classification module is used to perform initial text classification based on keyword group matching of the OCR-recognized text, and obtain the initial text classification result; An integrated scheduling output module is used to directly output the final image classification result based on a preset classification integration strategy and the consistency between the initial image classification result and the initial text classification result, or to call different preset classifiers to update the initial image classification result and / or the initial text classification result before outputting the final image classification result.

[0007] A third aspect of the present invention provides a computer device including at least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the image classification method for the claims materials described above.

[0008] A fourth aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform the above-described image classification method for claims materials.

[0009] Beneficial Effects: This invention discloses a method, apparatus, device, and medium for image classification of claims materials. Compared with existing technologies, this invention acquires image data of claims materials to be classified; extracts image features from the claims material image data and performs initial image classification to obtain initial image classification results; performs OCR recognition on the claims material image data to obtain OCR-recognized text; performs initial text classification based on keyword group matching of the OCR-recognized text to obtain initial text classification results; and, based on a preset classification integration strategy, directly outputs the final image classification result based on the consistency between the initial image classification result and the initial text classification result, or calls different preset classifiers to update the initial image classification result and / or the initial text classification result before outputting the final image classification result. This invention can be applied to business scenarios such as fintech and healthcare. By constructing a dual-dimensional initial classification system of images and text and combining it with an integrated decision-making strategy of multiple classifiers, it achieves intelligent classification of claims material images based on multi-dimensional features. It can flexibly and dynamically schedule classifiers to complete result verification and updates based on the consistency of classification results, effectively improving the accuracy and generalization ability of claims material image classification. Attached Figure Description

[0010] To more clearly illustrate the solutions in this invention, the accompanying drawings used in the description of the embodiments of this invention will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0011] Figure 1 A schematic diagram of an application environment for the image classification method for claims materials provided in an embodiment of the present invention; Figure 2 A flowchart of an image classification method for claims materials provided in an embodiment of the present invention; Figure 3 A schematic diagram of the functional modules of the image classification device for claims materials provided in an embodiment of the present invention; Figure 4 A schematic diagram of the hardware structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0012] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention is further described in detail below. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. The embodiments of the invention are described below in conjunction with the accompanying drawings.

[0013] The image classification method for claims materials provided in this embodiment of the invention can be applied to, for example... Figure 1In the application environment, it includes a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0014] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as financial clients, healthcare clients, web browser applications, search applications, instant messaging tools, email clients, and / or social media platform software, etc. (for example only).

[0015] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0016] Server 105 can be a server providing various services, such as a backend server supporting the content browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend server can analyze and process received user requests and other data, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices. Server 105 can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the shortcomings of traditional physical hosts and VPS services ("Virtual Private Server", or simply "VPS"), such as high management difficulty and weak business scalability. Server 105 can also be a server for a distributed system or a server combined with blockchain.

[0017] It should be noted that the image classification method for claim materials provided in this embodiment can generally be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103. Correspondingly, the image classification device for claim materials provided in this embodiment can also be located in the first terminal device 101, the second terminal device 102, or the third terminal device 103. Alternatively, the image classification method for claim materials provided in this embodiment can generally be executed by the server 105. Correspondingly, the image classification device for claim materials provided in this embodiment can generally be located in the server 105.

[0018] It should be understood that the number of terminal devices, networks, and servers listed above is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be used.

[0019] like Figure 2 As shown, the image classification method for claims materials provided in this embodiment of the invention specifically includes the following steps: S201. Obtain image data of the claims materials to be classified.

[0020] In this embodiment, the image data of the claims materials to be classified comes from online submissions and offline scanning and entry of claims. The image data formats include, but are not limited to, common image formats such as JPG, PNG, and PDF. The claims material image data can cover digitized images of various claims-related paper materials such as medical invoices, detailed expense lists, prescriptions, diagnostic reports, and examination reports. After acquiring the image data, basic preprocessing can be performed, including image denoising, size normalization, and contrast adjustment, to remove irrelevant noise interference in the image and standardize the image input specifications, providing a high-quality image data foundation for subsequent image feature extraction and OCR recognition.

[0021] For example, in the financial insurance claims business, images of car insurance claims materials uploaded by customers through the insurance company's APP, WeChat official account, or offline counter are received. These images include accident liability determination letters, repair invoices, damage assessment reports, and copies of vehicle registration certificates. The image formats include JPG, PNG, and PDF. All images undergo preprocessing such as noise reduction, size normalization to 800×1200 pixels, and contrast adjustment, and are uniformly stored in the claims image database to prepare for subsequent classification.

[0022] In the medical insurance reimbursement and claims business in the healthcare field, images of medical reimbursement materials uploaded by hospitals and patients can be received, including outpatient invoices, inpatient expense lists, prescriptions, disease diagnosis reports, examination reports, etc. Blurred handwritten prescription images are first deblurred, tilted invoice images are corrected, and the image format is standardized to JPG, etc., to complete the standardized acquisition of image data.

[0023] This embodiment enables the unified collection of digital images of claims materials by being compatible with multi-channel and multi-format image data input. At the same time, it can preprocess to remove image noise and standardize specifications, thereby improving the accuracy of subsequent identification from the source, reducing the interference of invalid data on classification results, and improving the overall digital efficiency of claims business.

[0024] S202. Extract the image features of the claim material image data and perform initial image classification to obtain the initial image classification result.

[0025] In this embodiment, preliminary classification and recognition processing is performed on the image data of claim materials in both image and text dimensions. When performing initial classification in the image dimension, the image data of claim materials is input into a pre-trained image classification model. The model matches the extracted image features with a preset image feature library of claim material categories and outputs the initial image classification result corresponding to the image data. The initial image classification result includes the specific claim material category and the corresponding classification confidence level, such as "medical invoice, confidence level 0.95" and "diagnostic report, confidence level 0.88".

[0026] Specifically, the image classification model can adopt the Swin-Transformer model. The Swin-Transformer model is centered on the window attention module and includes core modules such as image block embedding layer, multi-level Swin Transformer blocks (including window partitioning, shift window attention, feedforward network), global pooling layer, and classification head. Each level adopts a hierarchical stacked structure, and adjacent Swin Transformer blocks are connected through residual connections and layer normalization to achieve image feature extraction from local to global. In the initial image classification stage of claims material image classification, the Swin-Transformer model first divides the preprocessed claims material image data (such as medical invoices and car insurance claims) into fixed-size image blocks through an image block embedding layer, converting each image block into a low-dimensional feature vector. Subsequently, the feature vectors are input into a multi-level SwinTransformer block, which first divides the feature map into non-overlapping windows through window partitioning, performs self-attention calculation within the window, and then achieves cross-window feature interaction through shift window attention. Combined with a feedforward network, feature transformation is completed, and residual connections and layer normalization ensure the stability of feature transfer. After multiple rounds of layer processing, visual features such as layout, format, and texture of the image are captured (such as the table layout of the expense list and the image area features of the examination report). Finally, the extracted deep visual features are aggregated through a global pooling layer and input into the classification head, which matches them with the preset claims material category image feature library, and outputs the initial image classification result with confidence.

[0027] For example, in the financial field, the Swin-Transformer model can be used to extract image features from images of auto insurance claim materials. By capturing the layout features of repair invoices (such as the fixed layout of invoice codes and numbers), the location and format of the official seal on accident liability determination reports, and the table layout features of damage assessment reports, the extracted features are matched with a pre-set image feature library for auto insurance claim materials to output initial image classification results, such as "Repair Invoice, Confidence 0.96" and "Accident Liability Determination Report, Confidence 0.92". Similarly, in the healthcare field, image features can be extracted from images of medical reimbursement materials. This includes capturing the location of the hospital logo on outpatient invoices, the multi-row table features of expense lists, the image image area features of CT scan reports, and the handwritten area layout features of prescriptions. These features are then matched with a medical reimbursement material image feature library to output initial image classification results, such as "Inpatient Expense List, Confidence 0.94" and "ECG Report, Confidence 0.89".

[0028] This embodiment extracts image features from claims material image data and performs initial image classification to obtain initial image classification results. Utilizing a high-performance feature extraction model, it can accurately capture the visual features of claims material images. Compared with traditional models, it can effectively identify features such as the layout, format, and style of the materials, improving the accuracy of initial classification. It can also quickly output initial image classification results with confidence. Furthermore, image feature-based classification does not rely on text information and can effectively classify claims material images with unclear or blurred text, effectively compensating for the limitations of pure text classification.

[0029] S203. Perform OCR recognition on the image data of the claim materials to obtain OCR-recognized text.

[0030] In this embodiment, Optical Character Recognition (OCR) technology is used to perform text detection and recognition on the image data of claim materials. This allows for efficient text recognition of various character types, including printed and handwritten text. The specific recognition process involves, in sequence, image orientation determination, image rotation correction to a positive orientation, text region detection, and character recognition and stitching, ultimately yielding the OCR-recognized text corresponding to the image data. For example, the image orientation is first determined; if it is rotated 90, 180, or 270 degrees, it is corrected to a positive orientation. Then, a text detection algorithm is used to locate all text regions in the image, excluding non-text blank or patterned areas. Finally, character recognition is performed on the located text regions, converting the text in the image into editable digital text, thus obtaining the OCR-recognized text.

[0031] For example, OCR recognition is performed on invoices and damage assessment reports for auto insurance claims in the financial sector. This allows for the identification of text information such as "invoice name, invoice date, amount, and seller name" in repair invoices, and text such as "damage assessment items, damage assessment amount, and damage assessor name" in damage assessment reports, thereby obtaining structured OCR-recognized text.

[0032] For example, OCR recognition can be performed on images such as prescriptions and diagnostic reports for medical reimbursement. It can accurately recognize the "disease name, diagnosing physician, and consultation time" in printed diagnostic reports and the "drug name and dosage" in regular handwritten prescriptions to obtain OCR-recognized text.

[0033] This embodiment uses OCR recognition to convert visual text in claims material images into machine-recognizable structured text, providing data support for subsequent text-dimensional classification and becoming a reliable foundation for achieving dual-dimensional classification of images and text, thereby improving the efficiency and accuracy of subsequent keyword matching and text classification.

[0034] S204. Based on the OCR-recognized text, perform initial text classification by keyword group matching to obtain the initial text classification result.

[0035] In this embodiment, when performing initial classification at the text dimension, the initial text classification is based on keyword group matching of the OCR-recognized text obtained by OCR recognition. Specifically, the initial text classification of the OCR-recognized text is based on the matching principle of phrases first, then words. That is, the OCR-recognized text is first matched with preset keyword groups. If a specific keyword group is matched, the initial text classification result is directly determined; if no keyword group is matched, individual keywords are further statistically matched, and the category association weight of the keywords is combined to calculate the score of each category. The category with the highest score is taken as the initial text classification result. The initial text classification result also includes the specific claim material category and the corresponding classification confidence level.

[0036] For example, a keyword group mapping library containing various claim materials can be pre-built. For instance, the keyword group corresponding to "repair invoice" includes ["invoice code", "invoice number", "total price and tax"], and the keyword group corresponding to "prescription" is ["drug name", "dosage and administration", "prescribing physician"]. If the OCR-recognized text of the repair invoice is matched with the mapping library, and "invoice code: 131002200111, invoice number: 00568974, total price and tax: 5800 yuan" is recognized, then the keyword group of "repair invoice" is completely matched, and the initial text classification result is directly determined as "repair invoice, confidence level 0.95". If the OCR-recognized text of the prescription is matched with the mapping library, and "drug name: amoxicillin capsules, dosage and administration: 3 times a day, 1 capsule each time, prescribing physician: Zhang XX" is recognized, then the keyword group of "prescription" is completely matched, and the initial text classification result is output as "prescription, confidence level 0.93".

[0037] This embodiment performs initial text classification by prioritizing keyword group matching, which improves the anti-interference ability of text classification compared to single-word matching. It can effectively filter the influence of background text and irrelevant text on the classification results. Furthermore, the hierarchical matching of keyword groups and individual keywords allows for the adaptation of OCR recognition results with different text completeness during text classification. Even if the text is partially missing, effective classification can still be achieved through weighted calculation of single-word keywords, thereby achieving fast and accurate text-level classification.

[0038] S205. Based on a preset classification integration strategy, the final image classification result is directly output according to the consistency between the initial image classification result and the initial text classification result, or different preset classifiers are called to update the initial image classification result and / or the initial text classification result before outputting the final image classification result.

[0039] In this embodiment, the preset classification ensemble strategy is based on ensemble learning, which achieves better generalization performance than a single learner by constructing and combining multiple learners. In addition to initial image classification using an image classification model and initial text classification based on keyword matching, several other classifiers are pre-constructed, such as a text similarity retrieval classifier, an image similarity retrieval classifier, an LLM language large model classifier, and a VLM multimodal recognition classifier. After obtaining the initial image and text classification results, the consistency of the two classification results is evaluated based on the preset classification ensemble strategy to confirm the final output image classification result. That is, if the initial classification result is reliable, the final image classification result can be directly output for fast and efficient image classification. If the initial classification result is unreliable, for example, due to inconsistencies in the two classification results caused by diverse claim materials, other different preset classifiers are further invoked to update the initial image and / or initial text classification results before outputting the final image classification result. By employing a classification ensemble strategy to achieve cross-validation of image and text classification results and classifier scheduling, the accuracy and generalization ability of the final classification results are significantly improved compared to single-dimensional classification, effectively adapting to diverse claims material classification needs.

[0040] For example, in the classification of auto insurance claim materials in the financial field, if the initial image classification result is "repair invoice" and the initial text classification result is also "repair invoice", then the two are consistent, and "repair invoice" is directly used as the final classification result based on the preset classification integration strategy. As another example, in the classification of medical reimbursement materials in the healthcare field, if the initial image classification result of a CT examination report image is "examination report" and the initial text classification result is "diagnosis report", then the two are inconsistent, and other different classifiers, such as the VLM multimodal recognition classifier, are called for secondary classification based on the preset classification integration strategy to obtain a new text classification result of "examination report". After it is consistent with the image classification result, "examination report" is output as the final classification result.

[0041] In the above embodiments, this invention discloses an image classification method for claims materials. The method involves acquiring image data of the claims materials to be classified; extracting image features from the image data and performing initial image classification to obtain an initial image classification result; performing OCR recognition on the image data to obtain OCR-recognized text; performing initial text classification based on keyword matching of the OCR-recognized text to obtain an initial text classification result; and, based on a preset classification integration strategy, directly outputting the final image classification result based on the consistency between the initial image classification result and the initial text classification result, or updating the initial image classification result and / or the initial text classification result using different preset classifiers before outputting the final image classification result. This invention can be applied to business scenarios such as fintech and healthcare. By constructing a dual-dimensional initial classification system of images and text and combining it with an integrated decision-making strategy of multiple classifiers, it achieves intelligent image classification of claims materials based on multi-dimensional features. It can flexibly and dynamically schedule classifiers to complete result verification and updates based on the consistency of classification results, effectively improving the accuracy and generalization ability of claims material image classification.

[0042] In one embodiment, step S204 includes: S241. Perform a full match between the OCR-recognized text and a pre-built keyword group mapping library, wherein the keyword group mapping library stores at least one set of keywords corresponding to different categories of claim materials; S242. If the OCR-recognized text completely matches any set of keywords in a claim material category, then the completely matched claim material category is taken as the initial text classification result of the OCR-recognized text. S243. Otherwise, count the frequency of occurrence of individual keywords that are partially matched in the OCR-recognized text, and calculate the corresponding total category score by combining the category association weights pre-labeled for each keyword. S244. The category of claim materials corresponding to the highest total category score is taken as the initial text classification result of the OCR-recognized text.

[0043] In this embodiment, a keyword group mapping library is pre-constructed. This mapping library has a key-value pair structure, where the key represents the category of claim materials, including medical invoices, expense lists, prescriptions, diagnostic reports, examination reports, etc., and the value represents at least one set of exclusive keyword groups under the corresponding category. Each keyword group consists of two or more core words with category recognition. For example, the keyword group corresponding to a diagnostic report is ["disease diagnosis", "diagnosis opinion", "examination conclusion"]; the keyword group corresponding to a prescription is ["drug name", "dosage and administration", "prescribing physician"]; and the keyword group corresponding to a medical invoice is ["medical charges", "invoice", "receipt", "payment voucher"]. After the OCR-recognized text is segmented, it is fully matched with all keyword groups in the keyword group mapping library. That is, after the OCR-recognized text is segmented, it is compared one by one with the keyword groups under all categories in the keyword group mapping library to determine whether there is a completely matching keyword group.

[0044] If the word segmentation result of the OCR-recognized text contains any complete keyword group of a certain claim material category in the keyword group mapping library, it is determined to be a complete match, and the claim material category is directly used as the initial text classification result. For example, if the OCR-recognized text contains "Drug Name: Amoxicillin Capsules, Dosage and Administration: 3 times a day, 1 capsule each time, Prescribing Physician: Zhang XX", and the word segmentation completely matches the keyword group ["Drug Name", "Dosage and Administration", "Prescribing Physician"] corresponding to the prescription, then the initial text classification result is directly determined to be a prescription.

[0045] If the OCR-recognized text does not completely match any set of keywords in the keyword group mapping library, it enters the single keyword matching stage. That is, the keyword group mapping library simultaneously stores a single core keyword corresponding to each category of claim materials, and each keyword is pre-labeled with a category association weight. The weight value is set according to the closeness of the association between the keyword and the corresponding category, and the value ranges from 0 to 1. The higher the association, the greater the weight value. For example, the keyword "invoice number" has a weight of 1 for "medical invoice", the keyword "inspection" has a weight of 0.9 for "inspection report" and a weight of 0.1 for other categories; the keyword "amount" has a weight of 0.8 for "expense list" and a weight of 0.2 for other categories, and so on.

[0046] In the single keyword matching stage, for example, if the OCR recognizes the text as "blood routine white blood cell test results outpatient clinic", and there is no complete keyword group match, then the text is segmented, all individual keywords are extracted, the frequency of each keyword in the text is counted, and the total category score is calculated by combining the pre-labeled category association weights. Specifically, the total category score = Σ(keyword frequency × keyword category association weight). The category of claim materials with the highest total score is taken as the initial text classification result. If multiple categories have the same total score, the category with the highest total weight is selected as the result.

[0047] For example, after OCR recognition of the text "blood routine white blood cell test results outpatient clinic" and word segmentation, the keywords [blood routine, white blood cells, test, result, outpatient clinic] are obtained. The weights of each keyword corresponding to the test report are 0.8, 0.8, 0.9, 0.7, and 0.2, respectively, and the frequency of each keyword is 1. The total score of the test report is calculated as 0.8 + 0.8 + 0.9 + 0.7 + 0.2 = 3.4. The total scores of other categories are all lower than 3.4. Therefore, the initial text classification result is test report.

[0048] This embodiment achieves accurate text classification of claims materials through full matching of exclusive keyword groups, avoiding classification errors caused by overlapping individual keywords. For scenarios where no keyword group is matched, a weighted calculation method based on frequency of occurrence and category association weights is used to effectively solve the classification problem of incomplete text or lack of complete keyword groups in OCR recognition, improving the uniqueness and accuracy of text classification and reducing the ambiguity of classification results.

[0049] In one embodiment, step S205 includes: S251. Confirm whether the initial image classification result and the initial text classification result are consistent; S252. If the classifications are consistent, the initial image classification result and the initial text classification result are directly output as the final image classification result. S253. If the classifications are inconsistent, the best classifier for each update round shall be determined in turn according to the performance index of different preset classifiers. S254. Call the best classifier in each update round to perform corresponding classification updates on the image features or OCR-recognized text to obtain new image classification results or new text classification results. S255. After each update, compare the latest image classification result with the latest text classification result to see if they are consistent. Output the final image classification result only when the two classification results are consistent.

[0050] In this embodiment, when classifier scheduling and final image classification result output are performed based on a preset classification integration strategy, it is first confirmed whether the initial image classification result and the initial text classification result are consistent. The consistency judgment adopts the principle of complete matching of category labels. That is, when the claim material category labels of the initial image classification result and the initial text classification result are exactly the same, it is determined that the classification is consistent. At this time, there is no need to call other preset classifiers. The consistent claim material category is directly used as the final image classification result output, thereby ensuring the rapid classification of regular clear images and improving the overall processing efficiency.

[0051] If the initial classification results are inconsistent, the process proceeds to the subsequent classifier invocation and classification update stage. Each update round calls only one optimal classifier. The preset classifiers fall into two main categories: image classifiers and text classifiers. Image classifiers update classification based on initially extracted image features, outputting new image classification results; while text classifiers update classification based on OCR-recognized text, outputting new text classification results. Specific preset classifiers include text similarity retrieval classifiers, image similarity retrieval classifiers, LLM language model classifiers, and VLM multimodal recognition classifiers. Each classifier has different processing targets, optimization directions, and performance characteristics. For example, the LLM language model classifier is designed for OCR-recognized text with background text interference, while the image similarity retrieval classifier is designed for image data with insufficient image feature extraction. The optimal classifier for each update round is determined based on the performance metrics of different preset classifiers and current optimization needs. In other words, each update round only calls the single classifier with the highest overall score for image or text classification updates.

[0052] After the optimal classifier is confirmed in each update round, it is invoked to update the classification of image features or OCR-recognized text, outputting a new image classification result or a new text classification result. The new classification result includes the updated claim material category and classification confidence level, and overwrites the original corresponding initial classification result. After each round of classifier invocation and classification update, the latest image classification result and the latest text classification result are immediately compared. If they match, the classifier invocation stops, and the matching category is output as the final image classification result. If they still do not match, the process of confirming and invoking the optimal classifier in the next round begins, and this cycle continues until a matching classification result is obtained. Optionally, if all preset classifiers have been invoked and no matching result is obtained after the update, the category with the highest confidence level among all classification results (including the initial classification result and the results of each round of updates) is selected as the final image classification result.

[0053] In practice, the text similarity retrieval classifier includes a text embedding module, a text vector retrieval module, and a category matching module. When the text similarity retrieval classifier is called, the text embedding module performs semantic encoding on the OCR-recognized text of the claim material image, converting the text into a low-dimensional vector that can represent semantic features. Then, this low-dimensional vector is input into the text vector retrieval module. In a pre-built claim material category text vector database, the 10 sample vectors that are closest to the input text vector are found through vector retrieval algorithms (such as cosine similarity matching). Finally, the category matching module performs a weighted average calculation on the claim material categories corresponding to the 10 retrieved sample vectors (the higher the similarity, the greater the weight), and outputs a new text classification result.

[0054] The image similarity retrieval classifier includes an image feature input module, an image vector retrieval module, and a classification decision module. When the image similarity retrieval classifier is called, the image features extracted by the Swin-Transformer model are reused. These image features are directly obtained through the image feature input module, avoiding repeated feature extraction calculations. Subsequently, the image features are input to the image vector retrieval module, which retrieves several image samples with the highest similarity to the input vector from a preset image vector library for claims material categories, and calculates the similarity value between the input vector and each sample vector. Finally, the classification decision module performs a weighted average calculation based on the similarity values ​​to determine the material category corresponding to the image and outputs a new image classification result.

[0055] The LLM language large model classifier is based on the large language model architecture, which includes a text input layer, a semantic encoding layer, a prompt word parsing layer, a deep semantic understanding layer, and a classification output layer. The prompt word parsing layer is embedded between the semantic encoding layer and the deep semantic understanding layer to achieve accurate parsing of prompt words and filtering and guidance of core information. When the LLM language large model classifier is invoked, the text input layer receives OCR-recognized text and preset prompt words (such as "filter background irrelevant text in the text and determine the corresponding claim material category based only on core business information"). Subsequently, the semantic encoding layer performs joint semantic encoding on the input text and prompt words, converting them into semantic representations that the model can recognize. The prompt word parsing layer filters the encoded semantic features according to the prompt word instructions, filtering out invalid semantic information corresponding to background text and retaining the semantic features of the core business text. The deep semantic understanding layer performs in-depth analysis on the filtered core semantic features, capturing subtle semantic differences in the text (such as "total price and tax" corresponding to an invoice, and "responsibility division" corresponding to an accident liability determination letter). Finally, the classification output layer outputs accurate text classification results based on the semantic understanding results, completing the update of the initial text classification results.

[0056] This embodiment uses a classification integration strategy to compare the consistency of classification results across different dimensions and schedule and update the classifier. This ensures that regular clear images can be classified quickly, while also optimizing and updating the multi-classifier for claims materials that are difficult to classify accurately using conventional methods. After each update, a two-dimensional result comparison is performed to ensure timely capture of consistent results, thus balancing classification accuracy and processing efficiency.

[0057] For example, the initial image classification result is a cost list, and the initial text classification result is a prescription. At this time, the two classifications are inconsistent, triggering the classifier call and classification update process. In the first round, the best classifier is confirmed to be the LLM language large model classifier (text class). After the call, the new text classification result is output as a cost list. At this time, the latest image classification result is a cost list, and the latest text classification result is a cost list. The two are consistent, so the call stops and the cost list is output as the final image classification result.

[0058] In one embodiment, step S253 includes: S2531. Obtain the performance metrics of each preset classifier obtained from the pre-test, including accuracy, processing speed, concurrency and operating cost; S2532. Obtain the corresponding weight allocation strategy according to the current scenario requirements and confirm the weight of each performance indicator; S2533. Perform weighted calculations based on the performance indicators and corresponding weights of each preset classifier to obtain the scheduling performance score of each preset classifier. S2534. Based on the scheduling performance scores of each preset classifier, sort the scheduling priorities and determine the best classifier for each update round.

[0059] In this embodiment, each preset classifier is tested and verified with a large number of claims material image samples before being put into use, and the performance indicators of each classifier are obtained. Among them, the accuracy rate is the proportion of samples correctly classified by the classifier; the processing speed is the average processing time of a single image by the classifier; the concurrency is the amount of image data processed by the classifier at the same time; and the operating cost is the computing power, storage and other resource consumption of a single processing by the classifier. All the performance indicators of the preset classifiers are stored in the database for easy access at any time.

[0060] When determining the best classifier for each update round, not only are the performance metrics of each preset classifier obtained, but also multiple weight allocation strategies are pre-set to adapt to different business scenario requirements. For example, the high-precision priority strategy is suitable for scenarios where high accuracy is required for claims material classification, and accuracy has the highest weight in this strategy; the high-efficiency priority strategy is suitable for scenarios where claims business is in full swing and a large number of images need to be processed quickly, and processing speed and concurrency have the highest weights in this strategy; the low-cost priority strategy is suitable for routine claims processing scenarios, and operating cost has the highest weight in this strategy, and so on.

[0061] After obtaining the performance metrics and weight allocation strategy, the performance metrics are normalized, converting metrics of different dimensions into normalized scores between 0 and 1. Accuracy, processing speed, and concurrency are positive metrics; the higher the value, the higher the normalized score. Operating cost is a negative metric; the lower the value, the higher the normalized score. A weighted sum is calculated based on the normalized scores and corresponding weights of each performance metric to obtain the scheduling performance score for each preset classifier. All preset classifiers are sorted from highest to lowest scheduling performance score to obtain a classifier scheduling priority list. The best classifier in each update round is the classifier ranked first in the current priority list. If a classifier has already been used in a previous update round, it is removed from the priority list, and the remaining top-ranked classifier is selected as the best classifier in the next round, and so on.

[0062] This embodiment uses multi-dimensional classifier performance evaluation and scenario-based weight allocation strategies to ensure that the selected classifiers better reflect the overall performance of the classifiers and are adapted to actual business needs. This ensures that the best classifier in each round of updates is the optimal choice that has not yet been called, thus achieving orderly scheduling of all preset classifiers.

[0063] In one embodiment, after step S203, the method further includes: S301. Perform word segmentation on the OCR-recognized text and calculate the percentage of each individual character after word segmentation; S302. Match the OCR-recognized text with a preset list of rare characters, and calculate the proportion of rare characters based on the matching results; S303. Calculate the OCR quality score of the OCR-recognized text based on the proportion of individual characters and the proportion of rare characters.

[0064] In this embodiment, the effectiveness of OCR-recognized text is quantitatively evaluated through OCR quality scoring. This determines whether recognition errors exist due to image blur, handwriting, or incorrect orientation, providing a basis for deciding whether to initiate secondary recognition. Specifically, the OCR-recognized text is first segmented using a word segmentation tool, breaking it down into words, phrases, and individual characters. The number of individual characters and the total number of segmented words are counted. The percentage of individual characters after segmentation is calculated using the formula: Percentage of individual characters = Number of individual characters / Total number of segmented words. A higher percentage of individual characters indicates fewer effective words in the OCR-recognized text, potentially resulting in poorer recognition quality.

[0065] Furthermore, a pre-built list of uncommon characters is constructed, which includes uncommon characters, variant characters, and garbled characters that are not frequently used in claims business scenarios. All characters in the OCR-recognized text are matched character by character against the pre-built list of uncommon characters. The number of matched uncommon characters and the total number of characters in the OCR-recognized text are counted. The proportion of uncommon characters is calculated according to the formula: proportion of uncommon characters = number of uncommon characters / total number of characters. The higher the proportion of uncommon characters, the higher the error rate of OCR recognition and the worse the recognition quality.

[0066] The OCR quality score of the OCR-recognized text is calculated based on the proportion of individual characters and the proportion of rare characters. For example, corresponding weight coefficients α and β can be pre-configured for the proportion of individual characters and the proportion of rare characters. Then, the OCR quality score = 1 - (α × proportion of individual characters + β × proportion of rare characters), where α + β = 1. These values ​​can be adjusted according to the actual OCR recognition scenario. A higher score indicates better quality of the OCR-recognized text, while a lower score indicates poorer recognition quality, with issues such as incorrect image orientation, blurred images, and failure to recognize handwritten characters.

[0067] This embodiment objectively judges the quality of OCR recognition results by using two core indicators: the proportion of single characters and the proportion of rare characters. This solves the problem of not being able to quantitatively evaluate the quality of OCR recognition. It can identify low-quality OCR text caused by image orientation errors, image blurring, and handwriting recognition failures from the source, preventing low-quality text from entering the subsequent classification stage, reducing classification bias caused by text errors, and improving the overall classification accuracy.

[0068] For example, the OCR-recognized text is "Shijiazhuang Third Hospital Laboratory Report". After word segmentation, it becomes "Shijiazhuang / Third / Hospital / Laboratory / Report / Single". The number of individual characters is 1 (single), and the total number of characters is 12. The percentage of individual characters is 1 / 12 × 100% ≈ 8.33%. There are no rare characters matched, and the percentage of rare characters is 0. The OCR quality score is 1 - (8.33% × 0.6 + 0 × 0.4) ≈ 0.95 points, which is judged as high-quality recognized text.

[0069] If the OCR recognized text is "摄验医荥来耳邙槠拥铐摄住址", after word segmentation, it is all single characters. The number of single characters is 12, the total number of characters is 14, and the proportion of single characters is approximately 85.71%; the number of rare characters matched is 8, and the proportion of rare characters is approximately 57.14%; the OCR quality score = 1 - (85.71% × 0.6 + 57.14% × 0.4) ≈ 0.26 points, and it is determined as low-quality recognized text.

[0070] In one embodiment, after step S204, the method further includes: S401. Obtain the OCR quality score of the OCR recognized text, and confirm whether the OCR quality score is less than a preset threshold; S402. If it is less than the preset threshold, then call a preset multi-modal recognition classifier to perform secondary text recognition and classification on the claim settlement material image data, and output multi-modal recognition text and multi-modal text classification results; S403. After correspondingly updating the OCR recognized text and the initial text classification results to the multi-modal recognition text and the multi-modal text classification results, participate in the consistency comparison and result output of the classification integration strategy.

[0071] In this embodiment, the preset threshold of the OCR quality score is determined in advance according to the test results of a large number of OCR recognition samples. For example, the default threshold is 0.7, which can be adjusted according to actual business needs. Retrieve the OCR quality score corresponding to the OCR recognized text from the OCR quality assessment results, and compare it with the preset threshold. If the score is greater than or equal to the preset threshold, it indicates that the OCR recognition quality is good and the initial text classification result is valid, and directly enter the classification integration strategy stage; if the score is less than the preset threshold, it indicates that the OCR recognition quality is poor and the initial text classification result may be incorrect, then secondary text recognition and classification update need to be performed before the subsequent classification integration strategy, so as to intercept text classification errors caused by poor text recognition quality in advance.

[0072] Specifically, call a preset multi-modal recognition classifier to perform secondary text recognition and classification on the claim settlement material image data. The preset multi-modal recognition classifier is specifically a VLM multi-modal recognition classifier, which can simultaneously understand and process image features and text features, and has stronger recognition ability for claim settlement material images with poor OCR recognition effects such as blur, handwriting, and multiple languages. Call the VLM multi-modal recognition classifier, input the claim settlement material image data into the model, and the model performs secondary text recognition and classification through the fusion of visual features and text features, and finally outputs more accurate multi-modal recognition text and corresponding multi-modal text classification results, and this result includes the claim settlement material category and classification confidence.

[0073] After completing the secondary text recognition and classification, the original OCR-recognized text is replaced with multimodal recognition text, and the original initial text classification result is replaced with the multimodal text classification result. The updated text classification result serves as the new initial text classification result, and together with the initial image classification result obtained in step S202, it enters the classification integration strategy stage in subsequent step S205 to participate in the consistency comparison between the two, and outputs the final image classification result according to the rules of the classification integration strategy. It is understandable that if, after secondary text recognition and classification, there are inconsistencies between the two classification results, and other preset classifiers need to be called for text classification updates, then the multimodal recognition text is subjected to corresponding classification processing to ensure the reliability of the text classification object.

[0074] This embodiment uses the VLM multimodal recognition classifier to perform secondary text recognition on low-quality OCR text. By combining image features and text features, it has a stronger ability to adapt to complex scenarios compared to single OCR recognition. It makes up for the shortcomings of single OCR recognition in complex scenarios such as blurry, handwritten, and multilingual scenarios, thereby automatically updating the classification results of low-quality OCR text and realizing dynamic correction of text classification results.

[0075] In one embodiment, step S402 includes: S421. Input the preset chain prompt words and the image data of the claim materials into the multimodal recognition classifier; S422. Guided by the chain of prompts, the multimodal recognition classifier is used to perform secondary text recognition by fusing visual and textual features on the image data of the claims materials, and outputs the multimodal recognition text. S422. Then, perform secondary text classification based on the multimodal recognition text and visual features, and output the multimodal text classification result.

[0076] In this embodiment, when performing secondary text recognition and classification on the claim material image data using a multimodal recognition classifier, preset chain prompts and the claim material image data are input into the multimodal recognition classifier. The chain prompts are thought chain (COT) prompts, which provide step-by-step guidance instructions, such as: "Step 1: Accurately identify all text content in the image, including handwritten and printed text, ignoring background interference text; Step 2: Based on the identified text content, determine the type of material. Claim material types only include medical invoices, expense lists, prescriptions, diagnostic reports, and examination reports; Step 3: Output the identified complete text and the determined material type sequentially." These chain prompts, along with the claim material image data, are input into the VLM multimodal recognition classifier, guiding the model to process the data according to the preset steps.

[0077] Specifically, the VLM multimodal recognition classifier includes a visual feature extraction module, a text feature processing module, a multimodal fusion module, a thought chain prompt word parsing module, a secondary recognition module, and a fusion classification module. First, the image data of the claims materials and the preset chain prompts are input into the model. The visual feature extraction module extracts visual features from the images, capturing visual information such as character shape, layout, and text region position. The text feature processing module performs preliminary feature extraction on the existing OCR-recognized text. The features of both are input into the multimodal fusion module to complete cross-modal feature fusion. The chain prompt parsing module parses the chain prompts step by step, guiding the model to execute the task step by step. Under the guidance of the chain prompts, the secondary recognition module first focuses on identifying the blurred areas where OCR recognition failed based on the fused multimodal features, outputting complete and accurate multimodal recognized text. Then, the fusion classification module combines the semantic features of the multimodal recognized text with the previously obtained visual features and compares it with the preset claims material category feature library. Finally, the corresponding multimodal text classification result is output, completing the secondary text classification, replacing the original low-quality initial text classification result, and re-participating in the consistency comparison of the subsequent classification integration strategy.

[0078] This embodiment uses a thought chain (COT) prompt word to guide a multimodal classifier, breaking down secondary recognition and classification into step-by-step tasks. By simultaneously understanding and processing image and text information for secondary text recognition, it can focus on capturing blurry areas and handwritten areas where OCR recognition fails, outputting more complete and accurate multimodal recognition text, thereby improving the accuracy of secondary classification results.

[0079] It should be noted that there is no necessary order between the above steps. Those skilled in the art will understand from the description of the embodiments of the present invention that the above steps may have different execution orders in different embodiments, that is, they may be executed in parallel or in turn, etc.

[0080] Further reference Figure 3 As a response to the above Figure 2 The present invention provides an embodiment of an image classification device for claims materials, which is implemented in accordance with the method shown. Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0081] like Figure 3 As shown, the image classification device 30 for claims materials described in this embodiment includes: The acquisition module 301 is used to acquire image data of the claims materials to be classified; The initial image classification module 302 is used to extract image features from the image data of the claims materials and perform initial image classification to obtain the initial image classification result. OCR recognition module 303 is used to perform OCR recognition on the image data of the claim materials to obtain OCR recognized text; The initial text classification module 304 is used to perform initial text classification based on keyword group matching of the OCR-recognized text, and obtain the initial text classification result; The integrated scheduling output module 305 is used to directly output the final image classification result based on the consistency between the initial image classification result and the initial text classification result according to a preset classification integration strategy, or to call different preset classifiers to update the initial image classification result and / or the initial text classification result before outputting the final image classification result.

[0082] The module referred to in this invention is a series of computer program instruction segments that can perform specific functions. It is more suitable than a program for describing the image classification and execution process of claims materials. For specific implementation methods of each module, please refer to the corresponding method embodiments above, which will not be repeated here.

[0083] In one embodiment, the text initial classification module 304 includes: The full matching unit is used to perform a full matching between the OCR-recognized text and a pre-built keyword group mapping library, which stores at least one set of keywords corresponding to different categories of claim materials. The keyword group classification unit is used to take the fully matched claim material category as the initial text classification result of the OCR-recognized text if the OCR-recognized text completely matches any group of keywords of a claim material category. The keyword matching statistics unit is used to count the frequency of occurrence of partially matched individual keywords in the OCR-recognized text, and to calculate the corresponding category total score by combining the pre-labeled category association weight of each keyword. The keyword classification unit is used to take the claim material category with the highest total category score as the initial text classification result of the OCR-recognized text.

[0084] In one embodiment, the integrated scheduling output module 305 includes: A consistency verification unit is used to verify whether the initial image classification result and the initial text classification result are consistent. The direct output unit is used to directly output the initial image classification result and the initial text classification result as the final image classification result if the classifications are consistent. The classifier confirmation unit is used to confirm the best classifier for each update round in turn, based on the performance indicators of different preset classifiers, if the classifications are inconsistent. The scheduling and updating unit is used to call the best classifier in each update round to perform corresponding classification updates on the image features or OCR-recognized text, so as to obtain new image classification results or new text classification results. The update output unit is used to compare the latest image classification result with the latest text classification result after each update, and outputs the final image classification result when the two classification results match.

[0085] In one embodiment, the classifier confirmation unit includes: The performance acquisition unit is used to acquire the performance metrics of each preset classifier obtained from pre-testing, including accuracy, processing speed, concurrency, and operating cost. The weight confirmation unit is used to obtain the corresponding weight allocation strategy according to the current scenario requirements and confirm the weight of each performance indicator. The weighted calculation unit is used to perform weighted calculations based on the performance indicators and corresponding weights of each preset classifier to obtain the scheduling performance score of each preset classifier. The sorting confirmation unit is used to sort the scheduling priorities according to the scheduling performance scores of each preset classifier and confirm the best classifier for each update round.

[0086] In one embodiment, the device 30 further includes: The word segmentation module is used to segment the OCR-recognized text into words and calculate the proportion of each word after segmentation. The uncommon character module is used to match the OCR-recognized text with a preset uncommon character list and calculate the proportion of uncommon characters based on the matching results; The quality assessment module is used to calculate the OCR quality score of the OCR-recognized text based on the proportion of individual characters and the proportion of rare characters.

[0087] In one embodiment, the device 30 further includes: The scoring comparison module is used to obtain the OCR quality score of the OCR-recognized text and to confirm whether the OCR quality score is less than a preset threshold. The secondary text recognition and classification module is used to call a preset multimodal recognition classifier to perform secondary text recognition and classification on the image data of the claims materials if the value is less than a preset threshold, and output the multimodal recognition text and the multimodal text classification result. The text classification update module is used to update the OCR-recognized text and the initial text classification result to correspond to the multimodal recognized text and the multimodal text classification result, and then participate in the consistency comparison and result output of the classification integration strategy.

[0088] In one embodiment, the secondary text recognition and classification module includes: The input unit is used to input the preset chain prompt words and the image data of the claim materials into the multimodal recognition classifier; A multimodal text recognition unit is used to perform secondary text recognition by fusing visual and text features on the image data of the claims materials under the guidance of the chain prompt words, and output the multimodal recognized text. The multimodal text classification unit is used to perform secondary text classification based on the multimodal recognized text and visual features, and output the multimodal text classification result.

[0089] In the above embodiments, the present invention discloses an image classification device for claims materials. By constructing an initial classification system with two dimensions of images and text and combining it with an integrated decision-making strategy of multi-classifiers, it realizes intelligent classification of claims material images based on multi-dimensional features. It can flexibly schedule classifiers to complete result verification and updates according to the consistency of classification results, effectively improving the accuracy and generalization ability of claims material image classification.

[0090] Specific limitations regarding the image classification device for claim materials can be found in the above description of the image classification method for claim materials, and will not be repeated here. Each module in the aforementioned image classification device for claim materials can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0091] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0092] Another embodiment of the present invention provides a computer device, such as... Figure 4 As shown, the computer device 40 includes: One or more processors 401 and memory 402, Figure 4The following section uses a processor 401 as an example. The processor 401 and the memory 402 can be connected via a bus or other means. Figure 4 Taking the example of a connection between China and Israel via a bus.

[0093] The processor 401 is used to perform various control logics of the computer device 40. It can be any conventional processor, microprocessor, state machine, general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), microcontroller, ARM (Acorn RISC Machine) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of these components.

[0094] The memory 402, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions corresponding to the image classification method for claims materials in the embodiments of the present invention. The processor 401 executes various functional applications and data processing of the computer device 40 by running the non-volatile software programs, instructions, and units stored in the memory 402, thereby implementing the image classification method for claims materials in the above-described method embodiments.

[0095] Another embodiment of the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by one or more processors, perform the steps of the image classification method for claims materials in any of the above method embodiments.

[0096] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0097] Based on the above description of the embodiments, those skilled in the art will understand that the methods described in the embodiments can be implemented using software plus necessary general-purpose hardware platforms. Of course, they can also be implemented using hardware, but in many cases, the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0098] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The computer program can be stored in a non-volatile, computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The storage medium can be a memory, magnetic disk, floppy disk, flash memory, optical storage, etc.

[0099] It should be noted that any AI models, software tools, or components not belonging to this company appearing in the embodiments of this application are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this application has been authorized (with the knowledge and consent) by the relevant parties or has been fully authorized by all parties, and the executing entity may obtain it through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.

[0100] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for classifying claims materials by image, characterized in that, include: Obtain image data of the claims materials to be classified; Extract image features from the image data of the claims materials and perform initial image classification to obtain the initial image classification result; The image data of the claim materials is subjected to OCR recognition to obtain OCR-recognized text; Initial text classification is performed based on keyword group matching of the OCR-recognized text to obtain the initial text classification result; Based on a preset classification integration strategy, the final image classification result is directly output according to the consistency between the initial image classification result and the initial text classification result, or the final image classification result is output after calling different preset classifiers to update the initial image classification result and / or the initial text classification result.

2. The image classification method for claims materials according to claim 1, characterized in that, The initial text classification based on keyword group matching of the OCR-recognized text, to obtain the initial text classification result, includes: The OCR-recognized text is fully matched with a pre-built keyword group mapping library, which stores at least one set of keywords corresponding to different categories of claim materials. If the OCR-recognized text completely matches any set of keywords in a claim material category, then the completely matched claim material category will be used as the initial text classification result of the OCR-recognized text. Otherwise, the frequency of occurrence of individual keywords that partially match in the OCR-recognized text is counted, and the corresponding total category score is calculated by combining the pre-labeled category association weights of each keyword; The category of claim materials with the highest total score is used as the initial text classification result for the OCR-recognized text.

3. The image classification method for claims materials according to claim 1, characterized in that, The preset classification integration strategy directly outputs the final image classification result based on the consistency between the initial image classification result and the initial text classification result, or updates the initial image classification result and / or the initial text classification result by calling different preset classifiers before outputting the final image classification result, including: Confirm whether the initial image classification result and the initial text classification result are consistent; If the classifications are consistent, the initial image classification result and the initial text classification result are directly output as the final image classification result. If the classifications are inconsistent, the best classifier for each update round is determined sequentially based on the performance metrics of the different preset classifiers. The best classifier in each update round is invoked to perform corresponding classification updates on the image features or OCR-recognized text, thereby obtaining new image classification results or new text classification results. After each update, the latest image classification result is compared with the latest text classification result to see if they match. The final image classification result is output only when the two classification results match.

4. The image classification method for claims materials according to claim 3, characterized in that, The process of determining the best classifier for each update round based on the performance metrics of different preset classifiers includes: Obtain the performance metrics of each preset classifier obtained from pre-testing, including accuracy, processing speed, concurrency, and operating cost; Based on the current scenario requirements, obtain the corresponding weight allocation strategy and confirm the weight of each performance indicator; The scheduling performance score of each preset classifier is obtained by weighting the performance indicators and corresponding weights of each preset classifier. The scheduling priority is sorted according to the scheduling performance scores of each preset classifier, and the best classifier for each update round is determined.

5. The image classification method for claims materials according to claim 1, characterized in that, After performing OCR recognition on the image data of the claim materials to obtain OCR-recognized text, the method further includes: The OCR-recognized text is segmented into words, and the percentage of each individual character after segmentation is calculated. The OCR-recognized text is matched with a preset list of rare characters, and the proportion of rare characters is calculated based on the matching results. The OCR quality score of the OCR-recognized text is calculated based on the proportion of individual characters and the proportion of rare characters.

6. The image classification method for claims materials according to claim 1, characterized in that, After obtaining the initial text classification result by performing keyword group matching based on the OCR-recognized text, the method further includes: Obtain the OCR quality score of the OCR-recognized text and confirm whether the OCR quality score is less than a preset threshold; If the value is less than a preset threshold, a preset multimodal recognition classifier is invoked to perform secondary text recognition and classification on the claim material image data, and output multimodal recognition text and multimodal text classification results; After updating the OCR-recognized text and the initial text classification result to correspond to the multimodal recognized text and the multimodal text classification result, they are used for consistency comparison and result output in the classification integration strategy.

7. The image classification method for claims materials according to claim 6, characterized in that, The process involves calling a preset multimodal recognition classifier to perform secondary text recognition and classification on the claim material image data, outputting multimodal recognition text and multimodal text classification results, including: The preset chain of prompts and the image data of the claim materials are input into the multimodal recognition classifier; Guided by the chain of prompts, the multimodal recognition classifier first performs secondary text recognition by fusing visual and textual features on the image data of the claims materials, and outputs the multimodal recognition text. Then, based on the multimodal recognition text and visual features, a secondary text classification is performed, and the multimodal text classification result is output.

8. An image classification device for claims materials, characterized in that, include: The acquisition module is used to acquire image data of the claims materials to be classified; The initial image classification module is used to extract image features from the image data of the claims materials and perform initial image classification to obtain the initial image classification result. The OCR recognition module is used to perform OCR recognition on the image data of the claim materials to obtain OCR-recognized text. The initial text classification module is used to perform initial text classification based on keyword group matching of the OCR-recognized text, and obtain the initial text classification result; An integrated scheduling output module is used to directly output the final image classification result based on a preset classification integration strategy and the consistency between the initial image classification result and the initial text classification result, or to call different preset classifiers to update the initial image classification result and / or the initial text classification result before outputting the final image classification result.

9. A computer device, characterized in that, Includes at least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the image classification method for claims materials according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform the image classification method for claims materials as described in any one of claims 1-7.

Images