Image classification methods, image classification devices, computer equipment and storage media

Abstract

This application provides an image classification method, image classification device, computer equipment, and storage medium, belonging to the field of artificial intelligence technology. The method includes: acquiring a first sample image and a sample image category of a sample document; performing text recognition on the first sample image, where the recognized initial text sequence includes initial text characters; inputting the first sample image and the initial text sequence into an initial classification model, performing keyword classification processing on the initial text characters to obtain keyword prediction scores for the initial text characters; constructing an image character table vector of the first sample image based on the keyword prediction scores; determining the predicted image category of the first sample image based on the image character table vector; adjusting the parameters of the initial classification model based on the sample image category and the predicted image category, and classifying the target image based on the adjusted document image classification model. This application embodiment can improve the accuracy of image classification.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an image classification method, an image classification device, a computer device, and a storage medium. Background Technology

[0002] Currently, when classifying images such as examination reports and discharge summaries, the small differences in features between images necessitate a method that first understands the text content within the images. This involves generating a complete feature vector based on the text content of each image, and then classifying the images based on this feature vector. However, this method suffers from low accuracy. Therefore, developing an image classification method that improves classification accuracy has become a pressing technical challenge. Summary of the Invention

[0003] The main objective of this application is to propose an image classification method, an image classification device, a computer device, and a storage medium that can improve the accuracy of image classification.

[0004] To achieve the above objectives, a first aspect of this application proposes an image classification method, the method comprising:

[0005] Obtain the first sample image of the sample document and the sample image category of the first sample image;

[0006] The first sample image is subjected to text recognition to obtain an initial text sequence, the initial text sequence including initial text words;

[0007] The first sample image and the initial text sequence are input into a preset initial classification model, which includes a keyword recognition sub-model, a vector construction sub-model, and an image classification sub-model.

[0008] The initial text is classified according to the keyword recognition sub-model to obtain the keyword prediction score of the initial text.

[0009] Based on the vector construction sub-model and the keyword prediction score, the first sample image is used to construct a word table vector to obtain the image word table vector.

[0010] The image classification sub-model is used to perform image classification processing on the image word table vector to determine the predicted image category of the first sample image;

[0011] The parameters of the initial classification model are adjusted according to the sample image category and the predicted image category to obtain the document image classification model;

[0012] The document image classification model is used to classify the pre-acquired target images.

[0013] In some embodiments, before performing keyword classification processing on the initial text characters according to the keyword recognition sub-model to obtain the keyword prediction score of the initial text characters, the method further includes: training the keyword recognition sub-model, specifically including:

[0014] Obtain a preset number of second sample images, wherein the image category of the second sample images is the same as that of the sample images;

[0015] The first sample image and the second sample image are combined to determine the training data;

[0016] Text annotation is performed on the training data to obtain training labeled data;

[0017] The training data is input into a preset initial recognition model for keyword classification processing to predict labeled data;

[0018] The initial recognition model is adjusted based on the predicted labeled data and the training labeled data to obtain the keyword recognition sub-model.

[0019] In some embodiments, the training data includes a first candidate image and a second candidate image, and the step of text annotation of the training data to obtain training annotation data includes:

[0020] Perform text recognition on the first candidate image to obtain a first candidate text sequence;

[0021] Perform text recognition on the second candidate image to obtain the second candidate text sequence;

[0022] The first candidate text sequence is divided into text sequences based on preset text segmentation data to obtain the first segmentation word;

[0023] The second candidate text sequence is divided into text sequences based on the text segmentation data to obtain the second segmentation word;

[0024] The training data is labeled with text characters based on the first separator and the second separator to obtain the labeled training data.

[0025] In some embodiments, the step of performing text annotation on the training data based on the first delimiter word and the second delimiter word to obtain the training annotation data includes:

[0026] Based on the first delimiter word, keyword matching is performed on the second candidate text sequence to obtain the first annotation data of the second candidate image;

[0027] Based on the second delimiter, keyword matching is performed on the first candidate text sequence to obtain the second annotation data of the first candidate image;

[0028] The training annotation data is determined based on the first annotation data and the second annotation data.

[0029] In some embodiments, the step of constructing a word table vector for the first sample image based on the vector-based sub-model and the keyword prediction score to obtain an image word table vector includes:

[0030] Obtain a preset element text character and an initial character table vector. The initial character table vector includes character table elements, which are used to represent the prediction score of the element text character.

[0031] Perform text matching between the element text and the initial text to obtain the text matching result;

[0032] The initial character table vector is filled with character table vectors based on the text character matching results to obtain the image character table vector.

[0033] In some embodiments, the step of filling the initial character table vector with a character table vector based on the text character matching result to obtain the image character table vector includes:

[0034] If the text matching result indicates that the element text is the same as the initial text, then the initial prediction score of the element text is obtained;

[0035] The initial prediction score and the keyword prediction score are weighted and calculated to obtain the element prediction score of the word table element;

[0036] The initial word table vector is updated based on the element prediction score to obtain the image word table vector.

[0037] In some embodiments, the step of performing image classification processing on the image word table vector according to the image classification sub-model to determine the predicted image category of the first sample image includes:

[0038] Obtain the category word table vector of the sample image category;

[0039] The category similarity is calculated between the category word table vector and the image word table vector to obtain a category similarity score, and the predicted image category is determined based on the category similarity score.

[0040] To achieve the above objectives, a second aspect of this application provides an image classification apparatus, the apparatus comprising:

[0041] The image acquisition module is used to acquire the first sample image of the sample document and the sample image category of the first sample image;

[0042] The text recognition module is used to perform text recognition on the first sample image to obtain an initial text sequence, wherein the initial text sequence includes initial text characters;

[0043] The model input module is used to input the first sample image and the initial text sequence into a preset initial classification model, which includes a keyword recognition sub-model, a vector construction sub-model, and an image classification sub-model.

[0044] The keyword classification module is used to perform keyword classification processing on the initial text characters according to the keyword recognition sub-model to obtain the keyword prediction score of the initial text characters;

[0045] The character table vector construction module is used to construct a character table vector for the first sample image based on the vector construction sub-model and the keyword prediction score, so as to obtain the image character table vector.

[0046] The image classification module is used to perform image classification processing on the image word table vector according to the image classification sub-model to determine the predicted image category of the first sample image;

[0047] The parameter adjustment module is used to adjust the parameters of the initial classification model according to the sample image category and the predicted image category to obtain a document image classification model.

[0048] The document image classification module is used to classify the pre-acquired target images according to the document image classification model.

[0049] To achieve the above objectives, a third aspect of this application provides a computer device, comprising:

[0050] At least one memory;

[0051] At least one processor;

[0052] At least one computer program;

[0053] The at least one computer program is stored in the at least one memory, and the at least one processor executes the at least one computer program to implement the image classification method described in the first aspect above.

[0054] To achieve the above objectives, a fourth aspect of the present application provides a storage medium, which is a computer-readable storage medium storing a computer program for causing a computer to execute the image classification method described in the first aspect.

[0055] The image classification method, image classification apparatus, computer device, and storage medium proposed in this application first acquire a first sample image of a sample document and its sample image category. Text recognition is then performed on the first sample image to obtain an initial text sequence, which includes initial text characters. The first sample image and the initial text sequence are input into a preset initial classification model, which includes a keyword recognition sub-model, a vector construction sub-model, and an image classification sub-model. Then, keyword classification processing is performed on the initial text characters according to the keyword recognition sub-model to obtain keyword prediction scores. Based on the vector construction sub-model and the keyword prediction scores, a character table vector is constructed from the first sample image to obtain an image character table vector. Image classification processing is then performed on the image character table vector according to the image classification sub-model to determine the predicted image category of the first sample image. Afterwards, the parameters of the initial classification model are adjusted according to the sample image category and the predicted image category to obtain a document image classification model. Finally, the document image classification model is used to classify the pre-acquired target image. This application embodiment can improve the accuracy of image classification. Attached Figure Description

[0056] Figure 1 This is a first flowchart of the image classification method provided in the embodiments of this application;

[0057] Figure 2 The first sample image provided in this application embodiment is a schematic diagram of the report structure;

[0058] Figure 3 This is a second flowchart of the image classification method provided in the embodiments of this application;

[0059] Figure 4 yes Figure 3 The flowchart of step S330 in the process;

[0060] Figure 5 yes Figure 4 The flowchart of step S450 in the text;

[0061] Figure 6 yes Figure 1 The flowchart of step S150 in the middle;

[0062] Figure 7 yes Figure 6 The flowchart of step S630 in the middle;

[0063] Figure 8 yes Figure 1 The flowchart of step S160 in the process;

[0064] Figure 9This is a schematic diagram of the structure of the image classification device provided in the embodiments of this application;

[0065] Figure 10 This is a schematic diagram of the hardware structure of the computer device provided in the embodiments of this application. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0067] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0068] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0069] First, let's analyze some of the terms used in this application:

[0070] Artificial Intelligence (AI) is a new branch of computer science that studies, develops, and applies theories, methods, technologies, and systems to simulate, extend, and expand human intelligence. It aims to understand the essence of intelligence and produce intelligent machines that can react in a way similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thought. Furthermore, AI utilizes digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceiving the environment, acquiring knowledge, and using that knowledge to achieve optimal results.

[0071] Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in tasks such as target recognition, tracking, and measurement, and further performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image segmentation, image semantic understanding, image retrieval, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0072] Natural Language Processing (NLP): NLP uses computers to process, understand, and utilize human language (such as Chinese and English). NLP is a branch of artificial intelligence and an interdisciplinary field of computer science and linguistics, often referred to as computational linguistics. NLP includes syntactic analysis, semantic analysis, and discourse understanding. It is commonly used in machine translation, handwritten and printed character recognition, speech recognition and text-to-speech conversion, information retrieval, information extraction and filtering, text classification and clustering, sentiment analysis, and opinion mining. It involves data mining, machine learning, knowledge acquisition, knowledge engineering, artificial intelligence research, and linguistic research related to language computation.

[0073] Optical Character Recognition (OCR): The process of extracting text from an image using image algorithms.

[0074] BERT (Bidirectional Encoder Representation from Transformers): It is used to further enhance the generalization ability of word vector models, fully describe character-level, word-level, sentence-level, and even sentence-level relationship features, and is built on Transformer.

[0075] Currently, when classifying images such as examination reports and discharge summaries, the small differences in features between images necessitate a method that first understands the text content within the images. This involves generating a complete feature vector based on the text content of each image, and then classifying the images based on this feature vector. However, this method suffers from low accuracy. Therefore, developing an image classification method that improves classification accuracy has become a pressing technical challenge.

[0076] Based on this, embodiments of this application provide an image classification method, an image classification device, a computer device, and a storage medium, which can improve the accuracy of image classification.

[0077] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0078] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0079] The image classification method provided in this application relates to the field of artificial intelligence technology. The image classification method provided in this application can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms; the software can be an application implementing the image classification method, but is not limited to the above forms.

[0080] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers (PCs), minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0081] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0082] Please see Figure 1 , Figure 1 This is an optional flowchart of the image classification method provided in the embodiments of this application. In some embodiments of this application, Figure 1 The method described below may include, but is not limited to, steps S110 to S180. Figure 1 These eight steps will be explained in detail.

[0083] Step S110: Obtain the first sample image of the sample document and the sample image category of the first sample image;

[0084] Step S120: Perform text recognition on the first sample image to obtain an initial text sequence, which includes initial text characters;

[0085] Step S130: Input the first sample image and the initial text sequence into the preset initial classification model. The initial classification model includes a keyword recognition sub-model, a vector construction sub-model, and an image classification sub-model.

[0086] Step S140: Perform keyword classification processing on the initial text characters according to the keyword recognition sub-model to obtain the keyword prediction score of the initial text characters;

[0087] Step S150: Construct a word table vector for the first sample image based on the vector-based sub-model and keyword prediction score to obtain the image word table vector;

[0088] Step S160: Perform image classification processing on the image word table vector according to the image classification sub-model to determine the predicted image category of the first sample image;

[0089] Step S170: Adjust the parameters of the initial classification model according to the sample image category and the predicted image category to obtain the document image classification model;

[0090] Step S180: Classify the pre-acquired target image according to the document image classification model.

[0091] In step S110 of some embodiments, since paper documents are inconvenient to carry and difficult to store, related technologies can convert paper documents into image documents by photographing them, such as books, papers, journals, and certificates. To achieve a structured representation of the content in the image documents, thereby assisting in downstream operations such as manual information entry or automated underwriting and claims decision-making, the structuring of document images requires accurate identification of the document image category. For example, in underwriting and claims tasks, the sample document can be an image document such as an examination report or discharge record report, and the first sample image is an image obtained from the sample document through image acquisition methods such as photography. The sample image category is used to indicate the category to which the sample document belongs. This sample image category may include: discharge record category, admission record category, medical record cover page category, inpatient diagnosis certificate category, physical examination report category, laboratory report category, invoice category, birth certificate category, social security card category, etc.

[0092] For example, please refer to Figure 2 , Figure 2 This is the report of the target subject after undergoing an A-scan examination, essentially equivalent to the first sample image. The report includes a subject attribute information area 210, an examination content area 220, an examination result area 230, and a doctor's signature area 240. The subject attribute information area 210 represents the target subject's name, examination date, and department / location information. The examination content area 220 represents the report display of the target subject after the A-scan examination. The examination result area 230 represents the doctor's examination results based on the current report display.

[0093] It should be noted that, in some embodiments, the first sample image can be obtained by writing a web crawler or script program to selectively crawl data.

[0094] It should be noted that the training sample set constructed in this application includes multiple first sample images. For example, the training sample set includes 500 first sample images under each sample image category, and 200 first sample images under each sample image category can be used as test samples.

[0095] In step S120 of some embodiments, since the first sample image obtained through image acquisition methods such as taking a picture cannot be edited, this embodiment of the application can call a pre-trained document element recognition model and perform text recognition on the first sample image according to the document element recognition model to obtain document detection boxes for multiple document elements. Because the sample image categories have very obvious template characteristics—that is, although each hospital has some differences in describing elevation content, the content layout and key text distribution of the same sample image category have very strong similarities—the document elements identified by the document element recognition model include detection image blocks, text blocks, etc. The document detection box is used to represent the positional region of a document element on the first sample image. The initial text sequence is the text content in the first sample image; for example, if the initial text sequence is: "Examination Name: B Joint Anteroposterior and Lateral View (Left Side)", then the initial text word is each text word constituting the initial text sequence.

[0096] It should be noted that the embodiments of this application can use OCR to recognize text blocks, obtain the precise positions of each initial text sequence included in the text block, and thus generate document detection boxes for each initial text sequence. Subsequently, the embodiments of this application can specifically employ a text sequence recognition method using a Convolutional Recurrent Neural Network (CRNN).

[0097] It should be noted that you should refer to [link / reference]. Figure 2 Document elements can be text content in the object attribute information area 210, the inspection content area 220, the inspection result area 230, and the doctor's signature area 240.

[0098] It should be noted that the document detection box can be a regular shape such as a rectangle, square, circle, rhombus, or hexagon, or it can be an irregular shape. In order to extract the text content in the document detection box completely and efficiently, the embodiment of this application can set the shape of the document detection box to a rectangle.

[0099] In step S130 of some embodiments, since it is found during data analysis that key text in key regions of the image plays a crucial role in classification, and related technologies typically rely on statistical methods for keyword identification. However, statistical approaches involve setting numerous thresholds and parameters, and the threshold settings significantly impact the results. Therefore, this application proposes a keyword recognition model based on labeled data matching, combining a vector-based sub-model and an image classification sub-model, which improves both image classification accuracy and model interpretability.

[0100] In step S140 of some embodiments, this application embodiment classifies each text character in the initial text sequence as a keyword using a keyword recognition sub-model to determine whether the initial text character is a keyword. Each initial text character can correspond to a keyword label or a non-keyword label. The prediction output layer of the keyword recognition sub-model is normalized to a range of 0-1, yielding a keyword prediction score. This keyword prediction score characterizes the probability that the initial text character is a keyword. For example, when the keyword score threshold is set to 0.5, if the keyword prediction score is less than or equal to 0.5, it indicates that the initial text character belongs to a non-keyword label; if the keyword prediction score is greater than 0.5, it indicates that the initial text character belongs to a keyword label.

[0101] Please see Figure 3 , Figure 3 This is another optional flowchart of the image classification method provided in the embodiments of this application. In some embodiments of this application, before step S140, the image classification method provided in the embodiments of this application may further include: training a keyword recognition sub-model, specifically including but not limited to steps S310 to S350, which will be described below in conjunction with... Figure 3 These five steps will be explained in detail.

[0102] Step S310: Obtain a preset number of second sample images, wherein the image category of the second sample images is the same as that of the sample images;

[0103] Step S320: Combine the first sample image and the second sample image to determine the training data;

[0104] Step S330: Text annotation is performed on the training data to obtain the training labeled data;

[0105] Step S340: Input the training data into the preset initial recognition model for keyword classification processing and predict the labeled data;

[0106] Step S350: Adjust the parameters of the initial recognition model based on the predicted labeled data and the training labeled data to obtain the keyword recognition sub-model.

[0107] In step S310 of some embodiments, sample images are sampled from the training sample set to obtain a preset number of second sample images. The sampled second sample images belong to the same category as the original sample images.

[0108] In steps S320 and S330 of some embodiments, a first sample image of the same sample image category in the training sample set and a second sample image are combined. Specifically, if the preset quantity is 1, it means that the sample images in each sample image category are arranged and combined in pairs, and the training data obtained after combination is used to train the keyword recognition sub-model. Afterwards, the sample images in the training data are labeled to obtain training labeled data with keyword labels and non-keyword labels.

[0109] It should be noted that the preset quantity can also be flexibly adjusted according to actual needs to adjust the number of sample images in the training data.

[0110] In steps S340 and S350 of some embodiments, the initial recognition model and the keyword recognition sub-model have the same model structure and model parameters. This application embodiment can use a keyword recognition sub-model built based on BERT and Layoutlm. Loss values ​​are calculated on the predicted labeled data and the training labeled data to obtain the target labeled loss value. The parameters of the initial recognition model are continuously adjusted based on the target labeled loss value. When the adjusted model meets the preset termination conditions, a satisfactory keyword recognition sub-model is obtained.

[0111] It should be noted that the preset termination condition can be either the model training ends after all training data has been trained, or the model training ends when the target annotation loss value is less than the preset loss value threshold. No specific limitation is made here.

[0112] It should be noted that the labeled loss value can be calculated using cross-entropy loss function, 0-1 loss function, etc., without specific limitations or elaboration here.

[0113] It should be noted that if the preset quantity is 2, then the number of sample images in the training data is 3. Specifically, two different second sample images are obtained by sampling the training sample set, and the first sample image and each second sample image are combined to determine the training data. This application embodiment does not specifically limit the value of the preset quantity.

[0114] The embodiments of this application construct training data by combining images, which can effectively increase the amount of training data for the keyword recognition sub-model, thereby indirectly improving the accuracy of the model.

[0115] Please see Figure 4 , Figure 4 This is a flowchart illustrating the specific method of step S330 provided in an embodiment of this application. In some embodiments of this application, the training data includes a first candidate image and a second candidate image; therefore, step S330 may specifically include, but is not limited to, steps S410 to S450. The following describes the process in conjunction with... Figure 4 These five steps will be explained in detail.

[0116] Step S410: Perform text recognition on the first candidate image to obtain the first candidate text sequence;

[0117] Step S420: Perform text recognition on the second candidate image to obtain the second candidate text sequence;

[0118] Step S430: Divide the first candidate text sequence into text sequences according to preset text segmentation data to obtain the first segmentation word;

[0119] Step S440: Divide the second candidate text sequence into text sequences based on the text segmentation data to obtain the second segmentation word;

[0120] Step S450: Text annotation is performed on the training data based on the first separator word and the second separator word to obtain the training annotation data.

[0121] In steps S410 and S420 of some embodiments, the training data includes a first candidate image and a second candidate image. The first candidate image and the second candidate image are used to represent different sample images of the same sample image category collected in the training sample set. Specifically, if the preset quantity is 1, then the number of sample images in the training data is 2. The first candidate image is denoted as image a, and the second candidate image is denoted as image b. Assume that the first candidate text sequence obtained by performing text recognition on the first candidate image is: "Color Doppler Ultrasound Examination Report Name Zhang X Examination Item C Examination"; the second candidate text sequence obtained by performing text recognition on the second candidate image is: "Color Doppler Ultrasound Examination Report Name Li Y Examination Item C Examination". Since the initial recognition model is a statistical language model, the high-frequency words corresponding to the keyword tags will learn higher scores. Therefore, the words that exist simultaneously in both candidate images are the keywords of the images. For example, if a certain word exists in both image a and image b, then this word is the keyword, and the label of each word is a keyword label.

[0122] In steps S430 to S450 of some embodiments, the preset text delimiter data is used to represent the number of consecutive texts after division, and the text delimiter data can be set to 1 character, 2 characters, 3 characters, etc., as needed, without specific limitation here. In order to improve the matching accuracy of keywords, for example, the text delimiter data is set to 2 characters, and the first candidate text sequence is divided into text sequence to obtain the first delimiter word. The first delimiter word may include: "color, color, Doppler, Lechao, ultrasound, sound detection, inspection, report, report, notice, single surname, name, Zhang, Zhang X, X inspection, inspection, check item, project, item C, C inspection, inspection". Similarly, the second candidate text sequence is divided into text sequence to obtain the first delimiter word, and the second delimiter word may include: "color ultrasound, ultrasound detection, inspection, report, report, notice, single surname, name, Li, Li Y, Y inspection, inspection, check item, project, item C, C inspection, inspection". Then, the training data is annotated with text based on the first and second separator words to obtain the training annotation data corresponding to the training data.

[0123] This application embodiment, by labeling sample image types, enables keyword tagging through delimiter word matching and labeled image types, thus automatically mining keywords from sample images. A keyword recognition sub-model based on the Layoutlm model is then trained using the labeled data. The training process is simple and effectively improves keyword recognition accuracy while enhancing the model's interpretability.

[0124] It should be noted that when the training data includes three sample images, two of the sample images can be arbitrarily selected as the first candidate image and the second candidate image, and steps S410 to S450 can be executed.

[0125] Please see Figure 5 , Figure 5 This is a flowchart illustrating the specific method of step S450 provided in an embodiment of this application. In some embodiments of this application, step S450 may specifically include, but is not limited to, steps S510 to S530, as described below. Figure 5 These three steps will be explained in detail.

[0126] Step S510: Perform keyword matching on the second candidate text sequence based on the first delimiter word to obtain the first annotation data of the second candidate image;

[0127] Step S520: Perform keyword matching on the first candidate text sequence based on the second delimiter to obtain the second annotation data of the first candidate image;

[0128] Step S530: Determine the training labeled data based on the first labeled data and the second labeled data.

[0129] In steps S510 to S530 of some embodiments, keyword tags are denoted as L1, and non-keyword tags are denoted as 0. Therefore, this embodiment of the application performs keyword tagging on the first candidate text sequence and the second candidate text sequence, respectively. Specifically, keyword matching is performed on the second candidate text sequence based on the string matching algorithm and the first delimiter word derived from the first candidate text sequence. If the first delimiter word exists in the second candidate text sequence, the corresponding text character in the second candidate text sequence is denoted as L1; if the first delimiter word does not exist in the second candidate text sequence, the corresponding text character in the second candidate text sequence is denoted as 0. The text character tags are then concatenated based on the position of the text character in the text sequence to obtain the first annotation data. Similarly, keyword matching is performed on the first candidate text sequence based on the string matching algorithm and the second delimiter word derived from the second candidate text sequence. If the second delimiter word exists in the first candidate text sequence, the corresponding text character in the first candidate text sequence is denoted as L1; if the second delimiter word does not exist in the first candidate text sequence, the corresponding text character in the first candidate text sequence is denoted as L0. The text word tags are then concatenated based on their positions within the text sequence to obtain the second annotation data.

[0130] For example, the first candidate text sequence is: "Color Doppler Ultrasound Examination Report Name Zhang San Examination Item C Examination", and the first delimiter word may include: "color, color, doppler, ultrasound, ultrasound, sound examination, examination, report, report, single surname, name, Zhang, Zhang X, X examination, examination, item, item C, C examination, examination". The second candidate text sequence obtained by text recognition of the second candidate image is: "Color Doppler Ultrasound Examination Report Name Li Si Examination Item C Examination", and the second delimiter word can include: "Color Doppler Ultrasound, Ultrasound Examination, Examination, Report, Report, Report, Single Surname, Name, Name Li, Li Y, Y Examination, Examination, Item, C, C Examination, Examination". Therefore, after keyword matching of the second candidate text sequence according to the first delimiter word, the first annotation data corresponding to the second candidate text sequence is "L0, L0, L1, L1, L1, L1, L1, L1, L0, L0, L0, L1 ... After performing keyword matching on the first candidate text sequence based on the second delimiter, the second annotation data corresponding to the first candidate text sequence is obtained as "L0, L0, L0, L0, L0, L0, L0, L1, L1, L1, L1, L1, L1, L0, L0, L0, L1, L1, L1, L1, L1, L1, L1, L1, L1, L1, L1, L1, L1, L1.

[0131] It should be noted that the training labeled data includes first labeled data and second labeled data. The first candidate image from the training data is input into the initial recognition model for keyword classification, resulting in first predicted labeled data. A loss is calculated based on the first predicted labeled data and the first labeled data to obtain the first labeled loss value. The second candidate image from the training data is input into the initial recognition model for keyword classification, resulting in second predicted labeled data. A loss is calculated based on the second predicted labeled data and the second labeled data to obtain the second labeled loss value. The first labeled loss value and the second labeled loss value are then weighted to obtain the target labeled loss value. The parameters of the initial recognition model are adjusted based on the target labeled loss value to obtain the keyword recognition sub-model.

[0132] This application employs a keyword recognition sub-model built upon BERT and Layoutlm. It performs character-level keyword classification and annotates candidate images in the training data based on the delimiters obtained after text segmentation, generating an annotated dataset equal to the number of candidate images. This increases the amount of annotated data, thereby optimizing model performance. Furthermore, this application incorporates image features into the model input, effectively improving image classification accuracy.

[0133] It should be noted that when a text word is matched successfully multiple times, the text word is marked as L1, meaning that the markings will not be superimposed.

[0134] It should be noted that the string matching algorithm can be any of the following: multimodal Aho-Corasick automaton, hash matching algorithm (Rabin-Karp, RK algorithm), longest matching prefix and suffix substring algorithm (Knuth-Morris-Pratt, KMP algorithm), etc., without any specific limitation.

[0135] In step S150 of some embodiments, since the keyword prediction score is any value between 0 and 1, and this keyword prediction score is used to represent the confidence level that the text word is a keyword. For example, if the keyword prediction score is 0.92, then the initial text word is labeled as a keyword tag. In order to accurately predict the image category of the first sample image, embodiments of this application construct an image word table vector corresponding to the first sample image based on the keyword prediction score.

[0136] Please see Figure 6 , Figure 6 This is a flowchart illustrating the specific method of step S150 provided in the embodiments of this application. In some embodiments of this application, step S150 may specifically include, but is not limited to, steps S610 to S630, as described below. Figure 6 These three steps will be explained in detail.

[0137] Step S610: Obtain preset element text characters and initial character table vectors. The initial character table vector includes character table elements, which are used to represent the prediction scores of element text characters.

[0138] Step S620: Perform text matching between the element text and the initial text to obtain the text matching result;

[0139] Step S630: Fill the initial character table vector with the character table vector according to the text character matching result to obtain the image character table vector.

[0140] In steps S610 to S630 of some embodiments, the initial word table vector is a matrix vector of size 1*N, where N represents the number of element text characters, and these element text characters are text characters preset by the BERT-based language model. Text character matching is performed between the initial text characters and the element text characters in the initial word table vector, and the keyword prediction scores corresponding to the successfully matched initial text characters are filled into the corresponding positions in the initial word table vector to obtain the image word table vector.

[0141] This application embodiment expresses sample image vectors by setting word table vectors, and performs image classification based on the obtained image word table vectors. The classification process is simple and can effectively improve the classification efficiency of the model.

[0142] For example, the initial word table vector is a matrix vector of size 1*N, which includes N word table elements. These elements represent the predicted scores of the recorded element text characters. When the initial text character in the initial text sequence corresponding to the first sample image successfully matches the element text character, the predicted score of the element text character is updated.

[0143] Please see Figure 7 , Figure 7 This is a flowchart illustrating the specific method of step S630 provided in an embodiment of this application. In some embodiments of this application, step S630 may specifically include, but is not limited to, steps S710 to S730, as described below. Figure 7 These three steps will be explained in detail.

[0144] Step S710: If the text matching result indicates that the element text is the same as the initial text, then obtain the initial prediction score of the element text.

[0145] Step S720: The initial prediction score and the keyword prediction score are weighted and calculated to obtain the element prediction score of the word table element.

[0146] Step S730: Update the initial word table vector based on the element prediction score to obtain the image word table vector.

[0147] In steps S710 to S730 of some embodiments, text matching is performed between the initial text word and each element text word in the initial word table vector. If the text matching result indicates that an element text word is the same as the initial text word, it is determined whether the predicted score of the text word has already been stored in the element text word. If the initial prediction score is 0, the keyword prediction score of the initial text word is written to the position of the successfully matched element text word. If the initial prediction score is greater than 0, the keyword prediction score and the keyword prediction score of the initial text word are weighted and calculated, and the resulting element prediction score is filled into the position of the successfully matched element text word to update the obtained image word table vector. For example, the weight values ​​of the initial prediction score and the keyword prediction score can both be set to 0.5, which is equivalent to calculating the average of the initial prediction score and the keyword prediction score to determine the element prediction score.

[0148] It should be noted that for repeated initial text characters, the average of all keyword prediction scores for the initial text character is calculated, and the average score is filled into the position of the element text character that successfully matches the initial text character in the initial character table vector.

[0149] In step S160 of some embodiments, in order to more accurately predict the image category of the first sample image, the image word table vector is also set to a matrix vector with a matrix size of 1*N.

[0150] Please see Figure 8 , Figure 8 This is a flowchart illustrating the specific method of step S160 provided in an embodiment of this application. In some embodiments of this application, step S160 may specifically include, but is not limited to, steps S810 and S820, as described below. Figure 8 These two steps will be explained in detail.

[0151] Step S810: Obtain the category word table vector of the sample image category;

[0152] Step S820: Calculate the category similarity between the category word table vector and the image word table vector to obtain the category similarity score, and determine the predicted image category based on the category similarity score.

[0153] In steps S810 and S820 of some embodiments, a category word table vector for each sample image category is obtained. This category word table vector is obtained offline using a trained keyword recognition sub-model and a vector construction sub-model. For example, if a sample image category includes 500 sample images, these 500 sample images are input into the keyword recognition sub-model and the vector construction sub-model respectively to obtain template word table vectors for the sample images. Then, the vector mean of these 500 template word table vectors is calculated to determine the category word table vector for that sample image category. Assuming there are 20 sample image categories, category similarity is calculated for each category word table vector and image word table vector, resulting in 20 candidate similarity scores. Then, all candidate similarity scores are numerically compared, and the candidate similarity score with the highest score is taken as the category similarity score, and the image category corresponding to this category similarity score is the predicted image category. The matrix sizes of the category word table vector and the image word table vector are the same.

[0154] It should be noted that category similarity calculation can use any of the following methods, such as cosine similarity calculation, Euclidean distance calculation, or Manhattan distance, without specific limitations or elaborations here.

[0155] In step S170 of some embodiments, a loss value is calculated based on the sample image category and the predicted image category to obtain a category prediction loss value. The parameters of the initial classification model are continuously adjusted based on the category prediction loss value. When the adjusted model meets a preset termination condition, a satisfactory document image classification model is obtained. Therefore, the document image classification model and the initial classification model have the same model parameters and model structure. The preset termination condition for the initial classification model can refer to the preset termination condition during the training of the keyword recognition sub-model described above, and will not be specifically limited or elaborated upon here.

[0156] It should be noted that the category prediction loss value can be calculated using cross-entropy loss function, 0-1 loss function, etc., without specific limitations or elaboration here.

[0157] In step S180 of some embodiments, a target document is acquired, and an image acquisition method such as taking a picture of the target document is used to obtain a target image. When the image classification device of this application is installed on the terminal, the image classification device deploys a document image classification model. Therefore, when a document image classification operation is detected, the terminal generates a document image classification service request, which includes the obtained target image, and sends the document image classification service request to the image classification device. The image classification device responds to the document image classification service request and extracts the target image from the document image classification service request. Then, the target image is classified according to the document image classification model to determine the target image category, which is any one of the sample image categories.

[0158] Please see Figure 9 , Figure 9 This is a schematic diagram of the module structure of the image classification device provided in the embodiments of this application. The device can implement the image classification method of the above embodiments. The device includes an image acquisition module 910, a text recognition module 920, a model input module 930, a keyword classification module 940, a character vector construction module 950, an image classification module 960, a parameter adjustment module 970, and a document image classification module 980.

[0159] Image acquisition module 910 is used to acquire the first sample image of the sample document and the sample image category of the first sample image;

[0160] The text recognition module 920 is used to perform text recognition on the first sample image to obtain an initial text sequence, which includes initial text characters.

[0161] The model input module 930 is used to input the first sample image and the initial text sequence into the preset initial classification model. The initial classification model includes a keyword recognition sub-model, a vector construction sub-model, and an image classification sub-model.

[0162] The keyword classification module 940 is used to classify the initial text characters according to the keyword recognition sub-model and obtain the keyword prediction score of the initial text characters;

[0163] The character table vector construction module 950 is used to construct the character table vector of the first sample image based on the vector construction sub-model and keyword prediction score, so as to obtain the image character table vector.

[0164] The image classification module 960 is used to perform image classification processing on the image word table vector according to the image classification sub-model to determine the predicted image category of the first sample image;

[0165] The parameter adjustment module 970 is used to adjust the parameters of the initial classification model according to the sample image category and the predicted image category to obtain the document image classification model;

[0166] The document image classification module 980 is used to classify pre-acquired target images according to the document image classification model.

[0167] It should be noted that the image classification device in this application embodiment is used to implement the image classification method in the above embodiment. The image classification device in this application embodiment corresponds to the aforementioned image classification method. For the specific processing procedure, please refer to the aforementioned image classification method, which will not be repeated here.

[0168] This application also provides a computer device comprising: at least one memory, at least one processor, and at least one computer program. The at least one computer program is stored in the at least one memory, and the at least one processor executes the at least one computer program to implement any of the image classification methods described in the above embodiments. This computer device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0169] Please see Figure 10 , Figure 10 The illustration shows the hardware structure of a computer device according to another embodiment, the computer device comprising:

[0170] The processor 1010 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0171] The memory 1020 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1020 and called and executed by the processor 1010 using the image classification method of the embodiments of this application.

[0172] The input / output interface 1030 is used to implement information input and output;

[0173] The communication interface 1040 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0174] Bus 1050 transmits information between various components of the device (e.g., processor 1010, memory 1020, input / output interface 1030, and communication interface 1040);

[0175] The processor 1010, memory 1020, input / output interface 1030 and communication interface 1040 are connected to each other within the device via bus 1050.

[0176] This application also provides a storage medium, which is a computer-readable storage medium, storing a computer program for causing a computer to execute the image classification method described above.

[0177] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0178] This application provides an image classification method, image classification apparatus, computer device, and storage medium. The method involves acquiring a first sample image of a sample document and its sample image category, and performing text recognition on the first sample image to obtain an initial text sequence, which includes initial text characters. The first sample image and the initial text sequence are input into a preset initial classification model, which includes a keyword recognition sub-model, a vector construction sub-model, and an image classification sub-model. Then, the initial text characters are classified according to the keyword recognition sub-model to obtain keyword prediction scores. A character table vector is constructed from the first sample image based on the vector construction sub-model and the keyword prediction scores to obtain an image character table vector. Image classification is then performed on the image character table vector using the image classification sub-model to determine the predicted image category of the first sample image. Subsequently, the parameters of the initial classification model are adjusted based on the sample image category and the predicted image category to obtain a document image classification model. Finally, the document image classification model is used to classify pre-acquired target images. Data analysis reveals that document image classification is primarily based on keywords, while non-keywords have a negligible impact on classification. This application proposes a keyword recognition model based on labeled data matching, combined with a vector-based sub-model and an image classification sub-model. This improves both the accuracy and interpretability of image classification. For example, in the insurance claims field, when classifying medical record images, it enables structured analysis of the content within the image documents, thus assisting manual data entry and demonstrating strong practicality. Specifically, this application employs a keyword recognition sub-model based on BERT and Layoutlm, performing character-level keyword classification and incorporating image features into the model input, making it more suitable for document image classification. Furthermore, by setting character vectors to represent images and calculating similarity between the obtained image character vectors and preset category character vectors, image classification is achieved. This simplifies the classification process and effectively improves the model's classification efficiency.

[0179] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0180] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0181] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0182] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0183] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0184] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0185] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0186] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0187] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0188] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0189] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. An image classification method, characterized in that, The method includes: Obtain the first sample image of the sample document and the sample image category of the first sample image; The first sample image is subjected to text recognition to obtain an initial text sequence, the initial text sequence including initial text words; The first sample image and the initial text sequence are input into a preset initial classification model, which includes a keyword recognition sub-model, a vector construction sub-model, and an image classification sub-model. The initial text is classified according to the keyword recognition sub-model to obtain the keyword prediction score of the initial text. The keyword prediction score is used to characterize the prediction probability that the initial text is a keyword. Obtain a preset element text and an initial character table vector. The initial character table vector includes character table elements, which represent the prediction scores of the element text and the vector values ​​used for image classification. Perform text matching on the element text and the initial text to obtain a text matching result. If the text matching result indicates that the element text and the initial text are the same, obtain the initial prediction score of the element text. If the initial prediction score is 0, write the keyword prediction score of the initial text to the position of the successfully matched element text. If the initial prediction score is greater than 0, perform a weighted calculation on the initial prediction score and the keyword prediction score of the initial text, and fill the resulting element prediction score into the position of the successfully matched element text to update the image character table vector. Obtain the category word table vector of the sample image category, the category word table vector and the image word table vector have the same matrix size; calculate the category similarity between the category word table vector and the image word table vector to obtain the category similarity score, and determine the predicted image category of the first sample image based on the category similarity score; The parameters of the initial classification model are adjusted according to the sample image category and the predicted image category to obtain the document image classification model; The document image classification model is used to classify the pre-acquired target images.

2. The method according to claim 1, characterized in that, Before performing keyword classification processing on the initial text characters according to the keyword recognition sub-model to obtain the keyword prediction score of the initial text characters, the method further includes: training the keyword recognition sub-model, specifically including: Obtain a preset number of second sample images, wherein the image category of the second sample images is the same as that of the sample images; The first sample image and the second sample image are combined to determine the training data; Text annotation is performed on the training data to obtain training labeled data; The training data is input into a preset initial recognition model for keyword classification processing to predict labeled data; The initial recognition model is adjusted based on the predicted labeled data and the training labeled data to obtain the keyword recognition sub-model.

3. The method according to claim 2, characterized in that, The training data includes a first candidate image and a second candidate image. The step of text annotation on the training data to obtain training annotation data includes: Perform text recognition on the first candidate image to obtain a first candidate text sequence; Perform text recognition on the second candidate image to obtain the second candidate text sequence; The first candidate text sequence is divided into text sequences based on preset text segmentation data to obtain the first segmentation word; The second candidate text sequence is divided into text sequences based on the text segmentation data to obtain the second segmentation word; The training data is labeled with text characters based on the first separator and the second separator to obtain the labeled training data.

4. The method according to claim 3, characterized in that, The step of annotating the training data with text based on the first delimiter and the second delimiter to obtain the training labeled data includes: Based on the first delimiter word, keyword matching is performed on the second candidate text sequence to obtain the first annotation data of the second candidate image; Based on the second delimiter, keyword matching is performed on the first candidate text sequence to obtain the second annotation data of the first candidate image; The training annotation data is determined based on the first annotation data and the second annotation data.

5. An image classification device, characterized in that, The device includes: The image acquisition module is used to acquire the first sample image of the sample document and the sample image category of the first sample image; The text recognition module is used to perform text recognition on the first sample image to obtain an initial text sequence, wherein the initial text sequence includes initial text characters; The model input module is used to input the first sample image and the initial text sequence into a preset initial classification model, which includes a keyword recognition sub-model, a vector construction sub-model, and an image classification sub-model. The keyword classification module is used to perform keyword classification processing on the initial text characters according to the keyword recognition sub-model to obtain the keyword prediction score of the initial text characters. The keyword prediction score is used to characterize the prediction probability that the initial text characters are keywords. A character table vector construction module is used to obtain preset element text characters and an initial character table vector. The initial character table vector includes character table elements, which are used to represent the prediction scores of the element text characters. The prediction scores are used to represent the vector values ​​used by the initial character table vector for image classification. The module performs text character matching on the element text characters and the initial text characters to obtain a text character matching result. If the text character matching result indicates that the element text characters and the initial text characters are the same, the module obtains the initial prediction score of the element text characters. If the initial prediction score is 0, the keyword prediction score of the initial text characters is written into the position of the successfully matched element text characters. If the initial prediction score is greater than 0, the initial prediction score and the keyword prediction score of the initial text characters are weighted and calculated, and the resulting element prediction score is filled into the position of the successfully matched element text characters to update the obtained image character table vector. The image classification module is used to obtain the category word table vector of the sample image category, wherein the matrix size of the category word table vector and the image word table vector are the same; to calculate the category similarity between the category word table vector and the image word table vector to obtain the category similarity score, and to determine the predicted image category of the first sample image based on the category similarity score; The parameter adjustment module is used to adjust the parameters of the initial classification model according to the sample image category and the predicted image category to obtain a document image classification model. The document image classification module is used to classify the pre-acquired target images according to the document image classification model.

6. A computer device, characterized in that, include: At least one memory; At least one processor; At least one computer program; The at least one computer program is stored in the at least one memory, and the at least one processor executes the at least one computer program to perform: The method as described in any one of claims 1 to 4.

7. A storage medium, said storage medium being a computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is used to cause the computer to execute: The method as described in any one of claims 1 to 4.

Images