Document classification method and device, computer equipment and readable storage medium

Abstract

The invention relates to the technical field of big data analysis, and discloses a document classification method and device, computer equipment and a readable storage medium. The method comprises the following steps: acquiring a to-be-classified document and a to-be-classified image; performing feature extraction on the image to obtain an image feature extraction result; performing text feature extraction on the to-be-classified document to obtain a text feature extraction result; splicing the image feature extraction result and the text feature extraction result to obtain a splicing result; classifying the splicing result to obtain a classification result; and outputting the classification result. By implementing the method provided by the invention, diversified document formats can be effectively processed, and the material classification efficiency in an insurance claim settlement scene is remarkably improved.

Description

Technical Field

[0001] The present invention relates to big data, and more specifically to a document classification method, apparatus, computer equipment and readable storage medium. Background Art

[0002] In the insurance claim scenario, classifying the claim materials uploaded by users is a crucial link, which not only affects the subsequent storage process, but also plays a decisive role in the efficiency and accuracy of the entire claim process. The current traditional technology mainly relies on the classification method based on image texture. This method performs well in processing some fixed-format documents such as identity cards, vehicle registration certificates, and driver's licenses, because these documents have clear formats and structures, and the classification process is relatively simple.

[0003] However, traditional classification methods have obvious shortcomings when facing diverse claims materials. For example, many claims materials such as medical records, admission / discharge records, and surgical records do not have a unified format, and the document formats of various hospitals or medical institutions vary greatly. Traditional image texture analysis cannot adapt to these diverse formats, resulting in reduced accuracy of classification results. Traditional methods usually only focus on the image features of documents and ignore the semantic information of text content. In many cases, the classification of claims materials depends on the information contained in the document, such as disease name, treatment plan, date, etc., and the extraction and understanding of this information requires the help of natural language processing technology. With the increase in the types of claims materials, traditional models often need to be frequently updated and replaced to adapt to new document formats and types. This not only increases maintenance costs, but also affects the continuous use efficiency of the system. In existing technologies, due to insufficient classification accuracy, manual review and adjustment of classification results are often required, which increases labor costs and workload, and also prolongs the cycle of the claims process. With the expansion of claims business, traditional technologies have bottlenecks in processing large-scale data and are difficult to meet the rapidly growing business needs.

[0004] Therefore, it is necessary to design a new method that can effectively process diverse document formats and significantly improve the efficiency of material classification in insurance claims scenarios. Summary of the invention

[0005] The present invention provides a document classification method, apparatus, computer equipment and readable storage medium to solve the technical problems that traditional claims material classification methods face challenges of diversified formats, cannot accurately process document content, and lead to low classification efficiency, increased manual review and increased system maintenance costs.

[0006] In a first aspect, a document classification method is provided, comprising:

[0007] Obtaining documents to be classified and images to be classified;

[0008] Performing feature extraction on the image to obtain an image feature extraction result;

[0009] Performing text feature extraction on the document to be classified to obtain a text feature extraction result;

[0010] Splicing the image feature extraction result and the text feature extraction result to obtain a splicing result;

[0011] Classifying the splicing results to obtain classification results;

[0012] The classification result is output.

[0013] In a second aspect, a document classification device is provided, comprising:

[0014] An acquisition unit, used for acquiring documents to be classified and images to be classified;

[0015] An image feature extraction unit, used to extract features from the image to obtain an image feature extraction result;

[0016] A text feature extraction unit, used to extract text features from the document to be classified to obtain a text feature extraction result;

[0017] A splicing unit, used for splicing the image feature extraction result and the text feature extraction result to obtain a splicing result;

[0018] A classification unit, used for classifying the splicing result to obtain a classification result;

[0019] An output unit is used to output the classification result.

[0020] In a third aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the document classification method when executing the computer program.

[0021] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned document classification method are implemented.

[0022] The above document classification method obtains the document and image to be classified, and extracts features respectively to obtain the feature results of the image and text; splices the image features and text features to form a comprehensive feature vector; classifies the splicing results and outputs the final classification results; specifically, by obtaining and extracting the features of different document formats and images, splices the image features and text features to form a comprehensive feature vector. Then, the splicing results are classified to achieve fast and accurate document processing. Ultimately, this method significantly improves the classification efficiency of insurance claims materials and adapts to a variety of document formats.

[0023] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0025] Figure 1 A schematic diagram of an application scenario of the document classification method provided by an embodiment of the present invention;

[0026] Figure 2 A schematic diagram of a flow chart of a document classification method provided by an embodiment of the present invention;

[0027] Figure 3 A schematic diagram of a sub-process of a document classification method provided by an embodiment of the present invention;

[0028] Figure 4 A schematic diagram of a sub-process of a document classification method provided by an embodiment of the present invention;

[0029] Figure 5 A schematic diagram of a sub-process of a document classification method provided by an embodiment of the present invention;

[0030] Figure 6 A schematic block diagram of a document classification device provided by an embodiment of the present invention;

[0031] Figure 7 A schematic block diagram of an image feature extraction unit of a document classification device provided in an embodiment of the present invention;

[0032] Figure 8 A schematic block diagram of a text feature extraction unit of a document classification device provided in an embodiment of the present invention;

[0033] Fig. 9 A schematic block diagram of a classification unit of a document classification device provided in an embodiment of the present invention;

[0034] Fig.10 is a schematic diagram of a structure of a computer device in an embodiment of the present invention;

[0035] Fig.11 2 is another schematic diagram of the structure of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0038] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0039] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0040] See also Figure 1 and Figure 2 , Figure 1 A schematic diagram of an application scenario of the document classification method provided in an embodiment of the present invention. Figure 2 A schematic flow chart of a document classification method provided in an embodiment of the present invention. The document classification method is applied in a server. The server interacts with a terminal for data, obtains documents to be classified and images to be classified from the terminal, and then extracts image features using an image encoder by putting image slices of different sizes into the same batch with the global image. At the same time, a text encoder is used to extract corresponding text features. Finally, these two features are spliced ​​and applied comprehensively to image classification, so as to achieve the ability to effectively process a variety of document formats and significantly improve the efficiency of material classification in insurance claims scenarios.

[0041] Figure 2FIG. 1 is a flow chart of a document classification method provided by an embodiment of the present invention. Figure 2 As shown, the method includes the following steps S110 to S160.

[0042] S110, obtaining documents to be classified and images to be classified.

[0043] In this embodiment, the documents to be classified include electronic documents and paper documents, wherein the electronic documents are obtained from the internal system of the insurance company, online forms submitted by customers, emails, etc. (such as PDF, Word, etc.). The paper documents are converted into digital format by a scanner to ensure that the image is clear and easy to process later. Image acquisition includes

[0044] Allow users to upload images related to claims, such as loss photos, accident scene pictures, etc. through mobile phones or computers. Use web crawler technology to automatically grab relevant images from relevant websites or databases.

[0045] S120: Perform feature extraction on the image to obtain an image feature extraction result.

[0046] In this embodiment, the image feature extraction result refers to representative feature information extracted from the image through a series of technical means. These features can effectively represent the content of the image and are used for subsequent classification tasks.

[0047] In one embodiment, see Figure 3 , the above-mentioned step S120 may include steps S121 to S122.

[0048] S121, dividing the image into slices of different sizes, and putting the image and the slices into a batch.

[0049] In this embodiment, the input image is divided into multiple small areas (slices), and the size of each slice can be different (such as 32x32 pixels, 64x64 pixels, etc.); the original image and all slices are included in a batch to facilitate subsequent feature extraction and processing.

[0050] By slicing the image, more detailed local features can be captured, improving the representativeness and accuracy of the features. Small-sized slices take up fewer resources when calculating, allowing parallel processing and increasing processing speed. Slicing can make the model more robust and better adaptable to local deformation or occlusion in the image.

[0051] S122: Use an image encoder to extract features of the slice and the image to obtain an image feature extraction result.

[0052] In this embodiment, a deep learning model (such as a convolutional neural network (CNN)) is used as an encoder to extract features from each slice and the entire image, and the extracted features are converted into high-dimensional feature vectors or descriptors to represent the features of the slice or image.

[0053] Deep learning models can automatically learn and extract complex features, making them richer and able to capture subtle differences in images. Encoders can adapt to different types of image data and improve the flexibility of feature extraction. The extracted feature vectors can significantly improve the accuracy and efficiency of subsequent classification, retrieval and other tasks.

[0054] By segmenting the image and using the image encoder to extract features, effective image feature extraction results can be obtained. These results can not only improve the performance of the model, but also enhance the efficiency and accuracy of image processing, providing a solid foundation for subsequent applications. Both local features and global features are retained.

[0055] In addition, after feature extraction, the extracted features need to be further processed and analyzed using a visual mapper. Specifically, the functions of the visual mapper include:

[0056] Mapping high-dimensional feature vectors to low-dimensional space helps reduce computational complexity and storage requirements while maintaining important feature information.

[0057] By mapping features into 2D or 3D space, Visual Mapper can help researchers and engineers intuitively understand the distribution of features and their relationships among different categories.

[0058] In many cases, the mapped features can improve the performance of machine learning algorithms (such as classifiers) because low-dimensional features tend to be easier to separate and process.

[0059] Visual mappers can help identify natural clusters in data, thereby uncovering hidden patterns or trends.

[0060] When processing large-scale image data, the mapper can effectively reduce the dimension, making subsequent calculations and analysis more efficient.

[0061] Visual mappers can reduce the number of visual tokens generated.

[0062] Through these functions, Visual Mapper provides a deeper analysis and application basis for image feature extraction results.

[0063] S130: Perform text feature extraction on the document to be classified to obtain a text feature extraction result.

[0064] In this embodiment, the text feature extraction result refers to feature information extracted from the document to be classified through a series of technical means. These features can effectively represent the document content and are used for subsequent classification tasks.

[0065] In one embodiment, see Figure 4 , the above-mentioned step S130 may include steps S131 to S132.

[0066] S131, using OCR technology to recognize text information of the document to be classified.

[0067] In this embodiment, the OCR technology is used to scan and recognize the documents to be classified, and the text in the image is converted into a machine-readable text format. The recognized text information is organized into a structured format to remove noise and irrelevant information.

[0068] OCR technology can extract text information from images or scanned documents, greatly expanding the types of documents that can be processed. The automated recognition process greatly reduces the workload of manual input and improves processing efficiency. Modern OCR systems support multiple languages ​​and are suitable for document recognition in different languages.

[0069] S132: extracting text features from the text information using a text encoder to obtain a text feature extraction result.

[0070] In this embodiment, natural language processing (NLP) techniques and models (such as word embedding models, BERT, etc.) are used to extract features from the recognized text. The extracted features are converted into high-dimensional feature vectors or descriptors to represent the semantics and content of the document.

[0071] The text encoder can understand the context and semantic relationship of the text and extract richer and more meaningful features. Mapping high-dimensional text data to low-dimensional space helps reduce the computational burden and improve the efficiency of subsequent analysis. The extracted feature vector can significantly improve the accuracy and efficiency of subsequent classification, retrieval and other tasks.

[0072] By using OCR technology to identify text information and using text encoders to extract features, effective text feature extraction results can be obtained. These results provide a solid foundation for document classification and analysis, and help improve the automation and accuracy of information processing.

[0073] S140: splicing the image feature extraction result and the text feature extraction result to obtain a splicing result.

[0074] In this embodiment, the splicing result refers to the result obtained by splicing the image feature extraction result and the text feature extraction result.

[0075] Specifically, the image feature extraction result and the text feature extraction result are sequentially placed into an array according to a set order to obtain a splicing result.

[0076] There are usually two ways to determine the order of splicing image features and text features:

[0077] Image features come first, text features come second (e.g. [image features, text features]);

[0078] Text features come first, image features come second (e.g. [text features, image features]);

[0079] Use a programming language to concatenate two feature vectors along the specified axis to generate the final feature array.

[0080] By splicing image and text features, a comprehensive feature representation can be created to capture the relationship between image and text, which is helpful for multimodal learning. For multimodal tasks (such as image description generation, image and text retrieval, etc.), the spliced ​​feature vector can provide richer information for machine learning or deep learning models, improving the accuracy and robustness of the model. Combining the two features into one vector can simplify the subsequent model training and inference process, making the processing process more unified and efficient. This splicing method can be flexibly applied to a variety of tasks and scenarios, and is suitable for applications that need to consider both visual and text information, such as social media content analysis, product recommendations, etc. In some cases, the spliced ​​features can be further processed by dimensionality reduction techniques (such as PCA, t-SNE), thereby reducing the dimension of the feature space and improving the efficiency of subsequent processing.

[0081] Concatenating the image feature extraction results with the text feature extraction results in a set order can not only generate a comprehensive feature representation, but also improve model performance, simplify processing procedures, enhance adaptability, and ultimately improve the effect of multimodal tasks. This method has been widely used in modern machine learning and deep learning and is an effective strategy for processing complex data.

[0082] S150: classify the splicing result to obtain a classification result.

[0083] In this embodiment, the classification result refers to the category corresponding to the document and the image after combination. In this step, the main goal is to classify the previously spliced ​​image features and text features to obtain a clear category identification. The classification result will combine image and text information to provide richer contextual understanding.

[0084] Specifically, the splicing result is classified using a Zero-shot classification method to obtain a classification result.

[0085] The splicing result is compared with the features formed by splicing the pre-stored image feature extraction results with categories and the text feature extraction results to determine the classification result.

[0086] In one embodiment, see Figure 5 , the above-mentioned step S150 may include steps S151 to S152.

[0087] S151, calculating the cosine distance of the features formed by splicing the splicing result with the pre-stored image feature extraction results with categories and the text feature extraction results to obtain a calculation result.

[0088] In this embodiment, the concatenated feature result (image feature+text feature) is concatenated with the pre-stored image features and text features with categories to form a new feature vector.

[0089] The cosine distance formula is used to evaluate the similarity between the splicing result and each feature vector with a category. The cosine distance formula is: A and B represent two feature vectors. The closer the calculated result is to 1, the higher the similarity between the two vectors.

[0090] S152: Filter out the calculation results that meet the requirements from the calculation results, and determine the corresponding categories to obtain classification results.

[0091] In this embodiment, based on the calculated cosine distance results, calculation results with similarity higher than a certain threshold are screened out. This threshold can be adjusted according to specific tasks and data sets.

[0092] From the filtered calculation results, determine the corresponding category. Usually the category with the highest similarity is selected as the final classification result.

[0093] Zero-shot classification can directly utilize stored features without the need for additional training data, making the classification process more efficient, especially in scenarios where the number of categories is large or new categories appear frequently.

[0094] The concatenation of image and text features helps the model to comprehensively consider multiple information during classification, thereby improving classification accuracy and contextual understanding.

[0095] Since cosine distance is used as the similarity metric, it can flexibly adapt to different data distributions and feature spaces and is suitable for various types of tasks.

[0096] The implementation process of this method is relatively simple, the complexity of calculating the cosine distance is low, and the result can be obtained quickly, which is suitable for real-time or near real-time application scenarios.

[0097] The advantage of zero-shot learning is that it can effectively handle unseen categories by simply comparing new image and text features with existing categories without retraining the model.

[0098] Through comparative learning (contrastive learning), the model can better capture the feature differences between different categories, thereby improving the generalization ability of the model and adapting to more varied data inputs.

[0099] By classifying the splicing results and using cosine distance calculation and screening the results that meet the requirements, the classification task of multimodal data can be completed efficiently and flexibly. This method not only improves the accuracy of classification, but also has significant advantages in adaptability to new categories and ease of implementation. It is an effective strategy for processing complex data.

[0100] In addition, since the corresponding content can be stored in advance to enable the model to be updated, when a new category appears, a corresponding feature vector needs to be generated for it. This can be done by extracting image and text features from an existing dataset, or by training a feature extraction model with new data. A dynamically updateable feature library can also be maintained to store the feature vectors of all current categories. Whenever a new category is added, its feature vector is added to the library.

[0101] During the classification process, the cosine similarity is calculated using the spliced ​​features and all the category features in the feature library. A threshold can be set to filter out categories with high similarity to the spliced ​​features. Based on the classification results and the performance of the model, feedback can be collected in actual applications to further adjust and update the category features. For example, if the classification accuracy of a certain category is low, you can consider retraining the feature vector of that category.

[0102] In other embodiments, an incremental learning algorithm is used to process newly added categories, so that the model can adapt to new category characteristics without retraining. For example, some neural network architectures allow fine-tuning of new categories. The threshold of cosine similarity is dynamically adjusted according to the current category distribution and model performance to adapt to the characteristics of the newly added category.

[0103] Allow users to manually specify or adjust features and labels for new categories to improve the accuracy of dynamic updates.

[0104] In real-time recommendation systems, image classification applications, or text analysis, this ability to dynamically change categories can improve the adaptability and accuracy of the model, enabling it to respond to environmental changes and user needs in a timely manner.

[0105] Through the above steps, dynamic changes and flexible management of categories can be achieved to adapt to ever-changing data and application scenarios.

[0106] S160: Output the classification result.

[0107] In this embodiment, the classification result is output to the terminal for display.

[0108] The method of this embodiment uses a convolutional neural network (CNN) to extract texture features of the image, and uses natural language processing (NLP) technology, such as BERT or Word2Vec, to extract semantic features of the text. Image features and text features are fused, and a comprehensive feature vector is generated using methods such as splicing or weighted averaging to enhance information expression. A model that supports zero-shot learning, such as a prototype-based classifier, is used to classify new categories by comparing the similarity of features with existing categories, avoiding repeated training. The model can be fine-tuned using samples of new categories to further improve the classification accuracy of documents with complex structures, especially when images and text appear together. Implement a user feedback mechanism to continuously optimize feature extraction and fusion methods based on classification results to improve generalization capabilities.

[0109] This method can effectively utilize image and text information in complex medical documents and insurance documents, thereby improving classification performance.

[0110] Take insurance classification as an example: an insurance company receives a claim application submitted by a customer, including photos (such as images of the accident scene) and documents (such as accident reports, medical invoices, etc.).

[0111] The accident scene images submitted by customers are cut into multiple small segments and features are extracted using an image encoder. This can help identify the specific circumstances of the accident (such as the characteristics of the damaged vehicle).

[0112] OCR technology is used to extract text information from accident reports and invoices. The text encoder further extracts key features such as accident time, location, related expenses, etc.

[0113] The extracted image features and text features are combined in a set order to form a comprehensive feature vector.

[0114] Using the zero-shot classification method, the splicing result is compared with the pre-stored categorized claim case features to determine which category the application belongs to (such as "car damage claim", "medical claim", etc.).

[0115] Specifically, by calculating the cosine distance between the concatenated result and the stored feature, the closest claim category is determined and the final classification result is output.

[0116] Through automated processing, manual review time can be reduced and the claims process can be accelerated. Accurate feature extraction and classification can reduce errors caused by manual operations and improve customer satisfaction. The classified data can be used for subsequent analysis and decision support to help companies optimize claims policies.

[0117] Through this approach, insurance companies are able to effectively process and classify large amounts of claims documents and improve operational efficiency.

[0118] Let’s take another example of medical classification: medical institutions upload medical records (such as scanned PDF files) and related medical images (such as X-rays, CT images) to the system. The medical images are cut into slices of different sizes (for example, each CT image can be divided into multiple slices), and the features of each slice are extracted using an image encoder (such as a convolutional neural network). The text information in the medical record document (such as patient name, diagnosis information, treatment plan) is recognized using OCR technology, and the text features are extracted using a text encoder (such as the BERT model). The image feature extraction results and the text feature extraction results are spliced ​​into a feature vector in a set order. Zero-shot classification is used to compare and learn the splicing results with the pre-stored classified medical document features (for example, medical records, test reports, imaging results, etc.) to determine the classification of the document. The system outputs the classification results, such as classifying the document as "medical record", "test report" or "imaging result".

[0119] Through this method, the hospital can automatically classify the documents it receives every day, greatly improving the efficiency of document management, reducing the time of manual processing, and also reducing the error rate. Doctors and nurses can find the information they need more quickly, thereby improving the quality of medical services.

[0120] The above document classification method obtains the document and image to be classified, and extracts features respectively to obtain the feature results of the image and text; splices the image features and text features to form a comprehensive feature vector; classifies the splicing results and outputs the final classification results; specifically, by obtaining and extracting the features of different document formats and images, splices the image features and text features to form a comprehensive feature vector. Then, the splicing results are classified to achieve fast and accurate document processing. Ultimately, this method significantly improves the classification efficiency of insurance claims materials and adapts to a variety of document formats.

[0121] Figure 6 is a schematic block diagram of a document classification device 300 provided in an embodiment of the present invention. Figure 6 As shown, corresponding to the above document classification method, the present invention also provides a document classification device 300. The document classification device 300 includes a unit for executing the above document classification method, and the device can be configured in a server. Figure 6 The document classification device 300 includes an acquisition unit 301, an image feature extraction unit 302, a text feature extraction unit 303, a splicing unit 304, a classification unit 305 and an output unit 306.

[0122] The acquisition unit 301 is used to acquire the document to be classified and the image to be classified; the image feature extraction unit 302 is used to extract features from the image to obtain an image feature extraction result; the text feature extraction unit 303 is used to extract text features from the document to be classified to obtain a text feature extraction result; the splicing unit 304 is used to splice the image feature extraction result and the text feature extraction result to obtain a splicing result; the classification unit 305 is used to classify the splicing result to obtain a classification result; and the output unit 306 is used to output the classification result.

[0123] In one embodiment, if Figure 7 As shown, the image feature extraction unit 302 includes a slicing subunit 3021 and a coding subunit 3022 .

[0124] The slicing subunit 3021 is used to slice the image into slices of different sizes and put the image and the slices into one batch;

[0125] The encoding subunit 3022 is used to extract features of the slice and the image using an image encoder to obtain an image feature extraction result.

[0126] In one embodiment, if Figure 8 As shown, the text feature extraction unit 303 includes a recognition subunit 3031 and an extraction subunit 3032 .

[0127] The recognition subunit 3031 is used to recognize text information of the document to be classified by using OCR technology; the extraction subunit 3032 is used to extract text features from the text information by using a text encoder to obtain a text feature extraction result.

[0128] In one embodiment, the splicing unit 304 is used to sequentially place the image feature extraction result and the text feature extraction result into an array according to a set order to obtain a splicing result.

[0129] In one embodiment, the classification unit 305 is used to classify the splicing result by using a Zero-shot classification method to obtain a classification result.

[0130] In one embodiment, the classification unit 305 is used to compare and learn the features formed by splicing the splicing result with pre-stored image feature extraction results with categories and text feature extraction results to determine the classification result.

[0131] In one embodiment, if Fig. 9As shown, the classification unit 305 includes a distance calculation subunit 3051 and a screening subunit 3052 .

[0132] The distance calculation subunit 3051 is used to calculate the cosine distance of the feature formed by splicing the splicing result with the pre-stored image feature extraction results with categories and text feature extraction results to obtain a calculation result; the screening subunit 3052 is used to screen out the calculation results that meet the requirements from the calculation results and determine the corresponding categories to obtain a classification result.

[0133] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned document classification device 300 and each unit can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and brevity of description, it will not be repeated here.

[0134] Each module in the document classification device 300 can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0135] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.10 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the server side of a document classification method.

[0136] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Fig.11As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the client side of a document classification method

[0137] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0138] Acquire a document to be classified and an image to be classified; perform feature extraction on the image to obtain an image feature extraction result; perform text feature extraction on the document to be classified to obtain a text feature extraction result; splice the image feature extraction result and the text feature extraction result to obtain a splicing result; classify the splicing result to obtain a classification result; and output the classification result.

[0139] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0140] Acquire a document to be classified and an image to be classified; perform feature extraction on the image to obtain an image feature extraction result; perform text feature extraction on the document to be classified to obtain a text feature extraction result; splice the image feature extraction result and the text feature extraction result to obtain a splicing result; classify the splicing result to obtain a classification result; and output the classification result.

[0141] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0142] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0143] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0144] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

[0145] The non-Company software tools or components appearing in the embodiments of this application are merely examples and do not represent actual use.

Claims

1. A document classification method, characterized in that: include: Obtaining documents to be classified and images to be classified; Performing feature extraction on the image to obtain an image feature extraction result; Performing text feature extraction on the document to be classified to obtain a text feature extraction result; Splicing the image feature extraction result and the text feature extraction result to obtain a splicing result; Classifying the splicing results to obtain classification results; The classification result is output.

2. The document classification method according to claim 1, characterized in that: The extracting features from the image to obtain an image feature extraction result includes: Slicing the image into slices of different sizes, and combining the image and the slices into a batch; An image encoder is used to extract features of the slice and the image to obtain an image feature extraction result.

3. The document classification method according to claim 1, characterized in that: The extracting text features of the document to be classified to obtain a text feature extraction result includes: Using OCR technology to identify text information of the document to be classified; The text information is subjected to a text encoder to extract text features to obtain a text feature extraction result.

4. The document classification method according to claim 1, characterized in that: The splicing of the image feature extraction result and the text feature extraction result to obtain a splicing result includes: The image feature extraction result and the text feature extraction result are sequentially placed into an array according to a set order to obtain a splicing result.

5. The document classification method according to claim 1, characterized in that: The classifying the splicing results to obtain classification results includes: The splicing result is classified using a Zero-shot classification method to obtain a classification result.

6. The document classification method according to claim 5, characterized in that: The splicing result is classified using a Zero-shot classification method to obtain a classification result, including: The splicing result is compared with the features formed by splicing the pre-stored image feature extraction results with categories and the text feature extraction results to determine the classification result.

7. The document classification method according to claim 6, characterized in that: The step of comparing and learning the features formed by combining the splicing result with the pre-stored image feature extraction results with categories and the text feature extraction results to determine the classification result includes: Calculate the cosine distance of the features formed by splicing the splicing result with the pre-stored image feature extraction result with categories and the text feature extraction result to obtain a calculation result; Filter out the calculation results that meet the requirements from the calculation results, and determine the corresponding categories to obtain classification results.

8. A document classification device, characterized in that: include: An acquisition unit, used for acquiring documents to be classified and images to be classified; An image feature extraction unit, used to extract features from the image to obtain an image feature extraction result; A text feature extraction unit, used to extract text features from the document to be classified to obtain a text feature extraction result; A splicing unit, used for splicing the image feature extraction result and the text feature extraction result to obtain a splicing result; A classification unit, used for classifying the splicing result to obtain a classification result; An output unit is used to output the classification result.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the document classification method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the document classification method according to any one of claims 1 to 7 are implemented.

Images