Document image key information extraction method and device based on knowledge distillation

Abstract

The invention relates to the technical field of computers, and discloses a document image key information extraction method and device based on knowledge distillation, and the method comprises the steps: carrying out the detection recognition and sequence labeling of a sample document image through employing an initial teacher model, and obtaining the position information of a sample text, the content of the sample text, and the sequence labeling information of the sample; performing feature extraction based on the three pieces of information to obtain a sample text feature, a sample visual feature and a sample layout feature; obtaining a sample text sequence label based on the three features; generating sample training loss based on the sample text position information, the sample text content and the sample text sequence tag, and training the initial teacher model to obtain a teacher model; and performing knowledge distillation on the teacher model to obtain a student model so as to perform key information extraction on the target document image. According to the method, the complexity and resource consumption of the model are remarkably reduced while the key information extraction precision is ensured, and the model landing requirement can be met.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and device for extracting key information from document images based on knowledge distillation. Background Art

[0002] Document image key information extraction refers to the automatic extraction of specific information that users are concerned about in a given document image. It is one of the key tasks in the field of intelligent document analysis. It realizes the conversion from unstructured data to structured data by automatically extracting key information from document images. It is widely used in many fields such as document image understanding and visual question answering.

[0003] With the development of deep learning technology, neural networks are usually used to extract key information from document images, but only text features are used to extract key information, resulting in poor extraction results. In addition, the Transformer, convolutional neural network, and graph convolution technology used in existing models during extraction have greatly increased the number of model parameters, thereby increasing the requirements for the deployment environment when the model is applied, making it difficult to implement the model. Summary of the invention

[0004] In view of this, the present invention provides a method and device for extracting key information from document images based on knowledge distillation to solve the problem that the model performance is poor and cannot meet the implementation needs.

[0005] In a first aspect, the present invention provides a method for extracting key information from a document image based on knowledge distillation, the method comprising:

[0006] For any sample document image, the initial teacher model is used to detect, identify and sequence-label the sample document image, and the sample text position information, sample text content and sample sequence labeling information of the sample document image are obtained;

[0007] Using the initial teacher model, feature extraction is performed based on the sample text position information, sample text content and sample sequence annotation information of the sample document image to obtain the sample text features, sample visual features and sample layout features of the sample document image;

[0008] Using the initial teacher model, a sample text sequence label is obtained based on the sample text features, sample visual features and sample layout features of the sample document image;

[0009] Based on the sample text position information, sample text content and sample text sequence label of the sample document image, a sample training loss is generated, and based on the sample training loss, an initial teacher model is trained to obtain a teacher model;

[0010] Based on the initial student model, the teacher model is subjected to knowledge distillation to obtain a student model, and key information of the target document image is extracted based on the student model.

[0011] The method for extracting key information from document images based on knowledge distillation provided by an embodiment of the present invention can accurately detect and identify the text position and content in the document image by adopting an initial teacher model, and can obtain more detailed text structure information through sequence labeling, and then perform feature extraction based on the above information to obtain sample text features, sample visual features and sample layout features of the sample document image, which can more comprehensively represent the content and structure of the document image. By integrating multiple features, the generated text sequence labels are more accurate, not only containing the text content, but also reflecting the structure and position relationship of the text in the document. The initial teacher model is trained based on the difference between the information extracted by the model and the actual information to obtain the teacher model, and then knowledge distillation is performed on the teacher model, which can inherit the key knowledge of the teacher model and maintain high performance. While ensuring the accuracy of key information extraction, it can significantly reduce the complexity and resource consumption of the model, and can meet the needs of model implementation.

[0012] In an optional implementation, for any sample document image, the initial teacher model is used to detect, identify and sequence-label the sample document image to obtain sample text position information, sample text content and sample sequence labeling information of the sample document image, including:

[0013] Using the preprocessing layer of the initial teacher model, based on the optical character recognition technology, the sample document image is detected and recognized to obtain the sample text position information and sample text content;

[0014] The sample text content is annotated based on the sequence labeling method to obtain sample sequence labeling information.

[0015] The document image key information extraction method based on knowledge distillation provided in an embodiment of the present invention can accurately detect and identify the text content and position in the document image by using optical character recognition technology, and can obtain the structural information of the text in the document through sequence labeling. It not only provides the text content, but also provides the context and structure of the text, providing rich information support for subsequent feature extraction and key information extraction.

[0016] In an optional implementation, an initial teacher model is used to extract features based on sample text content and sample sequence annotation information of the sample document image to obtain sample text features of the sample document image, including:

[0017] Using the base training layer of the initial teacher model, the sample text content and sample sequence annotation information of the sample document image are segmented to obtain multiple sample subwords;

[0018] For any sample subword, generate the word embedding vector and position embedding vector of the sample subword;

[0019] A multi-head self-attention mechanism is used to obtain the feature vector corresponding to the sample subword based on the word embedding vector and position embedding vector of the sample subword;

[0020] The feature vectors corresponding to all sample subwords are used as sample text features of the sample document image.

[0021] The document image key information extraction method based on knowledge distillation provided by the embodiment of the present invention obtains multiple sample sub-words by segmenting the sample text content and sample sequence annotation information of the sample document image. The word segmentation process can divide the text into smaller units to facilitate subsequent feature extraction, and generate a word embedding vector and a position embedding vector for each sample sub-word, so that the model can better understand the structure and order of the text. The multi-head self-attention mechanism can capture the long-term dependencies and local context information in the text, thereby generating a richer feature representation. The generated sample text features are richer and more detailed, thereby improving the robustness and generalization ability of the model.

[0022] In an optional implementation, an initial teacher model is used to perform feature extraction based on sample text position information of a sample document image to obtain sample visual features and sample layout features of the sample document image, including:

[0023] Using the base training layer of the initial teacher model, the sample document image is subjected to feature extraction based on the convolutional neural network to obtain the sample image features;

[0024] Normalize the position information of the sample text to obtain a normalized feature vector;

[0025] The normalized feature vector is divided into multiple windows, and the multiple windows are processed using a multi-head self-attention mechanism to obtain the first feature vector;

[0026] Shifting multiple windows of the first feature vector, and processing the shifted multiple windows using a multi-head self-attention mechanism to obtain a second feature vector;

[0027] Normalize the second eigenvector to obtain a third eigenvector;

[0028] Alignment is performed based on the third feature vector and the position information of the sample text to obtain the sample visual features and the sample layout features.

[0029] The document image key information extraction method based on knowledge distillation provided by the embodiment of the present invention extracts features of sample document images through convolutional neural networks. The generated sample image features contain the visual information of the document image, and the sample text position information is normalized so that the position information is within a standardized range, which is convenient for subsequent feature processing and alignment. By dividing the normalized feature vector into multiple windows and processing it with a multi-head self-attention mechanism, the local context information in the window can be effectively captured to generate a first feature vector, and then multiple windows of the first feature vector are shifted and processed again with a multi-head self-attention mechanism. The generated second feature vector further enriches the feature representation, and after normalization again, alignment is performed based on the third feature vector and the sample text position information to obtain the sample visual features and sample layout features. The alignment process ensures the consistency and correlation of the visual features and the layout features, and improves the comprehensive representation capability of the features.

[0030] In an optional implementation, an initial teacher model is used to obtain a sample text sequence label based on sample text features, sample visual features, and sample layout features of a sample document image, including:

[0031] The sample text features, sample visual features and sample layout features of the sample document image are spliced ​​to obtain the sample comprehensive features;

[0032] Using the base training layer of the initial teacher model, the context information is extracted from the comprehensive features of the samples based on the bidirectional long short-term memory network to obtain the fourth eigenvector;

[0033] Conditional random fields are used to generate sample text sequence labels based on the fourth eigenvector.

[0034] The method for extracting key information from document images based on knowledge distillation provided by an embodiment of the present invention obtains sample comprehensive features by splicing sample text features, sample visual features and sample layout features of sample document images. The multimodal feature fusion method can make full use of text, visual and layout information to generate a more comprehensive and rich feature representation. Context information is extracted from the sample comprehensive features through a bidirectional long short-term memory network. The generated fourth feature vector not only contains the features of a single word, but also contains information about its previous and following contexts, making the feature representation more complete and accurate. The conditional random field is used to generate sample text sequence labels to ensure that the generated label sequence has high coherence and accuracy.

[0035] In an optional implementation, the sample document image carries a text content tag, a text position tag, and a sequence annotation tag;

[0036] Based on the sample text position information, sample text content and sample text sequence label of the sample document image, a sample training loss is generated, including:

[0037] Determining a first training loss based on a difference between the sample text position information and the text position label;

[0038] Determining a second training loss based on the difference between the sample text content and the text content label;

[0039] Determine a third training loss based on the difference between the sample text sequence label and the sequence annotation label;

[0040] The sum of the first training loss, the second training loss, and the third training loss is taken as the sample training loss.

[0041] The method for extracting key information from document images based on knowledge distillation provided by an embodiment of the present invention determines a first training loss based on sample text position information and a text position label, which helps the model to more accurately learn and predict the position information of the text in the image; determines a second training loss based on sample text content and a text content label, which helps the model to more accurately learn and predict the text content; determines a third training loss based on sample text sequence labels and sequence annotation labels, which helps the model to more accurately learn and predict the labels of the text sequence; and uses the sum of the above three losses as the sample training loss. Training is performed based on this, thereby improving the overall performance of the model and improving the accuracy and robustness of the model.

[0042] In an optional implementation, knowledge distillation is performed on the teacher model based on the initial student model to obtain a student model, including:

[0043] Select the target distillation layer from the teacher model;

[0044] Input any sample document image to the teacher model and the initial student model, obtain the output difference between the teacher model and the initial student model at the target distillation layer, and generate the first distillation loss;

[0045] Extract the sample document image based on the initial student model to obtain the initial text position information, initial text content and initial text sequence label;

[0046] Generate a second distillation loss based on the sample document image, the initial text position information, the initial text content, and the initial text sequence label;

[0047] Based on the first distillation loss and the second distillation loss, the initial student model is trained to obtain a student model.

[0048] The method for extracting key information from document images based on knowledge distillation provided by an embodiment of the present invention selects a target distillation layer from a teacher model, obtains the output difference between the teacher model and the initial student model at each target distillation layer, generates a first distillation loss, helps the student model to be consistent with the teacher model on the key layer, and thus inherits the capabilities of the teacher model, extracts initial text position information, initial text content, and initial text sequence labels based on the initial student model, and generates a second distillation loss based on this information, helps the student model to perform closer to the teacher model on actual tasks, combines the two distillation losses to train the initial student model, and obtains the final student model, so that the student model can be optimized in multiple aspects to improve the overall performance, while significantly reducing the size and computational complexity of the model, and is suitable for deployment on resource-constrained devices.

[0049] In a second aspect, the present invention provides a document image key information extraction device based on knowledge distillation, the device comprising:

[0050] The first determination module is used to detect, identify and sequence-label any sample document image using an initial teacher model to obtain sample text position information, sample text content and sample sequence labeling information of the sample document image;

[0051] An extraction module, for using an initial teacher model to perform feature extraction based on sample text position information, sample text content and sample sequence annotation information of the sample document image, to obtain sample text features, sample visual features and sample layout features of the sample document image;

[0052] A second determination module is used to obtain a sample text sequence label based on sample text features, sample visual features and sample layout features of the sample document image using an initial teacher model;

[0053] A training module, used for generating a sample training loss based on sample text position information, sample text content and sample text sequence label of the sample document image, and training the initial teacher model based on the sample training loss to obtain a teacher model;

[0054] The distillation module is used to perform knowledge distillation on the teacher model based on the initial student model to obtain a student model, and extract key information from the target document image based on the student model.

[0055] In a third aspect, the present invention provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the knowledge distillation-based document image key information extraction method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0056] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the document image key information extraction method based on knowledge distillation of the above-mentioned first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0058] Figure 1 is a flow chart of a method for extracting key information from a document image based on knowledge distillation according to an embodiment of the present invention;

[0059] Figure 2 is a flow chart of an initial teacher model according to an embodiment of the present invention;

[0060] Figure 3 is a flow chart of knowledge distillation according to an embodiment of the present invention;

[0061] Figure 4 is a structural block diagram of a device for extracting key information from a document image based on knowledge distillation according to an embodiment of the present invention;

[0062] Figure 5 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0064] With the development of deep learning technology, neural networks are usually used to extract key information from document images, but only text features are used to extract key information, resulting in poor extraction effects. In addition, the Transformer, convolutional neural network, and graph convolution technology used in existing models during extraction greatly increase the number of model parameters, thereby increasing the requirements for the deployment environment when the model is applied, resulting in difficulties in model implementation. The method for extracting key information from document images based on knowledge distillation provided by an embodiment of the present invention trains the initial teacher model by comprehensively using text features, visual features, and layout features to obtain a teacher model that accurately extracts key information, thereby performing knowledge distillation on the teacher model, which can inherit the key knowledge of the teacher model and maintain high performance. It can significantly reduce the complexity and resource consumption of the model while ensuring the accuracy of key information extraction, and can meet the needs of model implementation.

[0065] According to an embodiment of the present invention, an embodiment of a method for extracting key information from document images based on knowledge distillation is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0066] In this embodiment, a method for extracting key information from a document image based on knowledge distillation is provided, which can be used in electronic devices. Figure 1 is a flowchart of a method for extracting key information from a document image based on knowledge distillation according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0067] Step S101, for any sample document image, the initial teacher model is used to detect, identify and sequence annotate the sample document image, and obtain the sample text position information, sample text content and sample sequence annotation information of the sample document image. Specifically, a document image refers to a photo or scan of a paper document or other written material stored in digital form, which usually contains information such as text, charts, tables, handwritten notes, etc. The key information of a document image refers to data or text that has important value or meaning in a specific context. This information is usually an indispensable part of decision-making, analysis, recording or other business processes. The specific content of the key information depends on the application scenario and requirements. The key information extraction of the document image realizes the conversion of the key information from unstructured to structured, which is convenient for directly operating, processing, analyzing and other operations on the key information. The embodiment of the present invention first uses the initial teacher model to perform preliminary extraction on any sample document image, and obtains the sample text position information, sample text content and sample sequence annotation information, which provides a basis for model training and is also the basis for key information extraction.

[0068] Step S102, using the initial teacher model, based on the sample text position information, sample text content and sample sequence annotation information of the sample document image, feature extraction is performed to obtain the sample text features, sample visual features and sample layout features of the sample document image. Specifically, using the initial teacher model, based on the above-mentioned preliminary extracted information, a deeper level of feature information extraction is performed to obtain sample text features, sample visual features and sample layout features. The sample text features are feature representations of the text content, the sample visual features are visual features of the sample document image, such as color, texture, etc., and the sample layout features are layout features of the text in the sample document image, such as line spacing, column spacing, etc. In the related art, text features are simply used to extract key information. This method not only ignores the visual information contained in the texture, color, size, etc. in the image, but also ignores the relative layout information between the text segments, resulting in poor extraction effect. The embodiment of the present invention integrates multiple feature training models to extract key information to improve extraction accuracy.

[0069] Step S103, using the initial teacher model, based on the sample text features, sample visual features and sample layout features of the sample document image, obtain a sample text sequence label. Specifically, the sample text features, sample visual features and sample layout features are integrated, and the initial teacher model is used for further processing and prediction to obtain a sample text sequence label, which is mainly used to represent the specific attributes or categories of each word in the document image in the context, and can accurately identify the key entities and information in the document image, thereby achieving efficient information extraction and processing.

[0070] Step S104, based on the sample text position information, sample text content and sample text sequence label of the sample document image, a sample training loss is generated, and based on the sample training loss, the initial teacher model is trained to obtain a teacher model. Specifically, the sample document image carries real text position information, text content and text sequence label, and the sample training loss is generated based on the real information and the information obtained by the initial teacher model, which can quantify the key information extraction effect of the initial teacher model, so as to train the initial teacher model based on the sample training loss, and optimize the model parameters of the initial teacher model using algorithms such as back propagation or gradient descent. After optimizing the parameters, the next sample document image is continued to be used for training, that is, steps S101 to S104 are repeated until the number of training times reaches a preset threshold or the model parameters converge, and a teacher model that can accurately extract key information is obtained.

[0071] Step S105, based on the initial student model, knowledge distillation is performed on the teacher model to obtain a student model, and key information is extracted from the target document image based on the student model. Specifically, although the teacher model can accurately extract key information from the document image, its model structure includes multiple neural networks, and the model complexity is relatively large, which is not friendly to the implementation of the model. Therefore, the embodiment of the present invention performs knowledge distillation on the teacher model through an initial student model to learn the teacher model's ability to accurately extract key information, while having a lightweight model structure to facilitate the implementation of the model.

[0072] The method for extracting key information from document images based on knowledge distillation provided by an embodiment of the present invention can accurately detect and identify the text position and content in the document image by adopting an initial teacher model, and can obtain more detailed text structure information through sequence labeling, and then perform feature extraction based on the above information to obtain sample text features, sample visual features and sample layout features of the sample document image, which can more comprehensively represent the content and structure of the document image. By integrating multiple features, the generated text sequence labels are more accurate, not only containing the text content, but also reflecting the structure and position relationship of the text in the document. The initial teacher model is trained based on the difference between the information extracted by the model and the actual information to obtain the teacher model, and then knowledge distillation is performed on the teacher model, which can inherit the key knowledge of the teacher model and maintain high performance. While ensuring the accuracy of key information extraction, it can significantly reduce the complexity and resource consumption of the model, and can meet the needs of model implementation.

[0073] In this embodiment, a method for extracting key information from a document image based on knowledge distillation is provided, which can be used in the above-mentioned electronic device. The method specifically includes the following steps:

[0074] Step S201: for any sample document image, the initial teacher model is used to detect, identify and sequence-label the sample document image to obtain sample text position information, sample text content and sample sequence labeling information of the sample document image.

[0075] Specifically, the above step S201 includes:

[0076] Step S2011, using the preprocessing layer of the initial teacher model, based on the optical character recognition technology, the sample document image is detected and recognized to obtain the sample text position information and the sample text content. Specifically, the preprocessing layer detects the coordinate position of each text area in the sample document image, that is, the sample text position information, by using the OCR (Optical Character Recognition) technology. At the same time, the text string, that is, the sample text content, can be recognized.

[0077] Step S2012, annotate the sample text content based on the sequence annotation method to obtain sample sequence annotation information. Specifically, taking the BIO sequence annotation method as an example, B (Beginning) represents the beginning of the entity, I (Inside) represents the inside of the entity, and O (Outside) represents a word that does not belong to any entity. The BIO sequence annotation method is used to annotate the text content in the sample document image to obtain sample sequence annotation information. Assuming that the sample text content is "Xiao Ming works in a technology company in area A as a technical manager", define the entity types that need to be annotated: PER (Person): name; LOC (Location): location; ORG (Organization): organization name; POS (Position): position. Xiao Ming: is a person's name, so it is marked as B-PER; in: does not belong to any entity, so it is marked as O; region A: is a place, so it is marked as B-LOC; of: does not belong to any entity, so it is marked as O; a: does not belong to any entity, so it is marked as O; technology: is part of the name of a company, so it is marked as B-ORG; company: is part of the company name, so it is marked as I-ORG; work: does not belong to any entity, so it is marked as O; serve: does not belong to any entity, so it is marked as O; technology: is part of a position, so it is marked as B-POS; manager: is part of a position, so it is marked as I-POS. All punctuation marks are marked as O. By performing sequence annotation, the key entities in the text content can be accurately identified, providing a basis for subsequent information extraction and processing.

[0078] Step S202, using the initial teacher model, based on the sample text position information, sample text content and sample sequence annotation information of the sample document image, feature extraction is performed to obtain sample text features, sample visual features and sample layout features of the sample document image.

[0079] In some optional implementations, the above step S202 uses an initial teacher model to perform feature extraction based on the sample text content and sample sequence annotation information of the sample document image to obtain sample text features of the sample document image, including:

[0080] Step S2021, using the base training layer of the initial teacher model, the sample text content and sample sequence annotation information of the sample document image are segmented to obtain multiple sample subwords. Specifically, the base training layer is composed of an encoder and a decoder. Based on the encoder, the input sample text content and sample sequence annotation information are preliminarily processed, such as segmentation, to obtain multiple sample subwords. Common segmentation methods include Byte Pair Encoding (BPE), WordPiece, etc.

[0081] Step S2022, for any sample subword, generate a word embedding vector and a position embedding vector of the sample subword. Specifically, each sample subword is mapped to a vector space of fixed dimension to obtain a word embedding vector. The word embedding vector can capture the semantic information of the sample subword. At the same time, in order to retain the order information of the sample subword in the sentence in which it is located, each sample subword will also have a position embedding vector. The position embedding vector helps the model understand the positional relationship of the sample subwords.

[0082] Step S2023, using a multi-head self-attention mechanism, based on the word embedding vector and position embedding vector of the sample sub-word, obtain the feature vector corresponding to the sample sub-word. Specifically, the multi-head self-attention mechanism captures different aspects of information through multiple different attention heads. Each attention head focuses on different sub-word relationships, thereby enhancing the expressiveness of the model. Through the multi-head self-attention mechanism, combined with the word embedding vector and the position embedding vector, a feature vector for each sample sub-word is generated, which not only contains the semantic information of the sample sub-word, but also takes into account the positional relationship between the sample sub-words.

[0083] Step S2024: The feature vectors corresponding to all sample subwords are used as sample text features of the sample document image. Specifically, the feature vectors of all sample subwords are combined to form a text feature representation of the entire sample document image, which serves as one of the bases for accurately extracting key information.

[0084] In some optional implementations, the above step S202 uses an initial teacher model to perform feature extraction based on sample text position information of the sample document image to obtain sample visual features and sample layout features of the sample document image, including:

[0085] Step S2025, using the base training layer of the initial teacher model, extracting features of the sample document image based on the convolutional neural network to obtain sample image features. Specifically, the convolutional neural network in the encoder can extract local features of the sample document image through convolution operations to capture the visual information of the sample document image as one of the bases for accurately extracting key information.

[0086] Step S2026, normalize the position information of the sample text to obtain a normalized feature vector. Specifically, the LN layer (Linear layer) in the encoder converts the position information of each text area in the sample document image, such as the upper left corner coordinates, the lower right corner coordinates, etc., into a fixed range to obtain a normalized feature vector. Normalization helps the model better process data of different scales.

[0087] Step S2027, divide the normalized feature vector into multiple windows, and use the multi-head self-attention mechanism to process the multiple windows to obtain the first feature vector. Specifically, the normalized feature vector is divided into multiple windows according to a certain size and step size. The encoder captures the relationship between different windows through multiple different attention heads of W-MS (Windows Multi-head Self Attention). Each attention head focuses on different window relationships, thereby enhancing the expressiveness of the model. The feature vector processed by the multi-head self-attention mechanism not only contains the local information of the window, but also takes into account the relationship between windows.

[0088] Step S2028, shift the multiple windows of the first feature vector, and use the multi-head self-attention mechanism to process the shifted multiple windows to obtain the second feature vector. Specifically, the multiple windows of the first feature vector are shifted according to a certain step size to achieve information fusion in order to capture more contextual information. The encoder uses SW-MSA (ShiftWindows Multi-head Self Attention) to achieve information interaction between pixels of each text during the window shifting process. Each text is located in a different position, and the layout information is indirectly constructed during the shifting process, thereby obtaining the second feature vector and capturing a wider range of contextual information.

[0089] Step S2029, normalize the second feature vector to obtain a third feature vector. Specifically, the encoder normalizes the second feature vector again through the LN layer to make its feature expression more stable.

[0090] Step S20210, aligning based on the third feature vector and the sample text position information to obtain sample visual features and sample layout features. Specifically, the encoder uses ROI Align (Region of Interest Align) to assume that for each sample text area in the sample text position information, a region of interest (RoI) is defined, and its bounding box is (x1, y1, x2, y2). Each RoI is divided into several smaller sub-regions. In each sub-region, the coordinates of the center point of the sub-region are calculated, and the eigenvalue of the center point is obtained from the third eigenvector using a bilinear interpolation method. The eigenvalues ​​of the center point of each sub-region are pooled to generate a final fixed-size feature vector, and the feature vectors of all sub-regions included in the sample text position information are used as sample layout features.

[0091] Step S203, using the initial teacher model, based on the sample text features, sample visual features and sample layout features of the sample document image, obtains a sample text sequence label.

[0092] Specifically, the above step S203 includes:

[0093] Step S2031, the sample text features, sample visual features and sample layout features of the sample document image are spliced ​​to obtain sample comprehensive features. Specifically, the three features of the sample document image are spliced ​​to form sample comprehensive features of multimodal information, which helps to extract key information more accurately.

[0094] Step S2032, using the base training layer of the initial teacher model, extracting context information from the comprehensive features of the sample based on the bidirectional long short-term memory network to obtain a fourth feature vector. Specifically, the decoder of the base training layer can capture context information from both the front and back directions through the bidirectional long short-term memory network (Bidirectional Long Short-Term Memory, BiLSTM), thereby generating a richer feature representation, that is, the fourth feature vector.

[0095] Step S2033, using conditional random fields, to generate sample text sequence labels based on the fourth feature vector. Specifically, the encoder generates sample text sequence labels from the fourth feature vector through conditional random fields (CRF). The fourth feature vector is input into the CRF layer, and the CRF layer generates the most likely label sequence, i.e., the sample text sequence label, based on the dependency between the feature vector and the label.

[0096] In some optional embodiments, the sample text sequence tags provide a structured representation for the key information in the sample document image, making information extraction more efficient and accurate. Through these tags, the key information in the document image, that is, the data or text in the document image, can be accurately identified and extracted.

[0097] In some optional embodiments, Figure 2 is a flow chart of an initial teacher model according to an embodiment of the present invention, such as Figure 2As shown, the sample document image is input into the initial teacher model. The preprocessing layer obtains the sample text position information and sample text content through optical character recognition. Then, the sample sequence annotation information is obtained by annotating through the sequence annotation method. Then, the above three information are input into the base training layer. The base training layer first performs word segmentation, then generates an embedding vector, and then obtains the sample text features through the multi-head self-attention mechanism. At the same time, the base training layer obtains the sample visual features through the convolutional neural network, and then aligns the sample layout features through the combination of multiple neural networks. The combination includes a linear layer, a window multi-head self-attention mechanism, a shifted window multi-head self-attention mechanism and a linear layer. Then, the base training layer is spliced ​​based on the above three features, and then the sample text sequence label is obtained through the processing of the bidirectional long short-term memory network and the conditional random field, so that the key information in the sample document image can be obtained based on the sample text sequence label.

[0098] Step S204, based on the sample text position information, sample text content and sample text sequence label of the sample document image, generate a sample training loss, and train the initial teacher model based on the sample training loss to obtain a teacher model, wherein the sample document image carries a text content label, a text position label and a sequence annotation label.

[0099] In some optional implementations, the above step S204 generates a sample training loss based on the sample text position information, the sample text content and the sample text sequence label of the sample document image, including:

[0100] Step S2041, determines the first training loss based on the difference between the sample text position information and the text position label. Specifically, the sample text position information refers to the specific position information of the text in the sample document image determined by the initial teacher model. For example, "work" is located at the (x2, y2) position of the sample document image. The text position label is the actual position of the text carried by the sample document image in the sample document image. For example, "work" is located at the (x1, y1) position of the sample document image. By calculating the difference between the sample text position information and the text position label, such as by quantifying the difference through the mean square error loss, a first training loss is obtained to evaluate the accuracy of the model in predicting text positions.

[0101] Step S2042, based on the difference between the sample text content and the text content label, determine the second training loss. Specifically, the sample text content refers to the text content in the sample document image determined by the initial teacher model. The text content label is the actual content of the text carried by the sample document image in the sample document image. The difference between the sample text content and the text content label is calculated, and the difference is quantified, for example, by cross entropy loss to obtain a second training loss, which is used to evaluate the accuracy of the model in predicting text content.

[0102] Step S2043, based on the difference between the sample text sequence label and the sequence annotation label, determine the third training loss. Specifically, the sample text sequence label refers to the label of the text sequence determined by the initial teacher model, which represents the category of each word in the text content. The sequence annotation label is the true classification of the text sequence carried by the sample document image. The difference between the sample text sequence label and the sequence annotation label is calculated, and the difference is quantified by CRF loss, for example, to obtain the third training loss, which is used to evaluate the accuracy of the model in predicting sequence labels.

[0103] Step S2044, the sum of the first training loss, the second training loss and the third training loss is used as the sample training loss. Specifically, the three loss values ​​are added together to obtain a total loss value, which is used to evaluate the overall performance of the model when processing the current sample. By comprehensively considering losses in multiple aspects, it is ensured that the model can achieve good performance in different dimensions, thereby improving the overall training effect.

[0104] Step S205, based on the initial student model, knowledge distillation is performed on the teacher model to obtain a student model, and key information is extracted from the target document image based on the student model.

[0105] In some optional implementations, the above step S205 performs knowledge distillation on the teacher model based on the initial student model to obtain a student model, including:

[0106] Step S2051, select a target distillation layer from the teacher model. Specifically, the teacher model is a model that has been trained and has better performance, which is used to guide the learning of the student model. Some target distillation layers are selected from the teacher model, and the outputs of these layers will be used to guide the learning of the student model. The target distillation layer can usually provide rich feature information. The embodiment of the present invention takes the LN layer-W-MS layer-SW-MSA layer-LN layer in step S202 as an overall structure, and takes the overall structure as a target distillation layer. At the same time, the ROI Align layer in step S20210 is taken as a target distillation layer, and the BiLSTM layer-CRF layer in step S203 is taken as an overall structure, and takes the overall structure as a target distillation layer. Optionally, the selection of the above-mentioned target distillation layer is only an example and is not limited.

[0107] Step S2052, input any sample document image into the teacher model and the initial student model, obtain the output difference between the teacher model and the initial student model in the target distillation layer, and generate the first distillation loss. Specifically, the model structure of the initial student model is consistent with the teacher model, but the number of layers of each network is smaller than that of the teacher model. Select any sample document image, input the sample document image into the teacher model and the initial student model respectively, obtain the output results of the teacher model and the initial student model respectively when passing through each target distillation layer, determine the difference between the output results, and use any loss function to calculate the loss. The sum of the losses obtained from the three target distillation layers is used as the first distillation loss, which is used to evaluate the difference in model performance between the initial student model and the teacher model.

[0108] Step S2053: extract the sample document image based on the initial student model to obtain initial text position information, initial text content and initial text sequence labels. Specifically, the sample document image is processed by the initial student model to obtain initial text position information, initial text content and initial text sequence labels.

[0109] Step S2054, based on the sample document image, the initial text position information, the initial text content and the initial text sequence label, a second distillation loss is generated. Specifically, since the sample document image carries the real text content label, the text position label and the sequence annotation label. Referring to the above steps S2041 to S2043, the loss between the output of the initial student model and the true value is calculated respectively to obtain the second distillation loss, which is used to evaluate the performance of the initial student model in extracting key information.

[0110] Step S2055, based on the first distillation loss and the second distillation loss, the initial student model is trained to obtain a student model. Specifically, the first distillation loss and the second distillation loss are added to obtain a comprehensive loss value, and the initial student model is trained using the comprehensive loss value to adjust the model parameters. After each adjustment, steps S2052 to S2055 are repeated to achieve better results in both feature learning and task performance, thereby obtaining the final student model. The student model not only learns the ability of the teacher model to accurately extract key information, but also the model structure is lighter than the teacher model, which can save computing resources and computing time, and is conducive to the implementation of the model.

[0111] In some optional embodiments, Figure 3 is a flow chart of knowledge distillation according to an embodiment of the present invention. Figure 3As shown, the sample document image is input into the teacher model and the initial student model. After each target distillation layer, the difference between the output results of the teacher model and the initial student model is determined, and the loss is calculated. The losses corresponding to the three target distillation layers are added together as the first distillation loss. Based on the difference between the initial text sequence label obtained by the initial student model and the sequence annotation label carried by the sample document image, the second distillation loss is calculated. The initial student model is trained based on the sum of the two distillation losses to obtain the student model.

[0112] The method for extracting key information from document images based on knowledge distillation provided by an embodiment of the present invention can accurately detect and identify the text position and content in the document image by adopting an initial teacher model, and can obtain more detailed text structure information through sequence labeling, and then perform feature extraction based on the above information to obtain sample text features, sample visual features and sample layout features of the sample document image, which can more comprehensively represent the content and structure of the document image. By integrating multiple features, the generated text sequence labels are more accurate, not only containing the text content, but also reflecting the structure and position relationship of the text in the document. The initial teacher model is trained based on the difference between the information extracted by the model and the actual information to obtain the teacher model, and then knowledge distillation is performed on the teacher model, which can inherit the key knowledge of the teacher model and maintain high performance. While ensuring the accuracy of key information extraction, it can significantly reduce the complexity and resource consumption of the model, and can meet the needs of model implementation.

[0113] In this embodiment, a document image key information extraction device based on knowledge distillation is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0114] This embodiment provides a document image key information extraction device based on knowledge distillation. Figure 4 As shown, including:

[0115] The first determination module 401 is used to detect, identify and sequence-label any sample document image using an initial teacher model to obtain sample text position information, sample text content and sample sequence labeling information of the sample document image.

[0116] The extraction module 402 is used to use the initial teacher model to perform feature extraction based on the sample text position information, sample text content and sample sequence annotation information of the sample document image to obtain the sample text features, sample visual features and sample layout features of the sample document image.

[0117] The second determination module 403 is used to obtain a sample text sequence label based on the sample text features, sample visual features and sample layout features of the sample document image using the initial teacher model.

[0118] The training module 404 is used to generate a sample training loss based on the sample text position information, sample text content and sample text sequence label of the sample document image, and train the initial teacher model based on the sample training loss to obtain a teacher model.

[0119] The distillation module 405 is used to perform knowledge distillation on the teacher model based on the initial student model to obtain a student model, and extract key information from the target document image based on the student model.

[0120] In some optional implementations, the first determining module 401 includes:

[0121] The recognition unit is used to detect and recognize the sample document image using the preprocessing layer of the initial teacher model based on the optical character recognition technology to obtain the sample text position information and the sample text content.

[0122] The labeling unit is used to label the sample text content based on the sequence labeling method to obtain sample sequence labeling information.

[0123] In some optional implementations, the extraction module 402 includes:

[0124] The word segmentation unit is used to use the base training layer of the initial teacher model to segment the sample text content and sample sequence annotation information of the sample document image to obtain multiple sample sub-words.

[0125] The first generating unit is used to generate a word embedding vector and a position embedding vector of any sample sub-word.

[0126] The first determination unit is used to adopt a multi-head self-attention mechanism to obtain a feature vector corresponding to the sample subword based on the word embedding vector and the position embedding vector of the sample subword.

[0127] The second determining unit is used to use the feature vectors corresponding to all sample subwords as sample text features of the sample document image.

[0128] In some optional implementations, the extraction module 402 includes:

[0129] The first extraction unit is used to use the base training layer of the initial teacher model to extract features of the sample document image based on the convolutional neural network to obtain sample image features.

[0130] The first normalization unit is used to normalize the position information of the sample text to obtain a normalized feature vector.

[0131] The partitioning unit is used to partition the normalized feature vector into multiple windows, and process the multiple windows using a multi-head self-attention mechanism to obtain a first feature vector.

[0132] The shift unit is used to shift the multiple windows of the first feature vector, and use a multi-head self-attention mechanism to process the shifted multiple windows to obtain a second feature vector.

[0133] The second normalization unit is used to normalize the second eigenvector to obtain a third eigenvector.

[0134] The alignment unit is used to perform alignment based on the third feature vector and the sample text position information to obtain the sample visual feature and the sample layout feature.

[0135] In some optional implementations, the second determining module 403 includes:

[0136] The splicing unit is used to splice the sample text features, sample visual features and sample layout features of the sample document image to obtain the sample comprehensive features.

[0137] The third determination unit is used to adopt the base training layer of the initial teacher model to extract context information from the comprehensive features of the sample based on a bidirectional long short-term memory network to obtain a fourth feature vector.

[0138] The second generating unit is used to generate a sample text sequence label based on the fourth feature vector by using a conditional random field.

[0139] In some optional implementations, the sample document image carries a text content label, a text position label, and a sequence annotation label; the training module 404 includes:

[0140] The fourth determining unit is used to determine the first training loss based on the difference between the sample text position information and the text position label.

[0141] The fifth determining unit is used to determine the second training loss based on the difference between the sample text content and the text content label.

[0142] The sixth determining unit is used to determine the third training loss based on the difference between the sample text sequence label and the sequence annotation label.

[0143] The seventh determining unit is used to use the sum of the first training loss, the second training loss and the third training loss as the sample training loss.

[0144] In some optional embodiments, the distillation module 405 includes:

[0145] The selection unit is used to select the target distillation layer from the teacher model.

[0146] The third generating unit is used to input any sample document image into the teacher model and the initial student model, obtain the output difference between the teacher model and the initial student model at the target distillation layer, and generate a first distillation loss.

[0147] The second extraction unit is used to extract the sample document image based on the initial student model to obtain initial text position information, initial text content and initial text sequence label.

[0148] The fourth generating unit is used to generate a second distillation loss based on the sample document image, the initial text position information, the initial text content and the initial text sequence label.

[0149] The training unit is used to train the initial student model based on the first distillation loss and the second distillation loss to obtain the student model.

[0150] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0151] The document image key information extraction device based on knowledge distillation in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0152] The embodiment of the present invention also provides a computer device having the above Figure 4 The document image key information extraction device based on knowledge distillation is shown.

[0153] See also Figure 5 , Figure 5 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 5As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 10 is taken as an example.

[0154] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0155] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0156] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0157] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0158] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0159] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0160] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0161] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0162] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for extracting key information from document images based on knowledge distillation, characterized in that: The method comprises: For any sample document image, the initial teacher model is used to detect, identify and sequence-label the sample document image to obtain sample text position information, sample text content and sample sequence labeling information of the sample document image; Using the initial teacher model, based on the sample text position information, sample text content and sample sequence annotation information of the sample document image, feature extraction is performed to obtain sample text features, sample visual features and sample layout features of the sample document image; Using the initial teacher model, based on sample text features, sample visual features and sample layout features of the sample document image, a sample text sequence label is obtained; Based on the sample text position information, sample text content and sample text sequence label of the sample document image, a sample training loss is generated, and based on the sample training loss, the initial teacher model is trained to obtain a teacher model; Based on the initial student model, knowledge distillation is performed on the teacher model to obtain a student model, and key information is extracted from the target document image based on the student model.

2. The method according to claim 1, characterized in that For any sample document image, the initial teacher model is used to detect, identify and sequence-label the sample document image to obtain sample text position information, sample text content and sample sequence labeling information of the sample document image, including: Using the preprocessing layer of the initial teacher model, based on optical character recognition technology, the sample document image is detected and recognized to obtain the sample text position information and the sample text content; The sample text content is annotated based on a sequence annotation method to obtain the sample sequence annotation information.

3. The method according to claim 1, characterized in that The initial teacher model is used to extract features based on the sample text content and sample sequence annotation information of the sample document image to obtain sample text features of the sample document image, including: Using the base training layer of the initial teacher model, segmenting the sample text content and sample sequence annotation information of the sample document image to obtain a plurality of sample subwords; For any sample subword, generate a word embedding vector and a position embedding vector of the sample subword; Using a multi-head self-attention mechanism, based on the word embedding vector and position embedding vector of the sample subword, a feature vector corresponding to the sample subword is obtained; The feature vectors corresponding to all sample subwords are used as sample text features of the sample document image.

4. The method according to claim 1, characterized in that: The initial teacher model is used to extract features based on the sample text position information of the sample document image to obtain sample visual features and sample layout features of the sample document image, including: Using the base training layer of the initial teacher model, extracting features of the sample document image based on a convolutional neural network to obtain sample image features; Normalizing the position information of the sample text to obtain a normalized feature vector; Dividing the normalized feature vector into multiple windows, and processing the multiple windows using a multi-head self-attention mechanism to obtain a first feature vector; Shifting the multiple windows of the first feature vector, and processing the shifted multiple windows using a multi-head self-attention mechanism to obtain a second feature vector; Normalizing the second eigenvector to obtain a third eigenvector; The sample visual feature and the sample layout feature are obtained by aligning the third feature vector and the sample text position information.

5. The method according to claim 1, characterized in that: The initial teacher model is used to obtain a sample text sequence label based on sample text features, sample visual features and sample layout features of the sample document image, including: splicing the sample text features, sample visual features and sample layout features of the sample document image to obtain sample comprehensive features; Using the base training layer of the initial teacher model, extracting context information from the comprehensive features of the sample based on a bidirectional long short-term memory network to obtain a fourth feature vector; A conditional random field is used to generate the sample text sequence label based on the fourth feature vector.

6. The method according to claim 1, characterized in that The sample document image carries a text content label, a text position label and a sequence annotation label; The generating of the sample training loss based on the sample text position information, the sample text content and the sample text sequence label of the sample document image comprises: Determining a first training loss based on a difference between the sample text position information and the text position label; Determining a second training loss based on a difference between the sample text content and the text content label; Determining a third training loss based on a difference between the sample text sequence label and the sequence annotation label; The sum of the first training loss, the second training loss and the third training loss is used as the sample training loss.

7. The method according to claim 1, characterized in that Based on the initial student model, the teacher model is subjected to knowledge distillation to obtain a student model, including: selecting a target distillation layer from the teacher model; Input any sample document image to the teacher model and the initial student model, obtain the output difference between the teacher model and the initial student model at the target distillation layer, and generate a first distillation loss; Extracting the sample document image based on the initial student model to obtain initial text position information, initial text content and initial text sequence label; generating a second distillation loss based on the sample document image, the initial text position information, the initial text content, and the initial text sequence label; The initial student model is trained based on the first distillation loss and the second distillation loss to obtain the student model.

8. A document image key information extraction device based on knowledge distillation, characterized in that: The device comprises: The first determination module is used to detect, identify and sequence-label any sample document image using an initial teacher model to obtain sample text position information, sample text content and sample sequence labeling information of the sample document image; An extraction module, used to use the initial teacher model to perform feature extraction based on the sample text position information, sample text content and sample sequence annotation information of the sample document image to obtain sample text features, sample visual features and sample layout features of the sample document image; A second determination module is used to obtain a sample text sequence label based on sample text features, sample visual features and sample layout features of the sample document image using the initial teacher model; A training module, used to generate a sample training loss based on the sample text position information, the sample text content and the sample text sequence label of the sample document image, and train the initial teacher model based on the sample training loss to obtain a teacher model; The distillation module is used to perform knowledge distillation on the teacher model based on the initial student model to obtain a student model, and extract key information from the target document image based on the student model.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the document image key information extraction method based on knowledge distillation as described in any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the document image key information extraction method based on knowledge distillation as described in any one of claims 1 to 7.

Images