Text structure recognition method, device and equipment and storage medium

Abstract

The application discloses a text structure identification method, device and equipment and a storage medium. The method comprises the following steps: extracting features of to-be-processed file information to obtain to-be-processed file feature information; classifying the to-be-processed file feature information to obtain attribute values and attribute names of the to-be-processed file feature information and determine distance information of the attribute values and the attribute names; and identifying the to-be-processed file information based on the distance information through a preset information structure identification model. According to the application, the extracted feature information is classified to obtain attribute values and attribute names of to-be-processed files, and distance information between the attribute values and the attribute names is determined. The distance information is input into the preset information structure identification model as a parameter to identify the to-be-processed file, so that the preset text structure model does not need to be redesigned for different structured texts when facing different structured texts, and the efficiency of text structure identification can be improved.

Description

Technical Field

[0001] This invention relates to the field of information recognition technology, and in particular to a text structure recognition method, apparatus, device, and storage medium. Background Technology

[0002] Existing information structure recognition solutions all share a common pain point: when faced with structured text images in different layout formats, such as invoices, receipts, certificates, and documents, the layout format changes, requiring the corresponding information structure recognition model to be retrained or a corresponding template to be created, resulting in low efficiency in recognizing text information.

[0003] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main objective of this invention is to provide a text structure recognition method, apparatus, device, and storage medium, aiming to solve the technical problem of low recognition efficiency caused by the need to redesign the information structure model for different structured texts.

[0005] To achieve the above objectives, the present invention provides a text structure recognition method, which includes the following steps:

[0006] Feature extraction is performed on the information of the file to be processed to obtain the feature information of the file to be processed;

[0007] The feature information of the file to be processed is classified to obtain the attribute values ​​and attribute names of the feature information of the file to be processed, and the distance information between the attribute values ​​and the attribute names is determined.

[0008] Based on the distance information, the information of the file to be processed is identified using a preset information structured recognition model.

[0009] Optionally, the feature information of the file to be processed includes: word vector features;

[0010] The step of extracting features from the file information to be processed to obtain the file feature information includes:

[0011] Text information is extracted from the file information to be processed based on the character recognition model and the text detection model;

[0012] The semantic features of the text information are extracted according to a preset word vector model to obtain the word vector features.

[0013] Optionally, the distance information includes the relative distance between the attribute value and the attribute name;

[0014] The step of classifying the feature information of the file to be processed, obtaining the attribute values ​​and attribute names of the feature information of the file to be processed, and determining the distance information between the attribute values ​​and the attribute names includes:

[0015] The word vector features are classified according to the attribute classification network to obtain the attribute values ​​and attribute names of the feature information of the file to be processed;

[0016] Determine the relative distance between the attribute value and the attribute name.

[0017] Optionally, the distance information includes relative coordinate distance, and the text information includes text region coordinates;

[0018] The step of determining the distance information between the attribute value and the attribute name includes:

[0019] Determine the attribute value coordinates and attribute name coordinates based on the text region coordinates;

[0020] Determine the relative distance between the coordinates of the attribute value and the coordinates of the attribute name.

[0021] Optionally, the distance information includes the relative distance between word vectors;

[0022] The step of determining the distance information between the attribute value and the attribute name includes:

[0023] The word vectors for the attribute values ​​and the attribute names are determined based on the word vector features.

[0024] Determine the relative distance between the word vectors of the attribute value and the word vector of the attribute name.

[0025] Optionally, before the step of extracting semantic features from the text information according to a preset word vector model to obtain the word vector features, the method further includes:

[0026] A text knowledge base is constructed based on the text information, attribute names, and attribute values ​​of each structure, as well as the word vector features corresponding to the word vector features and the word segmentation results of the word vector features.

[0027] The text data in the text knowledge base is used as input data to train the word vector model, thereby obtaining a preset word vector model.

[0028] Optionally, the step of identifying the file information to be processed based on the distance information using a preset information structure recognition model includes:

[0029] The relative distances between the attribute values ​​and attribute names, the relative distances of the coordinates, and the relative distances of the word vectors are ranked according to the matching degree based on the preset information structure recognition model.

[0030] The matching degree ranking result is optimized according to the loss function to identify the information of the file to be processed. The loss function in the preset information structured recognition model is a loss function optimized according to the matching degree error between the attribute value and the attribute name.

[0031] The optimized loss function is as follows:

[0032]

[0033] In the formula, y (i) This indicates the actual tag to which the current attribute value belongs in the i-th attribute region. Z represents the predicted label of the current attribute value estimated by the model, belonging to the i-th attribute region, where m represents the total number of attribute regions (m). (i) This represents the product of the weight coefficient vector and the input features.

[0034] In addition, to achieve the above objectives, the present invention also proposes a text structure recognition device, which includes: a feature extraction module, a distance determination module, and an information recognition module;

[0035] The feature extraction module is used to extract features from the file information to be processed, and obtain the feature information of the file to be processed.

[0036] The distance determination module is used to classify the feature information of the file to be processed, obtain the attribute value and attribute name of the feature information of the file to be processed, and determine the distance information between the attribute value and the attribute name;

[0037] The information recognition module identifies the information of the file to be processed based on the distance information and through a preset information structure recognition model.

[0038] Furthermore, to achieve the above objectives, the present invention also proposes a text structure recognition device, which includes a memory, a processor, and a text structure recognition program stored in the memory and capable of running on the processor. The text structure recognition program is configured to implement the text structure recognition method as described above.

[0039] Furthermore, to achieve the above objectives, the present invention also proposes a storage medium storing a text structure recognition program, which, when executed by a processor, implements the text structure recognition method as described above.

[0040] This invention discloses a method, apparatus, device, and storage medium for text structure recognition. The method includes: extracting features from the information of a file to be processed to obtain feature information of the file; classifying the feature information of the file to be processed to obtain attribute values ​​and attribute names, and determining the distance information between the attribute values ​​and attribute names; and recognizing the information of the file to be processed based on the distance information using a preset information structure recognition model. This invention classifies the extracted feature information to obtain the attribute values ​​and attribute names of the file to be processed, determines the distance information between the attribute values ​​and attribute names, and inputs the distance information as a parameter into a preset information structure recognition model to perform structure recognition of the file to be processed. Therefore, when faced with different structured texts, it is not necessary to redesign the preset text structure recognition model for different structured texts; only the distance information between the attribute values ​​and attribute names needs to be determined and input into the preset text structure recognition model, thereby improving the efficiency of text structure recognition. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the structure of a text structure recognition device in the hardware operating environment involved in the embodiments of the present invention;

[0042] Figure 2 This is a flowchart illustrating the first embodiment of the text structure recognition method of the present invention;

[0043] Figure 3 This is a flowchart illustrating the second embodiment of the text structure recognition method of the present invention;

[0044] Figure 4 This is a flowchart illustrating the third embodiment of the text structure recognition method of the present invention;

[0045] Figure 5 This is a flowchart illustrating the fourth embodiment of the text structure recognition method of the present invention;

[0046] Figure 6 This is a semantic feature extraction diagram of an embodiment of the text structure recognition method of the present invention;

[0047] Figure 7 This is a diagram illustrating the construction of a preset word vector model in an embodiment of the text structure recognition method of the present invention.

[0048] Figure 8 This is a flowchart illustrating the overall text structure recognition process of an embodiment of the text structure recognition method of the present invention.

[0049] Figure 9 This is a flowchart of the relative distance processing in an embodiment of the text structure recognition method of the present invention;

[0050] Figure 10This is a flowchart illustrating the coordinate relative distance processing of an embodiment of the text structure recognition method of the present invention;

[0051] Figure 11 This is a structural block diagram of the first embodiment of the text structure recognition device of the present invention.

[0052] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0053] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0054] Reference Figure 1 , Figure 1 This is a schematic diagram of the text structure recognition device in the hardware operating environment involved in the embodiments of the present invention.

[0055] like Figure 1 As shown, the text structure recognition device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen, and optionally, it may also include a standard wired interface or a wireless interface. In this invention, the wired interface of the user interface 1003 may be a USB interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (RAM) or a non-volatile memory (NVM), such as a disk storage device. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0056] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the text structure recognition device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0057] like Figure 1 As shown, the memory 1005, which is identified as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a text structure recognition program.

[0058] exist Figure 1In the text structure recognition device shown, the network interface 1004 is mainly used to connect to the backend server and communicate data with the backend server; the user interface 1003 is mainly used to connect to the user equipment; the text structure recognition device calls the text structure recognition program stored in the memory 1005 through the processor 1001 and executes the text structure recognition method provided in the embodiment of the present invention.

[0059] Based on the above hardware structure, an embodiment of the text structure recognition method of the present invention is proposed.

[0060] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the text structure recognition method of the present invention, which presents the first embodiment of the text structure recognition method of the present invention.

[0061] Step S10: Extract features from the file information to be processed to obtain the feature information of the file to be processed.

[0062] It should be noted that the execution subject of this embodiment may be a computer software service device with data processing, network communication and program running functions, such as a text structure recognition device, or other electronic devices that can achieve the same or similar functions. This embodiment does not limit this.

[0063] Understandably, before feature extraction from the document information, preprocessing is necessary. This involves using two main models: image calibration and image denoising, to perform denoising and correction of distortions such as tilt. For example, if the input document image is a low-quality mobile phone photo, image denoising and distortion correction based on transmission changes are required. If the input document image is a high-resolution scan, subsequent steps can proceed directly.

[0064] It is understandable that the obtained feature information of the file to be processed can be text content, such as text and images, or text region coordinate information and word vector features. Word vector features can be obtained based on the text content. For example, the text content extracted from a bank receipt can be the account holder and the corresponding name of the account holder. The extracted text region coordinate information is the coordinate information of the account holder and the coordinate information of the corresponding name of the account holder.

[0065] Furthermore, in order to improve the accuracy of text structured information recognition, step S10 of this embodiment may include:

[0066] Text information is extracted from the file information to be processed based on the character recognition model and the text detection model;

[0067] The semantic features of the text information are extracted according to a preset word vector model to obtain the word vector features.

[0068] It should be noted that text region coordinates can be extracted from the file information to be processed based on a text recognition model, which can be a CRNN+CTC model; text content can be extracted from the file information to be processed based on a text detection model, which can be a DB model or an EAST model, etc.

[0069] It should be noted that this embodiment replaces image features with semantic features because semantic features do not change with the layout format, thus ensuring that the model does not require customized development.

[0070] It is understandable that semantic features can be used instead of image features. As long as two attribute values ​​are semantically the same or similar, their corresponding sub-vector features will be similar, and there is no need to customize the training model.

[0071] For ease of understanding, please refer to Figure 6 To explain, Figure 6 This is a semantic feature extraction graph. The graph extracts the same information from different layouts; that is, layout A and layout B have the same features. Semantic features of layouts A and B are extracted using a word vector model, resulting in word vector features A and B. Since word vector features A and B are identical, the results of identifying word vector feature A using information structure recognition model A and word vector feature B using information structure recognition model B are the same. Furthermore, information structure recognition models A and B are also identical.

[0072] Furthermore, in order to improve the accuracy of recognizing structured information in text, before the step of extracting semantic features from the text information according to a preset word vector model to obtain the word vector features, the method further includes:

[0073] A text knowledge base is constructed based on the text information, attribute names, and attribute values ​​of each structure, as well as the word vector features corresponding to the word vector features and the word segmentation results of the word vector features.

[0074] The text data in the text knowledge base is used as input data to train the word vector model, thereby obtaining a preset word vector model.

[0075] Understandably, building a text knowledge base allows for the input of text information from various structured text images, enabling the text knowledge base to cover various structured text images such as invoices, receipts, and documents, including machine-printed invoices, bank receipts, purchase orders, budget documents, and contract documents. It also allows for the input of paired attribute names and attribute values ​​with contextual relationships as objects in the knowledge base. Furthermore, it allows for the segmentation of paired text information using word segmentation tools, with the segmentation results then being input into the text knowledge base.

[0076] It should be noted that the construction of the text knowledge base is based on using the knowledge of the text knowledge base as the input corpus for training the word vector model, so as to minimize the word vector distance between the attribute name and its corresponding attribute value, as well as the word vector distance between attribute names of the same type.

[0077] For ease of understanding, please refer to Figure 7 To explain, Figure 7 To construct the pre-defined word vector model, the graph includes various text images, such as machine-printed invoices, bank receipts, purchase orders, budget documents, and contract documents, as well as texts with extended types, such as visa records, site survey records, and design change orders, to build a structured text knowledge base. The structured text knowledge base inputs text data into the word vector deep network for training and outputs a table of professional vitality values.

[0078] Step S20: Classify the feature information of the file to be processed, obtain the attribute values ​​and attribute names of the feature information of the file to be processed, and determine the distance information between the attribute values ​​and the attribute names.

[0079] It should be noted that existing information structure recognition models and technologies use image data and labeled absolute position coordinates as input data. Regardless of the image data or absolute position coordinates, both types of data will change for different text image types. Correspondingly, information structure recognition models for different text images need to be retrained and developed. Therefore, this embodiment uses the distance information between attribute values ​​and attribute names as input to the information structure recognition model, thus eliminating the need to customize and retrain the model when the text structure changes.

[0080] It should be noted that the classification of the feature information of the file to be processed can be based on the attribute classification network to classify the attribute values ​​and attribute names of the feature information of the file to be processed. Classification models such as softmax and CRNN can be used to classify the feature information of the file to be processed.

[0081] It should be noted that by detecting and distinguishing attribute values ​​and attribute names, the relative distance between attribute values ​​and attribute names can be calculated from various feature dimensions as the relative features of the structured recognition model, replacing the absolute features of attribute values ​​used only in existing technologies (including: image features of attribute values ​​and absolute position coordinates of attribute values ​​in the image). The relative distances of various features will not change with the layout format, ensuring that the model does not require customized development.

[0082] Step S30: Based on the distance information, the information of the file to be processed is identified by a preset information structured recognition model.

[0083] It should be noted that the loss function of existing information structure recognition models is a combination of two loss functions: a multi-class classification loss function and a regression loss function for the absolute position of attribute values. These two functions are jointly used to optimize and train the model, and the model labels are multiple classification categories and attribute region position coordinates. For images with different layouts, the multi-class categories and absolute position coordinates will change. Therefore, to address the problems caused by the aforementioned loss function, the loss function in the preset information structure recognition model of this embodiment is an optimized loss function based on the matching error between the attribute values ​​and the attribute names. The optimized loss function replaces the previously used joint loss function of multi-class classification and position regression, and the labels corresponding to the preset information structure recognition model are changed to 0 and 1. This transforms the model training technique customized for different layouts into a non-customized, general-purpose model training technique suitable for various layouts.

[0084] Furthermore, in order to improve the accuracy of the structured text recognition results, step S30 of this embodiment may include:

[0085] The relative distances between the attribute values ​​and attribute names, the relative distances of the coordinates, and the relative distances of the word vectors are ranked according to the matching degree based on the preset information structure recognition model.

[0086] The matching degree ranking result is optimized according to the loss function to identify the information of the file to be processed. The loss function in the preset information structured recognition model is a loss function optimized according to the matching degree error between the attribute value and the attribute name.

[0087] It should be noted that the optimized loss function is:

[0088]

[0089] In the formula, y (i) This indicates the actual tag to which the current attribute value belongs in the i-th attribute region. Z represents the predicted label of the current attribute value estimated by the model, belonging to the i-th attribute region, where m represents the total number of attribute regions (m). (i) This represents the product of the weight coefficient vector and the input features.

[0090] It is understandable that the distance information between attribute values ​​and attribute names includes the relative distance between attribute values ​​and attribute names, the relative distance between coordinates, and the relative distance between word vectors. The relative distance between attribute values ​​and attribute names, the relative distance between coordinates, and the relative distance between word vectors are input as parameters into the preset information structure recognition model.

[0091] It should be noted that Z (i) It can be represented as:

[0092] Z (i) =Xα(i) T ;

[0093] α(i)={α i,1 ,α i,2 ,α i,3 ,...,α i,n};

[0094] In the formula, α T X represents the weight coefficient vector, and X represents the input features (i.e., the relative distance between attribute values ​​and attribute names, the relative distance between coordinates, and the relative distance between word vectors).

[0095] It should be noted that matching ranking can be performed by considering the distance between the matching attribute value and the attribute name, the relative distance of coordinates, and the relative distance of word vectors. The highest and closest matching result is likely to be the attribute value corresponding to the attribute name. Furthermore, the optimized loss function will continuously optimize the ranking result to achieve the most accurate match.

[0096] It should be noted that existing technologies require pre-labeling before training the text structure recognition model for each type of image layout. This involves two aspects: first, labeling the absolute coordinates of the attribute value of the field to be classified within the image; and second, labeling the attribute name to which the attribute value belongs. This labeling process, which involves marking text boxes and positions on the image and adding classification labels, is labor-intensive and requires complex manual labeling. At least hundreds of images for each layout need to be labeled with both types of labels. Therefore, this embodiment proposes an automated intelligent learning platform that enables automated intelligent learning from labeling, model training, testing, and deployment, reducing manual costs and complexity. Furthermore, the automated online model training function further improves the model's coverage of new text image types and the model's accuracy.

[0097] For ease of understanding, please refer to Figure 8 To explain, Figure 8 A flowchart of the overall process for text structure recognition.Figure 8 First, the text image is preprocessed, including image calibration and denoising. Then, the text is detected and recognized. The text detection and recognition results are used for text feature extraction and classification through an attribute name and attribute information classification network. Based on the text extraction and classification results, structural recognition features are extracted, including attribute value / attribute name classification, relative distance between attribute values / attribute names, relative distance between coordinate positions of attribute values / attribute names, and relative distance between word vectors of attribute values / attribute names. A structured text knowledge base provides the basis for attribute value / attribute name classification. The relative distances between attribute values / attribute names, relative distances between positions, and relative distances between word vectors are input into a preset information structured recognition model. The preset information structured recognition model includes an optimized loss function and a matching degree ranking. At the same time, the preset information structured recognition model is trained and tested on an automatic intelligent learning platform, which includes sample labeling, model training, model testing, and model deployment. The preset information structured recognition model outputs structured recognition, that is, the classification of field information.

[0098] This embodiment extracts features from the file information to obtain its feature information; classifies the feature information to obtain its attribute values ​​and attribute names, and determines the distance between the attribute values ​​and attribute names; based on the distance information, it identifies the file information using a preset information structure recognition model. This invention classifies the extracted feature information to obtain the attribute values ​​and attribute names of the file to be processed, determines the distance between the attribute values ​​and attribute names, and inputs this distance information as a parameter into a preset information structure recognition model to perform structured recognition of the file. Therefore, when faced with different structured texts, it is not necessary to redesign the preset text structure recognition model for different structured texts; only the distance information between the attribute values ​​and attribute names needs to be determined and input into the preset text structure recognition model, thereby improving the efficiency of text structure recognition.

[0099] Reference Figure 3 , Figure 3 This is a flowchart illustrating the second embodiment of the text structure recognition method of the present invention, based on the above. Figure 2 The first embodiment shown presents a second embodiment of the text structure recognition method of the present invention.

[0100] In the second embodiment, step S20 includes:

[0101] Step S201: Classify the word vector features according to the attribute classification network to obtain the attribute values ​​and attribute names of the feature information of the file to be processed;

[0102] Step S202: Determine the relative distance between the attribute value and the attribute name.

[0103] For ease of understanding, please refer to Figure 9 To explain, Figure 9 This is a flowchart for relative distance processing. Figure 9 The attribute value region and attribute name region of the image to be identified are respectively input into the attribute classification network. The attribute classification network classifies the attribute values ​​and attribute names from the image to be identified. The relative distance between the attribute values ​​and attribute names is extracted as the relative feature of the information structure recognition model and input into the information structure recognition model to extract the attribute structure information.

[0104] This embodiment classifies the word vector features using an attribute classification network to obtain the attribute values ​​and attribute names of the feature information of the document to be processed; and determines the relative distance between the attribute values ​​and attribute names. This embodiment classifies attribute values ​​and attribute names using an attribute classification network and determines the relative distance between them, thus using this relative distance as input to a preset information structured recognition model. This eliminates the need to re-customize and retrain the preset information structured recognition model based on changes in page layout, improving the efficiency of structured text recognition.

[0105] Reference Figure 4 , Figure 4 This is a flowchart illustrating the third embodiment of the text structure recognition method of the present invention, based on the above. Figure 2 The first embodiment shown is followed by a third embodiment of the text structure recognition method of the present invention.

[0106] In the third embodiment, step S20 further includes:

[0107] Step S201': Determine the attribute value coordinates and attribute name coordinates based on the text region coordinates;

[0108] Step S202': Determine the relative distance between the coordinates of the attribute value and the coordinates of the attribute name.

[0109] It should be noted that the problem with the absolute position coordinate features used in the existing technology is that the absolute position coordinates of the attribute values ​​will change when different structured documents, certificates, invoices, and other structured documents and layout formats change. Therefore, this embodiment uses the relative distance of relative coordinates as the input of the preset information structure recognition model. In this embodiment, the relative distance between the position coordinates of the attribute value and its corresponding attribute name is the smallest, and it is applicable to any structured text image, which can ensure that the model will not be customized as the image layout changes.

[0110] It should be noted that, taking Euclidean distance as an example, the Euclidean distance between the i-th attribute value to be extracted and the n-th attribute name to be assigned is... <value i ,key n The formula for calculating > is:

[0111]

[0112] In the formula, (X i,value ,Y i,value () represents the absolute coordinates of the top-left corner of the i-th attribute value region to be extracted, (X) i,key ,Y i,key ) represents the absolute coordinates of the top left corner of the nth attribute name region to be assigned, value represents the attribute value, and key represents the attribute name.

[0113] For ease of understanding, please refer to Figure 10 To explain, Figure 10 This is a flowchart for processing relative distances between coordinates. Figure 10 The relative feature calculation is performed on the absolute position coordinates corresponding to the attribute value and the absolute position coordinates corresponding to the attribute name. The relative distance of the position coordinates is then input into the information structure recognition model to extract the structured information of the attribute value.

[0114] This embodiment determines the attribute value coordinates and attribute name coordinates based on the text region coordinates; it also determines the relative distance between the attribute value coordinates and the attribute name coordinates. This embodiment determines the relative distance based on the attribute value coordinates and attribute name coordinates, thus using this relative distance as input to a preset information structured recognition model. This eliminates the need to re-customize and retrain the preset information structured recognition model based on changes in page layout, thereby improving the efficiency of structured text recognition.

[0115] Reference Figure 5 , Figure 5 This is a flowchart illustrating the fourth embodiment of the text structure recognition method of the present invention, based on the above. Figure 2 The first embodiment shown presents a fourth embodiment of the text structure recognition method of the present invention.

[0116] In the fourth embodiment, step S20 further includes:

[0117] Step S201”: Determine the word vector of the attribute value and the word vector of the attribute name based on the word vector features;

[0118] Step S202”: Determine the relative distance between the word vectors of the attribute value and the word vector of the attribute name.

[0119] It should be noted that when NLP is used for text classification, it only uses the word vector features of the text to be classified as the model input, without introducing word vector features with category information. It calculates the relative distance between word vectors as the input of the classification model. Therefore, it calculates the relative distance between the word vectors of attribute values ​​and attribute names, replacing the existing text classification. The relative distance between the word vectors of attribute names and attribute values ​​is minimized. This feature is applicable to any structured text image and can also ensure that the model does not require customized development.

[0120] It should be noted that, taking cosine distance as an example, the cosine distance between the word vector feature corresponding to the k-th attribute value to be extracted and the word vector feature corresponding to the l-th attribute name to be assigned is... <value k ,key l The formula for calculating > is:

[0121]

[0122] In the formula, X i,value(k) Let X represent the word vector feature corresponding to the k-th attribute value to be extracted. i,key(k) This represents the word vector feature corresponding to the l-th attribute name to be assigned.

[0123] This embodiment determines the word vectors of the attribute value and the attribute name based on the word vector features; it also determines the relative distance between the word vectors of the attribute value and the attribute name. This embodiment determines the relative distance between the word vectors of the attribute value and the attribute name based on their respective word vectors, thus using this relative distance as input to a pre-defined information structured recognition model. This eliminates the need to re-customize and retrain the pre-defined information structured recognition model based on changes in page layout, thereby improving the efficiency of structured text recognition.

[0124] Furthermore, this embodiment of the invention also proposes a storage medium storing a text structure recognition program, which, when executed by a processor, implements the text structure recognition method as described above.

[0125] In addition, refer to Figure 11 The present invention also proposes a text structure recognition device, which includes: a feature extraction module 10, a distance determination module 20 and an information recognition module 30;

[0126] The feature extraction module 10 is used to extract features from the file information to be processed, and obtain the feature information of the file to be processed.

[0127] The distance determination module 20 is used to classify the feature information of the file to be processed, obtain the attribute value and attribute name of the feature information of the file to be processed, and determine the distance information between the attribute value and the attribute name;

[0128] The information recognition module 30, based on the distance information, identifies the information of the file to be processed through a preset information structure recognition model.

[0129] This embodiment extracts features from the file information to obtain its feature information; classifies the feature information to obtain its attribute values ​​and attribute names, and determines the distance between the attribute values ​​and attribute names; based on the distance information, it identifies the file information using a preset information structure recognition model. This invention classifies the extracted feature information to obtain the attribute values ​​and attribute names of the file to be processed, determines the distance between the attribute values ​​and attribute names, and inputs this distance information as a parameter into a preset information structure recognition model to perform structured recognition of the file. Therefore, when faced with different structured texts, it is not necessary to redesign the preset text structure recognition model for different structured texts; only the distance information between the attribute values ​​and attribute names needs to be determined and input into the preset text structure recognition model, thereby improving the efficiency of text structure recognition.

[0130] Based on the first embodiment of the text structure recognition device of the present invention described above, a second embodiment of the text structure recognition device of the present invention is proposed.

[0131] In this embodiment, the feature extraction module 10 is used to extract text information from the file information to be processed based on the character recognition model and the text detection model.

[0132] Furthermore, the feature extraction module 10 is also used to extract semantic features from the text information according to a preset word vector model to obtain the word vector features.

[0133] Furthermore, the distance determination module 20 is also used to classify the word vector features according to the attribute classification network to obtain the attribute values ​​and attribute names of the feature information of the file to be processed.

[0134] Furthermore, the distance determination module 20 is also used to determine the relative distance between the attribute value and the attribute name.

[0135] Furthermore, the distance determination module 20 is also used to determine the attribute value coordinates and attribute name coordinates based on the text region coordinates.

[0136] Furthermore, the distance determination module 20 is also used to determine the relative distance between the attribute value coordinates and the attribute name coordinates.

[0137] Furthermore, the distance determination module 20 is also used to determine the word vector of the attribute value and the word vector of the attribute name based on the word vector features.

[0138] Furthermore, the distance determination module 20 is also used to determine the relative distance between the word vectors of the attribute value and the word vector of the attribute name.

[0139] Furthermore, the feature extraction module 10 is also used to construct a text knowledge base based on the text information of each structure, the word vector features corresponding to the attribute names and attribute values, and the word segmentation results of the word vector features.

[0140] Furthermore, the feature extraction module 10 is also used to train the word vector model by using the text data in the text knowledge base as input data to obtain a preset word vector model.

[0141] Furthermore, the information recognition module 30 is also used to sort the relative distance between the attribute value and the attribute name, the relative distance of the coordinates and the relative distance of the word vector according to the preset information structure recognition model.

[0142] Furthermore, the information recognition module 30 is also used to optimize the matching degree ranking result according to the loss function to identify the information of the file to be processed. The loss function in the preset information structured recognition model is a loss function optimized according to the matching degree error between the attribute value and the attribute name.

[0143] Other embodiments or specific implementations of the text structure recognition device described in this invention can be found in the above-described method embodiments, and will not be repeated here.

[0144] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0145] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0146] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as a read-only memory image (ROM) / random access memory (RAM), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0147] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method of text structure recognition, characterized by, The text structure recognition method comprises the following steps: characteristic extraction is performed on the to-be-processed file information to obtain to-be-processed file characteristic information; the to-be-processed file characteristic information is classified to obtain attribute values and attribute names of the to-be-processed file characteristic information, and distance information of the attribute values and the attribute names is determined, the distance information comprising relative distances between attribute values and attribute names, coordinate relative distances and word vector relative distances; based on the distance information, the to-be-processed file information is recognized by a preset information structure recognition model.

2. The text structuring recognition method of claim 1, wherein, The to-be-processed file characteristic information comprises word vector characteristics. The step of performing characteristic extraction on the to-be-processed file information to obtain to-be-processed file characteristic information comprises: text information is extracted from the to-be-processed file information according to a character recognition model and a text detection model; semantic characteristic extraction is performed on the text information according to a preset word vector model to obtain the word vector characteristics.

3. The text structuring recognition method of claim 2, wherein, The distance information comprises relative distances between attribute values and attribute names. The step of classifying the to-be-processed file characteristic information to obtain attribute values and attribute names of the to-be-processed file characteristic information and determining distance information of the attribute values and the attribute names comprises: attribute values and attribute names of the to-be-processed file characteristic information are obtained by classifying the word vector characteristics according to an attribute classification network; relative distances between the attribute values and the attribute names are determined.

4. The text structuring recognition method of claim 2, wherein, The distance information comprises coordinate relative distances, and the text information comprises text region coordinates. The step of determining distance information of the attribute values and the attribute names comprises: attribute value coordinates and attribute name coordinates are determined according to the text region coordinates; the coordinate relative distances between the attribute value coordinates and the attribute name coordinates are determined.

5. The text structuring recognition method of claim 2, wherein, The distance information comprises word vector relative distances. The step of determining distance information of the attribute values and the attribute names comprises: word vectors of the attribute values and word vectors of the attribute names are determined according to the word vector characteristics; the word vector relative distances between the word vectors of the attribute values and the word vectors of the attribute names are determined.

6. The text structuring recognition method of claim 1, wherein, Before the step of performing semantic characteristic extraction on the text information according to a preset word vector model to obtain the word vector characteristics, the method further comprises: a text knowledge base is constructed according to word vector characteristics corresponding to text information, attribute names and attribute values of each structure and a word segmentation result of the word vector characteristics; text data in the text knowledge base is taken as input data of a word vector model to train the word vector model to obtain a preset word vector model.

7. The text structuring recognition method according to any one of claims 1 to 6, characterized in that, The step of recognizing the to-be-processed file information based on the distance information by a preset information structure recognition model comprises: the relative distances between the attribute values and the attribute names, the coordinate relative distances and the word vector relative distances are sorted according to matching degrees by the preset information structure recognition model. The matching degree ranking result is optimized according to a loss function to recognize the to-be-processed file information, and the loss function in the preset information structured recognition model is a loss function optimized according to a matching degree error between the attribute value and the attribute name; The optimized loss function is: where y (i) denotes the actual label of the current attribute value belonging to the i-th attribute region, denotes the predicted label of the current attribute value belonging to the i-th attribute region estimated by the model, m denotes the total number of attribute m regions, Z (i) denotes the product of the weight coefficient vector and the input features.

8. A text structuring recognition apparatus characterized by comprising: The text structured recognition device comprises a feature extraction module, a distance determination module and an information recognition module. The feature extraction module is configured to perform feature extraction on the to-be-processed file information to obtain to-be-processed file feature information. The distance determination module is configured to classify the to-be-processed file feature information to obtain an attribute value and an attribute name of the to-be-processed file feature information, and determine distance information of the attribute value and the attribute name, wherein the distance information comprises a relative distance, a coordinate relative distance and a word vector relative distance between the attribute value and the attribute name. The information recognition module recognizes the to-be-processed file information based on the distance information through a preset information structured recognition model.

9. A text structuring recognition device, characterized by The text structured recognition device comprises a memory, a processor and a text structured recognition program stored on the memory and executable on the processor, and the text structured recognition program, when executed by the processor, implements the steps of the text structured recognition method according to any one of claims 1 to 7.

10. A storage medium, characterized by The storage medium stores a text structured recognition program, and the text structured recognition program, when executed by the processor, implements the steps of the text structured recognition method according to any one of claims 1 to 7.

Images