A method, device and equipment for processing layout documents

Abstract

The present invention provides a method, device and equipment for processing layout documents. The method comprises: obtaining graphic segmentation data of the layout document; obtaining label data and vectorized data according to the graphic segmentation data; storing the graphic segmentation data in a first database, storing the label data in a second database, and storing the vectorized data in a third database according to a preset identifier of the layout document; reading target graphic segmentation data from the first database, reading target label data from the second database, and reading target vectorized data from the third database according to a received retrieval request; generating a target layout document according to the target graphic segmentation data, the target label data and the target vectorized data, and loading the target layout document. The present invention has the advantages of high retrieval efficiency and high retrieval accuracy for layout documents.

Description

Technical Field

[0001] The present invention relates to the technical field of layout document processing, and also to a layout document processing method, device and equipment. Background Art

[0002] The layout document format is an electronic document format with a fixed layout presentation effect. The presentation of the layout document is independent of the device. When reading, printing or printing on various devices, the presentation results of the layout are consistent. When searching for layout documents such as PDF (portable document format) or OFD (national standard for layout documents), it is necessary to convert the content in the layout document into text before searching. In this process, due to a certain degree of data loss during the graphic conversion process, the layout-related information in the layout document cannot be retained, and information such as tables and graphics is difficult to describe through text after conversion, and can only be discarded, so that information such as tables and graphics cannot be retrieved and used in the subsequent search process. Therefore, the existing retrieval efficiency and accuracy of layout documents are low. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a method, device and equipment for processing layout documents to improve the retrieval efficiency and accuracy of layout documents.

[0004] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0005] A first aspect of the present invention provides a method for processing a layout document, comprising:

[0006] Get the graphic segmentation data of the layout document;

[0007] Segment the data according to the graph to obtain label data and vectorized data;

[0008] According to the preset identifier of the layout document, the graphic segmentation data is stored in a first database, the label data is stored in a second database, and the vectorized data is stored in a third database;

[0009] According to the received search request, read the target graphic segmentation data from the first database, read the target label data from the second database, and read the target vectorized data from the third database;

[0010] A target layout document is generated according to the target graphic segmentation data, the target label data and the target vectorized data, and the target layout document is loaded.

[0011] Optionally, obtain the graphic segmentation data of the layout document, including:

[0012] Get layout document data;

[0013] The layout document data is input into a trained segmentation model for segmentation processing to obtain graphic segmentation data of the layout document. The segmentation model is obtained by training a preset network model based on collected layout document samples.

[0014] Optionally, segmenting the data according to the graph to obtain label data includes:

[0015] Performing text recognition on the graphic segmentation data to obtain text data;

[0016] Performing word segmentation processing on the text data to obtain word segmentation data;

[0017] According to the word segmentation data, label data of the layout document is determined.

[0018] Optionally, segmenting the data according to the graph to obtain vectorized data includes:

[0019] Performing angle adjustment on the graphic segmentation data to obtain adjusted original data;

[0020] Performing feature extraction on the adjusted original data to obtain first feature data;

[0021] Performing mapping processing on the first feature data to obtain second feature data;

[0022] The second characteristic data is converted to obtain vectorized data of the layout document.

[0023] Optionally, according to a preset identifier of the layout document, the graphic segmentation data is stored in a first database, the label data is stored in a second database, and the vectorized data is stored in a third database, including:

[0024] Adding the preset identifier of the layout document to the graphic segmentation data, the label data and the vectorized data respectively, to obtain the graphic segmentation data with the preset identifier, the label data with the preset identifier and the vectorized data with the preset identifier;

[0025] The graphic segmentation data with the preset identifier is stored in a first database, the label data with the preset identifier is stored in a second database, and the vectorized data with the preset identifier is stored in a third database.

[0026] Optionally, according to the received search request, reading the target graphic segmentation data from the first database, reading the target label data from the second database, and reading the target vectorized data from the third database includes:

[0027] Receiving a search request from a user; the search request includes text information and / or image information;

[0028] Preprocessing the search request to obtain a preprocessed search request;

[0029] According to the preprocessed retrieval request, the target graphic segmentation data is read from the first database, the target label data is read from the second database, and the target vectorized data is read from the third database.

[0030] Optionally, generating a target layout document according to the target graphic segmentation data, the target label data and the target vectorized data includes:

[0031] Determining a user template according to the target label data;

[0032] The user template is filled according to the target graphic segmentation data and the target vectorized data to generate a target layout document.

[0033] A second aspect of the present invention provides a layout document processing device, comprising:

[0034] An acquisition module, used for acquiring the graphic segmentation data of the layout document;

[0035] A processing module is used to obtain label data and vectorized data based on the graphic segmentation data; store the graphic segmentation data in a first database, store the label data in a second database, and store the vectorized data in a third database according to a preset identifier of the layout document; read target graphic segmentation data from the first database, read target label data from the second database, and read target vectorized data from the third database according to a received retrieval request; generate a target layout document based on the target graphic segmentation data, the target label data, and the target vectorized data, and load the target layout document.

[0036] According to a third aspect of the present invention, a computing device is provided, comprising: a processor and a memory storing a computer program, wherein when the computer program is executed by the processor, the method according to the first aspect is executed.

[0037] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions, which, when executed on a computer, causes the computer to execute the method described in the first aspect.

[0038] The above solution of the present invention includes at least the following beneficial effects:

[0039] The above-mentioned scheme of the present invention obtains the graphic segmentation data of the layout document, and then obtains the label data and the vectorized data, and then stores the graphic segmentation data in the first database, stores the label data in the second database, and stores the vectorized data in the third database. Then, according to the received retrieval request, the target graphic segmentation data is read from the first database, the target label data is read from the second database, and the target vectorized data is read from the third database, and the target layout document is generated and loaded. It not only retains the tables, graphics and other data in the layout document, but also can be used to respond to the user's retrieval request, which not only improves the comprehensiveness of the retrieval, but also improves the accuracy of the retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a flowchart of a method for processing a layout document in an embodiment of the present invention;

[0041] Figure 2 is a schematic diagram of layout document segmentation in an embodiment of the present invention;

[0042] Figure 3 It is a schematic diagram of the process of storing formatted documents in an embodiment of the present invention;

[0043] Figure 4 is a schematic diagram of a process of reading data according to a search request in an embodiment of the present invention;

[0044] Figure 5 It is a structural schematic diagram of a layout document processing device in an embodiment of the present invention.

[0045] Explanation of the accompanying drawings: 1-layout document data, 11-text, 12-table, 13-picture, 21-graphic segmentation data. DETAILED DESCRIPTION

[0046] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0047] like Figure 1 As shown, an embodiment of the present invention provides a method for processing a layout document, comprising the following steps:

[0048] Step 101, obtaining graphic segmentation data of a layout document;

[0049] Step 102, segmenting the data according to the graph to obtain label data and vectorized data;

[0050] Step 103, according to the preset identifier of the layout document, the graphic segmentation data is stored in a first database, the label data is stored in a second database, and the vectorized data is stored in a third database;

[0051] Step 104, according to the received search request, read the target graphic segmentation data from the first database, read the target label data from the second database, and read the target vectorized data from the third database;

[0052] Step 105 , generating a target layout document according to the target graphic segmentation data, the target label data and the target vectorized data, and loading the target layout document.

[0053] The method for processing layout documents in an embodiment of the present invention obtains the graphic segmentation data of the layout document, thereby obtaining label data and vectorized data, and then stores the graphic segmentation data in a first database, stores the label data in a second database, and stores the vectorized data in a third database. Then, according to the received retrieval request, the target graphic segmentation data is read from the first database, the target label data is read from the second database, and the target vectorized data is read from the third database, and the target layout document is generated and loaded. Not only does it retain the tables, graphics and other data in the layout document, but it can also be used to respond to the user's retrieval request, which not only improves the comprehensiveness of the retrieval, but also improves the accuracy of the retrieval.

[0054] In an optional embodiment of the present invention, step 101 includes:

[0055] Step 1011, obtaining layout document data;

[0056] Specifically, the layout document data can be document data in PDF (Portable Document Format) format, or OFD (Layout Document Standard) format, or other fixed formats such as layout or layout. Here, the layout document is characterized by a fixed layout, especially for periodical documents. In order to save layout space to the maximum extent, various annotation tips, knowledge points or pictures, as well as various tables, schematic diagrams, etc., will be inserted into the document.

[0057] Step 1012, input the layout document data into the trained segmentation model for segmentation processing to obtain graphic segmentation data of the layout document. The segmentation model is obtained by training a preset network model based on the collected layout document samples.

[0058] Specifically, Figure 2As shown, the layout document data 1 includes various types of information such as text 11, table 12, and picture 13. The segmentation model can segment the layout document data into segments or blocks to obtain multiple pictures of the layout document, each picture including at least one of text, table, and picture, etc. The multiple pictures are graphic segmentation data 21, so that the label data and vectorized data can be obtained according to the graphic segmentation data for storage and utilization. Generally, each picture generally contains a picture and a paragraph of text. In this embodiment, text, table, and picture are used as examples. In actual applications, the layout document data may contain other types of data, including but not limited to at least one of text, table, and picture, etc.

[0059] Here, in step 1012, the layout document data is input into the trained segmentation model for segmentation processing to obtain graphic segmentation data of the layout document, including:

[0060] Step 10121, extracting feature information from the layout document data to obtain feature information;

[0061] Specifically, the layout document data can be preprocessed by resizing, normalizing, etc. to ensure that the data meets the input requirements of the segmentation model. Extracting feature information from the layout document data is to extract features including color, texture, shape, etc. from the layout document data, and then perform feature fusion to generate a more representative feature map as feature information for subsequent processing.

[0062] Step 10122, predicting bounding box coordinates and confidence scores for each block or segment based on the extracted feature information;

[0063] Specifically, the prediction of the bounding box is not a direct regression to a fixed value, but a regression to a probability distribution. Taking the x-coordinate of the center point of the bounding box as an example, first preset a range of x-coordinates (for example, 0 to 18), and then insert multiple segmentation points (for example, 8) in this range to divide the range into multiple segments (for example, 9 segments). For each segment, predict a probability value, indicating the possibility that the x-coordinate of the center point of the bounding box falls within the segment. Finally, by calculating the weighted sum of these probability values ​​and the coordinates in the corresponding segment, the actual predicted value of the x-coordinate of the center point of the bounding box is obtained. The formula for the predicted horizontal coordinate of the center point of the bounding box is: Similarly, the y coordinate of the center point of the bounding box can be calculated ,width and height ,in, is the predicted value of the horizontal coordinate of the center point of the bounding box, is the horizontal coordinate of the center point of the ith segment, is the predicted value of the ordinate of the center point of the bounding box, is the width of the center point of the bounding box, is the predicted value of the height of the center point of the bounding box, is the ordinate of the center point of the ith segment, is the center point width of the ith segment, is the height of the center point of the ith segment, is the number of split points, is the predicted probability of the i-th segment. The confidence score indicates the confidence of the segmentation model that there is a target object in the predicted bounding box. Each candidate bounding box is classified to determine which target category it belongs to (if there are multiple categories) or whether it belongs to the background. A confidence matrix is ​​output to indicate the confidence of each candidate bounding box belonging to each category. The category with the highest confidence is selected as the prediction result, and the corresponding confidence score is used as the confidence score of the bounding box.

[0064] For example, for a candidate bounding box, the predicted probability distribution of the x-coordinate of its center point is [0.1, 0.2, 0.3, 0.1, 0.1, 0.1, 0.05, 0.03, 0.02] (corresponding to 9 segments), as well as the center coordinates of each segment (e.g. [0.5, 1.5, 2.5, ..., 8.5]). According to the formula The actual predicted value can be calculated At the same time, it is also predicted that the confidence score of the bounding box belonging to a certain target category is 0.8. Therefore, the prediction result of the candidate bounding box is: the center point coordinates are ( , ), with a width of , the height is , with a confidence score of 0.8.

[0065] In step 10123, in order to avoid repeated detection, bounding boxes with high overlap and low confidence can be filtered out; finally, the bounding box coordinates and confidence score of each block or segment are output.

[0066] Step 10124, extracting the area of ​​each block or segment from the original image (i.e., the layout document data) according to the bounding box coordinates and confidence scores output by the segmentation model, and these block or segment areas are the graphic segmentation data of the layout document. It should be noted that after extracting the area of ​​each block or segment, the images containing the area can be processed such as cropping, rotating, resizing, etc. to improve the accuracy of the graphic segmentation data.

[0067] Here, the document processing software can be used to traverse the bounding box coordinates output by the segmentation model. If the corresponding confidence score meets the preset threshold (for example, the confidence score corresponding to the bounding box coordinate is greater than 0.5), the corresponding image block is cropped from the layout document. When cropping, the upper left corner coordinate of the bounding box is calculated ( , ) and width ( ),high( ) to determine the cropping area, and then you can get the graphic segmentation data of the layout document.

[0068] Among them, the training process of the segmentation model includes:

[0069] Collect layout document samples, which are in the form of images and include blocks or segments that need to be segmented; divide the samples into training samples and verification samples, both of which include images and corresponding annotation files, wherein the annotation files are annotation files in XML format generated by annotating blocks or segments in the images; initialize the preset network model, including setting the initial learning rate in the training parameters to 0.01 to 0.1; use the training samples to train the initialized preset network model to obtain the trained preset network model; use the verification samples to verify the trained preset network model, and adjust the training parameters to improve the accuracy and generalization ability of the model; obtain the trained segmentation model according to the adjusted training parameters.

[0070] Here, the preset network model may include: a backbone network, a neck network, and a head network; and the initialized preset network model is trained using the training sample, including:

[0071] The training samples are input into the backbone network of the preset network model to extract image features and obtain image features; sufficiently rich feature information is extracted from the images of the training samples for further processing and decision-making by subsequent network layers.

[0072] The image features are input into the neck network of the preset network model for feature fusion to obtain a feature map; the neck network integrates the image features at multiple scales and constructs a feature pyramid, which helps capture objects of different sizes and improves the detection performance of small objects. If the image blocks or paragraphs in the layout document have different sizes and layouts, the feature fusion function of the neck network will help improve the recognition ability of these image blocks or paragraphs of different sizes.

[0073] The feature map is input into the head network of the preset network model for calculation to obtain the bounding box coordinates and confidence score of each block or segment. The calculation process is as described in step 10122 and will not be repeated here.

[0074] In an optional embodiment of the present invention, the step 102 of segmenting the data according to the graph to obtain label data includes:

[0075] Step 1021, performing text recognition on the graphic segmentation data to obtain text data;

[0076] Specifically, use image processing technology (such as edge detection, contour analysis, etc.) to find areas in the graphic segmentation data that may contain text, locate the specific position of the text in the image, and prepare for subsequent character segmentation and recognition; segment the detected text area into individual characters, which can be specifically segmented by features such as the spacing and connectivity between characters; extract feature information of each character, such as shape, texture, angle, etc.; compare the extracted character features with the preset character library, identify the content of each character, and obtain text data.

[0077] Step 1022, performing word segmentation processing on the text data to obtain word segmentation data;

[0078] Specifically, a word segmentation tool may be used to perform word segmentation processing on text data to obtain word segmentation data, thereby providing a basis for subsequently determining labels for the layout document.

[0079] Step 1023: Determine label data of the layout document according to the word segmentation data.

[0080] Specifically, multi-dimensional label data of layout documents can be formed through entity extraction (such as the directory, abstract, chapter title, etc. of the layout document), content summary extraction, etc.

[0081] In an optional embodiment of the present invention, step 102 includes:

[0082] Step 1024, performing angle adjustment on the graphic segmentation data to obtain adjusted original data;

[0083] Specifically, the image of the graphic segmentation data is adjusted to the correct angle through angle adjustment to improve the subsequent processing efficiency. Here, the graphic segmentation data can also be standardized, that is, the pixel values ​​of the graphic segmentation data are normalized to a fixed range (such as 0 to 1) to ensure better adaptability when processing images with different brightness and contrast; the graphic segmentation data can also be resized, that is, the size of the image is unified to improve processing efficiency.

[0084] Step 1025, performing feature extraction on the adjusted original data to obtain first feature data;

[0085] Specifically, different features, such as edge, texture, shape, etc., are extracted from the adjusted original data as first feature data, and the first feature data is represented in the form of a feature map.

[0086] Step 1026, mapping the first feature data to obtain second feature data;

[0087] Specifically, the amount of calculation can be reduced by reducing the spatial size of the feature map while retaining the main features of the image; then the multi-dimensional first feature data is converted into a one-dimensional vector, specifically, each row or column in the first feature data is expanded into a one-dimensional vector, namely, the second feature data.

[0088] Step 1027: convert the second feature data to obtain vectorized data of the layout document.

[0089] Specifically, the one-dimensional vectors of each row or column in the second feature data are concatenated to form a long feature vector as the vectorized data of the layout document.

[0090] In an optional embodiment of the present invention, step 103 includes:

[0091] Step 1031, respectively adding a preset identifier of the layout document to the graphic segmentation data, the label data and the vectorized data, to obtain graphic segmentation data with the preset identifier, label data with the preset identifier and vectorized data with the preset identifier;

[0092] Specifically, in order to facilitate the restoration of layout documents, the preset identifier of the layout document (which can be serial number 1, 2, 3..., or other unique identifiers that distinguish different layout documents) can be added to the graphic segmentation data, label data and vectorized data. When the layout document is subsequently restored, the graphic segmentation data, label data and vectorized data can be quickly and accurately read out according to the preset identifier of the layout document for restoration.

[0093] Step 1032: store the graphic segmentation data with the preset identifier in the first database, store the label data with the preset identifier in the second database, and store the vectorized data with the preset identifier in the third database.

[0094] Specifically, storing different data in different databases makes it easy to find the corresponding database according to the user's search request, which is conducive to improving the search efficiency.

[0095] In an optional embodiment of the present invention, step 104 includes:

[0096] Step 1041, receiving a search request from a user; the search request includes text information and / or image information;

[0097] Specifically, the user's search request may be text, or a picture, or a combination of text and picture, so as to adapt to the search requests of different users and improve adaptability.

[0098] Step 1042, preprocessing the search request to obtain a preprocessed search request;

[0099] Specifically, keywords may be extracted from the text in the search request, and the images in the search request may be converted into vectorized data, so as to facilitate subsequent reading of data from the database.

[0100] Step 1043, according to the preprocessed retrieval request, read the target graphic segmentation data from the first database, read the target label data from the second database, and read the target vectorized data from the third database.

[0101] Specifically, based on the keywords in the preprocessed retrieval request, the target graphic segmentation data corresponding to the keyword can be read from the first database, and the corresponding target label data can be read from the second database; based on the vectorized data of the image in the preprocessed retrieval request, the target vectorized data matching it can be read from the third database.

[0102] In an optional embodiment of the present invention, step 105 includes:

[0103] Step 1051, determining a user template according to the target tag data;

[0104] Specifically, according to the target tag data read from the second database, the layout, page layout and other format contents of the layout document can be determined, and then according to these contents, the user template can be determined.

[0105] Step 1052: Fill the user template according to the target graphic segmentation data and the target vectorized data to generate a target layout document.

[0106] Specifically, the target layout document can be obtained by filling the target graphic segmentation data and / or target vectorized data according to the layout, layout and other formats in the user template. At the same time, the target layout document can be loaded on the user side for the user to view.

[0107] like Figure 3 and Figure 4 As shown, a specific embodiment of the method for processing a layout document according to an embodiment of the present invention is:

[0108] Step 111, obtaining the graphic segmentation data of the layout document.

[0109] Layout files generally refer to files in PDF and OFD formats. Layout files can be divided into single-layer and double-layer files according to their structure and content. Single-layer layout files refer to files with only one level of structure, usually containing elements such as text, pictures, and tables, but these elements are mixed together and cannot be identified or edited separately. Double-layer layout files contain two levels: text layer and picture layer. Users can see the picture layer, but can select, copy, and search through the text layer.

[0110] The trained segmentation model is used to segment the various segments, images, tables and other information of the layout document in a graphical manner to obtain the graphic segmentation data of the layout document.

[0111] Step 112, obtaining label data and vectorized data.

[0112] For a single-layer layout file, there is only one level of information. After the layout document is segmented, the segmented image needs to be recognized for text, and the recognized text needs to be feature extracted, such as word segmentation, entity extraction, content summary extraction, etc., to form a multi-dimensional label text as the label value of this image; then the image is vectorized, so that the vectorized data of the image and the corresponding label data can be obtained. The processing method of the double-layer layout file is basically the same as that of the single-layer layout file, but there is one less step, because the double-layer layout file contains the text layer, so there is no need to do text recognition. After the label is extracted from the text corresponding to the segmented image, it can be directly associated with the image. The other vectorization methods are the same as those of the single-layer layout file.

[0113] In this embodiment, after obtaining the block and segmented image information, in addition to extracting the text labels, it is also necessary to perform graphic vectorization processing on the block images. The graphic data vectorization method can adopt a feature-based vectorization method, by extracting the color, texture, shape and other features of the image, and then converting these features into vectors. This method works better when processing simple and clear images. The other is the mapping relationship from raw pixels to high-level semantic features, thereby extracting more complex and abstract features. These feature vectors not only contain the underlying information of the image, but also contain high-level semantic information, so that the content of the image can be better represented. You can choose according to the actual situation, as long as the picture can be converted into vectorized data.

[0114] Step 113, store data.

[0115] The graphic segmentation data, label data and vectorized data of the layout document are stored separately in different databases. The graphic segmentation data of the layout document is stored in a dedicated image storage server in the form of images, and is associated with the label data and vectorized data using the preset identifier of the layout document.

[0116] Here, after the graphics are vectorized, a set of data (i.e., vectorized data) will be obtained. This set of data is generally in the form of a list. This list contains all the visual elements that have appeared in the document, including text, tables, graphics, etc. The data of each element is composed of the data of the associated graphics vectorization and text labels, and then this set of data is stored in the vector database. The storage and recall of vectorized data can be roughly divided into the following steps:

[0117] (1) Create an index and set the vector field

[0118] (2) Write the block and segment data: Write the block labels and graphic vectorized data obtained above to the specified index.

[0119] PUT index3 / _doc / 1

[0120] {

[0121] "doc_id": "0000001" # Document ID

[0122] "doc_pos": ["101","203","23","50"] # Position information

[0123] "doc_vector": [0.1, 0.2, 0.3, ..., 0.9] # Block segment vector value

[0124] "doc_tags": ["label1", "label2", "label3"]

[0125] }

[0126] Step 114, read data.

[0127] like Figure 4 As shown, the two search conditions of text label and graphic vectorization are used for cross-search, mainly for the accuracy of recall. These two search conditions represent different search dimensions, one is content, and the other is layout and style. After cross-comparison and search screening, more accurate search results can be obtained, which are more suitable for the search of layout documents.

[0128] Specifically, based on the search request input by the user, the corresponding label data can be retrieved from the database for the text in the search request, and then a vector comparison is performed between the vectorized data of the image in the search request and the vectorized data in the database, and they are sorted according to the similarity, and the n vectorized data with higher similarity are output, or converted into image data for output.

[0129] Step 115, generating and loading the target layout document.

[0130] After all the data is read out, the layout can be restored according to the application requirements, or a new layout document layout or layout can be created based on the content of the data to reorganize and generate a new layout document.

[0131] The method for processing layout documents in the embodiment of the present invention implements graphically based blocking and segmenting of layout documents, wherein the blocking and segmenting operations refer to edge cutting of the main text paragraphs, annotated texts, tables and schematic diagrams in the layout document to generate independent area division information, while maintaining and preserving the original layout association relationship to meet the user's reading habits.

[0132] The processing method of the layout document of the embodiment of the present invention can retain the positional relationship of the annotation information, small text segments, graphic tables, etc. in the layout document corresponding to the main text. After reading the data from the database, the original layout can be restored to the maximum extent through this information, and finally the layout document can be recalled as it is; a new layout document can also be regenerated according to the read multiple columns of data. It can also support the implementation of more value-added extensions and generation applications, such as encryption information, signature information, etc., which can be stored and retrieved in a targeted manner. The layout document can be applied to the RAG (retrieval enhanced generation) application, and the layout information of the layout document in the present invention is added, so that pictures, table links, etc. can be introduced into the generated results; the layout document supports many professional application fields, such as encryption information, signature information, etc., which can be stored and retrieved in a targeted manner.

[0133] like Figure 5 As shown, an embodiment of the present invention provides a layout document processing device 200, comprising:

[0134] An acquisition module 201 is used to acquire the graphic segmentation data of the layout document;

[0135] The processing module 202 is used to obtain label data and vectorized data based on the graphic segmentation data; store the graphic segmentation data in a first database, store the label data in a second database, and store the vectorized data in a third database according to a preset identifier of the layout document; read the target graphic segmentation data from the first database, read the target label data from the second database, and read the target vectorized data from the third database according to the received retrieval request; generate a target layout document based on the target graphic segmentation data, the target label data and the target vectorized data, and load the target layout document.

[0136] Optionally, obtain the graphic segmentation data of the layout document, including:

[0137] Get layout document data;

[0138] The layout document data is input into a trained segmentation model for segmentation processing to obtain graphic segmentation data of the layout document. The segmentation model is obtained by training a preset network model based on collected layout document samples.

[0139] Optionally, segmenting the data according to the graph to obtain label data includes:

[0140] Performing text recognition on the graphic segmentation data to obtain text data;

[0141] Performing word segmentation processing on the text data to obtain word segmentation data;

[0142] According to the word segmentation data, label data of the layout document is determined.

[0143] Optionally, segmenting the data according to the graph to obtain vectorized data includes:

[0144] Performing angle adjustment on the graphic segmentation data to obtain adjusted original data;

[0145] Performing feature extraction on the adjusted original data to obtain first feature data;

[0146] Performing mapping processing on the first feature data to obtain second feature data;

[0147] The second characteristic data is converted to obtain vectorized data of the layout document.

[0148] Optionally, according to a preset identifier of the layout document, the graphic segmentation data is stored in a first database, the label data is stored in a second database, and the vectorized data is stored in a third database, including:

[0149] Adding the preset identifier of the layout document to the graphic segmentation data, the label data and the vectorized data respectively, to obtain the graphic segmentation data with the preset identifier, the label data with the preset identifier and the vectorized data with the preset identifier;

[0150] The graphic segmentation data with the preset identifier is stored in a first database, the label data with the preset identifier is stored in a second database, and the vectorized data with the preset identifier is stored in a third database.

[0151] Optionally, according to the received search request, reading the target graphic segmentation data from the first database, reading the target label data from the second database, and reading the target vectorized data from the third database includes:

[0152] Receiving a search request from a user; the search request includes text information and / or image information;

[0153] Preprocessing the search request to obtain a preprocessed search request;

[0154] According to the preprocessed retrieval request, the target graphic segmentation data is read from the first database, the target label data is read from the second database, and the target vectorized data is read from the third database.

[0155] Optionally, generating a target layout document according to the target graphic segmentation data, the target label data and the target vectorized data includes:

[0156] Determining a user template according to the target label data;

[0157] The user template is filled according to the target graphic segmentation data and the target vectorized data to generate a target layout document.

[0158] The processing device for layout documents of the embodiment of the present invention obtains the graphic segmentation data of the layout document, and then obtains the label data and the vectorized data, and then stores the graphic segmentation data in a first database, stores the label data in a second database, and stores the vectorized data in a third database. Then, according to the received retrieval request, the target graphic segmentation data is read from the first database, the target label data is read from the second database, and the target vectorized data is read from the third database, and the target layout document is generated and loaded. Not only does it retain the tables, graphics and other data in the layout document, but it can also be used to respond to the user's retrieval request, which not only improves the comprehensiveness of the retrieval, but also improves the accuracy of the retrieval.

[0159] It should be noted that the device is a device corresponding to the above method, and all implementations in the above method embodiment are applicable to the embodiment of the device and can achieve the same technical effect. This embodiment will not be repeated.

[0160] The embodiment of the present invention further provides a computing device, comprising: a processor, and a memory storing a computer program, wherein when the computer program is executed by the processor, the method described in any one of the above embodiments is executed. All implementations in the above method embodiments are applicable to the embodiments of the device, and can achieve the same technical effects. They will not be described in detail in this embodiment.

[0161] The embodiment of the present invention further provides a computer-readable storage medium on which instructions are stored. When the instructions are executed on a computer, the computer executes the method as described in any one of the above embodiments. All implementations in the above method embodiments are applicable to the embodiments of the device and can achieve the same technical effects. They will not be described in detail in this embodiment.

[0162] It should be noted that in the apparatus and method of the present invention, it is obvious that each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of the present invention. In addition, the steps of performing the above series of processes can naturally be performed in chronological order according to the order of description, but do not necessarily need to be performed in chronological order. Some steps can be performed in parallel, crosswise or independently of each other.

[0163] It should be noted that, in the above-mentioned embodiments, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "include one..." do not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the implementation of the above-mentioned embodiments is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0164] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for processing a layout document, characterized in that: include: Get the graphic segmentation data of the layout document; Segment the data according to the graph to obtain label data and vectorized data; According to the preset identifier of the layout document, the graphic segmentation data is stored in a first database, the label data is stored in a second database, and the vectorized data is stored in a third database; According to the received search request, read the target graphic segmentation data from the first database, read the target label data from the second database, and read the target vectorized data from the third database; Generate a target layout document according to the target graphic segmentation data, the target label data and the target vectorized data, and load the target layout document; Wherein, obtaining the graphic segmentation data of the layout document includes: Get layout document data; The layout document data is input into the trained segmentation model for segmentation processing to obtain graphic segmentation data of the layout document, including: Extracting feature information from the layout document data to obtain feature information; wherein the color, texture, and shape features in the layout document data are extracted and feature fused to generate a feature map as feature information; Based on the extracted feature information, the bounding box coordinates and confidence scores of each block or segment are predicted; the formula for the predicted horizontal coordinates of the center point of the bounding box is: , the y coordinate of the center point of the bounding box ,width and height ,in, is the predicted value of the horizontal coordinate of the center point of the bounding box, is the horizontal coordinate of the center point of the ith segment, is the predicted value of the ordinate of the center point of the bounding box, is the predicted value of the width of the center point of the bounding box, is the predicted value of the height of the center point of the bounding box, is the ordinate of the center point of the ith segment, is the center point width of the ith segment, is the height of the center point of the ith segment, is the number of split points, is the predicted probability of the i-th segment, where 0, 1, ..., The segments are divided by the segmentation points; By filtering the bounding boxes, output the bounding box coordinates and confidence scores of each block or segment; Extracting the area of ​​each block or segment from the layout document data according to the output bounding box coordinates and confidence scores, i.e., the graphic segmentation data of the layout document; The step of generating a target layout document according to the target graphic segmentation data, the target label data and the target vectorized data includes: Determining a user template according to the target label data; The user template is filled according to the target graphic segmentation data and the target vectorized data to generate a target layout document.

2. The method for processing layout documents according to claim 1, characterized in that: According to the graphic segmentation data, label data is obtained, including: Performing text recognition on the graphic segmentation data to obtain text data; Performing word segmentation processing on the text data to obtain word segmentation data; According to the word segmentation data, label data of the layout document is determined.

3. The method for processing layout documents according to claim 1, characterized in that: According to the graphic segmentation data, vectorized data is obtained, including: Performing angle adjustment on the graphic segmentation data to obtain adjusted original data; Performing feature extraction on the adjusted original data to obtain first feature data; Performing mapping processing on the first feature data to obtain second feature data; The second characteristic data is converted to obtain vectorized data of the layout document.

4. A layout document processing device, characterized in that: include: An acquisition module, used for acquiring the graphic segmentation data of the layout document; A processing module, used to obtain label data and vectorized data according to the graphic segmentation data; store the graphic segmentation data in a first database, store the label data in a second database, and store the vectorized data in a third database according to a preset identifier of a layout document; read target graphic segmentation data from the first database, read target label data from the second database, and read target vectorized data from the third database according to a received retrieval request; generate a target layout document according to the target graphic segmentation data, the target label data, and the target vectorized data, and load the target layout document; Wherein, obtaining the graphic segmentation data of the layout document includes: Get layout document data; The layout document data is input into the trained segmentation model for segmentation processing to obtain graphic segmentation data of the layout document, including: Extracting feature information from the layout document data to obtain feature information; wherein the color, texture, and shape features in the layout document data are extracted and feature fused to generate a feature map as feature information; Based on the extracted feature information, the bounding box coordinates and confidence scores of each block or segment are predicted; the formula for the predicted horizontal coordinates of the center point of the bounding box is: , the y coordinate of the center point of the bounding box ,width and height ,in, is the predicted value of the horizontal coordinate of the center point of the bounding box, is the horizontal coordinate of the center point of the ith segment, is the predicted value of the ordinate of the center point of the bounding box, is the predicted value of the width of the center point of the bounding box, is the predicted value of the height of the center point of the bounding box, is the ordinate of the center point of the ith segment, is the center point width of the ith segment, is the height of the center point of the ith segment, is the number of split points, is the predicted probability of the i-th segment, where 0, 1, ..., The segments are divided by the segmentation points; By filtering the bounding boxes, output the bounding box coordinates and confidence scores of each block or segment; Extracting the area of ​​each block or segment from the layout document data according to the output bounding box coordinates and confidence scores, i.e., the graphic segmentation data of the layout document; The step of generating a target layout document according to the target graphic segmentation data, the target label data and the target vectorized data includes: Determining a user template according to the target label data; The user template is filled according to the target graphic segmentation data and the target vectorized data to generate a target layout document.

5. A computing device, characterized in that: include: A processor and a memory storing a computer program, wherein when the computer program is executed by the processor, the method according to any one of claims 1 to 3 is performed.

6. A computer-readable storage medium, characterized in that: Instructions are stored, and when the instructions are executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 3.

Images