Standard layout analysis and structured storage method based on intelligent identification

Abstract

The invention is suitable for the technical field of layout analysis and storage methods, and provides a standard layout analysis and structured storage method based on intelligent identification, which comprises the following operation steps: S1, data labeling: converting a standard pdf document related to the safety and reliability standard of a robot into a picture form, and carrying out binarization processing on the picture form to obtain a data label; removing the watermark of the image, and performing data annotation on the image to obtain a standard image; s2, coordinate information: carrying out layout analysis on the standard image obtained in S1 through a layout analysis network of multi-scale information fusion. According to the standard layout analysis and structured storage method based on intelligent identification, the retrieval time of the content of the robot standard document is shortened, directory structure information is effectively restored, the storage consumption of the document is reduced, the reading efficiency of the document is improved, a user can conveniently obtain the text information of the robot standard document, and the reading efficiency is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of layout analysis and storage methods, and in particular relates to a standard layout analysis and structured storage method based on intelligent recognition. Background Art

[0002] In the field of robot safety and reliability standardization, my country's system construction has continued to deepen, and based on the specific characteristics of industry development, it has continuously carried out the formulation and revision of standards. Some group standards and local standards have also been issued and implemented. Although my country has established a national standard public service platform to solve the problem of some users obtaining national standard texts, according to the survey results.

[0003] In the existing technology, users still face many difficulties in obtaining various standard texts. For example, it is difficult to obtain standard texts and the application of indicators is not intuitive enough. These problems hinder the deep integration of standards and product research and development, and thus limit the enabling role of standard digitization in promoting industrial development. Summary of the invention

[0004] The present invention provides a standard layout analysis and structured storage method based on intelligent recognition, aiming to solve the many difficulties that users still face in obtaining various types of standard texts, such as the difficulty in obtaining standard texts and the lack of intuitiveness in the application of indicators.

[0005] The present invention is implemented as follows: a standard layout analysis and structured storage method based on intelligent recognition, comprising the following steps: S1. Data labeling: Convert the standard PDF document related to the safety and reliability standards of the robot into an image, perform binarization on it to remove the image watermark, and label the image to obtain a standard image; S2, coordinate information: through the layout analysis network of multi-scale information fusion, the standard image obtained in S1 is analyzed to obtain the coordinate information and corresponding labels of each part of the content in the standard PDF document; S3, text recognition: perform text recognition on the text content of each area in the standard PDF document through a text recognition network to obtain the recognition results of each part of the text content; S4, directory information: the results identified in S3 are reorganized according to different categories using the directory information restoration method to obtain the directory structure information of the standard PDF document; S5. Information processing: Process the directory structure information obtained in S4 to form a data table containing text information of the standard PDF document, and store it in the constructed robot standard database.

[0006] Preferably, in S2, after the layout analysis, the coordinate information and corresponding tags of the title part and the text part in the standard PDF document are obtained.

[0007] Preferably, in S2, the standard images after layout analysis are classified based on content and layout structure.

[0008] Preferably, in S2, the classification specifically includes the following steps: S21, creating a data set based on a standard PDF document for use in the layout analysis network training, and manually annotating the image to obtain a standard image; S22, training the layout analysis network using the training set, and constraining the layout analysis network through a loss function to improve the accuracy of the analysis; S23. By integrating multi-scale information through a layout analysis network that fuses multi-scale information, feature discrimination is enhanced, and the content of a standard image obtained from a standard PDF document is accurately classified.

[0009] Preferably, in S21, the layout analysis network is constructed using the pytorch deep framework, and the network includes: a feature processing module, a multi-scale feature fusion module and a semantic preservation module.

[0010] Preferably, the calculation process of the layout analysis network is: The standard image is input into the feature processing module, and after n times of convolution operation, the features are extracted to obtain the original size. Figure 1 / 2, 1 / 4, 1 / 8, ... and 1 / 2 n Mapping characteristics of scale; The mapping features are then input into the multi-scale feature fusion module, where they are upsampled from top to bottom and fused with the features of the same size generated from bottom to top to obtain mapping features that fuse multi-scale information. The semantics preservation module is used to process the mapping features of the fused multi-scale information to obtain a labeling box of the text area, wherein the labeling box records the coordinate information and the corresponding label.

[0011] Preferably, the semantic preservation module includes a first module and a second module; the first module is used to predict probability mapping and obtain a probability map of mapping features that integrate multi-scale information; the second module is used to predict threshold mapping, obtain a threshold map of the probability map after binarization, obtain position information and label information based on the threshold map, and obtain a prediction box of the text area based on the position information and label information.

[0012] Preferably, in S2, the loss function used to optimize the layout analysis network training process is: .

[0013] Preferably, in S2, the layout analysis network is trained and optimized using stochastic gradient descent method and Adam optimizer.

[0014] Preferably, in S3, the text recognition is performed using the paddleOCR framework, and the text recognition model selected by paddleOCR is PP-OCRv4.

[0015] The beneficial effects of the present invention are: Compared with the prior art, the beneficial effects of the present invention are: a standard layout analysis and structured storage method based on intelligent recognition of the present invention reduces the retrieval time of the content of the robot standard document and effectively restores the directory structure information, reduces the storage consumption of the document, improves the reading efficiency of the document, facilitates users to obtain the text information of the robot standard document, improves the reading efficiency, and further improves the readers' understanding of the safety and reliability standards of the robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 The present invention is a flow chart of the standard layout analysis and structured storage method based on intelligent recognition. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0018] See also Figure 1 The present invention provides a technical solution: a standard layout analysis and structured storage method based on intelligent recognition, comprising the following steps: S1. Data labeling: Convert the standard PDF document related to the safety and reliability standards of the robot into an image, perform binarization on it to remove the image watermark, and label the image to obtain a standard image; The pdf document is converted into png image data, and then the image data is binarized through the OpenCV library to extract image features and remove the image watermark. At the same time, the image data is labeled to facilitate the subsequent image processing process.

[0019] S2, coordinate information: through the layout analysis network of multi-scale information fusion, the standard image obtained in S1 is analyzed to obtain the coordinate information and corresponding labels of each part of the content in the standard PDF document; The constructed training set is used to train the layout analysis network based on multi-scale information fusion, and then the trained layout analysis network is used to classify and predict the content of the standard image, obtain the coordinate information and semantic information of each part, and form a prediction box. In addition, to further ensure the accuracy, the classification results can be manually reviewed.

[0020] S3, text recognition: perform text recognition on the text content of each area in the standard PDF document through a text recognition network to obtain the recognition results of each part of the text content; The features of the image are extracted through the text recognition framework and the text content in the prediction box in S2 is recognized to form recognized text, which is convenient for subsequent information processing.

[0021] S4, directory information: the results identified in S3 are reorganized according to different categories using the directory information restoration method to obtain the directory structure information of the standard PDF document; Get the directory structure of PDF through the directory information restoration method.

[0022] Specifically, after recognition processing, the order relationship of general tags in the PDF text can be judged by the Y-axis coordinates of the predicted boxes of the content on the same page number. For example, titles such as "1 Introduction" and "2 Experiments" are in a level relationship, which can be judged by the Y-axis coordinate information in the text position information. However, the content labeled "Title" cannot be judged by a simple Y-axis coordinate, because the text may have a superior-subordinate relationship. For example, the content of Title is "1 Introduction" and "1.1 Basis for Topic Selection". Therefore, regular expressions are further used to match their subordinate relationships. For the first-level title, the regular expression \d? [\u4e00-\u9fa5]+ is used, where \d determines whether the text is a number, and a \d is added to the expression for each level down of the title; ? is used to determine whether the text is a space; \u4e00-\u9fa5 belongs to unicode encoding and is used to determine whether the text is Chinese characters. The purpose of class relationship reorganization is achieved through position information and regularized expressions, and the required directory structure information can be obtained in the end.

[0023] S5. Information processing: Process the directory structure information obtained in S4 to form a data table containing text information of the standard PDF document, and store it in the constructed robot standard database.

[0024] After obtaining the structural relationship between each part of the document, a database is created, the recognized content is converted into a data table, and stored in the database to provide an index to facilitate user queries. The information in the data table includes the name, number of pages, class name, parent title, text content, etc. of the PDF document. Once the user searches for keywords in the title, the title and all the contents contained in its subordinate titles, including the number of pages and corresponding text content, will pop up.

[0025] It reduces the retrieval time of the content of robot standard documents and effectively restores the directory structure information, reduces the storage consumption of documents, improves the reading efficiency of documents, facilitates users to obtain the text information of robot standard documents, improves reading efficiency, and further enhances readers' understanding of robot safety and reliability standards.

[0026] Intelligent recognition technology can automatically analyze document layouts, identify elements such as images, text, and tables, and store them in a structured manner, greatly reducing the need for manual operations. The automated processing process greatly increases document processing speed, especially for the rapid processing of large quantities of documents, where the advantages are even more obvious.

[0027] According to specific needs, the intelligent recognition system can be customized and developed to adapt to specific layout analysis and structured storage requirements. The structured stored data can be easily read and processed directly by subsequent systems or programs, which improves the reusability of the data. By improving processing speed and accuracy, intelligent recognition technology helps enterprises or individuals complete document processing tasks faster, thereby improving overall work efficiency.

[0028] Layout analysis is a key step in document processing, which involves area division, element recognition and structural analysis of document images. Layout analysis based on intelligent recognition mainly relies on OCR (optical character recognition) technology and its extended functions, such as layout analysis and understanding technology.

[0029] OCR technology scans document images, identifies text information in them, and converts them into editable text format. However, traditional OCR technology mainly focuses on text recognition, and has limited processing of layout structure and layout information in documents.

[0030] Furthermore, in S2, after the layout analysis, the coordinate information and corresponding tags of the title part and the text part in the standard PDF document are obtained.

[0031] Furthermore, in S2, the standard images after layout analysis are classified based on content and layout structure.

[0032] Furthermore, in S2, the classification specifically includes the following steps: S21, creating a data set based on a standard PDF document for use in the layout analysis network training, and manually annotating the image to obtain a standard image; In order to make the layout analysis network more accurate in classifying robot standard documents, a dataset of robot standard documents was created for training the layout analysis network. The training set contains 1,000 images of robot standard documents promoted by major Chinese associations, and the images are manually annotated. Among them, the dataset contains the header "Header", text "Text", picture title "Table caption", picture "Figure", table "Table", table title "Table caption", title "Title", and footer "Footer" in some robot standard documents, a total of 8 types of labels.

[0033] Layout analysis and understanding technology is a further extension based on OCR. It aims to perform structured analysis on the entire document or page in order to better extract and utilize information, including identifying titles, paragraphs, tables, graphics, pictures and other elements in the document, and analyzing their positional relationships and logical structures.

[0034] With the development of artificial intelligence technology, deep learning, computer vision and other technologies have been widely used in the field of layout analysis. These technologies can automatically learn the characteristics of document images and accurately identify various elements in them.

[0035] S22, training the layout analysis network using the training set, and constraining the layout analysis network through a loss function to improve the accuracy of the analysis; The layout analysis network was trained for 100 iterations using the dataset of robot standard documents created.

[0036] S23. By integrating multi-scale information through a layout analysis network that fuses multi-scale information, feature discrimination is enhanced, and the content of a standard image obtained from a standard PDF document is accurately classified.

[0037] The content in the PDF document image is classified through a layout analysis network based on multi-scale information fusion to form a prediction box and obtain the coordinate information and semantic information of each part. The coordinate information is modeled using a plane rectangular coordinate system, including the position information of the X-axis and the Y-axis. The classification results are further manually reviewed, and incorrect information is manually adjusted to change its category information, thereby further ensuring the accuracy of document content classification.

[0038] Furthermore, in S21, the layout analysis network is built using the pytorch deep framework, and the network includes: a feature processing module, a multi-scale feature fusion module, and a semantic preservation module.

[0039] Furthermore, the calculation process of the layout analysis network is as follows: the standard image is input into the feature processing module, and after n times of convolution operation, features are extracted. Taking 5 times as an example, the size of the original image is obtained. Figure 1 / 2, 1 / 4, 1 / 8, ... and 1 / 32 scale feature maps, and then input the feature maps into a multi-scale feature fusion module, complete upsampling from top to bottom, and fuse with the features of the same size generated from bottom to top, fuse multi-scale information in the features, predict targets of different sizes at different scales, and based on the final feature map obtained by the convolution upsampling operation, the semantic preservation module includes a first module and a second module, the first module is used to predict the probability map, and the second module is used to predict the threshold map, the probability map is processed by the threshold map, and the binary map is obtained after binarization, and the annotation box of the text area is obtained after processing.

[0040] Furthermore, in S2, the loss function used to optimize the layout analysis network training process is: .

[0041] in: L s represents the standard document semantics preservation loss, L b represents the probability mapping prediction loss, L t represents the threshold map prediction loss; α and β are hyperparameters and are set to 1 and 10 respectively.

[0042] Standard document semantics preservation loss L s and probability mapping prediction loss L b The composition is as follows:

[0043] in: S l is the sample set obtained through sampling, x i For the i The characteristics of the samples, y i For the i The predicted labels for samples.

[0044] Threshold Mapping Prediction Loss L t The composition is as follows:

[0045] Where: R d are all the pixels obtained after the annotation box is expanded. is the value of the i-th point in the threshold map output by the network, is the value of the i-th point in the threshold map calculated from the true label of the training set, and ‖ represents the L1 norm.

[0046] Furthermore, in S2, the layout analysis network is trained and optimized using the stochastic gradient descent method and the Adam optimizer.

[0047] Furthermore, in S3, the text recognition is performed using the paddleOCR framework, and the text recognition model selected by paddleOCR is PP-OCRv4.

[0048] The working principle and use process of the present invention are as follows: After the present invention is installed, the PDF document related to the safety and reliability standards of the robot is converted into a picture form, and is binarized to remove the picture watermark, and the image is data labeled. The layout analysis network S1 based on multi-scale information fusion performs layout analysis on the standard image obtained by processing the PDF document, and obtains the coordinate information and corresponding labels of each part. The text recognition network is used to perform text recognition on each classified area in the document, and the category relationship of the recognized results is reorganized using the directory information restoration method to obtain the directory structure information of the robot standard document. The obtained directory structure information is processed to form a data table containing the text information of the robot standard document, and it is stored in the constructed robot standard database.

[0049] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A standard layout analysis and structured storage method based on intelligent recognition, characterized in that: The steps include: S1. Data labeling: Convert the standard PDF document related to the safety and reliability standards of the robot into an image, perform binarization on it to remove the image watermark, and label the image to obtain a standard image; S2, coordinate information: through the layout analysis network of multi-scale information fusion, the standard image obtained in S1 is analyzed to obtain the coordinate information and corresponding labels of each part of the content in the standard PDF document; S3, text recognition: perform text recognition on the text content of each area in the standard PDF document through a text recognition network to obtain the recognition results of each part of the text content; S4, directory information: the results identified in S3 are reorganized according to different categories using the directory information restoration method to obtain directory structure information based on the standard PDF document; S5. Information processing: Process the directory structure information obtained in S4 to form a data table containing text information of the standard PDF document, and store it in the constructed robot standard database.

2. The method for analyzing and structuring standard layouts based on intelligent recognition according to claim 1, characterized in that: In S2, after the layout analysis, the coordinate information and corresponding tags of the title part and the text part in the standard PDF document are obtained.

3. The method for analyzing and structuring standard layouts based on intelligent recognition according to claim 1, characterized in that: In S2, the standard images after layout analysis are classified based on content and layout structure.

4. The method for analyzing and structuring standard layouts based on intelligent recognition according to claim 3, characterized in that: In S2, the classification specifically includes the following steps: S21, creating a data set based on a standard PDF document for use in the layout analysis network training, and manually annotating the image to obtain a standard image; S22, training the layout analysis network using a training set, and constraining the layout analysis network using a loss function; S23. The multi-scale information is integrated through a layout analysis network of multi-scale information fusion to classify the content of the standard image obtained from the standard PDF document.

5. The method for analyzing and structuring standard layouts based on intelligent recognition according to claim 4, characterized in that: In S21, the layout analysis network is built using the pytorch deep framework, and the network includes: a feature processing module, a multi-scale feature fusion module and a semantic preservation module.

6. The method for analyzing and structuring standard layouts based on intelligent recognition according to claim 5, characterized in that: The calculation process of the layout analysis network is: The standard image is input into the feature processing module. After n times of convolution operation to extract features, the sizes of the original image are 1 / 2, 1 / 4, 1 / 8, ... and 1 / 2. n Mapping characteristics of scale; The mapping features are then input into the multi-scale feature fusion module, where they are upsampled from top to bottom and fused with the features of the same size generated from bottom to top to obtain mapping features that fuse multi-scale information. The semantics preservation module is used to process the mapping features of the fused multi-scale information to obtain a labeling box of the text area, wherein the labeling box records the coordinate information and the corresponding label.

7. The method for analyzing and structuring standard layouts based on intelligent recognition according to claim 6, characterized in that: The semantic preservation module includes a first module and a second module; the first module is used to predict probability mapping and obtain a probability map of mapping features that integrates multi-scale information; the second module is used to predict threshold mapping, obtain a threshold map of the probability map after binarization, obtain position information and label information based on the threshold map, and obtain a prediction box of the text area based on the position information and label information.

8. The method for analyzing and structuring standard layouts based on intelligent recognition according to claim 1, characterized in that: In S2, the loss function used to optimize the layout analysis network training process is: 。 9. The method for analyzing and structuring standard layouts based on intelligent recognition according to claim 1, characterized in that: In S2, the layout analysis network is trained and optimized using the stochastic gradient descent method and the Adam optimizer.

10. The method for analyzing and structuring standard layouts based on intelligent recognition according to claim 1, characterized in that: In S3, the text recognition is performed using the paddleOCR framework, and the text recognition model selected by paddleOCR is PP-OCRv4.

Images