Automatic identification and extraction method for complex images and tables of railway geological files

Abstract

The invention discloses an automatic identification and extraction method for complex images and tables of railway geological files, which comprises the following steps of: firstly, automatically identifying whether a file to be extracted belongs to the type of a picture or a table by designing a classification method, and secondly, carrying out automatic data extraction by adopting different identification technologies according to an identification result. The method specifically comprises the following steps of: obtaining text description in an image by adopting an OCR (Optical Character Recognition) technology based on deep learning for data of a picture type, firstly identifying frame lines of a table for data similar to the table, and then identifying and extracting content of the table by adopting a table image identification method. The corpora are subjected to data cleaning and formatting processing, and a standard data format is constructed. The constructed method not only can efficiently identify and extract the text information in the railway geological file, but also can accurately analyze and structuralize the table data.

Description

Technical Field

[0001] The invention relates to the technical field of railway engineering, and in particular to a method for automatically identifying and extracting complex images and tables of railway geological documents. Background Art

[0002] In the field of railway engineering, adverse geological conditions such as landslides, debris flows, karst, faults, etc. are key factors affecting the safety, efficiency and cost of railway construction. Traditionally, the identification and handling of these conditions rely on the experience and intuition of engineers. This method is inefficient and has the risk of errors. Especially in the current process of digital transformation and intelligent evolution of enterprises, an automated document OCR recognition and extraction method is urgently needed to improve work efficiency. However, the large amount of geological data and documents involved in railway engineering are challenging to effectively integrate for decision-making.

[0003] In order to overcome these challenges, the present invention proposes an automatic document OCR recognition and extraction method based on deep learning, which uses optimized and improved OCR technology to automatically recognize and extract complex images and table contents in railway geological documents. The method based on the deep learning model constructed by the present invention can systematically and effectively parse the special formats such as pictures, formulas, tables, subscripts and subscripts, as well as text in the specifications and standards, and convert them into various types of editable data. Summary of the invention

[0004] The present application provides a method for automatically recognizing and extracting complex images and tables of railway geological documents, so as to solve the problem of inaccurate and difficult extraction of special formats such as pictures, formulas, tables, superscripts and subscripts, and text content in the process of compiling geological specifications and standards.

[0005] According to the first aspect, an embodiment provides a method for automatically identifying and extracting complex images and tables of railway geological documents, the method comprising:

[0006] Obtain railway geological scan files and pre-process the files;

[0007] Perform table or image type classification detection on the pre-processed files;

[0008] For table data, table preprocessing, table frame line detection, table frame line removal, table content recognition and table information extraction are performed in sequence to obtain table recognition results;

[0009] For image data, it is processed sequentially through the deep convolution layer, the recurrent layer, and the transcription layer to obtain the image recognition result;

[0010] The obtained table recognition results and image recognition results are integrated and normalized.

[0011] Furthermore, the railway geological scan files are obtained and preprocessed, including:

[0012] Perform noise reduction, binarization and tilt correction on the file.

[0013] Furthermore, the preprocessed files are classified and detected into table or image types, including:

[0014] A pre-trained classification model is used to classify the documents to be identified. The training method of the classification model includes: first, semi-automatic manual annotation of the data in the existing documents is performed to divide the document part in the table format and the document part in the image format; secondly, the classification model is trained using the annotated data and the effect is tested; the trained model is used to classify the documents to be identified.

[0015] Furthermore, the table frame line detection specifically includes:

[0016] First, the horizontal and vertical lines in the table image are extracted, and the extracted straight lines contain the general framework of the entire table;

[0017] Secondly, the roughly extracted frame is subjected to horizontal and vertical erosion and dilation operations to eliminate small objects and smooth the boundaries of large objects, and two horizontal and vertical images are obtained at the same time;

[0018] Finally, after the straight line coordinates in the two images are stored separately, the image merging operation is performed to complete the table frame line detection.

[0019] Furthermore, the table content recognition specifically includes:

[0020] The table structure segmentation and recognition model based on the SPLERGE algorithm is adopted.

[0021] Furthermore, the table information extraction specifically includes:

[0022] Use Tesseract-OCR font library to identify and extract the information in the table;

[0023] The identified results are integrated according to the format and order of the original table to obtain editable reconstructed table content.

[0024] Furthermore, the processing of the deep convolutional layer specifically includes:

[0025] The detected and annotated image is input into the deep convolution layer, and the text features are extracted through convolution calculation, activation calculation, pooling calculation and feature extraction, and finally output as a feature sequence.

[0026] Furthermore, the processing of the loop layer specifically includes:

[0027] The feature sequence is input into the recurrent layer, where a bidirectional Bi-LSTM network is used to further process the feature sequence. The output of the Bi-LSTM network is a high-dimensional feature vector sequence, where the vector of each time step represents the text feature at the corresponding position.

[0028] Furthermore, the transcription layer processing specifically includes:

[0029] The transcription layer uses the CTC model to process the output of the Bi-LSTM network. The CTC model converts the feature sequence into a text sequence and converts the continuous feature vector into discrete characters or words through a decoding operation. The output of the transcription layer is a verified and converted text sequence, which represents the text content recognized in the image and forms the final recognition result.

[0030] According to the second aspect, an embodiment provides a system for automatically identifying and extracting complex images and tables of railway geological documents, the system comprising:

[0031] A file preprocessing module is used to obtain railway geological scanning files and preprocess the files;

[0032] A classification and recognition module is used to classify and detect the table or image type of the pre-processed files;

[0033] The table extraction module is used to perform table preprocessing, table frame line detection, table frame line removal, table content recognition and table information extraction in sequence for table data to obtain table recognition results;

[0034] The image extraction module is used to process image data through the deep convolution layer, the recurrent layer and the transcription layer in sequence to obtain image recognition results;

[0035] The result integration module is used to integrate and normalize the obtained table recognition results and image recognition results.

[0036] According to the third aspect, an embodiment provides an electronic device, the device comprising: a processor and a memory;

[0037] The memory is used to store one or more program instructions;

[0038] The processor is used to run one or more program instructions to execute the steps of the method for automatically identifying and extracting complex images and tables of railway geological documents as described in any of the above items.

[0039] According to the fourth aspect, an embodiment provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of a method for automatically identifying and extracting complex images and tables of railway geological documents as described in any of the above items are implemented.

[0040] The present application provides a method for automatically identifying and extracting complex images and tables of railway geological documents, which has the following beneficial effects:

[0041] (1) The present invention addresses the problem of poor recognition of geological standard documents by conventional OCR, and significantly improves the automation and intelligence level of geological scan file processing by constructing automatic recognition technology for complex images and tables. In particular, the geological scan file processing system of the present invention can automatically perform tasks such as SVM classification of document content, image and table preprocessing, feature extraction, table frame line detection and removal, and content recognition. This automated process greatly reduces manual operations and improves the speed and accuracy of data processing.

[0042] (2) Through the automatic recognition and application of images and tables, the present invention effectively reduces the workload of staff and improves decision-making efficiency. At the same time, through automated and intelligent data extraction and analysis, the present invention improves the quality of data and ensures the smooth progress of railway projects.

[0043] (3) Through this method, a system can be constructed to convert standard files in PDF or various image formats into word format. Since the standard files contain special formats such as pictures, formulas, tables, subscripts and subscripts, as well as text, the system can provide accurate file content extraction solutions, thereby improving the safety and efficiency of railway engineering. The geological scanning file processing system integrates advanced image processing and machine learning technologies to achieve automated, precise and standardized processing of geological data, bringing significant benefits to research and engineering practice in the field of geology. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A flowchart of a method for automatically identifying and extracting complex images and tables of railway geological documents provided by one embodiment of the present invention;

[0045] Figure 2 An overall framework flow chart of a method for automatically identifying and extracting complex images and tables of railway geological documents provided by one embodiment of the present invention;

[0046] Figure 3 A schematic diagram of the logical structure of a system for automatically identifying and extracting complex images and tables in railway geological documents provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0047] The present invention is further described in detail below by specific embodiments in conjunction with the accompanying drawings. Wherein similar elements in different embodiments adopt associated similar element numbers. In the following embodiments, many detailed descriptions are for making the present application better understood. However, those skilled in the art can easily recognize that some features can be omitted in different situations, or can be replaced by other elements, materials, methods. In some cases, some operations related to the present application are not shown or described in the specification, this is to avoid the core part of the present application being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0048] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various implementations. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a required sequence, unless otherwise specified that a certain sequence must be followed.

[0049] The present invention focuses on building a geological document OCR automatic extraction system, which uses advanced artificial intelligence technology, especially for the automatic processing and analysis of data in the field of geology, aiming to provide a comprehensive and efficient solution to improve the processing flow of geological documents, thereby improving the accuracy of information extraction, and further assisting the digital and intelligent development and transformation of related enterprises.

[0050] The first embodiment of the present invention provides a method for automatically identifying and extracting complex images and tables of railway geological documents. Figure 1 and Figure 2 Provide detailed explanation.

[0051] like Figure 1 As shown, in step S100, a railway geological scan file is obtained and pre-processed.

[0052] Noise is generated during the acquisition and transmission of documents. The presence of noise has a great impact on the quality of the image, such as causing image blur and reduced recognition accuracy. The image preprocessing process is divided into three parts, namely noise reduction, binarization and tilt correction. The details are as follows:

[0053] S110, noise reduction. The common noises in the scanned image are mainly salt and pepper noise, Gaussian noise, Poisson noise, and multiplicative noise. In view of these noises, this embodiment adopts the Gaussian filtering method in the traditional digital image to perform noise reduction on the image. Gaussian filtering is a linear smoothing filter, which converts the image into a combination of frequency domain processing and time domain processing, and performs average weighted calculation on the image to be identified, so that each pixel on the image is calculated by itself and other adjacent pixel values, thereby filtering out the noise information.

[0054] S120, binarization. Binarization is to convert the values ​​in all pixel matrix of the image to 255 (white) or 0 (black), so that the whole image presents the effect of only black and white. Binarization converts the grayscale image into a binary image. This step can significantly reduce the interference information in the image. This embodiment uses the adaptive threshold binarization cv2.adaptiveThreshold function to binarize the image. This function is more effective in removing the background and extracting useful information of the foreground.

[0055] S130, tilt correction. Scanned document data is inevitably tilted when photographed, which will interfere with the subsequent table processing, especially when the text cannot be divided into individual characters, thereby reducing the recognition accuracy. Therefore, the tilted image needs to be corrected during the preprocessing process. This embodiment adopts Hough transform, and its basic implementation principle is to first identify the geometric figures in the image, detect the corresponding straight line therefrom, and then determine the degree of inclination of the figure by calculating the inclination angle of the straight line, and finally perform rotation correction; since the frame line of the table image is horizontal and vertical, Hough transform is used to perform tilt correction on the table image, thereby obtaining a good recognition result.

[0056] like Figure 1 As shown, in step S200, the pre-processed file is classified and detected as a table or image type.

[0057] Table and image position detection. Specifically, firstly, based on the existing document data, the data in the document is semi-automatically annotated manually with the help of PPOCRLabel. The annotated content is mainly to divide the table part and other document parts in picture format. Secondly, the SVM algorithm is used to train the annotated data and test the effect. The training set and test set are divided into 80% and 20%, and finally a robust model with an accuracy rate of more than 90% is obtained, which is used to classify the subsequent data. The images to be recognized are classified, and different subsequent processes are entered according to the type of the image to be recognized. The OCR system needs to preprocess the image to be recognized to improve the accuracy of recognition in subsequent processes.

[0058] like Figure 1 As shown, in step S300, for table data, table preprocessing, table frame line detection, table frame line removal, table content recognition and table information extraction are performed in sequence to obtain a table recognition result.

[0059] Specifically, according to the classification results obtained in the above steps, if the classified category belongs to table-type data, the identification base is extracted according to the following steps to finally obtain the recognition result of the table content. The specific process is as follows:

[0060] S310, table preprocessing. In order to extract the classified table data more accurately, based on the first step, the table image is tilted again, and the Hough transform is also used to effectively optimize the data after the first step of image tilt correction.

[0061] S320, table frame line inspection. Correctly extracting grid lines is the basis for dividing grid units, which is related to whether the grid information can be correctly divided, and is the most critical link in the automatic processing of grids. This embodiment adopts a Hough transform algorithm based on image morphology improvement. The algorithm first uses the Hough transform algorithm to extract the horizontal and vertical lines in the table image. The extracted straight lines contain the general framework of the entire table. Secondly, the roughly extracted frame is respectively subjected to corrosion operations and expansion operations of the horizontal and vertical lines to achieve the effect of eliminating small objects and smoothing the boundaries of large objects, and two horizontal and vertical images are obtained at the same time. Finally, after the straight line coordinates in the two images are stored separately, the image merging operation is performed to complete the inspection of the table frame.

[0062] S330, table frame line removal. In order to eliminate the influence of table borders on table field character recognition, the detected table borders need to be deleted before recognition. Another important task in the table recognition process is to delete the detected borders from the original table image. Considering the simple and stable geometric features of the table frame, this embodiment adopts an image analysis technology based on simple prior knowledge, and here this embodiment adopts a method based on corrosion and expansion in morphology to remove lines.

[0063] S340, table content recognition.

[0064] The author of the SPLERGE algorithm for identifying table structures by bidirectional splitting and merging points out that some tables have a large amount of blank areas, and it is difficult to identify the cell borders of the table using only the local features of the cells or pixels. Considering this characteristic of the table, this embodiment adopts the table structure splitting and merging recognition model - the SPLERGE algorithm.

[0065] The segmentation model of the SPLERGE algorithm consists of a shared fully convolutional network (SFCN), a row projection network (RPN), and a column projection network (CPN). In order to expand the local field of view of SFCN without changing the size of the feature map to obtain more local information fusion, a dilated convolution layer (DC) is used in the last layer of SFCN. At the same time, the concept of projection pooling is proposed in RPN and CPN. The concept of "projection pooling" is an important improvement in the network structure. This pooling operation is proposed in the row projection network (RPN) and the column projection network (CPN). It allows the model to expand the receptive field and obtain more local information without changing the size of the feature map. In RPN and CPN, projection pooling is to aggregate row and column features by averaging along the rows and columns of pixels respectively, so that local row and column features can be propagated throughout the entire image. The final segmentation model obtains the 0 / 1 probability that each row / column pixel belongs to the logical segmentation area of ​​the row / column.

[0066] Projection pooling is achieved by averaging along rows and columns of pixels, which helps to aggregate row and column features. This pooling operation allows local row and column features to propagate throughout the image, allowing the model to exploit the global structural information of the table. This is especially important for processing tables that contain a large amount of blank areas, because it may be difficult to accurately identify the cell borders of the table using only the local features of cells or pixels. At the same time, the introduction of projection pooling in the SPLERGE algorithm is an important improvement to the network structure, which enables the model to better process the global structural information in the table image, thereby improving the accuracy of table structure recognition.

[0067] The merge model of the SPLERGE algorithm receives the original image and the results from the segmentation model at the same time, and assembles the vertical adjacency matrix D and the horizontal adjacency matrix R according to the structure of the original image, where the element D of the adjacency matrix is i,j and R i,j They represent the probability that cell (i,j) needs to be merged with cell (i+1,j) and (i,j+1) respectively. Finally, through the learning of convolutional neural network, the potential features of the table grid structure are learned. The average pooling operation after the convolution layer is called grid pooling. The output of the merging model is a merged table structure, in which adjacent cells are appropriately merged according to the merging probability to form the final table layout.

[0068] S350, table information extraction. Since the characters on the form inherently contain numbers, English, Chinese, special characters, etc., according to the characteristics of these data, this embodiment uses the Tesseract-OCR character library to recognize the table information.

[0069] S360, table recognition result: According to the recognition result of S350, the recognition result is integrated according to the format and order of the original table to obtain editable reconstructed table content.

[0070] like Figure 1 As shown, in step S400, the image data is processed in sequence through a deep convolution layer, a recurrent layer, and a transcription layer to obtain an image recognition result.

[0071] The purpose of table-based text detection is to identify the text area in the image, and then recognize the image result of the text detection and convert it into corresponding text. According to the classification result obtained in the above step S200, if the classified category belongs to image-type data, the recognition base is extracted according to the following steps to finally obtain the recognition result of the image content. The specific process is as follows:

[0072] S410, deep convolutional layer.

[0073] The detected and annotated image is input into the deep convolution layer, and the text features are extracted through convolution calculation, activation calculation and pooling calculation. The detailed steps of this operation are as follows: Convolution calculation, in the deep convolution layer, multiple convolution kernels (or filters) are used to slide on the image to perform convolution operations. The convolution kernel is element-wise multiplied with the local area of ​​the image and then summed to generate feature maps, which capture the local features in the image, such as the edges and corners of the text. Activation calculation, the output of the convolution layer is usually passed through a nonlinear activation function, such as ReLU (Rectified Linear Unit), to introduce nonlinear characteristics. The role of the activation function is to increase the expressive power of the model and help the network learn more complex features. Pooling calculation, use the pooling layer (such as maximum pooling or average pooling) to downsample the feature map, reduce the size of the feature map, thereby reducing the amount of calculation and extracting more abstract feature representations, while increasing the invariance to image displacement. Feature extraction, through the above convolution, activation and pooling operations, the deep convolution layer can extract text features in the image, which will then be used in the next step of text recognition. Output feature sequence: The final output of the deep convolutional layer is a feature sequence, which contains the key visual information of the text in the input image and provides rich feature input for the subsequent recognition network.

[0074] S420, circulation layer.

[0075] After being integrated into a feature sequence, it is input into the recurrent layer, and the text recognition prediction result is obtained through the bidirectional long short-term memory network. The recurrent layer is a key component in the text recognition network. It uses the bidirectional long short-term memory network (Bi-LSTM) to process the feature sequence extracted from the deep convolution layer. The following is a detailed description of the process: The above S41 serialized features are input into the recurrent layer. The recurrent layer can process sequence data and capture dynamic features in time series. In the recurrent layer, the feature sequence is further processed using a bidirectional LSTM network. LSTM is a special type of recurrent neural network (RNN) that can learn long-term dependencies and avoid the gradient disappearance problem. In particular, Bi-LSTM contains two LSTM layers, which process the input sequence in forward and reverse directions respectively. The forward LSTM processes from the beginning to the end of the sequence, while the reverse LSTM processes from the end to the beginning of the sequence. Through the processing of the bidirectional LSTM, the network can simultaneously consider the contextual information, enhance the expressiveness of the features, and provide rich contextual information for text recognition. After the Bi-LSTM layer, the representation of the feature sequence is further refined and enhanced, and these representations are then used to generate prediction results for text recognition. The output of the Bi-LSTM network is a continuous sequence of vectors, and the vectors at each time step represent the text features at the corresponding position. These vectors are then sent to the transcription layer for the final recognition conversion. Finally, the output of the recurrent layer is used to predict the text sequence, that is, to recognize the text content in the image and generate text recognition prediction results. In this way, the system can effectively recognize text in various fonts, sizes, and directions.

[0076] S430, transcription layer.

[0077] The prediction results are then verified by the CTC model of the transcription layer and converted into the final recognition results. The transcription layer mentioned in the document is the last stage in the text recognition network. It uses the CTC (Connectionist Temporal Classification) model to convert the output of the recurrent layer into the final text recognition result. The output of the Bi-LSTM layer is passed to the transcription layer, which is usually a high-dimensional feature vector sequence. The vector of each time step contains the potential features of the corresponding text position. The transcription layer uses the CTC model to process the output of the Bi-LSTM. CTC is a loss function for sequence recognition. It can map the features of the input sequence to the label of the output sequence, even if the lengths of the input and output sequences are different. The CTC model can handle spaces and repeated labels. It scores each possible output sequence in a probability distribution manner and selects the sequence with the highest probability as the prediction result. The CTC model converts the feature sequence into a text sequence. This process involves a decoding operation, which converts the continuous feature vector into discrete characters or words. The output of the transcription layer is a verified and converted text sequence, which represents the text content recognized in the image and forms the final recognition result.

[0078] S440, image recognition result: Based on the recognition result obtained in S430, the recognition result is integrated according to the order of the characters in the original image to obtain an editable reconstructed image recognition result.

[0079] like Figure 1 As shown, in step S500, the obtained table recognition results and image recognition results are integrated and normalized.

[0080] By making comprehensive use of the results of all the above processes, the necessary information is filtered out according to the order in which the tables and other texts in the original document and images appear, and the identified data is integrated into an editable file such as word or txt.

[0081] Through the above steps, the automatic OCR extraction of complex images and tables in railway geological documents can be completed, which will help the digitalization of the railway geological field, improve procurement efficiency, save a lot of manpower costs, and provide support for the digitalization of railway geology.

[0082] The automatic OCR model proposed in the present invention combines artificial intelligence algorithms, which can perform pattern recognition on standard documents in PDF or various image formats, and convert the recognized content into editable files according to their corresponding formats, thereby providing accurate and real-time decision-making suggestions, thereby improving the efficiency of railway engineering document processing and increasing the efficiency of the digitalization process. By integrating poor geological data and document information, the constructed automatic OCR recognition and extraction model can not only serve as an intelligent consulting assistant, but also perform poor geological information analysis, which significantly improves the digital efficiency of geological information analysis in the railway system and helps the digitalization and intelligent development of enterprises. It has the following specific innovations:

[0083] (1) The present invention trains and optimizes the model based on the characteristics of the geological industry. The system integrates multiple steps such as document classification, image preprocessing, feature extraction, table detection and content recognition to form a comprehensive automated processing flow, which improves the model's adaptability to geological documents and recognition accuracy.

[0084] (2) The present invention firstly uses support vector machine (SVM) to automatically classify document content, which may improve the accuracy and efficiency of classification. Secondly, preprocessing and nonlinear mapping techniques are used in the processing of image files, including image contraction and expansion operations. These techniques may be used to improve image quality and provide better image input for feature extraction. Nonlinear mapping is used in the feature extraction stage, which may help the system capture more complex image features and improve recognition accuracy. Table processing uses table frame line detection and removal, table image content recognition and other technologies, specifically including the processing of geological table documents, by detecting and removing table frame lines, improving the recognition accuracy of table content.

[0085] (3) Finally, the recognition results of tables and images are normalized to ensure that the output data format is unified to facilitate subsequent storage, analysis, and use.

[0086] Corresponding to the above-disclosed method for automatically identifying and extracting complex images and tables of railway geological documents, the embodiment of the present invention further discloses a system for automatically identifying and extracting complex images and tables of railway geological documents, such as Figure 3 As shown, it specifically includes:

[0087] A file preprocessing module is used to obtain railway geological scanning files and preprocess the files;

[0088] A classification and recognition module is used to classify and detect the table or image type of the pre-processed files;

[0089] The table extraction module is used to perform table preprocessing, table frame line detection, table frame line removal, table content recognition and table information extraction in sequence for table data to obtain table recognition results;

[0090] The image extraction module is used to process image data through the deep convolution layer, the recurrent layer and the transcription layer in sequence to obtain image recognition results;

[0091] The result integration module is used to integrate and normalize the obtained table recognition results and image recognition results.

[0092] It should be noted that for the detailed description of the system for automatic recognition and extraction of complex images and tables of railway geological documents provided in an embodiment of the present invention, reference can be made to the relevant description of the method for automatic recognition and extraction of complex images and tables of railway geological documents provided in an embodiment of the present application, which will not be repeated here.

[0093] In addition, an embodiment of the present invention also provides an electronic device, which includes: a processor and a memory; the memory is used to store one or more program instructions; the processor is used to run one or more program instructions to execute the steps of a method for automatically identifying and extracting complex images and tables of railway geological documents as described in any of the above items.

[0094] It should be noted that for the detailed description of an electronic device provided in an embodiment of the present invention, reference can be made to the relevant description of a method for automatically recognizing and extracting complex images and tables of railway geological documents provided in an embodiment of the present application, which will not be repeated here.

[0095] In addition, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of a method for automatically identifying and extracting complex images and tables of railway geological documents as described in any of the above items are implemented.

[0096] It should be noted that for the detailed description of a computer-readable storage medium provided in an embodiment of the present invention, reference can be made to the relevant description of a method for automatically recognizing and extracting complex images and tables of railway geological files provided in an embodiment of the present application, which will not be repeated here.

[0097] Those skilled in the art will appreciate that all or part of the functions of the various methods in the above-mentioned embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, and the storage medium can include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to implement the above-mentioned functions. For example, the program is stored in the memory of the device, and when the program in the memory is executed by the processor, all or part of the above-mentioned functions can be implemented. In addition, when all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can also be stored in a storage medium such as a server, another computer, disk, optical disk, flash disk or mobile hard disk, and can be downloaded or copied and saved in the memory of the local device, or the system of the local device is updated, and when the program in the memory is executed by the processor, all or part of the functions in the above-mentioned embodiments can be implemented.

[0098] The above specific examples are used to illustrate the present invention, which are only used to help understand the present invention and are not used to limit the present invention. For those skilled in the art to which the present invention belongs, some simple deductions, modifications or substitutions can be made according to the idea of ​​the present invention.

Claims

1. A method for automatically identifying and extracting complex images and tables of railway geological documents, characterized in that: The method comprises: Obtain railway geological scan files and pre-process the files; Perform table or image type classification detection on the pre-processed files; For table data, table preprocessing, table frame line detection, table frame line removal, table content recognition and table information extraction are performed in sequence to obtain table recognition results; For image data, it is processed sequentially through the deep convolution layer, the recurrent layer, and the transcription layer to obtain the image recognition result; The obtained table recognition results and image recognition results are integrated and normalized.

2. The method for automatically identifying and extracting complex images and tables of railway geological documents as claimed in claim 1, characterized in that: Obtain railway geological scan files and pre-process the files, including: Perform noise reduction, binarization and tilt correction on the file.

3. The method for automatically identifying and extracting complex images and tables of railway geological documents as claimed in claim 1, characterized in that: The pre-processed files are classified and tested by table or image type, including: A pre-trained classification model is used to classify the documents to be identified. The training method of the classification model includes: first, semi-automatic manual annotation of the data in the existing documents is performed to divide the document part in the table format and the document part in the image format; secondly, the classification model is trained using the annotated data and the effect is tested; the trained model is used to classify the documents to be identified.

4. The method for automatically identifying and extracting complex images and tables of railway geological documents as claimed in claim 1, characterized in that: Table frame line detection, including: First, the horizontal and vertical lines in the table image are extracted, and the extracted straight lines contain the general framework of the entire table; Secondly, the roughly extracted frame is subjected to horizontal and vertical erosion and dilation operations to eliminate small objects and smooth the boundaries of large objects, and two horizontal and vertical images are obtained at the same time; Finally, after the straight line coordinates in the two images are stored separately, the image merging operation is performed to complete the table frame line detection.

5. The method for automatically identifying and extracting complex images and tables of railway geological documents as claimed in claim 1, characterized in that: Table content recognition, including: The table structure segmentation and recognition model based on the SPLERGE algorithm is adopted.

6. The method for automatically identifying and extracting complex images and tables of railway geological documents as claimed in claim 1, characterized in that: Table information extraction, including: Use Tesseract-OCR font library to identify and extract the information in the table; The identified results are integrated according to the format and order of the original table to obtain editable reconstructed table content.

7. The method for automatically identifying and extracting complex images and tables of railway geological documents as claimed in claim 1, characterized in that: The processing of the deep convolutional layer includes: The detected and annotated image is input into the deep convolution layer, and the text features are extracted through convolution calculation, activation calculation, pooling calculation and feature extraction, and finally output as a feature sequence.

8. The method for automatically identifying and extracting complex images and tables of railway geological documents as claimed in claim 7, characterized in that: The processing of the loop layer includes: The feature sequence is input into the recurrent layer, where a bidirectional Bi-LSTM network is used to further process the feature sequence. The output of the Bi-LSTM network is a high-dimensional feature vector sequence, where the vector of each time step represents the text feature at the corresponding position.

9. The method for automatically identifying and extracting complex images and tables of railway geological documents as claimed in claim 8, characterized in that: Transcription layer processing, including: The transcription layer uses the CTC model to process the output of the Bi-LSTM network. The CTC model converts the feature sequence into a text sequence and converts the continuous feature vector into discrete characters or words through a decoding operation. The output of the transcription layer is a verified and converted text sequence, which represents the text content recognized in the image and forms the final recognition result.

10. An automatic recognition and extraction system for complex images and tables of railway geological documents, characterized in that: The system comprises: A file preprocessing module is used to obtain railway geological scanning files and preprocess the files; A classification and recognition module is used to classify and detect the table or image type of the pre-processed files; The table extraction module is used to perform table preprocessing, table frame line detection, table frame line removal, table content recognition and table information extraction in sequence for table data to obtain table recognition results; The image extraction module is used to process image data through the deep convolution layer, the recurrent layer and the transcription layer in sequence to obtain image recognition results; The result integration module is used to integrate and normalize the obtained table recognition results and image recognition results.

Images