Water power engineering document text vectorization conversion method based on spatial semantic correction

Abstract

The application discloses a hydropower engineering document text vectorization conversion method based on spatial semantic correction, constructs a hydropower engineering document automatic scanning instrument, and realizes automatic conversion of paper documents into picture formats. A hydropower engineering construction text interpretation dataset and a domain knowledge-based professional corpus DKPC are constructed, the text category and the layout background of the hydropower engineering construction document are acquired, a text detection model HC-PSENet based on pixel-level segmentation is developed, the feature extraction performance of a PP-HGNet backbone network enhanced model is introduced, a text recognition model HC-CRNN based on feature enhancement is established, the accuracy of character recognition of a ResNet backbone network and a C-CTC Loss enhanced model is adopted, and a model is trained based on a parameter-based transfer learning strategy, so as to avoid the problems of sample class imbalance and model overfitting. Finally, the automatic interpretation device of the paper document of the hydropower engineering construction is formed based on the instrument and the algorithm, and the intelligent level of text interpretation is improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent construction technology, specifically to a method for vectorizing hydropower engineering document text based on spatial semantic correction. Background Technology

[0002] Faced with a large volume of complex hydropower engineering construction documents, the project typically employs manual interpretation to extract information from non-editable documents. This method is excessively time-consuming and its accuracy is unpredictable, especially when dealing with complex document layouts where the required time and accuracy cannot be precisely quantified.

[0003] The emergence of deep learning has led to groundbreaking advancements in computer vision and pattern recognition. Text interpretation technology has expanded in depth and breadth, with deep learning-based methods showing significant improvements in both accuracy and efficiency, becoming increasingly adaptable to documents across various domains with diverse characters and complex backgrounds. Some researchers have applied these advancements to general fields such as invoices and medical records, balancing the performance and time consumption of text interpretation models by improving feature extraction and feature fusion modules.

[0004] Although the development of deep learning has significantly improved the accuracy and efficiency of text interpretation methods in general domains, the sheer number of text categories and the complexity of character classification in hydropower engineering documents remain significant challenges. Furthermore, the Chinese characters in hydropower engineering texts exhibit complex stroke structures and contain similar-looking words, making it difficult to distinguish between similar characters based on visual features such as stroke outlines, connectivity, and radicals. This places higher demands on text recognition models for extracting character features. These limitations of general domain technologies persist in hydropower engineering documents with their diverse layouts and complex character categories, severely hindering the development and progress of intelligent text processing technology in the water conservancy and hydropower industry.

[0005] Text interpretation is a crucial component of pattern recognition, comprising two main tasks: text detection and text recognition. In text detection, the complexity and diversity of text types in hydropower engineering construction documents—including paragraphs, table names, and footers—need text detection models to focus on extracting multi-scale text line features. In text recognition, unlike English documents composed of only 26 Latin letters, hydropower engineering construction documents contain a vast number of text categories and more complex character classifications. Furthermore, the Chinese characters in hydropower engineering construction texts exhibit complex stroke structures and contain similar-looking words; similar characters lack clear distinction in visual features such as stroke outlines, connectivity, and radicals, placing higher demands on text recognition models for character feature extraction.

[0006] Although relatively mature OCR technologies exist that can convert uncompilable text, the adaptability of existing OCR technologies to the specific characteristics of texts in the hydropower engineering field needs improvement. Furthermore, a complete automated text conversion system is currently lacking; documents such as books still require manual scanning followed by conversion using commercial software, indicating that automation levels need further development. Summary of the Invention

[0007] To address the aforementioned issues, this invention provides a method for vectorizing hydropower engineering document text based on spatial semantic correction, enabling the automated conversion of unstructured text and image data from hydropower engineering construction into editable text. Based on automatic scanning equipment and intelligent algorithms, this method efficiently and quickly transforms uneditable text existing on paper into editable electronic documents, automating the reuse of uneditable resources from hydropower engineering construction and significantly enhancing data service capabilities in the field of water conservancy and hydropower engineering.

[0008] To achieve the above-mentioned technical features, the objective of this invention is as follows: a method for vectorizing hydropower engineering document text based on spatial semantic correction, comprising the following steps:

[0009] Step 1: Build an automatic scanning device for hydropower project construction documents:

[0010] This automatic text scanner automatically flips through and scans construction documents for hydropower projects, creating image documents. This automatic text scanner then enables the automatic conversion of paper-based construction documents for hydropower projects into image-based documents.

[0011] Step 2, construct a dataset for interpreting construction texts of hydropower projects:

[0012] By batch processing uneditable hydropower engineering construction text into image format, the unstructured text images in hydropower engineering construction documents are further segmented, and data augmentation is used to expand the training samples of the text interpretation model.

[0013] Step 3: Construct the HC-PSENet model for detecting construction text in hydropower projects. Input unstructured text images, and output text regions containing location information in units of text lines through text region feature extraction, fusion, and progressive scale expansion post-processing.

[0014] Step 4: Analyze the domain knowledge of hydropower engineering construction documents, supplement professional terms and special characters to form a professional corpus;

[0015] Step 5: Establish the HC-CRNN text recognition model for hydropower engineering construction. Input the detected text region, and sequentially go through the processes of extracting text features, predicting text sequences, and mapping text labels to output text information in the form of editable characters.

[0016] Step 6: Introduce a two-stage transfer learning strategy. Without requiring a large amount of labeled data and training costs, the learned knowledge or model parameters can be applied to relevant fields, enabling the generalized model to quickly adapt to personalized needs.

[0017] Step 7: Construct a vectorization conversion device for hydropower engineering construction documents. Establish an information flow between the automatic text scanning machine and the intelligent algorithm. After scanning, the scanned image documents are transmitted to the background algorithm. After text detection and text recognition, the vectorized documents are output to the user.

[0018] Step 1 also includes the following steps:

[0019] Step 1-1, Document Preparation:

[0020] The automatic text scanner includes a power port and a document inlet. The power port is plugged in, the document inlet is opened and the paper document to be scanned is placed in the slot, and the document inlet is closed. When the document inlet is closed, the page turning brush will adjust its direction through the page turning axis so that the page turning brush is close to the document. In order to prevent the pages of perfect bound documents from curling up, a page fixing device is added to fix the document pages, thereby completing the preparation work for scanning the document.

[0021] Steps 1-2: Start the scan:

[0022] After clicking the start button, the instrument will start operating. Before starting, select single-sided or double-sided document using the selection button to accommodate different forms of paper documents. During the scanning process, the side lights will be used to supplement the scanning process. The top scanning camera will take pictures of two pages of the document. The images will be automatically recorded in the order of the images. When integrating the images at the end, the file will be saved according to the original page order of the text.

[0023] Steps 1-3, automatic page turning:

[0024] Since paper documents may have multiple pages, an automatic page-turning component is added to the instrument. After the scanning camera performs a single scan, the sensor automatically sends a signal, the page fixing device automatically pops up using a pivot, and the page-turning brush automatically starts to rotate, rolling up the document pages. Using the running device consisting of the page-turning pivot, slider, and base, the page is unfolded from right to left to complete the page turn. After the page turn is completed, the page-turning brush will automatically retract to avoid affecting the text scanning rotation. At the same time, when the sensor detects that the page turn is complete, the page fixing device will automatically lower to fix the two ends of the page, ensuring the flatness of the paper document after the page turn.

[0025] Steps 1-4, scanning complete:

[0026] After repeating steps 1-2 and 1-3 until all documents have been scanned, the scanned image documents are integrated and saved in sequence, and then fed into the subsequent text vectorization decoding algorithm.

[0027] Step 2 also includes the following steps:

[0028] Step 2-1: Batch process the uneditable hydropower engineering construction text into image format and uniformly adjust the resolution to a uniform resolution;

[0029] Step 2-2: Use the OpenCV toolbox to segment unstructured text images in hydropower engineering construction documents, and construct a text detection dataset and a text recognition dataset for hydropower engineering construction documents;

[0030] Steps 2-3 involve using Gaussian blur and contrast adjustment data augmentation methods to expand the training set in the hydropower engineering construction text interpretation dataset multiple times, thereby improving the training accuracy of the model while avoiding overfitting.

[0031] Step 3 also includes the following steps:

[0032] Step 3-1, Construct the PSENet text detection model:

[0033] PSENet is a segmentation-based text detection model that effectively solves the problem of difficult segmentation of neighboring text instances through a progressive scale expansion algorithm. At the same time, it achieves accurate localization of text regions of arbitrary shapes based on pixel-level segmentation, making it suitable for text detection tasks in hydropower engineering construction documents.

[0034] Step 3-2, Construction of the feature extraction layer for the hydropower engineering construction text detection model:

[0035] The feature extraction layer, based on the PP-HGNet backbone network, is responsible for extracting features at different levels of the image. The input image first passes through the Stem image processing module, which improves the model's feature extraction capability without increasing the number of model parameters. It then sequentially passes through four sets of convolutional layers, Conv Stage 1 to Conv Stage 4, to obtain feature maps. The output feature map sizes are 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original input image size, respectively. The Conv Stage consists of HG Block modules containing a large number of standard convolutions, and a learnable downsampling layer, LDS Layer, is introduced to reduce the number of model parameters and computational cost.

[0036] Step 3-3, Feature Fusion Construction of Hydropower Engineering Construction Text Detection Model:

[0037] Inspired by the feature pyramid network, the feature fusion layer combines shallow location information with high-level semantic features using a top-down and lateral connection method to obtain four 256-channel feature maps P2 to P5. The four feature maps are then fused into a 1024-channel feature map F using Equation (1).

[0038] F = P2||Up ×2 (P3)||||Up ×4 (P4)||Up ×8 (P5) (1);

[0039] In the formula: || represents the concatenation operation, Up ×2 Up ×4 Up ×8 These represent upsampling times of 2, 4, and 8, respectively.

[0040] Steps 3-4: Construction of feature maps for hydropower engineering construction text detection model:

[0041] The feature map F is reduced to 256 channels after passing through a 3×3 convolutional layer, a batchnorm layer, and a ReLU layer. Then, it is passed through n 1×1 convolutional layers, an upsampling layer, and a sigmoid layer to obtain n semantic segmentation results S1 to S2. n Where S1 represents the text instance segmentation result with the smallest kernel, S n This represents the original segmentation result with the largest kernel. Finally, the asymptotic expansion algorithm is used to sequentially segment S1 to S... n Scale expansion is performed to obtain the final hydropower engineering construction text detection result R;

[0042] Steps 3-5: Construction of the loss function for the hydropower engineering construction text detection model:

[0043] The loss function of the HC-PSENet construction text detection model for hydropower projects takes into account both the region category and the position of the text detection box, and the expression is shown in Equation (2).

[0044] L=λL f +(1-λ)L s (2);

[0045] In the formula: L f L represents the loss of a complete text instance, used to distinguish between text and non-text regions; s The loss represents the text kernel instance and is used to determine the border positions of the text region; λ is the balance coefficient.

[0046] Steps 3-6: Construct the HC-PSENet model for detecting construction text in hydropower projects.

[0047] The PP-HGNet backbone network is introduced into the PSENet model to construct the HC-PSENet hydropower engineering construction text detection model, which enhances the ability of the text detection model to extract multi-scale text line features. The network structure of the hydropower engineering construction text detection model is divided into three parts: feature extraction layer, feature fusion layer and output layer.

[0048] Step 4 also includes the following steps:

[0049] Step 4-1: Analyze the character categories of hydropower engineering construction documents, which are divided into four categories: basic text, professional text, superscript and subscript characters, and special characters. Basic text consists of text that is widely used in most fields of documents, professional text refers to text that appears frequently in hydropower engineering construction, and superscript and subscript characters and special characters refer to mathematical variables and chemical terms that are frequently used in hydropower engineering construction documents.

[0050] Step 4-2: Construct a domain-knowledge-based professional corpus. Add basic text, professional text, subscript and superscript characters, and special characters to the professional corpus to obtain accurate domain text expressions. The domain-knowledge-based professional corpus fills the gaps in the original corpus in terms of professional text, exponent units, and special symbols, and enhances the model's adaptability to hydropower engineering construction documents.

[0051] Step 5 also includes the following steps:

[0052] Step 5-1, introduce the ResNet network:

[0053] The ResNet network introduces residual modules and uses ResNet-50 as the backbone network of the HC-CRNN text recognition model for hydropower engineering construction. The overall network structure is divided into 5 stages, consisting of 49 convolutional layers and 1 fully connected layer.

[0054] Step 5-2, construct the improved loss function C-CTC Loss:

[0055] CTC Loss is a loss function used to process variable-length output sequences, and its formula is shown in Equation (3). Center Loss increases the inter-class distance and decreases the intra-class distance by optimizing the distance between the feature vector and the class center, which can effectively improve the model's ability to distinguish similar samples, and its formula is shown in Equation (4). Based on this, Center Loss is introduced into CTC Loss to form C-CTC Loss, as shown in Equation (5).

[0056] L CTC =-∑logp(l|y) (3);

[0057]

[0058] L C-CTC =L CTC +λL Center (5);

[0059] Step 5-3: Construct the HC-CRNN text recognition model for hydropower engineering construction based on the improved CRNN:

[0060] The CRNN model mainly consists of three parts: convolutional layers, recurrent layers, and transcription layers. The convolutional layers use 3×3 convolutional kernels to slide windows across image features, outputting a series of feature maps and extracting sequence features. The recurrent layers use a Bi-LSTM model to predict feature sequences and output predicted text label information. The transcription layers use CTC Loss to merge duplicate characters and remove whitespace labels to form the final text sequence result. To enhance the accuracy of the CRNN model in recognizing characters in hydropower engineering construction texts, a ResNet backbone network is introduced into the convolutional layers. A professional corpus of hydropower engineering construction texts is constructed based on domain knowledge as the basis for recurrent layer category prediction. Furthermore, Center Loss is integrated into the transcription layers to improve CTCLoss, achieving accurate recognition of similar-looking characters.

[0061] Step 6 also includes the following steps:

[0062] Step 6-1: Apply the parameter-based transfer learning method to the hydropower engineering construction text interpretation model. By finding shared parameter information from the source domain and the target domain, the transfer is achieved, avoiding overfitting of the model training under small datasets.

[0063] Step 6-2: For the text detection subtask, the source domain is the ICDAR2015 text detection dataset, and the target domain is the labeled hydropower engineering construction text detection dataset.

[0064] Step 6-3: For the text recognition subtask, the source domain is the MJSynth and SynthText text recognition datasets, and the target domain is the hydropower engineering construction text recognition dataset.

[0065] Step 6-4: For the sub-tasks of hydropower engineering construction text detection and text recognition, the pre-trained model that has been trained in the source domain is selected to initialize the model weights, and the model parameters are further adjusted in the target domain using a fine-tuning strategy. At the same time, since a new corpus of hydropower engineering construction professional language has been established, the output character categories of the fully connected layer of the text recognition model have changed, and the output layer of the text recognition model needs to be retrained.

[0066] Step 7 also includes the following steps:

[0067] Step 7-1: Automatic scanning of paper documents for hydropower engineering construction:

[0068] First, the paper document is placed into the automatic text scanner to convert the paper document into an image format. The converted file is then transferred to the intelligent algorithm module.

[0069] Step 7-2: Perform vectorization conversion using a text intelligent interpretation algorithm.

[0070] In this step, the image-text transmitted from step 7-1 is processed. After text detection and text recognition, the image document is transformed into a vector document.

[0071] Step 7-3, Vectorize the document output:

[0072] In step 7-2, the image document is converted to vector form. In this step, the vectorized text is sorted in order, and the images and tables are inserted in order. Finally, the text is converted to vector form and output after integration.

[0073] The present invention has the following beneficial effects:

[0074] This invention proposes a vectorization method for hydropower engineering documents based on spatial semantic correction, aiming to automate the conversion of unstructured text and image data from hydropower engineering construction into editable text. Based on automatic scanning equipment and intelligent algorithms, it efficiently and quickly transforms uneditable text existing on paper into editable electronic documents, enabling the automated reuse of uneditable resources from hydropower engineering construction and significantly improving data service capabilities in the field of water conservancy and hydropower engineering. Attached Figure Description

[0075] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0076] Figure 1 Process for interpreting construction documents for hydropower projects.

[0077] Figure 2 Automatic scanner for hydropower engineering construction documents.

[0078] Figure 3 Hydropower Engineering Construction Text Interpretation Dataset

[0079] Figure 4 A hydropower engineering construction text detection model HC-PSENet is integrated with PP-HGNet.

[0080] Figure 5 A professional corpus of hydropower engineering construction texts based on domain knowledge.

[0081] Figure 6 ResNet-50 network architecture.

[0082] Figure 7HC-CRNN, a hydropower engineering construction text recognition model based on an improved CRNN.

[0083] Figure 8 Visualization effect of the hydropower project construction text interpretation model. Detailed Implementation

[0084] The embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0085] Example 1:

[0086] This paper proposes a method for vectorizing hydropower engineering documents based on spatial semantic correction. An automatic scanning instrument for hydropower engineering documents is constructed to automatically convert paper documents to image formats. A dataset for interpreting hydropower engineering construction text and a domain-knowledge-based professional corpus, DKPC, are built to obtain the text categories and layout backgrounds of hydropower engineering construction documents. A pixel-level segmentation-based text detection model, HC-PSENet, is developed, and the PP-HGNet backbone network is introduced to enhance the model's feature extraction performance. A feature-enhanced text recognition model, HC-CRNN, is established, and a ResNet backbone network and C-CTCLoss are used to enhance the model's accuracy in recognizing characters. A parameter-based transfer learning strategy is employed to train the model to avoid imbalance between sample classes and model overfitting. Finally, an automated interpretation device for hydropower engineering construction paper documents is developed based on the instrument and algorithm, improving the intelligence level of text interpretation.

[0087] Example 2:

[0088] This invention belongs to the field of intelligent text mining for hydropower engineering construction, and provides a method for vectorizing hydropower engineering document text. The specific steps are as follows, and the overall approach is described below. Figure 1 .

[0089] Step 1: Construct an automatic scanning device for hydropower engineering construction documents. After automatic page turning and scanning, the documents are converted into image documents. This device enables the automatic conversion of paper-based hydropower engineering construction documents into image-based documents, such as… Figure 2 As shown.

[0090] Step 1-1, Document Preparation. The automatic text scanner includes a power port m and a document input g. Power port m is plugged in, and the document input g is opened to place the paper document to be scanned into the slot. The document input g is then closed. When the document input g is closed, the page-turning brush c will adjust its direction via the page-turning axis h, ensuring that the page-turning brush c is in close contact with the document. To prevent pages of perfect bound documents from curling up, a page-fixing device d is added to fix the document pages, thus completing the preparation work for scanning the document.

[0091] Steps 1-2: Start scanning. After clicking the start button n, the instrument will begin operating. Before starting, select single-sided or double-sided document using the selection button p to accommodate different paper document formats. During the scanning process, the side lights e and f provide supplementary lighting. The top scanning cameras a and b capture images of two pages of the document respectively. The captured images are automatically recorded in order, and the file is saved according to the original page order during final integration.

[0092] Steps 1-3: Automatic Page Turning. Since paper documents may contain multiple pages, an automatic page turning component is added to the instrument. After a single scan by the scanning camera, the sensor q automatically sends a signal, the page fixing device d automatically springs up using the pivot l, and the page turning brush c automatically starts rotating, rolling up the document pages. Using the operating mechanism consisting of the page turning pivot h, slide bar j, and base k, the page is unfolded from right to left, completing the page turn. After the page turn is complete, the page turning brush c automatically retracts to avoid affecting the text scanning rotation. Simultaneously, when the sensor q detects the completion of the page turn, the page fixing device d automatically lowers, fixing both ends of the page to ensure the flatness of the paper document after the page turn.

[0093] Steps 1-4: Scanning complete. Steps 1-2 and 1-3 are repeated until the entire document is scanned. The scanned image documents are then integrated and saved in sequence, and fed into the subsequent text vectorization decoding algorithm.

[0094] Step 2: Construct a dataset for interpreting hydropower engineering construction text. Batch process the uneditable hydropower engineering construction text into image format. Further, segment the unstructured text images from the hydropower engineering construction documents and use data augmentation to expand the training samples for the text interpretation model.

[0095] Step 2-1: Batch process the uneditable hydropower construction text into image format, with a uniform resolution of 300dpi.

[0096] Step 2-2: Use the OpenCV toolbox to segment unstructured text images in hydropower engineering construction documents, and construct a text detection dataset and a text recognition dataset for hydropower engineering construction documents.

[0097] Steps 2-3 involved using data augmentation methods such as Gaussian blur and contrast adjustment to expand the training set of the hydropower engineering construction text interpretation dataset by a factor of 6. This improved the model's training accuracy while avoiding overfitting. Figure 3 As shown.

[0098] Step 3: Construct the HC-PSENet model for detecting text in hydropower engineering construction. Input unstructured text images, and through post-processing of text region feature extraction, fusion, and progressive scale expansion, output text regions containing location information in text line units.

[0099] Step 3-1: Construct the PSENet text detection model. PSENet is a segmentation-based text detection model that effectively solves the problem of difficult segmentation of neighboring text instances through a progressive scale expansion algorithm. It also achieves accurate localization of text regions of arbitrary shapes based on pixel-level segmentation, making it suitable for text detection tasks in hydropower engineering construction documents.

[0100] Step 3-2: Construction of the Feature Extraction Layer for the Hydropower Engineering Construction Text Detection Model. The feature extraction layer, based on the PP-HGNet backbone network, is responsible for extracting features at different levels of the image. The input image first passes through the Stem image processing module, aiming to improve the model's feature extraction capability without increasing the number of model parameters. Further, feature maps are sequentially obtained through four sets of convolutional layers, ConvStage 1 to ConvStage 4, with output feature map sizes of 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original input image size, respectively. The ConvStage mainly consists of HG Block modules containing a large number of standard convolutions, while a learnable downsampling layer (LDS Layer) is introduced to reduce the number of model parameters and computational cost.

[0101] Step 3-3, Feature fusion construction of hydropower engineering construction text detection model. Inspired by the feature pyramid network, the feature fusion layer combines shallow location information with high-level semantic features using a top-down and horizontal connection method to obtain four 256-channel feature maps P2 to P5. The four feature maps are then fused into a 1024-channel feature map F using equation (1).

[0102] F = P2||Up ×2 (P3)||||Up ×4 (P4)||Up ×8 (P5) (1);

[0103] In the formula: || represents the concatenation operation, Up ×2 Up ×4 Up ×8 These represent upsampling times of 2, 4, and 8, respectively.

[0104] Steps 3-4: Construction of feature maps for the hydropower engineering construction text detection model. The feature map F is reduced to 256 channels after passing through a 3×3 convolutional layer, a Batchnorm layer, and a ReLU layer. It is then passed through n 1×1 convolutional layers, an upsampling layer, and a Sigmoid layer to obtain n semantic segmentation results S1 to S2.n Where S1 represents the text instance segmentation result with the smallest kernel, S n This represents the original segmentation result with the largest kernel. Finally, the asymptotic expansion algorithm is used to sequentially segment S1 to S... n Scale expansion is performed to obtain the final hydropower engineering construction text detection result R;

[0105] Steps 3-5: Construction of the loss function for the hydropower engineering construction text detection model. The loss function of the HC-PSENet hydropower engineering construction text detection model constructed in this embodiment considers both the region category and the position of the text detection box, and the expression is shown in Equation (2);

[0106] L=λL f +(1-λ)L s (2);

[0107] In the formula: L f L represents the loss of a complete text instance, used to distinguish between text and non-text regions; s λ represents the loss of the text kernel instance, used to determine the border position of the text region; λ is the balance coefficient, which is set to 0.7 in this embodiment.

[0108] Steps 3-6 involve constructing the HC-PSENet text detection model for hydropower engineering construction documents. PSENet is a segmentation-based text detection model that effectively solves the problem of segmenting neighboring text instances through a progressive scale expansion algorithm. It also achieves accurate localization of text regions of arbitrary shapes based on pixel-level segmentation, making it suitable for text detection tasks in hydropower engineering construction documents. To more accurately detect multi-scale text line location information in hydropower engineering construction documents, a PP-HGNet backbone network is introduced into the PSENet model to construct the HC-PSENet text detection model, enhancing the model's ability to extract multi-scale text line features. The network structure of the hydropower engineering construction text detection model consists of three parts: a feature extraction layer, a feature fusion layer, and an output layer, as follows: Figure 4 As shown.

[0109] Step 4: Analyze the domain knowledge of hydropower engineering construction documents, supplement professional terms and special characters to form a professional corpus.

[0110] Step 4-1: Analyze the character categories of the hydropower engineering construction documents, which are divided into four categories: basic text, professional text, subscript and superscript characters, and special characters. Figure 5 As shown. The basic text consists of text widely used in most fields of documentation, while the specialized text refers to text that appears frequently in hydropower engineering construction. Subscripts, superscripts, and special characters refer to characters commonly used in hydropower engineering construction documents, such as mathematical variables and chemical terms. "C90", etc.

[0111] Step 4-2 involves constructing a domain-knowledge-based professional corpus. This study adds basic text, professional text, subscripts and superscripts, and special characters to the professional corpus to obtain accurate domain-specific textual expressions. The domain-knowledge-based professional corpus fills the gaps in the original corpus regarding professional text, exponential units, and special symbols, enhancing the adaptability of the proposed model to hydropower engineering construction documents.

[0112] Step 5: Establish the HC-CRNN text recognition model for hydropower engineering construction. Input the detected text region, and sequentially go through processes such as extracting text features, predicting text sequences, and mapping text labels to output text information in the form of editable characters.

[0113] Step 5-1: Introduce the ResNet network. The ResNet network addresses the vanishing and exploding gradient problems of deep neural networks by introducing residual modules, possessing stronger feature extraction capabilities than VGG16. ResNet-50 is used as the backbone network of the HC-CRNN text recognition model for hydropower engineering construction. The overall network structure is divided into 5 stages, consisting of 49 convolutional layers and 1 fully connected layer, as follows... Figure 6 As shown.

[0114] Step 5-2: Construct the improved loss function C-CTC Loss. CTC Loss is a loss function used to process variable-length output sequences, and its formula is shown in Equation (3). Center Loss increases the inter-class distance and decreases the intra-class distance by optimizing the distance between feature vectors and class centers, which can effectively improve the model's ability to distinguish similar samples, as shown in Equation (4). Therefore, Center Loss is introduced into CTC Loss to form C-CTC Loss, as shown in Equation (5).

[0115] L CTC =-∑logp(l|y) (3);

[0116]

[0117] L C-CTC =L CTC +λL Center (5);

[0118] Step 5-3: Construct the HC-CRNN text recognition model for hydropower engineering construction based on an improved CRNN, such as... Figure 3As shown, the CRNN model mainly consists of three parts: convolutional layers, recurrent layers, and transcription layers. The convolutional layers use 3×3 convolutional kernels to slide windows across image features, outputting a series of feature maps and extracting sequence features. The recurrent layers use a Bi-LSTM model to predict feature sequences and output predicted text labels. The transcription layers use CTC Loss to merge duplicate characters and remove whitespace labels, forming the final text sequence result. To enhance the accuracy of the CRNN model in recognizing characters in hydropower engineering construction texts, this study introduces a ResNet backbone network into the convolutional layers, constructs a professional corpus of hydropower engineering construction texts based on domain knowledge as the basis for recurrent layer category prediction, and integrates Center Loss in the transcription layers to improve CTCLoss, achieving accurate recognition of similar-looking characters.

[0119] Step 6: Introduce a two-stage transfer learning strategy. This addresses the issues of limited annotation information and weak computational power in hydropower engineering construction texts under big data contexts. Instead of requiring large amounts of labeled data and training costs, it applies learned knowledge or model parameters to other related fields, enabling the generalized model to quickly adapt to personalized needs.

[0120] Step 6-1: Apply the parameter-based transfer learning method to the hydropower engineering construction text interpretation model. By finding shared parameter information from the source domain and the target domain, the transfer is achieved, avoiding the problem of model overfitting under small datasets.

[0121] Step 6-2: For the text detection subtask, the source domain is the ICDAR2015 text detection dataset, and the target domain is the hydropower engineering construction text detection dataset annotated in this study.

[0122] Step 6-3: For the text recognition subtask, the source domain is the MJSynth and SynthText text recognition datasets, and the target domain is the hydropower engineering construction text recognition dataset.

[0123] Step 6-4: For the sub-tasks of hydropower engineering construction text detection and text recognition, the pre-trained models already trained in the source domain are selected to initialize the model weights, and the model parameters are further adjusted in the target domain using a fine-tuning strategy. Meanwhile, since a new corpus of hydropower engineering construction terminology has been established, the output character categories of the fully connected layer of the text recognition model have changed; therefore, the output layer of the text recognition model needs to be retrained.

[0124] Step 7: Construct a vectorization conversion device for hydropower engineering construction documents. Establish an information flow between the automatic text scanning machine and the intelligent algorithm. Scanned image documents are fed into the background algorithm, where text detection and recognition are performed to generate vectorized documents, which are then output to the user.

[0125] Step 7-1: Automatic scanning of paper documents for hydropower engineering construction. First, the paper documents are placed into the scanning device to convert them into image format. The converted files are then transferred to the intelligent algorithm module.

[0126] Step 7-2 involves vectorization using a text intelligent interpretation algorithm. This step processes the image-based text received in Step 7-1. After text detection and recognition, the image document is converted into a vector form.

[0127] Step 7-3, Vectorized Document Output. Following the vectorization of the image document achieved in Step 7-2, this step involves sorting the vectorized text in sequence, inserting images, tables, etc., in that order, and finally integrating them to achieve the vectorization and output of the text.

[0128] Example 3:

[0129] Taking a hydropower project as an example, the superiority of the method and device for vectorization of hydropower project documents based on spatial semantic correction is verified.

[0130] 1. Dataset Construction:

[0131] Based on the aforementioned non-editable hydropower engineering construction documents, a hydropower engineering construction text interpretation dataset was constructed using PPOCRLabel, comprising two sub-datasets: text detection and text recognition. The text detection dataset, annotated in this study, consists of image paths and text coordinate information, locating text positions line by line, totaling 2547 annotated data sets. The text recognition dataset primarily focuses on character information, composed of primitive image paths and text line position information, containing 76189 characters including Chinese, English, and special text.

[0132] Based on transfer learning, the text recognition model modifies the character output categories according to a professional corpus, thus still requiring a large dataset to assist the model in learning the features of each text category. This study uses hydropower engineering construction documents as a foundation and employs the data synthesis tool Style Text to generate a dataset of professional hydropower engineering construction text and special characters, alleviating the problems of insufficient sample size and imbalanced class distribution in the text interpretation dataset. Furthermore, data augmentation methods such as Gaussian blur and contrast adjustment are used to expand the training set in the hydropower engineering construction text interpretation dataset by 6 times, improving the model's training accuracy while avoiding overfitting.

[0133] 2. Evaluation indicators:

[0134] Evaluating the performance of a hydropower engineering construction text interpretation model requires considering both text detection and text recognition, specifically the matching degree of detected text boxes and the accuracy of recognized characters. For the text detection task, the interaction ratio (IoU) is used as the criterion, and metrics such as precision, recall, and harmonic mean (Hmean) are employed to evaluate the performance of the text detection model. The calculation methods for these evaluation metrics are as follows:

[0135]

[0136]

[0137]

[0138]

[0139] In the formula: area(C) is the area of ​​the detected text box region; area(G) is the area of ​​the standard text box region; TP is the number of detected text boxes that exist and are predicted; FP is the number of detected text boxes that do not exist but are predicted; FN is the number of detected text boxes that exist but are not predicted.

[0140] Text recognition tasks require consideration not only of character recognition accuracy but also of the impact of misclassification, missed recognition, and misjudgment on the text line. Therefore, this study evaluates the performance of text recognition models from both character and text line perspectives, using metrics such as accuracy (Acc) and normalized edit distance (NED). The calculation methods for these text recognition evaluation metrics are as follows:

[0141]

[0142]

[0143] Where: N r To correctly identify the number of characters; N all N is the total number of characters to be recognized; N is the total number of text boxes to be detected; s i Represents the actual value of the string; Represents the predicted value of the string; For string s i and The larger value of the length; Represents string s i and The edit distance is calculated by obtaining the sequence s. i Convert to sequence The minimum number of single-character edits (insert, delete, or replace) required.

[0144]

[0145] 3. Model Construction and Result Analysis:

[0146] All model experiments involved in this study were conducted on the same computer device, with Windows 10 Professional operating system, Python 3.8 programming language, and a CPU of [CPU details missing]. Core™ i5-13600KF, GPU is The GeForce RTX™ 4070 was used to build and test the hydropower engineering construction text interpretation model using the PaddlePaddle framework. Each epoch employed a strategy of shuffling the dataset order during training. The configuration parameters for the training and validation phases of the hydropower engineering construction text interpretation model were determined based on relevant field literature; specific parameter information is shown in Table 1.

[0147] Table 1 Configuration Parameters for the Hydropower Engineering Construction Text Interpretation Model

[0148]

[0149] HC-PSENet and HC-CRNN were trained for 150 epochs each, with validation performed after each epoch. An early stopping strategy was implemented based on 50 epochs. The models achieved optimal validation performance at epoch 44 and 139, respectively, with validation accuracies of 98.08% and 90.63%. The model with the smallest validation loss was selected as the optimal model for testing. HC-PSENet achieved a detection precision of 97.57%, a detection recall of 99.35%, and a harmonic mean of 0.985; HC-CRNN achieved a recognition precision of 90.62% and a standard edit distance of 0.989.

[0150] To visually demonstrate the advantages of our research model in interpreting hydropower engineering construction texts, we selected three types of hydropower engineering construction text images and visualized the model's interpretation performance on the test images, such as... Figure 8 As shown. Among them, Figure 8 (a) consists only of text. Figure 8 (b) contains both text and numbers. Figure 8(c) contains text, numbers, and special characters. As shown in the figure, the number of text lines detected by HC-PSENet and the predicted text line outlines are basically consistent with the actual annotation information, demonstrating the adaptability of the detection model in this study to hydropower engineering construction documents. The HC-CRNN model can correctly identify most of the text, numbers, and special characters, and the text order is output according to the normal document reading order, indicating that the recognition model in this study has good performance in distinguishing and classifying characters.

[0151] 4. Ablation test results:

[0152] To verify the effectiveness of the improvement strategy for the hydropower engineering construction text interpretation model, the same training and testing conditions as in Table 1 were set. The model ablation experiment was conducted by controlling the variables of the improvement module. The indicators in Equations (7) to (11) were used to evaluate the impact of each improvement strategy on the model accuracy. The ablation experiment results of the hydropower engineering construction text detection model and the text recognition model are shown in Table 2 and Table 3, respectively.

[0153] Table 2. Results of HC-PSENet ablation experiments

[0154]

[0155] Table 3. Results of HC-CRNN ablation experiments

[0156]

[0157] Note: "√" indicates that the module has been added to the model, and "-" indicates that this operation is not performed.

[0158] Table 2 shows the impact of adopting transfer learning strategies and introducing the PP-HGNet backbone network on the performance of the PSENet model in detecting hydropower engineering construction text. Experiment 1 presents the test results of the baseline model PSENet, while Experiments 2 and 3 verify the impact of transfer learning and PP-HGNet on the baseline model's performance, respectively. Comparing Experiments 1 and 2, it can be seen that introducing the transfer learning strategy into the baseline model improves the model's accuracy by 4.12%, verifying that transfer learning can effectively improve the baseline model's ability to detect lines of hydropower engineering construction text. The results of Experiment 3 show that compared to the original backbone network MobileNetV3, PP-HGNet has stronger feature extraction capabilities and can obtain richer text location information in the feature map. Experiment 4 presents the test results of HC-PSENet with the introduction of the transfer learning strategy. As shown in the table, the model's accuracy increased to 97.57% and the H-mean reached 0.985 under the transfer learning strategy, indicating that the HC-PSENet proposed in this study performs better overall in the hydropower engineering construction text detection task.

[0159] To quantify the improvement effects of transfer learning strategies, specialized corpora, ResNet backbone, and C-CTC Loss on the performance of CRNN models in recognizing hydropower engineering construction text, this study designed nine sets of experiments to verify the impact of each improvement measure on the model's recognition accuracy, as shown in Table 3. In Experiment 5, the baseline model's accuracy in recognizing hydropower engineering construction text was only 1.95%, while the accuracy of the baseline model after introducing transfer learning reached 76.17%, indicating that the model overfitted without introducing transfer learning, thus verifying the importance of transfer learning strategies for complex multi-classification tasks with small sample datasets. Experiments 7-9 verified the impact of specialized corpora, ResNet network, and C-CTC Loss on the baseline model, respectively. The results show that each improvement measure effectively improves the accuracy of CRNN models in recognizing characters. Among them, introducing the ResNet backbone network significantly improves the model's recognition accuracy, reflecting the importance of the model's ability to extract character features for distinguishing and classifying characters in text recognition tasks. The improvements to the professional corpus and C-CTC Loss increased the recognition accuracy of the baseline model by 1.61% and 2.35%, respectively, confirming that increasing the number of text categories in the corpus and improving the loss function can effectively improve network performance in character classification. Overall, with the support of these improvements, HC-CRNN (Experiment 13) achieved a character recognition accuracy of 90.62% and a standard edit distance of 0.989, demonstrating its ability to accurately interpret hydropower engineering construction texts.

[0160] 5. Performance comparison of multiple models:

[0161] To further verify the superiority of the proposed method, the hydropower engineering construction text interpretation dataset constructed in this study was used as an example. Using the same configuration parameters as the hydropower engineering construction text interpretation model, four mainstream text detection models and four typical text recognition models were selected for comparison. Equations (7) to (11) were used as evaluation indicators, and the test results of each model are shown in Tables 4 and 5.

[0162] Table 4 Comparison Results of Text Detection Models

[0163]

[0164] Table 5 Comparison Results of Text Recognition Models

[0165]

[0166] Table 4 shows the performance of various typical text detection models on the test set of hydropower engineering construction text interpretation. Among them, HC-PSENet achieved a harmonic mean of 0.985, outperforming the other four methods in overall text detection performance. As shown in Table 5, HC-CRNN significantly outperformed the other four recognition methods in both accuracy and standard edit distance, indicating that it is more suitable for character recognition tasks in hydropower engineering construction text.

[0167] 6. Intelligent Text Conversion Device:

[0168] First, the paper document is placed into a scanner to convert it into an image format. The converted file is then transferred to the intelligent algorithm module. After text detection and recognition, the image document is converted into a vector format. In this step, the vectorized text is sorted in order, and images, tables, etc., are inserted in sequence. Finally, the data is integrated to achieve the vectorization conversion and output of the text.

Claims

1. A method for vectorizing hydropower engineering document text based on spatial semantic correction, characterized in that, Includes the following steps: Step 1: Build an automatic scanning device for hydropower project construction documents: This automatic text scanner automatically flips through and scans construction documents for hydropower projects, creating image documents. This automatic text scanner then enables the automatic conversion of paper-based construction documents for hydropower projects into image-based documents. Step 2, construct a dataset for interpreting construction texts of hydropower projects: By batch processing uneditable hydropower engineering construction text into image format, the unstructured text images in hydropower engineering construction documents are further segmented, and data augmentation is used to expand the training samples of the text interpretation model. Step 3: Construct the HC-PSENet model for detecting construction text in hydropower projects. Input unstructured text images, and output text regions containing location information in units of text lines through text region feature extraction, fusion, and progressive scale expansion post-processing. Step 4: Analyze the domain knowledge of hydropower engineering construction documents, supplement professional terms and special characters to form a professional corpus; Step 5: Establish the HC-CRNN text recognition model for hydropower engineering construction. Input the detected text region, and sequentially go through the processes of extracting text features, predicting text sequences, and mapping text labels to output text information in the form of editable characters. Step 6: Introduce a two-stage transfer learning strategy; Step 6-1: Apply the parameter-based transfer learning method to the hydropower engineering construction text interpretation model. By finding shared parameter information from the source domain and the target domain, the transfer is achieved, avoiding overfitting of the model training under small datasets. Step 6-2: For the text detection subtask, the source domain is the ICDAR2015 text detection dataset, and the target domain is the labeled hydropower engineering construction text detection dataset. Step 6-3: For the text recognition subtask, the source domain is the MJSynth and SynthText text recognition datasets, and the target domain is the hydropower engineering construction text recognition dataset. Step 6-4: For the sub-tasks of hydropower engineering construction text detection and text recognition, the pre-trained models that have been trained in the source domain are selected to initialize the model weights, and the model parameters are further adjusted in the target domain using a fine-tuning strategy. At the same time, since the professional corpus of hydropower engineering construction has been re-established, the output character categories of the fully connected layer of the text recognition model have changed, and the output layer of the text recognition model needs to be retrained. Step 7: Construct a vectorization conversion device for hydropower engineering construction documents. Establish an information flow between the automatic text scanning machine and the intelligent algorithm. After scanning, the scanned image documents are transmitted to the background algorithm. After text detection and text recognition, the vectorized documents are output to the user.

2. The method for vectorizing hydropower engineering document text based on spatial semantic correction according to claim 1, characterized in that, Step 1 also includes the following steps: Step 1-1, Document Preparation: The automatic text scanner includes a power port (m) and a document inlet (g). The power port (m) is plugged in, the document inlet (g) is opened and the paper document to be scanned is placed in the slot, and the document inlet (g) is closed. When the document inlet (g) is closed, the page turning brush (c) will adjust its direction through the page turning axis (h) so that the page turning brush (c) is in close contact with the document. In order to prevent the pages of perfect bound documents from curling up, a page fixing device (d) is added to fix the document pages, thereby completing the preparation work for scanning the document. Steps 1-2: Start the scan: After clicking the start button (n), the instrument will start operating. Before starting, select single-sided or double-sided document by using the selection button (p) to adapt to different forms of paper documents. During the scanning process, supplementary lighting is provided by the side lights (e, f). The scanning cameras (a, b) on the top will take pictures of two pages of the document respectively. The images will be automatically recorded in order and saved according to the original page order of the text when they are finally integrated. Steps 1-3, automatic page turning: For multi-page paper documents, an automatic page-turning component is set in the instrument. After the scanning camera performs a single scan, the sensing device (q) will automatically send a signal, the page fixing device (d) will automatically pop up using the rotating shaft (l), and the page-turning brush (c) will automatically start running, rolling up the document pages. Using the running device composed of the page-turning rotating shaft (h), the slide bar (j), and the base (k), the page is unfolded from right to left to complete the page-turning of the text. After the page-turning is completed, the page-turning brush (c) will automatically retract to avoid affecting the text scanning conversion. At the same time, when the sensing device (q) senses that the page-turning is complete, the page fixing device (d) will automatically lower to fix the two ends of the page, ensuring the flatness of the paper document after the page-turning. Steps 1-4, scanning complete: After repeating steps 1-2 and 1-3 until all documents have been scanned, the scanned image documents are integrated and saved in sequence, and then fed into the subsequent text vectorization decoding algorithm.

3. The method for vectorizing hydropower engineering document text based on spatial semantic correction according to claim 1, characterized in that, Step 2 also includes the following steps: Step 2-1: Batch process the uneditable hydropower engineering construction text into image format and adjust it to a uniform resolution; Step 2-2: Use the OpenCV toolbox to segment unstructured text images in hydropower engineering construction documents, and construct a text detection dataset and a text recognition dataset for hydropower engineering construction documents; Steps 2-3 involve using Gaussian blur and contrast adjustment data augmentation methods to expand the training set in the hydropower engineering construction text interpretation dataset multiple times, thereby improving the training accuracy of the model while avoiding overfitting.

4. The method for vectorizing hydropower engineering document text based on spatial semantic correction according to claim 1, characterized in that, Step 3 also includes the following steps: Step 3-1, Construct the PSENet text detection model: PSENet is a segmentation-based text detection model that effectively solves the problem of difficult segmentation of neighboring text instances through a progressive scale expansion algorithm. At the same time, it achieves accurate localization of text regions of arbitrary shapes based on pixel-level segmentation, making it suitable for text detection tasks in hydropower engineering construction documents. Step 3-2, Construction of the feature extraction layer for the hydropower engineering construction text detection model: The feature extraction layer, based on the PP-HGNet backbone network, is responsible for extracting features at different levels of the image. The input image first passes through the Stem image processing module, which improves the model's feature extraction capability without increasing the number of model parameters. It then sequentially passes through four sets of convolutional layers, Conv Stage 1 to Conv Stage 4, to obtain feature maps. The output feature map sizes are 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original input image size, respectively. The Conv Stage consists of HG Block modules containing a large number of standard convolutions, and a learnable downsampling layer, LDS Layer, is introduced to reduce the number of model parameters and computational cost. Step 3-3, Feature Fusion Construction of Hydropower Engineering Construction Text Detection Model: Inspired by the feature pyramid network, the feature fusion layer uses a top-down and lateral connection approach to combine shallow location information with high-level semantic features, resulting in four 256-channel feature maps. Furthermore, the four feature maps are fused into a 1024-channel feature map using equation (1). ; （1）； In the formula: This indicates a splicing operation. , , These represent upsampling times of 2, 4, and 8, respectively. Steps 3-4: Construction of feature maps for hydropower engineering construction text detection model: Feature map After passing through a 3×3 convolutional layer, a Batchnorm layer, and a ReLU layer, the dimensionality is reduced to 256 channels, and then sequentially passed through... A 1×1 convolutional layer, an upsampling layer, and a sigmoid layer are used to obtain... semantic segmentation results ,in, This represents the text instance segmentation result with the smallest kernel. This represents the original segmentation result with the largest kernel. Finally, an asymptotic expansion algorithm is used to sequentially segment... Scale expansion was performed to obtain the final inspection results of the hydropower engineering construction text. ; Steps 3-5: Construction of the loss function for the hydropower engineering construction text detection model: The loss function of the HC-PSENet hydropower engineering construction text detection model considers both the region category and the text detection box position, and the expression is shown in equation (2). （2）； In the formula: The loss represents the total loss of a complete text instance, used to distinguish between text and non-text regions; The loss represents the text kernel instance and is used to determine the border position of the text region; This is the balance coefficient; L This is the total loss function; Steps 3-6: Construct the HC-PSENet model for detecting construction text in hydropower projects. The PP-HGNet backbone network is introduced into the PSENet model to construct the HC-PSENet hydropower engineering construction text detection model, which enhances the ability of the text detection model to extract multi-scale text line features. The network structure of the hydropower engineering construction text detection model is divided into three parts: feature extraction layer, feature fusion layer and output layer.

5. The method for vectorizing hydropower engineering document text based on spatial semantic correction according to claim 1, characterized in that, Step 4 also includes the following steps: Step 4-1: Analyze the character categories of hydropower engineering construction documents, which are divided into four categories: basic text, professional text, subscript and superscript characters, and special characters. Step 4-2: Construct a domain-knowledge-based professional corpus. Add basic text, professional text, subscript and superscript characters, and special characters to the professional corpus to obtain accurate domain text expressions. The domain-knowledge-based professional corpus fills the gaps in the original corpus in terms of professional text, exponent units, and special symbols, and enhances the model's adaptability to hydropower engineering construction documents.

6. The method for vectorizing hydropower engineering document text based on spatial semantic correction according to claim 1, characterized in that, Step 7 also includes the following steps: Step 7-1: Automatic scanning of paper documents for hydropower engineering construction: First, the paper document is placed into the automatic text scanner to convert the paper document into an image format. The converted file is then transferred to the intelligent algorithm module. Step 7-2: Perform vectorization conversion using a text intelligent interpretation algorithm. In this step, the image-text transmitted from step 7-1 is processed. After text detection and text recognition, the image document is transformed into a vector document. Step 7-3, Vectorize the document output: In step 7-2, the image document is converted to vector form. In this step, the vectorized text is sorted in order, and the images and tables are inserted in order. Finally, the text is converted to vector form and output after integration.

Images