Information recognition method, device and system for construction documents

Abstract

The application provides a construction file information identification method, device and system, the method is applied to an industrial computer; the method comprises the following steps: loading a target drawing in a construction file; splitting the loaded target drawing into a plurality of sub-drawings according to pages; displaying a previewed target sub-drawing according to a page number and a zoom ratio selected by a user; automatically searching for and marking a target identification area in the target sub-drawing, and / or marking the target identification area in response to a user's identification area editing operation on the target sub-drawing; performing character recognition on the marked target identification area in the target sub-drawing through a preset information identification model, and saving the character recognition information corresponding to the target identification area. The application can automatically split the loaded drawing, preview the sub-drawing, mark the identification area, recognize the character and save the recognition result, realize intelligent information identification of the construction file, improve the information identification accuracy and the identification efficiency, and further improve the construction efficiency.

Description

Technical Field

[0001] This application relates to the field of nuclear island station technology, and in particular to a method, apparatus and system for identifying information in construction documents. Background Technology

[0002] Construction drawing information is the data foundation for all construction management information systems in a nuclear power project and is therefore crucial. Throughout the entire nuclear power installation and construction process, due to the large number of drawings and the massive amount of information, data input often becomes a constraint affecting the project progress. Furthermore, in recent years, quality incidents and problems caused by data entry errors by data entry personnel have been rampant.

[0003] Currently, the data transmission for nuclear island installation projects mainly involves the design institute transmitting completed PDF drawings to the construction unit. The construction unit then manually inputs the tabular information from the drawings to facilitate on-site construction and the development of corresponding construction plans. However, this method becomes increasingly inefficient as the volume of documents surges. Prolonged manual data entry leads to a high error rate, making it difficult to detect errors in the data. These errors directly impact the construction site, resulting in low efficiency. Furthermore, repetitive work reduces employee morale and overall productivity. This manual data transmission method is far from meeting the demands of on-site construction. Summary of the Invention

[0004] The purpose of this application is to provide a method, device, and system for information recognition of construction documents, which can automatically split loaded drawings, preview sub-drawings, mark recognition areas, recognize text, and save recognition results, thereby realizing intelligent information recognition of construction documents, improving the accuracy and efficiency of information recognition, and thus improving construction efficiency.

[0005] In a first aspect, this application provides a method for information recognition in construction documents, applied to an industrial control computer; the method includes: loading a target drawing from a construction document; the drawing includes at least one of the following: construction drawings, tables, and text; splitting the loaded target drawing into multiple sub-drawings by page; displaying a preview of the target sub-drawings according to the page number and scaling ratio selected by the user; automatically finding and marking target recognition areas in the target sub-drawings using an AI algorithm, and / or, in response to the user's editing operation on the recognition areas of the target sub-drawings, marking the target recognition areas; performing text recognition on the marked target recognition areas in the target sub-drawings using a preset information recognition model to obtain the text recognition information corresponding to the target recognition areas; and saving the text recognition information corresponding to the target recognition areas.

[0006] Furthermore, the above-mentioned step of loading the target drawings in the construction file includes: in response to the user's selection operation of the target construction file, loading all the drawings in the target construction file as the target drawings; or, in response to the user's selection operation of the target drawings in the target construction file, loading the selected target drawings.

[0007] Furthermore, the aforementioned information recognition model includes a text detection model and a text recognition model; the step of recognizing the target recognition area marked in the target sub-drawing through the preset information recognition model includes: detecting the text position in the target recognition area through the text detection model in the information recognition model, recognizing the text at the detected text position through the text recognition model in the information recognition model, and obtaining the text recognition information corresponding to the target recognition area.

[0008] Furthermore, the above-mentioned step of saving the text recognition information corresponding to the target recognition area includes: saving the text recognition information corresponding to the target recognition area to an Excel file with the same name as the target drawing and with the page number suffix added.

[0009] Furthermore, the training process of the above information recognition model is as follows: A training sample set is obtained; the samples in the training sample set include: drawings labeled with text location data, text image frames, and the text content corresponding to the text image frames; based on the text location data of the samples in the training sample set, a preset text detection model is trained to obtain a text detection model; based on the text image frames and the text content corresponding to the text image frames of the samples in the training sample set, a preset text recognition model is trained to obtain a text recognition model; the text detection model and the text recognition model are used as the information recognition model; when new sample data is obtained, the information recognition model is trained through incremental training mode to obtain an updated information recognition model.

[0010] Furthermore, the aforementioned information recognition model includes: an encoder, a decoder, and an objective function; the encoder includes: a two-dimensional convolutional neural network, a two-dimensional batch normalization layer, and a ReLU-like function; the decoder includes: a recurrent neural network with a gating mechanism and a fully connected neural network; the objective function includes a CTC-connected temporal classification function.

[0011] Furthermore, the hyperparameter design during the training of the above model includes: increasing the number of convolutional kernels layer by layer in multiples of 2 during convolutional encoding; setting the text height to 32 pixels; selecting fonts based on GB / T 14691-93; and conducting comparative ablation experiments on the following parameters: 4-8 convolutional layers, 16-256 convolutional kernels, the number of convolutional groups with the number of kernels or half the number of kernels, point-wise and depth-wise, and 1-2 recursive layers.

[0012] Secondly, this application also provides an information recognition device for construction documents, which is applied to an industrial control computer. The device includes: a drawing loading module, a drawing preview module, a recognition area management module, and a recognition saving module. The drawing loading module is used to load target drawings from construction documents. The drawings include at least one of the following: construction drawings, tables, and text. The drawing preview module is used to split the loaded target drawings into multiple sub-drawings by page. Based on the page number and scaling ratio selected by the user, the previewed target sub-drawings are displayed. The recognition area management module is used to automatically find and mark target recognition areas in the target sub-drawings using AI algorithms, and / or, in response to user editing operations on the recognition areas of the target sub-drawings, mark the target recognition areas. The recognition saving module is used to perform text recognition on the marked target recognition areas in the target sub-drawings using a preset information recognition model to obtain the text recognition information corresponding to the target recognition areas. The text recognition information corresponding to the target recognition areas is then saved.

[0013] Thirdly, this application also provides an information recognition system for construction documents, the system comprising: an industrial control computer and a training server; the training server is used to train an information recognition model and transmit the information recognition model to the industrial control computer; the industrial control computer is used to recognize construction information of the construction documents to be recognized based on the information recognition model, using the method described in the first aspect.

[0014] Fourthly, this application also provides a computer-readable storage medium storing computer-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the method described in the first aspect above.

[0015] The information recognition method, apparatus, and system for construction documents provided in this application are applied to an industrial control computer. The method includes: loading target drawings from a construction document; the drawings include at least one of the following: construction drawings, tables, and text; splitting the loaded target drawings into multiple sub-drawings by page; displaying a preview of the target sub-drawings based on the page number and scaling ratio selected by the user; automatically finding and marking target recognition areas in the target sub-drawings using an AI algorithm, and / or, in response to user editing operations on the recognition areas of the target sub-drawings, marking the target recognition areas; performing text recognition on the marked target recognition areas in the target sub-drawings using a preset information recognition model to obtain the text recognition information corresponding to the target recognition areas; and saving the text recognition information corresponding to the target recognition areas. This application can automatically split loaded drawings, preview sub-drawings, mark recognition areas, perform text recognition, and save recognition results, achieving intelligent information recognition of construction documents, improving the accuracy and efficiency of information recognition, and thus improving construction efficiency. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 A flowchart illustrating a method for identifying information in construction documents, as provided in this application embodiment;

[0018] Figure 2 A schematic diagram illustrating the training process of a text detection model provided in an embodiment of this application;

[0019] Figure 3 A schematic diagram illustrating the training process of a character recognition model provided in an embodiment of this application;

[0020] Figure 4 A schematic diagram of an activation function provided in an embodiment of this application;

[0021] Figure 5 A structural block diagram of a construction document information recognition device provided in this application embodiment;

[0022] Figure 6 This application provides a schematic diagram of the structure of a drawing loading module according to an embodiment of the present application.

[0023] Figure 7 This is a schematic diagram of the structure of a drawing preview module provided in an embodiment of this application;

[0024] Figure 8 This application provides a schematic diagram of the structure of an identification area management module.

[0025] Figure 9 This is a schematic diagram of the structure of an identification and storage module provided in an embodiment of this application. Detailed Implementation

[0026] The technical solutions of this application will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] To address the shortcomings of existing construction methods and reduce human error while improving the efficiency of construction drawing information collection, this application provides a method, device, and system for information recognition of construction documents. This overcomes the deficiencies of existing manual input methods by using an industrial control computer to input drawings from construction documents. Through automatic splitting of the loaded drawings, previewing sub-drawings, marking recognition areas, text recognition, and saving recognition results, intelligent information recognition of construction documents is achieved, improving the accuracy and efficiency of information recognition, thereby increasing construction efficiency.

[0028] To facilitate understanding of this embodiment, a method for identifying construction document information disclosed in this application embodiment will first be described in detail.

[0029] Figure 1 A flowchart of a method for identifying information in construction documents provided in this application embodiment, the method being applied to an industrial control computer; the method specifically includes the following steps:

[0030] Step S102: Load the target drawing from the construction file; the drawing must include at least one of the following: construction drawing, table, or text;

[0031] The target drawings in the above-loaded construction files can recognize vector and scanned PDF and TIF format construction drawings. These drawings may include construction drawings, tables, text, etc. The construction drawings may also contain text, and the tables may also contain text.

[0032] The aforementioned industrial computer is equipped with a visual interface, allowing users to import and select different PDF files, and manually select text regions, detect, recognize, and export results using function buttons.

[0033] Step S104: Split the loaded target drawing into multiple sub-drawings by page;

[0034] A construction file may contain multiple construction drawings, and each construction drawing may include multiple pages of sub-drawings. Therefore, after loading the target drawing into the construction file, it can be automatically split into multiple sub-drawings page by page.

[0035] Step S106: Display the preview target sub-drawing according to the page number and zoom level selected by the user;

[0036] For example, if a user selects page 3 and a zoom level of 1.2, the target sub-drawing on page 3 will be displayed in the preview area at a 1.2x magnification for the user to view.

[0037] Step S108: Automatically find and mark the target recognition area in the target sub-drawing using AI algorithm, and / or, in response to the user's editing operation on the recognition area of ​​the target sub-drawing, mark the target recognition area;

[0038] If the user does not select or edit a region, the system will automatically find and mark the target identification area in the target sub-drawing, such as a table area filled with text or a construction drawing area marked with text. Alternatively, the user can directly select the target identification area in the previewed target sub-drawing. Or, after the system automatically finds and marks the target identification area, it will respond to the user's adjustment and editing operations on the target identification area, and use the adjusted and edited selected area as the target identification area.

[0039] Step S110: Using a preset information recognition model, perform text recognition on the target recognition area marked in the target sub-drawing to obtain the text recognition information corresponding to the target recognition area.

[0040] The aforementioned information recognition model can be trained on an industrial control computer, or it can be trained on a training server and then imported into the industrial control computer. In this embodiment, the sample set used to train the model is an expandable adversarial sample library, which can upgrade the model's recognition capabilities. The information recognition model is a recognition model based on printed text lines, capable of recognizing text content in the target area.

[0041] Step S112: Save the text recognition information corresponding to the target recognition area.

[0042] The construction document information recognition method provided in this application can address the differences in drawings from different fields, industries, and layouts, and can effectively extract content by defining a custom recognition area. It automatically collects drawing data, stores it in a middleware database, and maintains integration with the construction management system. This improves the efficiency and accuracy of information recognition and extraction from construction documents, thereby increasing construction efficiency.

[0043] This application also provides another method for identifying construction document information, which is implemented based on the above embodiments; this embodiment focuses on describing the model training process and the possible implementation methods of the above steps.

[0044] The training process of the above information recognition model is as follows:

[0045] Obtain a training sample set; the samples in the training sample set include: drawings labeled with text location data, text image boxes, and the text content corresponding to the text image boxes; based on the text location data of the samples in the training sample set, train a preset text detection model to obtain a text detection model; based on the text image boxes and the text content corresponding to the text image boxes of the samples in the training sample set, train a preset text recognition model to obtain a text recognition model; use the text detection model and the text recognition model as an information recognition model; when new sample data is obtained, train the information recognition model through incremental training mode to obtain an updated information recognition model.

[0046] In practice, to effectively improve the accuracy of drawing recognition, a sample library, namely the training sample set mentioned above, is established by analyzing the information on the drawings and extracting images of the fields to be recognized. First, the text content to be recognized in the PDF drawing is selected using an annotation tool to form the sample library. During the drawing recognition process, the information on the drawing is recognized and compared with the sample information in the sample library, effectively improving the recognition accuracy.

[0047] First, based on the text location data marked in the drawing, a text detection model is trained. If the data is new, the model is trained incrementally. The specific process is as follows: Figure 2 As shown. Then, based on the text images cut out from the drawing and the corresponding labeled text content data, a text recognition model is trained. If it is new data, an incremental training mode is used, such as... Figure 3 As shown in the image. Finally, the generated model is deployed online.

[0048] The aforementioned information recognition model includes an encoder, a decoder, and an objective function. The encoder comprises a two-dimensional convolutional neural network, a two-dimensional batch normalization layer, and a ReLU-like function. The decoder comprises a recurrent neural network with a gating mechanism and a fully connected neural network. The encoder is used to perform high-dimensional abstraction and encoding on the original input feature map, extracting local and global feature information that can effectively distinguish characters. The decoder is used to effectively decode the high-dimensional input features according to the decoding direction, extracting local and global feature information that can effectively distinguish the relationship between characters. The objective function is used to set the convergence target for the training process and quantify the convergence degree of each iteration, providing the gradient error of backpropagation in the form of derivatives.

[0049] The hyperparameter design for training the above model includes: increasing the number of convolutional kernels layer by layer in multiples of 2 during convolutional encoding; setting the text height to 32 pixels; selecting fonts based on GB / T 14691-93; and conducting comparative ablation experiments on the following parameters: 4-8 convolutional layers, 16-256 convolutional kernels, the number of convolutional groups with the number of kernels or half the number of kernels, point-wise and depth-wise, and 1-2 recursive layers.

[0050] Point-wise convolution typically refers to convolution operations performed independently on each pixel in a convolutional neural network, such as a 1x1 convolution. Point-wise convolution can be viewed as independently adjusting the features of each channel without changing the dimension of the feature map, but it can enhance the expressive power of the features.

[0051] Depth-wise convolution refers to performing convolution operations on each channel of the input feature map separately, with each channel having an independent convolution kernel. This allows the model to learn the features of each channel without increasing the number of parameters. This operation is typically used to reduce the computational cost and number of parameters in the model, thereby improving its efficiency.

[0052] The feature extraction process is optimized by combining these two convolution methods. First, depth-wise convolution is used to extract features independently in each channel, and then point-wise convolution is used to merge these features. Alternatively, depth-wise convolution can be performed first and then point-wise convolution in some layers to enhance the model's ability to capture features.

[0053] The training process will be explained in detail below:

[0054] To meet the needs of rapid training and deployment, the training framework uses PyTorch, the programming language is Python, and the hardware is based on a GPU (NVIDIA GeForce RTX 2080Ti) server.

[0055] For sequence-to-sequence (Seq2Seq) recognition tasks, the overall model structure consists of three parts: an encoder, a decoder, and an objective function. The encoder performs high-dimensional abstraction and encoding of the original input feature map, extracting local and global feature information that can effectively distinguish characters. The decoder effectively decodes the high-dimensional input features along the decoding direction, extracting local and global feature information that can effectively distinguish the relationships between characters. The objective function sets a convergence target for the entire training process and quantifies the convergence degree of each iteration, providing the gradient error for backpropagation in the form of derivatives.

[0056] Since the input text line is a 256-level grayscale image, i.e., a single-channel byte image, a 2D convolutional neural network is chosen for effective feature encoding during the encoding stage to effectively capture text shape features and avoid inefficient manual feature design. Simultaneously, to accelerate model convergence and further approximate the independent and identically distributed assumption, ensuring that the input to each layer maintains approximately the same distribution and zero mean, batch normalization is incorporated. Furthermore, to accelerate model convergence and address the vanishing and exploding gradient problems, a ReLU-like activation function is adopted.

[0057] Since the reading order of text lines is horizontal, meaning the width corresponds to the decoding order, a recurrent neural network (RNN) is chosen for effective feature decoding in the decoding stage to capture the features encoded by the convolutional neural network and adapt to different font widths, font spacing, and font deformations. Furthermore, to address the vanishing and exploding gradient problems, a RNN with a gating mechanism is employed. Finally, to provide activation values ​​for different character categories, a fully connected neural network is used to output the activation values ​​for each character.

[0058] Since it is difficult to pinpoint the position of each character in the training data and to perform accurate character segmentation during use, CTC (Connectionist Temporal Classification) is chosen as the objective function to address the problem that the input and output sequences are difficult to correspond one-to-one, thereby improving recognition accuracy.

[0059] The following sections provide a detailed explanation of convolutional neural networks, normalization, activation operations, recurrent neural networks, fully connected neural networks, and objective functions:

[0060] (1) Convolutional Neural Network:

[0061] The input to the aforementioned 2D convolutional neural network is a tensor with an N×C×H×W structure, where N is the number of data sets during training, C is the number of channels, H is the height, and W is the width. The output tensor retains its structure, with N remaining constant, while C, H, and W may change according to the convolution parameters. The 2D convolution formula is as follows:

[0062]

[0063] Here, ★ represents the cross-correlation operation. out represents the output feature map; bias represents the bias term, used to adjust the output of the convolutional layer; weight represents the weight of the convolutional kernel, which determines the filtering characteristics of the convolutional operation; input represents the input feature map, i.e., the original image or the output of the previous layer; k represents the kernel size, which determines the receptive field size of the convolutional operation.

[0064] The parameters involved in the 2D convolution process are: kernel size, which determines the local receptive field of the convolution operation; stride, which represents the step size of the convolution kernel; padding, which adds extra pixels to the edges of the input feature map to control the size of the output feature map; dilation, which represents the spacing between elements in the convolution kernel, which can increase the receptive field without increasing the number of parameters; and group, which groups the input channels for convolution, which can reduce the number of parameters and computational cost.

[0065] (2) Normalization operation:

[0066] The input to the above normalization operation is a tensor with an N×C×H×W structure, where N is the number of data points in a training dataset, C is the number of channels, H is the height, and W is the width. The output tensor retains its structure, and N, C, H, and W also remain unchanged. The formula for the 2D batch normalization operation is as follows:

[0067]

[0068] Where γ and β are learnable parameters, E is the expectation (mean), and Var is the variance. γ represents the scaling factor, a learnable parameter used to scale the standardized output; β represents the offset, a learnable parameter used to offset the standardized output; E represents the expectation (mean), a statistic used to calculate the mean; ∈ represents a small constant used for numerical stability; and Var represents the variance, a statistic used to calculate the variance.

[0069] The 2D batch normalization operation is performed channel by channel.

[0070] (3) Activation operation:

[0071] The input to the activation function is a tensor of structure N×C×H×W, where N is the number of data sets during training, C is the number of channels, H is the height, and W is the width. The tensor at output maintains the same structure, and N, C, H, and W also remain unchanged.

[0072] The activation function chosen is LeakyReLU, and its formula and graph are as follows: Figure 4 As shown:

[0073] Here, negative_slope is the slope coefficient of the negative value region.

[0074] LeakyReLU operations are performed point by point.

[0075] (4) Recurrent Neural Network

[0076] The input to a recurrent neural network is a tensor with a T×N×O structure, where T is the length of the time axis, N is the number of data sets during training, and O is the number of neurons. The tensor at output maintains the same structure, with T and N remaining unchanged, while O may change according to the hidden unit parameters.

[0077] The recurrent neural network selects a GRU with a gating mechanism, and the formula is as follows:

[0078] r t =σ(W ir x t +b ir +W hr h (t-1) +bhr )

[0079] z t =σ(W iz x t +b iz +W hz h (t-1) +b hz )

[0080] n t =tanh(W in x t +b in +r t *(W hn h (t-1) +b hn ))

[0081] h t =(1-z) t )*n t +z t *h (t-1) ;

[0082] Here, * represents the Hadamard product operation, which refers to the element-wise multiplication operation. The parameter involved in the GRU process is the number of hidden units, which determines the complexity and capacity of the GRU model.

[0083] ir: Input to the reset gate, used to control the effect of the hidden state of the previous time step on the current time step;

[0084] hr: The output of the reset gate, usually represented as rt. It is a value between 0 and 1 used to adjust the hidden state h of the previous time step. (t-1) Impact on the current time step;

[0085] xt: Input for the current time step;

[0086] zt: Update gate, used to control the degree to which new information is updated;

[0087] rt: Reset gate, used to decide whether to ignore or retain information from the previous time step;

[0088] ht: Candidate hidden state;

[0089] nt: The hidden state at the current time step;

[0090] hn: The updated hidden state;

[0091] in: Input for the current time step;

[0092] iz: Input for updating and resetting doors;

[0093] σ: Activation function.

[0094] (5) Fully connected neural networks

[0095] The input to a fully connected neural network is an N×O tensor, where N is the number of data points in a training dataset and O is the number of neurons. The output tensor retains its structure, with N remaining constant while O may change according to the hidden unit parameters. The fully connected formula is as follows:

[0096] y = xA T +b;

[0097] Where A represents the weight matrix, used to connect the output of the previous layer and the input of the current layer; b represents the bias vector, used to adjust the output of the fully connected layer; and T represents the transpose operation, used to transpose the weight matrix.

[0098] The parameter involved in the fully connected process is the number of hidden units (hidden size).

[0099] (6) Objective function

[0100] The objective function is CTC. Assuming the outputs at different time steps are independent, the probability formula for any output sequence π is as follows:

[0101]

[0102] Where π represents the output sequence; p(π|x) represents the probability of outputting the sequence π given the labeled text sequence x. T is the sequence length, and πt represents all elements in π up to time step t.

[0103] The goal is to maximize the probability of the labeled text sequence, which means optimizing the model parameters during training so that the probability of the output sequence of the model is maximized compared with the labeled text sequence.

[0104] During training, the goal is to maximize the probability of labeled text sequences.

[0105] In this embodiment of the application, the hyperparameter design includes the following aspects:

[0106] 1) Incremental kernel strategy

[0107] To effectively preserve discriminative high-dimensional features during low-dimensional to high-dimensional encoding, convolutional encoding increases the number of convolutional kernels layer by layer in multiples of 2. By increasing the number of convolutional kernels layer by layer in multiples of 2, discriminative high-dimensional features can be effectively preserved during low-dimensional to high-dimensional encoding. This design helps the model capture features of different granularities at different levels, thereby improving the model's recognition ability.

[0108] 2) Character height

[0109] Due to the unique characteristics of printed optical characters, a text height of 32 pixels is sufficient for text differentiation. Setting the text height to 32 pixels satisfies the requirements for text differentiation while avoiding unnecessary calculations and improving the model's efficiency.

[0110] 3) Data simulation

[0111] Font selection was based on GB / T 14691-93. This ensured the quality and diversity of the training data, which helps improve the model's generalization ability.

[0112] 4) Ablation test

[0113] To select the model with the strongest classification ability, ablation experiments need to be conducted on the following parameters:

[0114] Number of convolutional layers (4-8)

[0115] Convolution kernel number (16-256)

[0116] Number of convolutional groups (number of kernels or half the number of kernels)

[0117] Point-wise and depth-wise (pw+dw and pw+dw+pw)

[0118] Recursion depth (1-2)

[0119] By conducting comparative ablation experiments, the model parameter combination with the strongest classification ability can be selected to optimize model performance. The purpose of these hyperparameter designs is to optimize the model training process, improve the model's recognition accuracy and efficiency, and ensure that the model can achieve better results in practical applications.

[0120] After the model is trained, the model parameters are exported, and the model is JIT (Just In Time) tracked and packaged, and then loaded and predicted in Libtorch.

[0121] Based on the overall method workflow, the corresponding apparatus can be divided into four functionally independent modules, see [link to relevant documentation]. Figure 5 As shown, these are the drawing loading module 52, the drawing preview module 54, the recognition area management module 56, and the recognition and saving module 58, respectively.

[0122] The drawing loading module 52 is responsible for loading drawings. Users can load all drawings in a folder at once, or load only a few selected drawings from a folder. If incorrect drawing information is found, the user can clear the loaded drawings.

[0123] The drawing preview module 54 is responsible for splitting the loaded drawings into pages and displaying the drawing information according to the page number and zoom level selected by the user.

[0124] The recognition area management module 56 is responsible for marking the recognition areas in the selected drawing. It can automatically find and mark the recognition areas through AI algorithms, or manually edit the marked recognition areas.

[0125] The recognition and saving module 58 recognizes the recognition modules marked in the drawing and saves the recognition results.

[0126] The following details the specific process of another construction document information identification method provided in this application embodiment, based on the above four modules:

[0127] (1) The drawing loading module loads the target drawings from the construction files.

[0128] Drawing loading is part of the software's preparation phase. Users determine which drawings require information collection and then load those drawings into the software. The drawing loading module has three sub-functions: loading drawings, adding drawings, and clearing drawings. Figure 6 As shown:

[0129] Loading drawings means that users can load all PDF drawings from a folder. The software will clear any already loaded drawings before loading any new ones. After the user triggers the load drawing operation, the software will display a folder selection dialog box. After the user selects the folder to load, the software will load all PDF files in that folder.

[0130] Adding drawings means that users can choose to load a subset of PDF drawings from a folder into the software. This operation will not delete already loaded drawings. After the user triggers the add drawings operation, the software will pop up a file selection dialog box. After the user selects the files to be loaded, the software will load the selected files.

[0131] Clearing drawings means that users can clear the loaded drawings.

[0132] In other words, the industrial control computer can respond to the user's selection of a target construction file and load all drawings in the target construction file as the target drawings; or, respond to the user's selection of a target drawing in the target construction file and load the selected target drawing. Additionally, it can respond to the user's clear drawing operation and clear the loaded drawings.

[0133] (2) The drawing preview module splits the loaded target drawing into multiple sub-drawings by page; and displays the preview target sub-drawings according to the page number and scaling ratio selected by the user.

[0134] After the drawing information is loaded into the software, the software will split the drawing into pages and display the images of the user-specified pages for preview, providing a foundation for the identification area management module. The drawing preview module includes two sub-functions: drawing splitting and drawing preview, such as... Figure 7 As shown:

[0135] Loaded drawings in PDF format will have multiple pages. The drawing preview module will split the loaded PDF drawing into different sub-drawings page by page. Subsequent previewing, recognition area management, and recognition saving are all performed on a specific sub-drawing page.

[0136] When previewing drawings, you can select which page to preview and zoom in / out on the selected page for the best preview effect. This helps avoid errors caused by unclear drawings.

[0137] (3) The identification area management module automatically finds and marks the target identification area in the target sub-drawing through AI algorithm, or responds to the user's editing operation of the identification area of ​​the target sub-drawing and marks the target identification area.

[0138] The identification area management module is the core module of the system. There are two ways to determine the identification area: one is to automatically identify and mark the identification area in the drawing through AI algorithms, and the other is for the user to determine and modify the identification area through software. The former is completed by computer calculation, which can reduce a lot of manual operation, while the latter is a supplement and improvement to the former, correcting some errors and mistakes of the AI ​​algorithm.

[0139] The recognition area management module has two sub-functions: AI algorithm determines the recognition area and user edits the recognition area, such as... Figure 8 As shown:

[0140] AI algorithms can identify and mark which areas in a drawing need to be identified, which can be done by computer, reducing the need for user intervention.

[0141] Once the software enters the recognition area editing mode, users can modify the size and position of the recognition area determined by the AI ​​algorithm, and can also add or delete a recognition area.

[0142] (4) The recognition and saving module performs text recognition on the target recognition area marked in the target sub-drawing through the preset information recognition model, and obtains the text recognition information corresponding to the target recognition area; and saves the text recognition information corresponding to the target recognition area.

[0143] Specifically, the OCR recognition module within the information recognition model performs text recognition and extraction on the target recognition area to obtain the corresponding text recognition information. In practice, the aforementioned OCR recognition module includes a text detection model and a text recognition model. The steps for recognizing the marked target recognition area in the target sub-drawing using the preset information recognition model include: detecting the text position in the target recognition area using the text detection model within the information recognition model; and recognizing the text at the detected text position using the text recognition model within the information recognition model to obtain the text recognition information corresponding to the target recognition area. Finally, the text recognition information corresponding to the target recognition area is saved to an Excel file with the same name as the target drawing and a page number suffix for easy viewing by the user.

[0144] The recognition and saving module is one of the software's output modules, consisting of two sub-functions: OCR recognition and result saving. The OCR recognition function identifies the regions defined in the recognition area management module and converts the image into text information; the result saving function saves the recognition results as an Excel spreadsheet file according to a predefined format. The sub-functions of the recognition and saving module are as follows: Figure 9 As shown.

[0145] The embodiments of this application can automate information and data collection, significantly reduce personnel input, effectively eliminate human error, shorten the data collection cycle, and save investment costs.

[0146] Taking 180,000 drawings as an example, manual input can only input a maximum of 600 drawings per day, working 22 days a month. It would take 10 input operators 14 months to complete the task, and 5 people are needed for review. Using this system, 800 drawings can be recognized per day, working 22 days a month, and the task can be completed in 10 months. Only 5 people are needed for post-maintenance and review, saving more than 600,000 yuan in labor costs.

[0147] This system uses AI to build a massive library of engineering drawing samples, possessing the largest number of nuclear engineering drawing information samples in China. It has developed a neural network learning algorithm, and on average, it takes less than 1 minute to complete data collection for each drawing, with an accuracy rate of 99.5%. It also has a 100% error correction function, effectively eliminating human error and significantly improving the efficiency of reading and inputting drawings. It is suitable for data collection of drawing and material information in various fields, especially in the field of nuclear engineering construction.

[0148] In the construction document information recognition method provided in this application embodiment, the industrial control computer allows users to load construction drawings of various formats and split the drawings by page, allowing users to preview and operate on a page-by-page basis. The AI ​​algorithm autonomously determines which areas in the drawings need to be recognized and marks them, or allows users to edit the recognized areas. After the recognition areas of the drawings are determined, the text within those areas can be recognized, and the recognition results can be saved to an Excel spreadsheet file. This greatly improves the accuracy and efficiency of information recognition and extraction, thereby improving construction efficiency.

[0149] Based on the above method embodiments, this application also provides an information recognition device for construction documents, which is applied to an industrial control computer; see also Figure 5 As shown, the device includes: a drawing loading module 52, a drawing preview module 54, a recognition area management module 56, and a recognition and saving module 58; wherein, the drawing loading module 52 is used to load target drawings from construction files; the drawings include at least one of the following: construction drawings, tables, and text; the drawing preview module 54 is used to split the loaded target drawings into multiple sub-drawings by page; and displays the previewed target sub-drawings according to the page number and scaling ratio selected by the user; the recognition area management module 56 is used to automatically find and mark target recognition areas in the target sub-drawings using AI algorithms, and / or, in response to the user's editing operation on the recognition areas of the target sub-drawings, mark the target recognition areas; the recognition and saving module 58 is used to perform text recognition on the marked target recognition areas in the target sub-drawings using a preset information recognition model, obtain the text recognition information corresponding to the target recognition areas, and save the text recognition information corresponding to the target recognition areas.

[0150] Furthermore, the drawing loading module 52 is used to respond to the user's selection operation of the target construction file and load all the drawings in the target construction file as target drawings; or, in response to the user's selection operation of the target drawings in the target construction file, load the selected target drawings.

[0151] Furthermore, the aforementioned information recognition model includes a text detection model and a text recognition model; the recognition and storage module 58 is used to detect the text position in the target recognition area through the text detection model in the information recognition model, and to recognize the text at the detected text position through the text recognition model in the information recognition model, thereby obtaining the text recognition information corresponding to the target recognition area.

[0152] Furthermore, the aforementioned recognition and saving module 58 is used to save the text recognition information corresponding to the target recognition area to an Excel file with the same name as the target drawing and with the page number suffix added.

[0153] Furthermore, the above-mentioned device also includes: a model training module, used to perform the following training process of the information recognition model: acquiring a training sample set; the samples in the training sample set include: drawings labeled with text image boxes and text content corresponding to the text image boxes; training a preset text recognition model based on the samples in the training sample set to obtain an information recognition model; when new sample data is acquired, training the information recognition model through incremental training mode to obtain an updated information recognition model.

[0154] Furthermore, the aforementioned information recognition model includes an encoder, a decoder, and an objective function. The encoder includes a two-dimensional convolutional neural network, a two-dimensional batch normalization layer, and a ReLU-like function. The decoder includes a recurrent neural network with a gating mechanism and a fully connected neural network. The encoder is used to perform high-dimensional abstraction and encoding on the original input feature map, extracting local and global feature information that can effectively distinguish characters. The decoder is used to effectively decode the high-dimensional input features according to the decoding direction, extracting local and global feature information that can effectively distinguish the relationship between characters. The objective function is used to set the convergence target for the training process and quantify the convergence degree of each iteration, providing the gradient error of backpropagation in the form of derivatives.

[0155] Furthermore, the hyperparameter design during the training of the above model includes: increasing the number of convolutional kernels layer by layer in multiples of 2 during convolutional encoding; setting the text height to 32 pixels; selecting fonts based on GB / T 14691-93; and conducting comparative ablation experiments on the following parameters: 4-8 convolutional layers, 16-256 convolutional kernels, the number of convolutional groups with the number of kernels or half the number of kernels, point-wise and depth-wise, and 1-2 recursive layers.

[0156] The device provided in this application embodiment has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts of the device embodiment not mentioned can be referred to the corresponding content in the aforementioned method embodiment.

[0157] Based on the above method embodiments, this application also provides an information recognition system for construction documents, see [link to relevant documentation]. Figure 9 As shown, the system includes an industrial control computer 92 and a training server 94; the training server 94 is used to train an information recognition model and transmit the information recognition model to the industrial control computer 92; the industrial control computer 92 is used to identify construction information of the construction documents to be identified based on the information recognition model, using the method described in the method embodiment.

[0158] The system provided in this application embodiment has the same implementation principle and technical effects as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the system embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0159] This application also provides a computer-readable storage medium storing computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions cause the processor to implement the above-described method. For specific implementation details, please refer to the foregoing method embodiments, which will not be repeated here.

[0160] The computer program products of the methods, apparatus, and electronic devices provided in the embodiments of this application include a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the preceding method embodiments. For specific implementations, please refer to the method embodiments, which will not be repeated here.

[0161] Unless otherwise specifically stated, the relative steps, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of this application.

[0162] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0163] In the description of this application, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0164] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the technical scope disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A method for identifying information in construction documents, characterized in that, The method is applied to an industrial control computer; the method includes: Load the target drawings into the construction file; the drawings shall include at least one of the following: construction drawings, tables, and text; Split the loaded target drawing into multiple sub-drawings by page; The target sub-drawing is displayed in preview based on the page number and zoom level selected by the user; The AI ​​algorithm automatically finds and marks the target recognition area in the target sub-drawing, and / or, in response to the user's editing operation on the recognition area of ​​the target sub-drawing, marks the target recognition area; Using a pre-defined information recognition model, text recognition is performed on the marked target recognition areas in the target sub-drawing to obtain the text recognition information corresponding to the target recognition areas. The information recognition model includes an encoder, a decoder, and an objective function. The encoder is used to perform high-dimensional abstraction and encoding on the input original feature map, extracting local and global feature information that can effectively distinguish characters. The decoder is used to effectively decode the input high-dimensional features according to the decoding direction, extracting local and global feature information that can effectively distinguish the relationship between characters. The objective function is used to set a convergence target for the training process and quantify the convergence degree of each iteration, providing the gradient error of backpropagation in the form of derivatives. The encoder includes a two-dimensional convolutional neural network, a two-dimensional batch normalization layer, and a ReLU-like function. The decoder includes a recurrent neural network with a gating mechanism and a fully connected neural network. The objective function includes a CTC connection temporal classification function. Save the text recognition information corresponding to the target recognition area; The hyperparameter design of the information recognition model during training includes: increasing the number of convolutional kernels layer by layer in multiples of 2 during convolutional encoding; setting the text height to 32 pixels; selecting fonts based on GB / T 14691-93; and conducting comparative ablation experiments on the following parameters: 4-8 convolutional layers, 16-256 convolutional kernels, the number of convolutional groups with the number of kernels or half the number of kernels, point-wise and depth-wise, and 1-2 recursive layers.

2. The method according to claim 1, characterized in that, The steps for loading the target drawings into the construction file include: In response to the user's selection of a target construction file, all drawings in the target construction file are loaded as target drawings; or, in response to the user's selection of a target drawing in the target construction file, the selected target drawing is loaded.

3. The method according to claim 1, characterized in that, The information recognition model includes a text detection model and a text recognition model; the step of recognizing the marked target recognition area in the target sub-drawing using the preset information recognition model includes: The text detection model in the information recognition model is used to detect the text position in the target recognition area, and the text at the detected text position is recognized by the text recognition model in the information recognition model to obtain the text recognition information corresponding to the target recognition area.

4. The method according to claim 1, characterized in that, The step of saving the text recognition information corresponding to the target recognition area includes: Save the text recognition information corresponding to the target recognition area to an Excel file with the same name as the target drawing and a page number suffix.

5. The method according to claim 1, characterized in that, The training process of the information recognition model is as follows: Obtain a training sample set; the samples in the training sample set include: drawings labeled with text position data, text image frames, and text content corresponding to the text image frames; Based on the text position data of the samples in the training sample set, a preset text detection model is trained to obtain a text detection model; based on the text image boxes of the samples in the training sample set and the text content corresponding to the text image boxes, a preset text recognition model is trained to obtain a text recognition model; the text detection model and the text recognition model are used as information recognition models; When new sample data is acquired, the information recognition model is trained using incremental training mode to obtain an updated information recognition model.

6. An information recognition device for construction documents, characterized in that, The device is applied to an industrial control computer; the device includes: a drawing loading module, a drawing preview module, a recognition area management module, and a recognition and saving module; wherein... The drawing loading module is used to load target drawings from construction files; the drawings include at least one of the following: construction drawings, tables, and text. The drawing preview module is used to split the loaded target drawing into multiple sub-drawings by page; and to display the previewed target sub-drawings according to the page number and zoom ratio selected by the user. The identification area management module is used to automatically find and mark the target identification area in the target sub-drawing using AI algorithms, and / or, in response to the user's editing operation on the identification area of ​​the target sub-drawing, mark the target identification area; The recognition and storage module is used to perform text recognition on the marked target recognition areas in the target sub-drawing using a preset information recognition model, to obtain the text recognition information corresponding to the target recognition areas; and to store the text recognition information corresponding to the target recognition areas. The information recognition model includes an encoder, a decoder, and an objective function. The encoder is used to perform high-dimensional abstraction and encoding on the input original feature map, extracting local and global feature information that can effectively distinguish characters. The decoder is used to effectively decode the input high-dimensional features according to the decoding direction, extracting local and global feature information that can effectively distinguish the relationship between characters. The objective function is used to set a convergence target for the training process and quantify the convergence degree of each iteration, providing the gradient error of backpropagation in the form of derivatives. The encoder includes a two-dimensional convolutional neural network, a two-dimensional batch normalization layer, and a ReLU-like function. The decoder includes a recurrent neural network with a gating mechanism and a fully connected neural network. The objective function includes a CTC connection temporal classification function. The hyperparameter design of the information recognition model during training includes: increasing the number of convolutional kernels layer by layer in multiples of 2 during convolutional encoding; setting the text height to 32 pixels; selecting fonts based on GB / T 14691-93; and conducting comparative ablation experiments on the following parameters: 4-8 convolutional layers, 16-256 convolutional kernels, the number of convolutional groups with the number of kernels or half the number of kernels, point-wise and depth-wise, and 1-2 recursive layers.

7. An information recognition system for construction documents, characterized in that, The system includes an industrial control computer and a training server; the training server is used to train an information recognition model and transmit the information recognition model to the industrial control computer; the industrial control computer is used to perform construction information recognition on the construction documents to be recognized based on the information recognition model and using the method described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions that, when invoked and executed by a processor, cause the processor to perform the method described in any one of claims 1 to 5.

Images