A Deep Learning-Based Intelligent Image Recognition and Semantic Understanding System
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANJING HUININGJIE INFORMATION TECH CO LTD
- Filing Date
- 2025-12-26
- Publication Date
- 2026-05-26
Smart Images
Figure CN122090475A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and in particular to a deep learning-based intelligent image recognition and semantic understanding system. Background Technology
[0002] The drawing and document system serves as a crucial interactive platform between power supply companies and customers, carrying core documents such as renovation plans, construction drawings, and project schedules. Among these, various engineering forms (such as material summary tables, equipment lists, and parameter tables) act as structured carriers of key information, serving as direct bases for construction procurement, cost accounting, and service implementation, as well as important evidence for internal business collaboration and responsibility definition.
[0003] Currently, there are three prominent problems in the intelligent management of table information: First, there is a lack of automatic and unified extraction standards. Table styles vary in different drawings, and manual collection easily leads to incomplete information and chaotic formats, affecting subsequent data analysis and decision-making. Second, the verification of table content (such as the consistency of the number and name of equipment in the master table and sub-tables) relies too much on the experience of professionals and visual comparison, which is labor-intensive, time-consuming, and prone to errors, making it difficult to meet the needs of large-scale project review. Third, existing technologies for table recognition are mostly limited to the level of text extraction (OCR), lacking in-depth analysis of the table's logical structure (cell merging, row and column relationships) and intelligent understanding of the semantic meaning of text within cells (such as synonymous device names), resulting in low levels of automated verification.
[0004] Therefore, it is urgent to build a deep learning-based intelligent document recognition and semantic understanding system to achieve automatic positioning of table areas in engineering drawings, accurate analysis of internal structures, accurate extraction and association of text content, and ultimately achieve semantic-level automatic verification of key information, significantly improving the efficiency and accuracy of document management. Summary of the Invention
[0005] The purpose of this invention is to provide a deep learning-based intelligent document recognition and semantic understanding system, which aims to achieve automatic positioning of table areas in engineering drawings, accurate analysis of internal structures, accurate extraction and association of text content, and ultimately achieve semantic-level automatic verification of key information, significantly improving the efficiency and accuracy of document management.
[0006] To achieve the above objectives, the present invention provides a deep learning-based intelligent image recognition and semantic understanding system, comprising a table object recognition module, a table structure recognition module, a text recognition and cell correspondence module, and a device name intelligent verification module; the table object recognition module, the table structure recognition module, the text recognition and cell correspondence module, and the device name intelligent verification module are connected sequentially. The table object recognition module is used to locate and identify table areas from the input engineering drawing image; The table structure recognition module is used to parse the table area output by the table object recognition module and identify the logical structure and spatial relationship of its internal cells. The text recognition and cell mapping module is used to perform text recognition on the cells output by the table structure recognition module and associate the recognized text with its corresponding cell. The intelligent device name verification module is used to perform semantic similarity calculation and consistency verification on device names from different tables output by the text recognition and cell correspondence module.
[0007] The table object recognition module includes a target detection unit and an output parsing unit; the output parsing unit is connected to the target detection unit. The target detection unit is used to detect the input image using a deep learning model based on the YOLOv10 architecture and output the detection results containing the table region. The output parsing unit is used to receive the detection results, parse the bounding box coordinate information therein to determine the table position and output it to the table structure recognition module.
[0008] The table structure recognition module includes an image preprocessing unit, a structure parsing unit, and a large table processing unit; the image preprocessing unit, the structure parsing unit, and the large table processing unit are connected in sequence. The image preprocessing unit is used to perform grayscale and binarization processing on the table region image output by the table object recognition module to prepare for structural analysis. The structure parsing unit is used to simultaneously predict the spatial coordinates of each cell and its logical row and column position in the table using an improved LORE algorithm model. The large table processing unit is used to control the structure parsing unit to perform segmentation recognition and result merging operations on the table when the size of the table area image exceeds a preset threshold.
[0009] The text recognition and cell correspondence module includes an OCR text recognition unit, a coordinate sorting and matching unit, and a cross-cell text processing unit. The OCR text recognition unit uses the RapidOCR engine to recognize the text content in the image and outputs the bounding box coordinates of each text region. The coordinate sorting and matching unit is used to sort the cell coordinates output by the table structure recognition module to determine the row and column structure of the table, and calculate the positional relationship between the center point coordinates of the text area and the cell bounding box, so as to accurately associate the text content with the corresponding cell. The cross-cell text processing unit is used to detect text regions whose bounding boxes span multiple cells, merge the relevant cell regions, and then re-identify them.
[0010] The device name intelligent verification module includes a semantic vectorization unit, a similarity calculation unit, and an optimization unit; the semantic vectorization unit, the similarity calculation unit, and the optimization unit are connected in sequence. The semantic vectorization unit is used to convert the text-based device name into a high-dimensional semantic vector representation by using a BERT pre-trained model and combining the first and last layer average pooling strategy. The similarity calculation unit is used to calculate the comprehensive similarity between two device names. The comprehensive similarity is a weighted sum of semantic similarity based on BERT and traditional cosine similarity based on word segmentation. The optimization unit is used to optimize the weight adjustment factor in the weighted sum using an improved cuckoo search algorithm, and provides the optimized factor to the similarity calculation unit.
[0011] The improved cuckoo search algorithm incorporates a dynamic step size adjustment strategy based on a sine function and a random position update strategy that simulates the catfish effect.
[0012] The improved LORE algorithm model integrates the segmentation, recognition, and merging processes of the large table processing unit into the original LORE framework.
[0013] The coordinate sorting and matching unit uses a judgment condition that includes a configurable pixel tolerance value when determining whether the text belongs to a cell.
[0014] This invention discloses a deep learning-based intelligent image recognition and semantic understanding system. The system includes a table object recognition module for locating and identifying table regions from input engineering drawing images; a table structure recognition module for parsing the table regions output by the table object recognition module and identifying their internal cell logical structure and spatial relationships; a text recognition and cell correspondence module for performing text recognition on the cells output by the table structure recognition module and associating the recognized text with their respective cells; and a device name intelligent verification module for performing semantic similarity calculation and consistency verification on device names from different tables output by the text recognition and cell correspondence module. Through these methods, the system achieves automatic location of table regions in engineering drawings, accurate parsing of internal structures, accurate extraction and association of text content, and ultimately, automatic semantic verification of key information, significantly improving the efficiency and accuracy of image management. Attached Figure Description
[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.
[0016] Figure 1 This is a schematic diagram of the structure of a deep learning-based image intelligent recognition and semantic understanding system according to the present invention.
[0017] Figure 2 This is a structural schematic diagram of the table object recognition module of the present invention.
[0018] Figure 3 This is a schematic diagram of the table structure recognition module of the present invention.
[0019] Figure 4 This is a structural diagram of the text recognition and cell correspondence module of the present invention.
[0020] Figure 5 This is a structural schematic diagram of the intelligent device name verification module of the present invention.
[0021] Figure 6 This is a diagram of the YOLOv10 system framework.
[0022] Figure 7 This is a diagram of the improved LORE algorithm framework.
[0023] Figure 8 This is a diagram illustrating the First-last layer average pooling operation.
[0024] Figure 9 This is the flowchart of the improved Cuckoo algorithm.
[0025] Figure 10It is the training iteration curve.
[0026] Figure 11 This is the recognition result of the YOLOv10 model.
[0027] Figure 12 This is a comparison diagram of the recognition process of the algorithm in this application and the original LORE algorithm.
[0028] Figure 13 These are the iterative curves for the three algorithms.
[0029] 1-Table object recognition module, 2-Table structure recognition module, 3-Text recognition and cell correspondence module, 4-Device name intelligent verification module, 101-Target detection unit, 102-Output parsing unit, 201-Image preprocessing unit, 202-Structure parsing unit, 203-Large table processing unit, 301-OCR text recognition unit, 302-Coordinate sorting and matching unit, 303-Cross-cell text processing unit, 401-Semantic vectorization unit, 402-Similarity calculation unit, 403-Optimization unit. Detailed Implementation
[0030] This invention provides a deep learning-based intelligent image recognition and semantic understanding system, comprising a table object recognition module 1, a table structure recognition module 2, a text recognition and cell correspondence module 3, and a device name intelligent verification module 4; the table object recognition module 1, the table structure recognition module 2, the text recognition and cell correspondence module 3, and the device name intelligent verification module 4 are connected sequentially. The table object recognition module 1 is used to locate and recognize the table area from the input engineering drawing image; The table structure recognition module 2 is used to parse the table area output by the table object recognition module 1 and identify the logical structure and spatial relationship of its internal cells. The text recognition and cell correspondence module 3 is used to perform text recognition on the cells output by the table structure recognition module 2, and associate the recognized text with its corresponding cell. The device name intelligent verification module 4 is used to perform semantic similarity calculation and consistency verification on device names from different tables output by the text recognition and cell correspondence module 3.
[0031] In this embodiment, the table object recognition module 1 is used to locate and identify table areas from the input engineering drawing image; the table structure recognition module 2 is used to parse the table area output by the table object recognition module 1 and identify the logical structure and spatial relationship of its internal cells; the text recognition and cell correspondence module 3 is used to perform text recognition on the cells output by the table structure recognition module 2 and associate the recognized text with its corresponding cell; the device name intelligent verification module 4 is used to perform semantic similarity calculation and consistency verification on the device names from different tables output by the text recognition and cell correspondence module 3. Through the above methods, automatic location of table areas in engineering drawings, accurate parsing of internal structures, accurate extraction and association of text content are achieved, and finally, semantic-level automatic verification of key information is realized, significantly improving the efficiency and accuracy of drawing management.
[0032] Furthermore, the table object recognition module 1 includes a target detection unit 101 and an output parsing unit 102; the output parsing unit 102 is connected to the target detection unit 101. The target detection unit 101 is used to detect the input image using a deep learning model based on the YOLOv10 architecture and output the detection result containing the table region. The output parsing unit 102 is used to receive the detection results, parse the bounding box coordinate information therein, determine the table position, and output it to the table structure recognition module 2.
[0033] In this embodiment, YOLOv10 is the latest version of the YOLO series, offering higher detection accuracy and speed. The model's backbone network employs an enhanced Cross-Stage Partial Network (CSPNet), designed to optimize gradient flow and reduce computational redundancy, thereby improving feature extraction efficiency. In the neck layer of the network structure, a Path Aggregation Network (PAN) layer is introduced to effectively fuse multi-scale features, thus enhancing the model's ability to handle targets of different sizes. YOLOv10's head architecture employs a "many-to-one" design. During training, the model generates multiple predictions for each target, providing rich supervision information to improve learning accuracy. During inference, the model selects the optimal prediction for each target using a "many-to-one" approach, avoiding the use of non-maximum suppression (NMS). This design significantly reduces inference latency and improves overall efficiency.
[0034] The framework of the model is as follows Figure 6As shown. YOLOv10 provides the following output parameters for each detected bounding box: P(obj): Confidence that the target exists within the bounding box.
[0035] P(class|obj): The probability that the target within the bounding box belongs to a specific class.
[0036] Bx, By, Bw, Bh: The coordinates, width, and height of the bounding box.
[0037] Furthermore, the table structure recognition module 2 includes an image preprocessing unit 201, a structure parsing unit 202, and a large table processing unit 203; the image preprocessing unit 201, the structure parsing unit 202, and the large table processing unit 203 are connected in sequence. The image preprocessing unit 201 is used to perform grayscale and binarization processing on the table area image output by the table object recognition module 1 to prepare for structural analysis. The structure parsing unit 202 is used to simultaneously predict the spatial coordinates of each cell and its row and column logical position in the table using an improved LORE algorithm model. The large table processing unit 203 is used to control the structure parsing unit 202 to perform segmentation recognition and result merging operations on the table when the size of the table area image exceeds a preset threshold.
[0038] In this embodiment, the improved LORE algorithm model integrates the segmentation, recognition, and merging processes of the large table processing unit 203 into the original LORE framework; LORE uses a convolutional neural network (CNN) as its backbone to extract visual features of table cells from the input image. Subsequently, two regression heads are used to predict the spatial and logical locations of the cells, respectively. This method specifically utilizes cascaded regressions and employs inter-cell and intra-cell supervision mechanisms to model the dependencies and constraints between logical locations. However, the study found that this method has certain limitations when processing large-sized images. Therefore, for engineering drawings with a length or width exceeding 2500 pixels, we introduce a segmentation processing method. Figure 7 The overall processing flow is demonstrated as follows: Image preprocessing: The image is converted to grayscale and binarized to generate a black and white image for subsequent contour detection; Table frame detection: Horizontal and vertical morphological operations are used to extract the table frames; Frame merging: Horizontal and vertical frames are merged to form a complete table frame. Subsequently, each frame in the table is detected and its boundary is extracted; Table segmentation: The table is segmented based on the detected frames to divide specific table regions.
[0039] Furthermore, the text recognition and cell correspondence module 3 includes an OCR text recognition unit 301, a coordinate sorting and matching unit 302, and a cross-cell text processing unit 303; The OCR text recognition unit 301 uses the RapidOCR engine to recognize the text content in the image and outputs the bounding box coordinates of each text region. The coordinate sorting and matching unit 302 is used to sort the cell coordinates output by the table structure recognition module 2 to determine the row and column structure of the table, and calculate the positional relationship between the center point coordinates of the text area and the cell bounding box, so as to accurately associate the text content with the corresponding cell. The cross-cell text processing unit 303 is used to detect text regions whose bounding boxes span multiple cells, merge the relevant cell regions, and then re-identify them.
[0040] In this embodiment, the coordinate sorting and matching unit 302 uses a judgment condition that includes a configurable pixel tolerance value when determining whether the text belongs to a cell. The improved LORE model is used to identify the coordinates of the four corner points of each cell in a table. When processing tables in an image, the coordinates of each cell first need to be sorted to determine its row and column structure. Assume the coordinates of the top-left corner of each cell in the table are... The coordinates of the lower right corner are Then, sort the cells from left to right according to their coordinates. Let the set of cell coordinates be... Among them Arrange in ascending order to determine the column position, as shown in equation (1): ; Similarly, each cell The position can be used to determine the position of the row. Press Sort in ascending order, as shown in equation (2): ; By sorting the coordinates of all cells, the row and column positions of each cell in the table can be accurately determined. When processing images using RapidOCR, its output includes the coordinates of the four boundary points of each text region, which are... , , and These points form the bounding box of the text region. To determine whether text belongs to a specific cell, the coordinates of the center point of the text bounding box must first be calculated. As shown in equation (3): ; By comparing the center point of the text bounding box with the boundary of the table cell, it can be determined whether the text is inside a certain cell. Let's assume the coordinates of the top-left corner of a table cell are... The coordinates of the lower right corner are If the coordinates of the center point of the text bounding box satisfy the condition of equation (4), then it can be determined that the text is located within that cell: ; To address the issue of slight text overflow, a tolerance value can be introduced into the judgment condition. Even if the text boundary slightly exceeds the cell boundary, it can still be considered to belong to that cell. By setting an appropriate tolerance value, the judgment condition can be adjusted to equation (5): ; In this paper, the tolerance value is set to 2 pixels. Text that spans multiple cells needs to be processed separately. This can be identified by detecting whether the boundary of the text region exceeds the range of a single cell. If the coordinates of any point of the text bounding box exceed the range of a cell, it is determined that the text spans multiple cells, as shown in Equation (6): ; If this condition is met, multiple relevant cell ranges need to be extracted and processed as a single unit. Then, OCR recognition is re-performed on these merged cell ranges to ensure accurate recognition of text content spanning multiple cells.
[0041] Furthermore, the intelligent device name verification module 4 includes a semantic vectorization unit 401, a similarity calculation unit 402, and an optimization unit 403; the semantic vectorization unit 401, the similarity calculation unit 402, and the optimization unit 403 are connected in sequence. The semantic vectorization unit 401 is used to convert the text-based device name into a high-dimensional semantic vector representation by using a BERT pre-trained model and combining the first and last layer average pooling strategy. The similarity calculation unit 402 is used to calculate the comprehensive similarity between two device names. The comprehensive similarity is a weighted sum of semantic similarity based on BERT and traditional cosine similarity based on word segmentation. The optimization unit 403 is used to optimize the weight adjustment factor in the weighted sum using an improved cuckoo search algorithm, and provide the optimized factor to the similarity calculation unit 402.
[0042] In this embodiment, the improved cuckoo search algorithm introduces a dynamic step size adjustment strategy based on a sine function and a random position update strategy that simulates the catfish effect. By using the table extraction, structure recognition, and text recognition methods described in the previous section, the unstructured table data in the original engineering drawings can be transformed into structured two-dimensional table data. In engineering drawings, tables are divided into material summary tables and sub-tables. Taking a substation project as an example, the summary table contains the main transformer equipment, distribution equipment, and materials required for the project. Currently, the matching operation mainly relies on manual methods. This paper proposes an intelligent verification method, the core of which is semantic similarity matching based on equipment names. Since the summary table and sub-tables are usually recorded by different personnel and there are differences in description habits, a semantic matching method is needed to verify the consistency of equipment quantities. Cosine similarity: Similarity metrics are used to calculate the degree of similarity between individuals, usually measured by distance. The smaller the similarity value, the greater the distance; the larger the similarity value, the smaller the distance. In text similarity measurement, the most commonly used method is cosine similarity. It measures the difference between individuals by calculating the cosine value of the angle between two vectors in space. When the cosine value approaches 1, the angle approaches 0, indicating that the two vectors are more similar; when the cosine value approaches 0, the angle approaches 90 degrees, indicating that the two vectors are less similar. The calculation formula of cosine similarity is shown in Equation (7): ; In Equation 7, and are two i-dimensional vectors respectively, representing the cosine similarity between vector and vector. However, calculating cosine similarity based only on word segmentation has certain limitations, especially when dealing with acronyms. For example, in equipment names, "main equipment" is often used to represent "busbar equipment". If segmented by words, the similarity between "main equipment" and "busbar equipment" is 0, while when calculated by character segmentation, the similarity is 0.7. Relying solely on cosine similarity to match equipment names often makes it difficult to capture the semantic features between names. In this case, this paper proposes a semantic similarity matching method based on BERT to make full use of the powerful capabilities of large models. BERT is a pre-trained language model based on the deep learning framework Transformer and uses self-attention mechanism, which can effectively capture long-term dependencies in sequence data. By learning the relationships between words in the text, BERT can extract rich semantic information.
[0043] Since BERT processes the input text based on words, its output is the vector representation of each word in the text. To generate the overall vector representation of a sentence, pooling operations are required. Common pooling methods include CLS, Pooler, last-layer average pooling, and first-and-last-layer average pooling. This paper selects first-and-last-layer average pooling, as shown in Figure 8 By taking the average of all word vector representations of the first layer and the last layer to generate the vector representation of the sentence.
[0044] For the merged device name vectors, the semantic similarity is calculated using the cosine similarity method. The similarity obtained using the above method is denoted as... , representing two i-dimensional vectors and Semantic similarity between them.
[0045] Using only cosine similarity or semantic similarity to calculate the matching degree between two device names may have certain limitations. Therefore, this paper proposes a comprehensive scoring method to more accurately assess the similarity between phrases, as shown in equation (8):
[0046] α and β are the adjustment factors.
[0047] Adjustment factors are crucial for improving the accuracy of similarity calculations; therefore, constructing reasonable adjustment factors based on text characteristics is particularly important. To this end, this paper proposes an improved cuckoo search algorithm to optimize semantic similarity adjustment factors, making them more suitable for engineering drawing scenarios.
[0048] The Cuckoo Search Algorithm is a population-based intelligent optimization algorithm inspired by the competitive behavior of cuckoos choosing nests during their breeding season. Cuckoos tend to lay their eggs in high-quality nests to increase their breeding success rate. This algorithm simulates this process, iteratively updating the location of solutions and gradually optimizing their quality. In the initial stage, a set of initial solutions is randomly generated as nest locations. Subsequently, based on the quality of the solutions and the attractiveness of the nests, the cuckoos choose new locations. High-quality solutions attract more cuckoos, while low-quality solutions are likely to be eliminated. Through continuous iteration, the Cuckoo Search Algorithm eventually converges to the optimal solution.
[0049] The Cuckoo Search algorithm has been widely applied in various fields and has achieved remarkable results. For example, in the allocation of regional energy consumption quotas in China, the algorithm optimized the weight calculation, significantly improving the fairness and effectiveness of the allocation scheme, thereby effectively promoting the rational allocation of resources. In the field of precision medicine, the algorithm was applied to biomedical literature retrieval, successfully improving the accuracy and efficiency of the retrieval. Furthermore, the algorithm has been improved and demonstrated good performance in solving the Traveling Salesman Problem (TSP) and optimizing glass cutting paths.
[0050] However, the traditional cuckoo search algorithm updates the solution using a fixed step size, which easily leads to local optima when approaching local optima. To overcome this problem, this paper introduces a sinusoidal optimization strategy, which dynamically adjusts the search step size to perform a broader global search in the initial stage and accurately approximates the optimal solution in subsequent stages. The dynamic update strategy of the sinusoidal function can be expressed as formula (9): ; In equation (9), The position vector of the next generation of cuckoo individuals; This represents the current position vector of the individual cuckoo. This is the position vector of the current optimal solution; It is a random number that controls the step size, and its value range is... ; It is an angle factor, usually determined by random generation, with a value range of [value range missing]. .
[0051] To further enhance the algorithm's ability to avoid local optima, this paper introduces a stochastic strategy known as the "catfish effect." The catfish effect simulates the stimulation of population vitality by introducing new randomly generated individuals (i.e., "catfish"), thereby effectively preventing the algorithm from converging prematurely. Specifically, this strategy reactivates population vitality by periodically and randomly updating the positions of some individuals. This process can be described by equation (10): ; In equation (10), This indicates the location of the individual cuckoo after a random update. A new position generated randomly; yes Random numbers between; This indicates the current position of individual cuckoo i.
[0052] The specific steps to improve the cuckoo search algorithm are as follows: (1) Define the problem space First, define the search space of the problem, that is, the upper and lower bounds of the decision variables. Assume the problem contains i decision variables, each variable... The range of values is .
[0053] (2) Randomly initialize the population The population of the cuckoo search algorithm consists of multiple nests, each representing a potential solution (i.e., a location). Population initialization typically follows these steps: a. Determine the population size N, which is the number of nests; b. For each nest, randomly generate a solution vector within the defined search space; c. Assume the solution space is If it is dimensional, then each solution can be represented as a dimensional vector Each component All within their corresponding range Randomly generated internally.
[0054] The formula for population initialization is shown in equation (11): ; In equation (11), Uniformly distributed random numbers within the interval ensure that the initial population is used throughout the solution space. , evenly distributed within.
[0055] (3) Set algorithm parameters After generating the initial population, it is necessary to set the key parameters of the cuckoo search algorithm to ensure that the algorithm can run effectively: a. Probability of discovery : Indicates the probability that the host bird will find the cuckoo's egg, typically taking a value within the range of . This parameter determines the proportion of solutions that are replaced in the algorithm. A higher discovery probability may lead to a decrease in population diversity, while a lower discovery probability may slow down the convergence speed.
[0056] b. Maximum number of iterations: Sets the termination condition for the algorithm, usually determined by the maximum number of iterations or the convergence accuracy of the objective function value.
[0057] (4) Assess the initial population After generating the initial population, the solution corresponding to each nest needs to be evaluated. This is done through the objective function. Calculate the fitness value for each solution. The fitness value is the basis for selecting the best solution or replacing the worse solution.
[0058] In this paper, the objective function is optimized using the accuracy of similarity matching, and its formula is shown in equation (12): ; in," "Indicates the number of correctly similar matches calculated, " "" indicates the total number of similar matches calculated.
[0059] (5) Generate new solutions In each iteration, the Cuckoo Search algorithm generates new solutions through random walks and uses the Lévy flight strategy to generate new candidate solutions. Lévy flight is a stochastic process with small step size transition characteristics, which is characterized by its ability to balance local search and global search, as shown in equation (13): ; in, This indicates the generated new solution; These are random numbers generated by the Lévy distribution. It is typically used to maintain global search capabilities; is the step size factor. In this paper, to further improve the algorithm performance, the step size factor is replaced with a sinusoidal optimization strategy, thus updating equation (13) to equation (14): ; For nests selected based on discovery probability, two strategies are employed: random new solution generation and the catfish effect. Random new solutions are used to increase solution diversity, while the catfish effect generates new solutions through local perturbations, further enhancing the algorithm's ability to escape local optima.
[0060] After each iteration, the global optimal solution is updated based on the fitness values of all evaluated solutions, that is, the solution with the highest fitness value is selected from all nests as the current optimal solution.
[0061] (6) Check the termination condition and output the optimal solution. The iterative process continues until a termination condition is met. Common termination conditions include the following two cases: reaching the maximum number of iterations, or the change in the global optimum is less than a preset threshold, indicating that the algorithm has converged. Finally, the algorithm outputs the current global optimum. The overall flow of the improved cuckoo search algorithm is as follows: Figure 9 As shown.
[0062] Example: 1. Table area recognition In this study, we used the YOLOv10 model to perform an image classification task. The model was trained for 100 epochs with a batch size of 16 and input images of 640 × 640 pixels. The training dataset contained 2041 sample images, while the validation dataset contained 466 images used to evaluate the model's performance. The main task of the model was to identify six categories: engineering stamps, equipment diagrams, chart annotations, tables, text, and signatures. Specific parameter settings are shown in Table 1.
[0063] Table 1 Parameter Settings ; from Figure 10The training iteration curves shown indicate that as the number of training iterations increases, various metrics (P, R, mAP50, mAP50-95) gradually stabilize. This demonstrates that the algorithm continuously optimizes during training, gradually improving its performance. Ultimately, the model reaches its optimal performance after approximately 100 iterations, with precision (P) of 0.832, recall (R) of 0.752, mAP50 of 0.788, and mAP50-95 of 0.52. This demonstrates the model's strong object detection capability, particularly its high mAP50 and stable precision, reflecting good recognition accuracy and overall stability. However, the relatively low mAP50-95 suggests that the model's recognition efficiency may decrease at higher IOU thresholds, but the overall performance remains satisfactory and can meet the recognition needs of most scenarios.
[0064] As shown in Table 2, the performance differences among the different categories are significant. Firstly, the "Engineering Chapters" category performed best, with a precision (P) of 0.917, a recall (R) of 0.842, an mAP50 of 0.912, and an mAP50-95 of 0.593, indicating that this category is easily recognized by the model and exhibits high accuracy and stability. The "Tables" category also performed well, with a precision (P) of 0.891, a recall (R) of 0.899, an mAP50 of 0.922, and an mAP50-95 of 0.677, indicating that the model performed relatively stably in this category.
[0065] The precision and recall for the "Equipment Image" category were 0.857 and 0.855, respectively, with mAP50 at 0.901 and mAP50-95 at 0.555, indicating that the model performed well in this category, but slightly worse than the "Engineering Stamp" and "Table" categories. In the "Signature" category, the model's precision (P) was 0.876, but the recall (R) was relatively low at only 0.655, with mAP50 and mAP50-95 at 0.703 and 0.526, respectively, indicating a certain degree of insufficient recall in this category.
[0066] In contrast, the recognition performance of the "Chart Annotation" and "Text" categories was relatively weak. The recall rate for the "Chart Annotation" category was only 0.495, resulting in mAP50 and mAP50-95 scores of 0.576 and 0.328, respectively. This is likely due to the significant diversity of this category in the sample images. The "Text" category, on the other hand, had a precision (P) of 0.687, a recall (R) of 0.766, an mAP50 score of 0.716, and an mAP50-95 score of 0.442. Although its overall performance was not as good as the other categories, its recall was more balanced.
[0067] Overall, the YOLOv10 model performs well in recognizing "engineering stamps," "tables," and "equipment diagrams," but there is still room for improvement in recognizing "chart annotations" and "text." Future efforts can focus on data augmentation or model optimization to further enhance the recognition capabilities for these less accurate categories, thereby achieving better overall performance.
[0068] Table 2. Statistics of Classification Indicators ; 2. Table area recognition comparison test The experiment compared and tested Baidu's ppstructure v2, the original LORE, and the improved LORE proposed in this paper. The test involved 200 engineering document PDFs, from which table structures were identified, and the content was extracted and stored in Excel. The experimental comparison results are shown in Table 3. Table 3 Comparison results of the three algorithms ; As can be seen from the comparative data in Table 3, the proposed method achieves the best table extraction accuracy, reaching 91.61%. Of the 155 extracted tables, 142 had their structure and content accurately parsed, demonstrating extremely high accuracy. In comparison, the original LORE algorithm has an accuracy of 89.03%, slightly lower than the proposed method, but still significantly better than PP-Structure v2's 72.26%. This result indicates that the proposed method has significant advantages in table structure and content parsing, enabling more efficient and accurate extraction of table information, and showing good application potential.
[0069] like Figure 12 As shown, the algorithm proposed in this paper significantly improves the accuracy of table processing by segmenting, recognizing, and then merging tables. Compared with the traditional LORE algorithm, this method shows a clear advantage in complex table segmentation. Specifically, the LORE algorithm is prone to structural loss or content misalignment when processing multi-level and complex table structures, while the proposed method effectively preserves the integrity of the original table structure and content through splitting and accurate recognition, ensuring that the merged table is closer to the format and layout of the original data. This "segmentation-recognition-merging" strategy significantly improves the accuracy of table recognition, and is especially suitable for tables with many fields and complex structures, demonstrating excellent application value.
[0070] 3. Semantic matching of device names based on the improved Cuckoo Search algorithm The key parameter settings for the improved cuckoo search algorithm and particle swarm optimization algorithm in the optimization summation problem are shown in Table 4. In the cuckoo search algorithm, the population size is set to 25, the individual dimension is 2, the optimal summation value range is [0, 100], the discovery probability is 0.25, the number of iterations is 30, and the update step size is 0.1. In the particle swarm optimization algorithm, the population size is also 25, the individual dimension is 2, the optimal summation value range is also [0, 100], the maximum speed is set to 1.5, the inertia weight is 0.8, and both the self-cognition factor and the social cognition factor are 1.5. In addition, a total of 474 self-labeled device name pairs were used to train the semantic matching parameters.
[0071] Table 4 Parameter values of the improved algorithm ; from Figure 13 As can be seen, after 30 iterations, the improved Cuckoo Search (ICS) algorithm significantly outperforms the traditional Cuckoo Search (CS) and Particle Swarm Optimization (PSO) algorithms in optimization performance, demonstrating stronger continuous optimization capabilities. In the first 20 iterations, the fitness of ICS rapidly improves, surpassing CS and PSO, and eventually stabilizes at 0.253609, higher than CS's 0.253128 and PSO's 0.25264. This indicates that after multiple iterations, ICS can get closer to the optimal solution and achieve better fitness performance. The final optimized parameters are α = 32.3 and β = 11.8, which are then applied to semantic similarity calculation.
[0072] The overall low fitness rate is due to the failure to fully consider the characteristics of different equipment categories, instead mixing all equipment names before matching and scoring. In reality, the equipment list can be divided into different categories such as main transformer equipment, 110kV equipment, and 10kV equipment. By performing semantic similarity matching based on categories and removing the corresponding equipment names from the database after a successful match, the accuracy of matching and overall fitness performance can be significantly improved, with matching accuracy approaching 100%.
[0073] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.
[0074] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.
Claims
1. A deep learning-based intelligent image recognition and semantic understanding system, characterized in that, It includes a table object recognition module, a table structure recognition module, a text recognition and cell correspondence module, and a device name intelligent verification module; the table object recognition module, the table structure recognition module, the text recognition and cell correspondence module, and the device name intelligent verification module are connected in sequence; The table object recognition module is used to locate and identify table areas from the input engineering drawing image; The table structure recognition module is used to parse the table area output by the table object recognition module and identify the logical structure and spatial relationship of its internal cells. The text recognition and cell mapping module is used to perform text recognition on the cells output by the table structure recognition module and associate the recognized text with its corresponding cell. The intelligent device name verification module is used to perform semantic similarity calculation and consistency verification on device names from different tables output by the text recognition and cell correspondence module.
2. The image intelligent recognition and semantic understanding system based on deep learning as described in claim 1, characterized in that, The table object recognition module includes a target detection unit and an output parsing unit; the output parsing unit is connected to the target detection unit. The target detection unit is used to detect the input image using a deep learning model based on the YOLOv10 architecture and output the detection results containing the table region. The output parsing unit is used to receive the detection results, parse the bounding box coordinate information therein to determine the table position and output it to the table structure recognition module.
3. The image-based intelligent recognition and semantic understanding system according to claim 2, characterized in that, The table structure recognition module includes an image preprocessing unit, a structure parsing unit, and a large table processing unit; the image preprocessing unit, the structure parsing unit, and the large table processing unit are connected in sequence. The image preprocessing unit is used to perform grayscale and binarization processing on the table region image output by the table object recognition module to prepare for structural analysis. The structure parsing unit is used to simultaneously predict the spatial coordinates of each cell and its logical row and column position in the table using an improved LORE algorithm model. The large table processing unit is used to control the structure parsing unit to perform segmentation recognition and result merging operations on the table when the size of the table area image exceeds a preset threshold.
4. The image intelligent recognition and semantic understanding system based on deep learning as described in claim 3, characterized in that, The text recognition and cell correspondence module includes an OCR text recognition unit, a coordinate sorting and matching unit, and a cross-cell text processing unit; The OCR text recognition unit uses the RapidOCR engine to recognize the text content in the image and outputs the bounding box coordinates of each text region. The coordinate sorting and matching unit is used to sort the cell coordinates output by the table structure recognition module to determine the row and column structure of the table, and calculate the positional relationship between the center point coordinates of the text area and the cell bounding box, so as to accurately associate the text content with the corresponding cell. The cross-cell text processing unit is used to detect text regions whose bounding boxes span multiple cells, merge the relevant cell regions, and then re-identify them.
5. The image intelligent recognition and semantic understanding system based on deep learning as described in claim 4, characterized in that, The intelligent device name verification module includes a semantic vectorization unit, a similarity calculation unit, and an optimization unit; the semantic vectorization unit, the similarity calculation unit, and the optimization unit are connected in sequence. The semantic vectorization unit is used to convert the text-based device name into a high-dimensional semantic vector representation by using a BERT pre-trained model and combining the first and last layer average pooling strategy. The similarity calculation unit is used to calculate the comprehensive similarity between two device names. The comprehensive similarity is a weighted sum of semantic similarity based on BERT and traditional cosine similarity based on word segmentation. The optimization unit is used to optimize the weight adjustment factor in the weighted sum using an improved cuckoo search algorithm, and provides the optimized factor to the similarity calculation unit.
6. The image-based intelligent recognition and semantic understanding system according to claim 5, characterized in that, The improved cuckoo search algorithm introduces a dynamic step size adjustment strategy based on a sine function and a random position update strategy that simulates the catfish effect.
7. The image-based intelligent recognition and semantic understanding system according to claim 6, characterized in that, The improved LORE algorithm model integrates the segmentation, recognition, and merging processes of the large table processing unit into the original LORE framework.
8. The image intelligent recognition and semantic understanding system based on deep learning as described in claim 7, characterized in that, The coordinate sorting and matching unit uses a judgment condition that includes configurable pixel tolerance values when determining whether text belongs to a cell.