Drawing text line matching method and system based on U-Net double-layer fusion model

Abstract

The invention discloses a drawing line matching method and system based on a U-Net double-layer fusion model, and relates to the technical field of intelligent image processing, and the method comprises the steps: detecting and marking the hotlink connection between a lead or a component in an electrical secondary drawing and a character label through a U-Net model; based on a thermodynamic diagram response algorithm, matching the character labels with the wires or the components through the middle bridge; lines in the drawing are vectorized through an image processing technology, and the connection relation of the lines in the drawing is extracted and analyzed. According to the drawing text line matching method based on the U-Net double-layer fusion model, the double-layer model is constructed and combined with the U-Net network, so that the risks of false detection and missing detection are reduced, the drawing understanding efficiency is improved, the complexity and errors of manual operation are reduced by automatically identifying and processing the drawing information, and the user experience is improved. The storage and management pressure of the transformer substation on a large amount of drawing data is relieved, the management efficiency of the drawing data is improved, and better effects are achieved in the aspects of recognition precision, understanding efficiency and management efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent image processing, and in particular to a drawing text and line matching method based on a U-Net double-layer fusion model. Background Art

[0002] In recent years, the construction of substations has been accelerating, and new substations and renovation and expansion projects have been put into operation on a large scale. As the source of the substation engineering database, electrical secondary drawings are the data basis for power grid construction, operation and management. In field applications, they play an important role in the development of inspection, operation and maintenance, etc. The substation secondary system contains a large number of electrical equipment and complex logical control relationships. The drawings involved are of various types and large in number, which brings about the problem of difficult drawing management. At present, due to the lack of supporting tools, the management of substation secondary engineering drawings still relies on a large amount of manual processing, with high labor costs and lack of intelligent drawing recognition methods. The existing technology lacks a complete set of methods for identifying characters and icons in drawings and matching text lines, so there is no way to manage the drawings. How to achieve semantic understanding of electrical secondary drawings, improve the management efficiency and quality of substation secondary engineering drawings, and realize the standardization and intelligence of substation secondary engineering drawings generation has become an important issue in the current smart grid construction. At present, there are mature technologies for detecting and recognizing text in specific scenarios, and traditional or deep learning methods can be used to extract components, lines and other elements in drawings. However, there is currently a lack of technology for text labels belonging to lines in drawings, and it is impossible to achieve professional intelligent recognition of complete sets of electrical secondary drawings. For professional electrical engineers, they can intuitively understand the relationship between lines and text based on their experience, but intelligent matching of text and lines in drawings is still an engineering problem. Summary of the invention

[0003] In view of the above-mentioned problems, the present invention is proposed.

[0004] Therefore, the technical problems solved by the present invention are: the existing cloud service platform calculation method has high labor costs, lacks intelligent drawing recognition, and how to improve the management efficiency and quality of substation secondary engineering drawings.

[0005] To solve the above technical problems, the present invention provides the following technical solutions: a drawing text-line matching method based on a U-Net double-layer fusion model, comprising detecting and marking the hotlink connection between the wires or components and the text labels in the electrical secondary drawings through the U-Net model; matching the text labels with the wires or components through an intermediate bridge based on a heat map response algorithm; vectorizing the lines in the drawings through image processing technology, and extracting and analyzing the connection relationship of the lines in the drawings.

[0006] As a preferred solution of the drawing text line matching method based on the U-Net double-layer fusion model described in the present invention, wherein: the detection and marking through the U-Net model includes in the encoding stage, the convolution layer and the pooling layer work together to reduce the spatial resolution of the feature map, and abstract the high-order feature representation of the image at the same time. Each convolution layer captures the local and global features in the image through a combination of convolution operations, batch normalization and activation functions, and introduces a jump connection mechanism. At each level of the encoder, the feature map is fused with the feature map of the middle level of the decoder. The decoder part is composed of a series of convolution layers and upsampling layers, and the feature map output by the encoder is restored to the same size as the original image. At the last layer of the U-Net architecture, a convolution layer is used to convert the fused feature map into an output channel number equal to the number of segmentation target categories. Through the activation function, the feature map is converted into a probability distribution that each pixel belongs to a different category. According to the maximum probability principle, each pixel is assigned to its respective category to generate a pixel-level segmentation result.

[0007] As a preferred solution of the drawing text line matching method based on the U-Net double-layer fusion model described in the present invention, wherein: the detection and marking by the U-Net model also includes calculating the network loss by combining the pixel-level Softmax with the cross entropy loss function, and the weighted cross entropy loss is used to evaluate the difference between the predicted image and the real image label. Each pixel is assigned a different weight coefficient, and the coefficient reflects the importance of each pixel. For each pixel, the unnormalized prediction value is calculated, and the unnormalized value is converted into a probability distribution by the Softmax function. The probability value of each pixel reflects the possibility of the category to which it belongs. The cross entropy loss is calculated by the real category label and the predicted probability value, and the whole image is integrated and calculated, and the second-order derivative of the predicted probability is introduced. By calculating the second-order derivative of the predicted probability and summing them, the smoothness of the predicted distribution is evaluated. The smaller the second-order derivative, the smoother the prediction result. The contribution of the second-order derivative to the loss is adjusted by the regularization parameter. The weighted cross entropy loss and the spatial regularization loss are combined to construct a loss evaluation framework expressed as:

[0008]

[0009] in, is the weighted cross entropy loss, is the spatial regularization loss, α i is the weight coefficient of the i-th pixel, y i is the true category label of the i-th pixel, z i is the unnormalized predicted value of the ith pixel, is the prediction probability of a pixel, β is the smoothing parameter, Ω is the domain of the image, γ is the regularization parameter, x k is the spatial coordinate of the image.

[0010] As a preferred solution of the drawing text and line matching method based on the U-Net double-layer fusion model described in the present invention, wherein: the detection and marking through the U-Net model also includes extracting the connection between the wire or component and the text label based on the U-Net hotlink model, converting the original electrical secondary drawing data into an image file, the drawing is a three-channel image with a white background area and black lines or component primitives, and the distance between the wire or component and the text label is 5-8 pixels. Before training, the grayscale value of the image with an original bit depth of 8 and a number of channels of 3 is reduced to one-half to obtain an initial drawing image. Based on the initial drawing image, a red straight line with a width of 5 pixels is marked between the wire or component and the text label to obtain a preliminary red marking image, and the pixels at the red mark are retained by screening the red channel grayscale value to obtain a single-channel image with the marked part being white and the remaining part being black.

[0011] As a preferred solution of the drawing text and line matching method based on the U-Net double-layer fusion model described in the present invention, wherein: the matching of text labels with wires or components through intermediate bridges includes outputting marks between wires or components and text labels based on the upper-layer hotkink model, outputting a binary mask image with a grayscale value of 255 for the area covered by the mark and a background area of ​​0, wherein each area covered by a mark in the binary mask image is an intermediate bridge, and Gaussian blur processing and threshold processing are performed on the binary mask image, and the intermediate bridges are converted into heat map areas, and each intermediate bridge is projected with a text label and a wire or component, presenting a one-to-one relationship, and when there are multiple text labels on a wire connected to the optical cable outside the terminal block, the same wire line is projected to multiple bridges.

[0012] As a preferred solution of the drawing text and line matching method based on the U-Net double-layer fusion model described in the present invention, wherein: the matching of the text label with the wire or component through the intermediate bridge also includes calculating the arrival cost based on pixel summation, the text label is a rectangular area, the label rectangular area is marked and cropped with the heat map area, and the pixels are summed after cropping. If there is no intermediate bridge in the area, the heat map of the area is black, indicating that the text label and the intermediate bridge are unreachable. The larger the pixel sum value, the greater the relationship between the text label and the intermediate bridge. The pixel matrix is ​​expressed as:

[0013] C=A+B

[0014] c ij =a ij +b ij

[0015] Among them, A is the label image pixel matrix, B is the heat map pixel matrix, C is the result matrix after adding A and B, c ijFor each element of the matrix C, the wire or component obtains the intermediate bridge through calculation. If it is projected on the same intermediate bridge as the text label, it is determined that the wire or component corresponds to the text label. According to the characteristics of the drawing, the matching is guided by manual marking in the upper model, and the matching problem is converted into a data-controllable problem. The deep learning model is combined to match text, components and lines.

[0016] As a preferred solution of the drawing text line matching method based on the U-Net double-layer fusion model described in the present invention, wherein: the extraction and analysis of the connection relationship of the lines in the drawing includes extracting the connection relationship of the drawing by vectorizing the line skeleton, converting the detected lines into vector form, extracting the relationship between the lines by analyzing the vectorized lines, binarizing the lines in the drawing, the grayscale value is 0 or 1, and the lines are refined. After the refinement, the image is processed by a convolution operation. If the result of the convolution operation is 1, it is the end point of the line. If the result of the convolution operation is greater than 3, it is the intersection of the line. If the result of the convolution operation is 2, it is the pixel point in the middle part of the line segment. After extracting the end point and intersection of the line, find the neighboring pixel points in the middle part of the connected line segment based on an end point on the line segment, and repeat the operation until the next end point or intersection is found. The extraction of the starting point and connection relationship of the line segment is completed, which is expressed as:

[0017]

[0018] Among them, N i is the gray value of the i-th neighbor pixel, IEP is used to determine whether the current pixel is an endpoint, ICP is used to determine whether the current pixel is an intersection, and IMP is used to determine whether the current pixel is an intermediate point.

[0019] Another object of the present invention is to provide a drawing text and line matching system based on the U-Net double-layer fusion model, which can extract and analyze the connection relationship between lines through an extraction and analysis module, solving the current problem that it is difficult to automatically extract and analyze the line relationship in electrical secondary drawings.

[0020] As a preferred solution of the drawing text and line matching system based on the U-Net double-layer fusion model described in the present invention, it includes: a detection and marking module, a judgment and matching module, and an extraction and analysis module; the detection and marking module is used to detect and mark the hotlink connection between the wires or components and the text labels in the electrical secondary drawings through the U-Net model; the judgment and matching module is used to match the text labels with the wires or components through the intermediate bridge based on the heat map response algorithm; the extraction and analysis module is used to vectorize the lines in the drawings through image processing technology, and extract and analyze the connection relationship of the lines in the drawings.

[0021] A computer device includes a memory and a processor, wherein the memory stores a computer program, and is characterized in that the processor executes the computer program to implement a step of a drawing text line matching method based on a U-Net double-layer fusion model.

[0022] A computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of a drawing text line matching method based on a U-Net double-layer fusion model are implemented.

[0023] Beneficial effects of the present invention: The drawing text and line matching method based on the U-Net double-layer fusion model provided by the present invention improves the recognition accuracy of key information in the secondary construction drawings of the substation by constructing a double-layer model and combining the U-Net network, reduces the risk of false detection and missed detection, introduces a line vectorization method, effectively extracts the line information in the drawing, solves the problem of standardized storage of line information, facilitates the storage and management of the connection relationship of electrical equipment, further enhances the intelligence of the drawing information, improves the efficiency of drawing understanding, and provides strong support for the intelligence of substation work. By automatically identifying and processing drawing information, the tediousness and errors of manual operation are reduced, the storage and management pressure of the substation for a large amount of drawing materials is reduced, and the management efficiency of the drawing materials is improved. The present invention achieves better results in recognition accuracy, understanding efficiency and management efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0025] Figure 1 An overall flow chart of a drawing text and line matching method based on a U-Net double-layer fusion model provided for the first embodiment of the present invention.

[0026] Figure 2 A schematic diagram of endpoints on lines of a drawing text line matching method based on a U-Net two-layer fusion model provided in the first embodiment of the present invention.

[0027] Figure 3 A schematic diagram of points in the middle of lines of a drawing text line matching method based on a U-Net double-layer fusion model provided in the first embodiment of the present invention.

[0028] Figure 4 A schematic diagram of line intersections of a drawing text line matching method based on a U-Net double-layer fusion model provided in the first embodiment of the present invention.

[0029] Figure 5 An original image of the segmentation effect of a drawing text and line matching method based on a U-Net double-layer fusion model provided in the second embodiment of the present invention.

[0030] Figure 6 A terminal strip diagram segmentation effect diagram of a drawing text and line matching method based on a U-Net double-layer fusion model provided in the second embodiment of the present invention.

[0031] Figure 7 A U-Net segmentation effect diagram of a drawing text and line matching method based on a U-Net double-layer fusion model provided in the second embodiment of the present invention.

[0032] Figure 8 A Refinnet segmentation effect diagram of a drawing text and line matching method based on a U-Net double-layer fusion model provided in the second embodiment of the present invention.

[0033] Fig. 9 A PSPNet segmentation effect diagram of a drawing text and line matching method based on a U-Net double-layer fusion model provided in the second embodiment of the present invention.

[0034] Fig.10 An FCN segmentation effect diagram of a drawing text line matching method based on a U-Net double-layer fusion model provided in the second embodiment of the present invention.

[0035] Fig.11 An overall module diagram of a drawing text and line matching system based on a U-Net two-layer fusion model provided for the third embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0037] Example 1

[0038] Reference Figure 1-Figure 4 , as an embodiment of the present invention, provides a drawing text line matching method based on a U-Net double-layer fusion model, comprising:

[0039] S1: Detect and mark the hotlink connections between wires or components and text labels in electrical secondary drawings through the U-Net model.

[0040] Furthermore, detection and labeling by U-Net model are included in the encoding stage.

[0041] It should be noted that the convolution layer and the pooling layer work together to reduce the spatial resolution of the feature map and abstract the high-order feature representation of the image. Each convolution layer captures the local and global features in the image through a combination of convolution operations, batch normalization and activation functions. A skip connection mechanism is introduced. At each level of the encoder, the feature map is fused with the feature map of the decoder. The decoder part consists of a series of convolution layers and upsampling layers, which restore the feature map output by the encoder to the same size as the original image. In the last layer of the U-Net architecture, a convolution layer is used to convert the fused feature map into an output channel number equal to the number of segmentation target categories. Through the activation function, the feature map is converted into a probability distribution that each pixel belongs to a different category. According to the maximum probability principle, each pixel is assigned to its respective category to generate a pixel-level segmentation result.

[0042] It should also be noted that the ability to autonomously learn and output masks directly from input label images effectively avoids the tedious process of manually designing feature extractors, making the end-to-end learning process more concise and efficient, and significantly improving the effectiveness of engineering applications. At the same time, the flexibility of the U-Net model structure enables it to easily cope with complex and changeable data distributions and has strong adaptability. The U-Net structure can capture rich contextual information at different levels. This feature is particularly critical for accurately extracting pixel-level information such as wires, components and their corresponding text labels in electrical secondary drawings.

[0043] Furthermore, detection and labeling by the U-Net model also includes calculating the network loss by combining pixel-level Softmax with a cross-entropy loss function.

[0044] It should be noted that the weighted cross entropy loss is used to evaluate the difference between the predicted image and the real image label. Each pixel is assigned a different weight coefficient, which reflects the importance of each pixel. For each pixel, the unnormalized predicted value is calculated and converted into a probability distribution through the Softmax function. The probability value of each pixel reflects the possibility of the category to which it belongs. The cross entropy loss is calculated by the real category label and the predicted probability value. The whole image is integrated and the second-order derivative of the predicted probability is introduced. The smoothness of the predicted distribution is evaluated by calculating the second-order derivative of the predicted probability and summing them. The smaller the second-order derivative, the smoother the prediction result. The contribution of the second-order derivative to the loss is adjusted by the regularization parameter. The weighted cross entropy loss and the spatial regularization loss are combined to construct a loss evaluation framework expressed as:

[0045]

[0046] in, is the weighted cross entropy loss, is the spatial regularization loss, α i is the weight coefficient of the i-th pixel, y i is the true category label of the i-th pixel, z i is the unnormalized predicted value of the ith pixel, is the prediction probability of a pixel, β is the smoothing parameter, Ω is the domain of the image, γ is the regularization parameter, x k is the spatial coordinate of the image.

[0047] It should also be noted that in the context of the big data era, deep learning models can usually demonstrate excellent performance on large-scale data sets. When there is sufficient and accurately labeled electrical secondary drawing data, the U-Net model can fully learn the complex patterns of text in the drawings. Compared with traditional algorithms, it is more adaptable to diverse application scenarios and provides strong technical support for the accurate recognition of electrical secondary drawings.

[0048] Furthermore, detecting and labeling through the U-Net model also includes extracting the connection between the wire or component and the text label based on the U-Net hotlink model.

[0049] It should be noted that the original electrical secondary drawing data is converted into an image file. The drawing is a three-channel image with a white background area and black lines or component graphics. The distance between the wire or component and the text label is 5-8 pixels. Before training, the grayscale value of the original image with a bit depth of 8 and a number of channels of 3 is reduced to one-half to obtain the initial drawing image. Based on the initial drawing image, a red straight line with a width of 5 pixels is marked between the wire or component and the text label to obtain a preliminary red marking map. The pixels at the red mark are retained by screening the red channel grayscale value to obtain a single-channel image with the marked part being white and the remaining part being black.

[0050] It should also be noted that most electrical secondary drawings are large-scale images with a pixel value of 4000×3000. For large images, if only global features are focused on, the model will ignore local details. By slicing the original drawings into slices with a pixel size of 768×768, the model can focus on both global and local features at the same time, thereby reducing the risk of overfitting. This study sets the overlap of the slices to 30% of the side length. Each slice contains its own local information, and through overlapping slices, the model can capture the associations and contextual relationships between these local information, and learn the features of the image from multiple angles and scales. This multi-scale learning can improve the generalization ability of the model and make it better adapt to targets of different sizes and shapes.

[0051] S2: Based on the heat map response algorithm, the text label is matched with the wire or component through the intermediate bridge.

[0052] Furthermore, matching the text label with the wire or component through the intermediate bridge includes outputting a mark between the wire or component and the text label based on an upper-level hotkink model.

[0053] It should be noted that the output mark covers a binary mask image with a grayscale value of 255 for the area covered and 0 for the background area. The area covered by each mark in the binary mask image is an intermediate bridge. Gaussian blur and threshold processing are performed on the binary mask image, and the intermediate bridges are transformed into heat map areas. A text label and a wire or component are projected on each intermediate bridge, showing a one-to-one relationship. When there are multiple text labels on a wire connecting the optical cable outside the terminal block, the same wire line is projected onto multiple bridges.

[0054] It should also be noted that, since it is difficult to define the distance between elements in a drawing, the present invention uses a heat map distance instead of a spatial distance, and introduces an intermediate layer between text labels and wires or components, turning the problem into a data-controllable problem.

[0055] It should also be noted that, based on the mapping rules, the same text label can only serve one component target or wire, so the method of the present invention needs to restrict the heat map to which the text label belongs.

[0056] Furthermore, matching the text label with the wire or component through the intermediate bridge also includes calculating the arrival cost based on pixel summation.

[0057] It should be noted that the text label is a rectangular area. The label rectangular area is marked and cropped with the heat map area. After cropping, the pixels are summed. If there is no intermediate bridge in the area, the heat map of the area is black, indicating that the text label and the intermediate bridge are unreachable. The larger the pixel sum value, the greater the relationship between the text label and the intermediate bridge. The pixel matrix is ​​expressed as:

[0058] C=A+B

[0059] c ij =a ij +b ij

[0060] Among them, A is the label image pixel matrix, B is the heat map pixel matrix, C is the result matrix after adding A and B, c ijFor each element of the matrix C, the wire or component obtains the intermediate bridge through calculation. If it is projected on the same intermediate bridge as the text label, it is determined that the wire or component corresponds to the text label. According to the characteristics of the drawing, the matching is guided by manual marking in the upper model, and the matching problem is converted into a data-controllable problem. The deep learning model is combined to match text, components and lines.

[0061] It should also be noted that, since there are situations where different heat map regions are close to each other and the heat map regions have a certain area size, if a text label is projected onto multiple heat map regions at the same time, the one with the smallest cost is selected.

[0062] S3: Vectorize the lines in the drawing through image processing technology, and extract and analyze the connection relationship of the lines in the drawing.

[0063] Furthermore, extracting and analyzing the connection relationship of the lines in the drawing includes extracting the connection relationship of the drawing by vectorizing the line skeleton.

[0064] It should be noted that the detected lines are converted into vector form, and the relationship between the lines is extracted by analyzing the vectorized lines. The lines in the drawing are binarized, and the grayscale value is 0 or 1. The lines are thinned, and the image is processed by convolution operation after thinning. If the result of the convolution operation is 1, it is the end point of the line. If the result of the convolution operation is greater than 3, it is the intersection of the line. If the result of the convolution operation is 2, it is the pixel point in the middle part of the line segment. After extracting the end point and intersection of the line, the neighbor pixel point in the middle part of the connected line segment is found based on an end point on the line segment. The operation is repeated until the next end point or intersection is found. The extraction of the starting point and connection relationship of the line segment is completed, which is expressed as:

[0065]

[0066] Among them, N i is the gray value of the i-th neighbor pixel, IEP is used to determine whether the current pixel is an endpoint, ICP is used to determine whether the current pixel is an intersection, and IMP is used to determine whether the current pixel is an intermediate point.

[0067] It should also be noted that Figure 2-Figure 4 The grayscale values ​​of the endpoint, middle point, intersection point and eight-neighborhood of the line are shown respectively. The end point of the line has only one neighbor pixel with a grayscale value in the eight-neighborhood, the pixel point in the line has two neighbor pixels with grayscale values ​​in the eight-neighborhood, and the intersection point of the line has at least 3 pixel points with grayscale values ​​in the eight-neighborhood.

[0068] Example 2

[0069] Reference Figure 5-10, which is an embodiment of the present invention, provides a drawing text line matching method based on the U-Net double-layer fusion model. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0070] In order to evaluate the effect of U-Net model in segmenting images, the U-Net segmentation model was compared with FCN, RefineNet, and PSPNet.

[0071] As shown in Table 1, in the image segmentation task, the pixel accuracy of the U-Net model reached 0.9982, showing a significant advantage. The reason for its best effect is mainly attributed to its unique network architecture, which allows the model to capture global context information while retaining local detail information. Through the encoder-decoder structure, U-Net can effectively extract multi-scale features of the image and fuse these features through jump connections, thereby improving the model's pixel-level classification accuracy. In addition, the simplicity and efficiency of U-Net also make it perform well in practical applications. In contrast, although other network models perform well in some indicators, they often have limitations. For example, although FCN has a high pixel accuracy, it is weak in capturing context information, resulting in low recall rate and Dice coefficient. Although RefineNet and PSPNet have improved in capturing context information, they are still inferior to U-Net in some detail processing. Therefore, considering all evaluation indicators, the U-Net model shows the best effect in the image segmentation task.

[0072] Visually analyze the segmentation effect of the hotlink model and verify the detection effect of the model by comparing the location of the red heat map area output by the algorithm, such as Figure 5 As shown in Figure 1, the U-Net algorithm has the best segmentation effect for the longer junctions between lines and text and the shorter junctions between components and text. Figure 6 As shown in Figure 2, the RefineNet algorithm can only detect the shorter boundaries between components and text, but the segmentation effect between lines and text is poor, such as Figure 7 and Figure 8As shown in the figure, the segmentation and positioning effects of PSPNet and FCN are similar, but FCN has better segmentation effect than PSPNet at the longer junction between text and lines. This phenomenon shows that although PSPNet captures contextual information of different scales through the pyramid pooling module, this multi-scale information fusion may not be as direct as FCN when processing long areas. FCN can balance local and global contextual information to a certain extent through continuous convolution and deconvolution operations, so as to perform better in long area segmentation. The network structure of FCN is relatively simple and does not introduce overly complex modules, which enables it to maintain high computational efficiency and stability when processing long areas. More complex network structures such as PSPNet may face more challenges when processing long areas, but FCN focuses more on recovering the category of each pixel from abstract features compared to U-net, and has weaker ability to recover image details. In general, U-Net introduces skip connections to fuse the feature maps of different levels of the encoder part with the corresponding levels of the decoder part. This design enables U-Net to make full use of feature information at different levels, improve segmentation accuracy and detail recovery ability, and have stronger robustness.

[0073] Table 1 Comparison of different model indicators

[0074] Model Pixel Accuracy Recall Accuracy Intersection and Union Dice coefficient FCN 0.9976 0.8373 0.7963 0.8532 0.7326 RefineNet 0.9954 0.8356 0.8037 0.8908 0.7756 PSPNet 0.9903 0.8398 0.8156 0.8793 0.7524 U-Net 0.9982 0.8834 0.8139 0.8874 0.7895

[0075] Example 3

[0076] Reference Fig.11 , as an embodiment of the present invention, provides a drawing text line matching system based on the U-Net double-layer fusion model, including: a detection and marking module, a matching determination module, and an extraction and analysis module.

[0077] The detection and marking module is used to detect and mark the hotlink connection between wires or components and text labels in electrical secondary drawings through the U-Net model; the judgment and matching module is used to match text labels with wires or components through an intermediate bridge based on the thermal map response algorithm; the extraction and analysis module is used to vectorize the lines in the drawings through image processing technology, and extract and analyze the connection relationship of the lines in the drawings.

[0078] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0079] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0080] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0081] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc. It should be noted that the above embodiments are only used to illustrate the technical solution of the present invention and are not limited. Although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

[0082] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A drawing text line matching method based on U-Net double-layer fusion model, characterized in that: include: Detect and mark the hotlink connection between wires or components and text labels in electrical secondary drawings through the U-Net model; Based on the heat map response algorithm, the text label is matched with the wire or component through the intermediate bridge; The lines in the drawing are vectorized through image processing technology, and the connection relationship of the lines in the drawing is extracted and analyzed.

2. The drawing text and line matching method based on the U-Net double-layer fusion model according to claim 1, characterized in that: The detection and labeling by the U-Net model includes that in the encoding stage, the convolution layer and the pooling layer work together to reduce the spatial resolution of the feature map and abstract the high-order feature representation of the image. Each convolution layer captures the local and global features in the image through a combination of convolution operations, batch normalization and activation functions, and introduces a jump connection mechanism. At each level of the encoder, the feature map is fused with the feature map of the decoder. The decoder part is composed of a series of convolution layers and upsampling layers. The feature map output by the encoder is restored to the same size as the original image. At the last layer of the U-Net architecture, a convolution layer is used to convert the fused feature map into an output channel number equal to the number of segmentation target categories. Through the activation function, the feature map is converted into a probability distribution that each pixel belongs to a different category. According to the maximum probability principle, each pixel is assigned to its respective category to generate a pixel-level segmentation result.

3. The drawing text and line matching method based on the U-Net double-layer fusion model as claimed in claim 2, characterized in that: The detection and labeling by the U-Net model also includes calculating the network loss by combining the pixel-level Softmax with the cross entropy loss function. The weighted cross entropy loss is used to evaluate the difference between the predicted image and the real image label. Each pixel is assigned a different weight coefficient, and the coefficient reflects the importance of each pixel. For each pixel, the unnormalized prediction value is calculated, and the unnormalized value is converted into a probability distribution by the Softmax function. The probability value of each pixel reflects the possibility of the category to which it belongs. The cross entropy loss is calculated by the real category label and the predicted probability value, and the whole image is integrated. The second-order derivative of the predicted probability is introduced, and the smoothness of the predicted distribution is evaluated by calculating the second-order derivative of the predicted probability and summing them. The smaller the second-order derivative, the smoother the prediction result. The contribution of the second-order derivative to the loss is adjusted by the regularization parameter. The weighted cross entropy loss and the spatial regularization loss are combined to construct a loss evaluation framework expressed as: in, is the weighted cross entropy loss, is the spatial regularization loss, α i is the weight coefficient of the i-th pixel, y i is the true category label of the i-th pixel, z i is the unnormalized predicted value of the ith pixel, is the prediction probability of a pixel, β is the smoothing parameter, Ω is the domain of the image, γ is the regularization parameter, x k is the spatial coordinate of the image.

4. The drawing text and line matching method based on the U-Net double-layer fusion model as claimed in claim 3, characterized in that: The detection and marking by the U-Net model also includes extracting the connection between the wire or component and the text label based on the U-Net hotlink model, converting the original electrical secondary drawing data into an image file, the drawing is a three-channel image with a white background area and black lines or component graphics, and the distance between the wire or component and the text label is 5-8 pixels. Before training, the grayscale value of the image with an original bit depth of 8 and a number of channels of 3 is reduced to one-half to obtain an initial drawing image. Based on the initial drawing image, a red straight line with a width of 5 pixels is marked between the wire or component and the text label to obtain a preliminary red marking image, and the pixels at the red mark are retained by screening the red channel grayscale value to obtain a single-channel image with the marked part being white and the remaining part being black.

5. The drawing text and line matching method based on the U-Net double-layer fusion model as claimed in claim 4, characterized in that: The method of matching the text label with the wire or component through the intermediate bridge includes outputting the mark between the wire or component and the text label based on the upper-layer hotkink model, outputting a binary mask image in which the grayscale value of the area covered by the mark is 255 and the background area is 0, and the area covered by each mark in the binary mask image is the intermediate bridge. Gaussian blur processing and threshold processing are performed on the binary mask image, and the intermediate bridge is converted into a heat map area. A text label and a wire or component are projected on each intermediate bridge, presenting a one-to-one relationship. When there are multiple text labels on a wire connected to the optical cable outside the terminal block, the same wire line is projected to multiple bridges.

6. The drawing text and line matching method based on the U-Net double-layer fusion model as claimed in claim 5, characterized in that: The matching of the text label with the wire or component through the intermediate bridge also includes calculating the arrival cost based on the pixel summation. The text label is a rectangular area. The label rectangular area is aligned with the cropped heat map area. After cropping, the pixels are summed. If there is no intermediate bridge in the area, the heat map of the area is black, indicating that the text label and the intermediate bridge are unreachable. The larger the pixel sum value, the greater the relationship between the text label and the intermediate bridge. The pixel matrix is ​​expressed as: C=A+B c ij =a ij +b ij Among them, A is the label image pixel matrix, B is the heat map pixel matrix, C is the result matrix after adding A and B, c ij For each element of the matrix C, the wire or component obtains the intermediate bridge through calculation. If it is projected on the same intermediate bridge as the text label, it is determined that the wire or component corresponds to the text label. According to the characteristics of the drawing, the matching is guided by manual marking in the upper model, and the matching problem is converted into a data-controllable problem. The deep learning model is combined to match text, components and lines.

7. The drawing text and line matching method based on the U-Net double-layer fusion model according to claim 6, characterized in that: The extraction and analysis of the line connection relationship of the drawing includes extracting the line connection relationship of the drawing by line skeleton vectorization, converting the detected lines into vector form, extracting the relationship between the lines by analyzing the vectorized lines, binarizing the lines in the drawing, and the grayscale value is 0 or 1, and the lines are thinned. After the thinning process, the image is processed by a convolution operation. If the result of the convolution operation is 1, it is the end point of the line. If the result of the convolution operation is greater than 3, it is the intersection of the line. If the convolution operation result is 2, it is the pixel point in the middle part of the line segment. After extracting the end point and intersection of the line, find the neighbor pixel point in the middle part of the connected line segment based on an end point on the line segment, and repeat the operation until the next end point or intersection is found. The extraction of the starting point and the line connection relationship of the line segment is completed, which is expressed as: Among them, N i is the gray value of the i-th neighbor pixel, IEP is used to determine whether the current pixel is an endpoint, ICP is used to determine whether the current pixel is an intersection, and IMP is used to determine whether the current pixel is an intermediate point.

8. A system using the drawing text and line matching method based on the U-Net double-layer fusion model as claimed in any one of claims 1 to 7, characterized in that: It includes a detection and marking module, a determination and matching module, and an extraction and analysis module; The detection and marking module is used to detect and mark the hotlink connection between the wire or component and the text label in the electrical secondary drawing through the U-Net model; The determination and matching module is used to match the text label with the wire or component through the intermediate bridge based on the thermal map response algorithm; The extraction and analysis module is used to vectorize the lines in the drawing through image processing technology, and to extract and analyze the connection relationship of the lines in the drawing.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the drawing text and line matching method based on the U-Net double-layer fusion model described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the drawing text and line matching method based on the U-Net double-layer fusion model described in any one of claims 1 to 7 are implemented.

Images