A cad terminal map and sleeve map image character recognition and matching method and system

Abstract

The application provides a CAD terminal diagram and sleeve image character recognition and matching method and system, comprising: inputting the pre-processed standardized CAD terminal diagram and the enhanced sleeve image into a shared feature extractor of a character detection and recognition model to extract low-level shared features, then mapping the features of different domains to the same vector space through migration learning in a domain self-adaptive module, updating the shared feature extractor through back propagation to extract high-level shared features, and at the same time, judging the feature classification of the two types of images through real domain labels in the domain self-adaptive module, and then inputting the images into a three-channel backbone network to obtain the character recognition output of the terminal diagram and the sleeve after feature extraction and decoding; synchronously detecting the corner of the standardized CAD terminal diagram, and reconstructing the context logical relationship with the character recognition output of the terminal diagram, and then matching the formatted sleeve character recognition output obtained by arrangement to output a matching result. The application can recognize and match the characters of the CAD terminal diagram and the sleeve image.

Description

Technical Field

[0001] This invention relates to the field of terminal positioning detection and sleeve matching technology, and in particular to a method and system for character recognition and matching of CAD terminal diagrams and sleeve images. Background Technology

[0002] In electrical systems, precise matching of terminals is a core guarantee for ensuring the safe operation of power equipment. Facing the technical challenges of terminal positioning and bushing matching in substation cabinets, using deep learning technology to perform content recognition and matching between electrical drawings and physical images has become an effective solution.

[0003] However, existing character detection and recognition models suffer from low directional accuracy in handling small target characters in high-resolution CAD terminal diagrams and tilted characters in sleeve images. This makes them unable to handle multiple tasks with different data architectures, resulting in low accuracy in recognizing tilted characters in sleeve images and small characters in high-resolution CAD terminal diagrams. Furthermore, high-resolution CAD terminal diagrams contain highly complex multi-level nested cable tray structures or non-standard corner designs. Conventional image processing methods struggle to accurately analyze the actual physical path of the cable and its intricate topological connections with terminal numbers, hindering the reconstruction of the internal logical connections of the device. In addition, the significant differences in data type standards between high-resolution CAD terminal diagrams and sleeve images make it difficult for models trained on high-resolution CAD terminal diagrams to be directly adapted to sleeve images, resulting in a significant bottleneck in model generalization ability. Summary of the Invention

[0004] To address the technical problems existing in the prior art, the present invention provides a method and system for character recognition and matching between CAD terminal diagrams and sleeve images, the technical solution of which is as follows:

[0005] On the one hand, a method for character recognition and matching between CAD terminal diagrams and sleeve images is provided, the method comprising:

[0006] S1. Collect images of bushings inside the substation cabinet, label the collected bushing images and CAD terminal diagrams respectively, and divide them into training set and validation set;

[0007] S2. Perform data preprocessing on the CAD terminal diagrams and sleeve images in the training set to obtain standardized CAD terminal diagrams and enhanced sleeve images;

[0008] S3. Classify the standardized CAD terminal diagram and the enhanced sleeve image into CAD domain and sleeve domain. Each time, obtain the same number of images from their respective type domains and input them into the character detection and recognition model. The character detection and recognition model includes a shared feature extractor, a domain adaptive module, and a three-channel backbone network.

[0009] The shared feature extractor first extracts low-level shared features. The domain adaptation module maps features from different domains to the same vector space through transfer learning and updates the parameters of the shared feature extractor through backpropagation, enabling the shared feature extractor to extract high-level shared features. Meanwhile, in the domain adaptation module, feature classification is performed on the two types of images using real domain labels. If it is a CAD domain feature, it is sent to the CAD terminal diagram character channel of the three-channel backbone network. If it is a sleeve domain feature, it is sent to the sleeve character channel and tilt correction channel of the three-channel backbone network. After feature extraction and feature decoding by the three-channel backbone network, CAD terminal diagram character recognition output and sleeve character recognition output are obtained.

[0010] S4. Synchronously input the standardized CAD terminal diagram into the corner detection model to perform corner detection. After obtaining the position label of the corner box, reconstruct the contextual logic relationship with the character recognition output of the CAD terminal diagram to obtain the formatted CAD terminal diagram character recognition output.

[0011] S5. Organize the sleeve character recognition output to obtain a formatted sleeve character recognition output;

[0012] S6. Match the formatted CAD terminal diagram character recognition output with the formatted sleeve character recognition output, and output the matching result.

[0013] Optionally, step S2 involves data preprocessing of the CAD terminal diagrams in the training set, specifically including:

[0014] The CAD terminal diagram is converted into a grayscale image, then the image is binarized to separate the bright effective area from the dark background. Then, a morphological erosion operation is applied, which is to erode the positions of all characters in the list in the image, so that the white area becomes smaller and the black area becomes larger.

[0015] Analyze the empty connected regions in the eroded image, statistically analyze the attributes of each region, find all empty connected regions and sort them, and retain the largest empty connected region as the background.

[0016] Find the outlines and hierarchical structure in the background, extract the list outlines, draw the outlines, obtain the bounding boxes of the outlines, save the image extracted from each outline and record the cropping position;

[0017] After cutting out each list individually, sort them from left to right, detect the length and width of each list, cut the longer side along the grid line of the shorter side to ensure that the cutting does not affect the characters, record the number of each cut list block to ensure that the cut list blocks can be reconstructed into the original image later, and use each cut list block as a standardized CAD terminal diagram.

[0018] The S2 step involves data preprocessing of the sleeve images in the training set, specifically including:

[0019] The actual acquired sleeve image is input, aligned with the resolution of the CAD terminal diagram list, and perspective distortion is corrected through affine transformation. Then, the sleeve image is grayscaled and binarized to extract a grid structure similar to the list in the CAD terminal diagram. A position mapping table is established based on the cutting coordinates of the CAD terminal diagram to ensure that the position of the sleeve character area corresponds to the character on the CAD terminal diagram, thus obtaining the enhanced sleeve image.

[0020] Optionally, the processing procedure of the shared feature extractor is as follows:

[0021] The standardized CAD terminal diagram and the enhanced sleeve image are first captured by a 7×7 convolutional layer to initially capture global structural features, and then spatial downsampling is performed by max pooling to reduce computation and enhance translation invariance.

[0022] The processed features are fed into a group of residual blocks consisting of four residual blocks: in each residual block, the features are passed through a 1×1 convolutional layer, a 3×3 convolutional layer and a 1×1 convolutional layer in sequence, while the original input is matched for dimension through parallel shortcut connections. The output of the main path is added to the output of the shortcut connections to form the output of each residual block.

[0023] After processing through four residual blocks to avoid gradient vanishing and preserve the original information, the output is then processed by the ReLU activation function and then compressed through a global average pooling layer. The final output feature vector serves as the shared feature.

[0024] Optionally, the processing procedure of the domain adaptation module is as follows:

[0025] The shared features first pass through the gradient reversal layer, which performs the opposite operation in the forward and backward propagation stages. Through adversarial training, the shared feature extractor is induced to undergo parameter updates to feed back training and extract high-level shared features.

[0026] After passing through the gradient reversal layer, the feature is input into the domain classifier: the feature dimension is compressed by global average pooling, the feature is mapped by a fully connected layer, processed by ReLU activation function, the class transformation of the fully connected layer and the probability output is obtained by Softmax normalization. The output is the domain prediction probability of whether the feature comes from the CAD domain or the casing domain.

[0027] When training the domain classifier, because there are more samples in the CAD domain and fewer samples in the tubing domain, transfer learning is used to map the features of different domains to the same vector space, eliminate low-level differences between domains, establish a bridge from the CAD domain to the tubing domain, so that the knowledge learned in the CAD domain can be transferred to the tubing domain, making it difficult for the domain classifier to distinguish which domain the features come from, thereby driving the shared feature extractor to extract high-level shared features.

[0028] The reversed features are then transformed by 1×1 convolution, batch normalization is performed to accelerate convergence, and ReLU activation function is used to process them before entering the three-channel backbone network after feature type determination.

[0029] Optionally, the three-channel backbone network feature extraction includes: CAD terminal diagram character channel, sleeve character channel, and tilt correction channel;

[0030] The processing procedure for the CAD terminal diagram character channel is as follows:

[0031] The CAD domain features are sequentially subjected to three parallel sets of 3×3 dilated convolutions with 1×1 convolution to adjust the channel dimension, 4x upsampling to restore the resolution, and dilation rate rate=1 / 2 / 4 to capture multi-scale features. The final feature concatenation outputs CAD terminal image character features.

[0032] The processing procedure for the sleeve character channel is as follows:

[0033] The sleeve domain features are first processed by an STN spatial transformation network, which automatically learns to correct for small tilt angles in the input image, making subsequent feature extraction more robust. The STN spatial transformation network first predicts a 2×3 affine transformation matrix through a localization network. Then use the affine transformation matrix Geometric correction is performed using the following formula:

[0034]

[0035] in It is a 2×3 affine transformation matrix containing translation, rotation, and scaling transformation parameters; (x, y) are the pixel coordinates in the input image, and (x', y') are the coordinates of the corresponding pixel in the corrected image;

[0036] After spatial transformation, the transformed feature maps are normalized. The normalized feature maps are then fed into four parallel angular convolutional layers. These four angular convolutional layers share the same structure, but their kernel parameters are learned independently. The key point is that these convolutional layers themselves do not physically rotate. Their design philosophy is to allow the network to automatically learn and capture feature patterns related to the 0°, 45°, 90°, and 135° directions, respectively. The feature maps extracted by these four convolutional layers are stitched together to obtain stitched features. Then, the stitched features are aligned with the correction features obtained from the tilt correction channel to obtain the sleeve image character features.

[0037] Optionally, the processing procedure for the tilt correction channel is as follows:

[0038] To correct large tilt angles in the input image, a spatial saliency analysis module is first used: a 1×1 convolution is used for feature transformation to reduce dimensionality, followed by batch normalization to stabilize the training process, and the ReLU activation function is added to enhance feature representation. Then, a single-channel feature map is generated through a 1×1 convolution, and a heatmap is generated through the Sigmoid activation function. The values ​​are all between [0, 1]. The larger the value, the more important the corresponding spatial location is for judging the tilt angle of the image. It can suppress the interference of background or irrelevant areas and focus on the area that can truly reflect the directionality of the characters.

[0039] The heat map is multiplied element-wise with the feature map fed into the tilt correction channel to obtain a weighted feature map;

[0040] The weighted feature map is fed into the angle prediction module: first, global flat pooling is used to generate a global feature vector, then a fully connected layer is used to predict the angle and output the original angle value. The Tanh activation function limits the output value to [-1, 1], and multiplies it by the maximum allowed angle π / 4 to obtain the output predicted rotation angle. ;

[0041] The predicted rotation angle Input geometry correction module: First use the predicted rotation angle Calculate the rotation matrix The formula is as follows:

[0042]

[0043] This rotation matrix Instead of directly multiplying by the image pixel values, it defines a spatial coordinate transformation, generates a virtual spatial coordinate system, and creates a blank grid matrix of the same size as the input feature map. Each element position (i,j) corresponds to the target coordinate system position (x'_d, y'_d). Through inverse coordinate mapping, it utilizes... The source point corresponding to the target point on the weighted feature map is calculated. Bilinear sampling is performed on the floating-point coordinates (x, y) on the weighted feature map to generate feature values ​​of the target position. Finally, the corrected feature is obtained. The corrected feature is then aligned with the spliced ​​feature obtained from the sleeve character channel to obtain the sleeve image character feature.

[0044] Optionally, the three-channel backbone network feature decoding includes:

[0045] The CAD terminal diagram character features are decoded using bidirectional LSTM to obtain timing information, multi-head self-attention to reconstruct key structures, layer normalization to stabilize the decoding process, and dynamic convolution to generate pixel space mapping, ultimately resulting in a CAD terminal diagram character recognition output with coordinates.

[0046] The sleeve image character features are captured by forward LSTM and backward LSTM working together to capture bidirectional context. The features are fused and integrated with the bidirectional decoding results. Dynamic convolution is used to perform spatial coordinate regression and character generation to obtain sleeve character recognition output with coordinates.

[0047] Optionally, the corner detection model uses the edge lines in the list to detect corners next to the list, and the processing procedure is as follows:

[0048] The characters in the standardized CAD terminal diagram are painted white, leaving only the grid lines to help detect corners;

[0049] The image after being painted white is binarized to clearly distinguish between grid lines and background;

[0050] The binarized image is then inverted to white, making the grid lines white and the background black. The white grid lines are then skeletonized, reducing their width to a single pixel to accurately represent the central axis of the lines. After another inversion, the intersections of the grid lines are clearly exposed, and these intersections have significant pixel differences from the points on other line segments.

[0051] Calculate the centroid coordinates of the small area where these intersections are located, create a small rectangle, add the rectangle to the corner box list, record and output the position coordinates of these corner boxes.

[0052] Optionally, the context logic relationship reconstruction in step S4 specifically includes:

[0053] Consider the center point of each corner box as a key coordinate node. These nodes are naturally distributed at the grid intersections of the list. Although there is no explicit line drawing operation, logically, the center points of adjacent corner boxes implicitly define the rows and columns of the grid. This set of center points of corner boxes itself forms an implicit coordinate network, providing a benchmark for subsequent spatial relationship judgments. In the corner coordinate network, find the character box closest to the center point of a certain corner box: vertically downwards, find the character box horizontally aligned with the center point of this corner box; horizontally, find the character box that is closest to the center point of this corner box. The character boxes are vertically aligned with the center point of each corner box, and a small threshold is defined for both directions. These thresholds define the tolerance range for alignment and are used to determine whether a character is in the row or column defined by the center point of the corner box. Through this search based on the nearest corner and horizontal and vertical thresholds, the character boxes are assigned to a specific grid area defined by the center point of the corner box. The character box found in the vertical direction is defined as a cable, the nearest character box found in the horizontal direction is defined as a cable tray, and the second nearest is defined as a terminal number. The search for cable trays, cables, and numbers is performed by associating the coordinate network with the corner position.

[0054] After a successful search, the logical relationship between the regions is reconstructed. A simple Excel spreadsheet is created, with the first column being the number column, the second column being the searched cable tray column, and the third column being the searched cable column. The corresponding horizontal and vertical coordinates can be added later.

[0055] To ensure the accuracy of terminal number identification, the terminal numbers are then subject to digital characteristic constraints to strictly guarantee that they are purely numerical and to eliminate misidentification of non-numerical interference items.

[0056] Finally, the formatted CAD terminal diagram character recognition output is obtained.

[0057] On the other hand, a character recognition and matching system for CAD terminal diagrams and sleeve images is provided, the system comprising:

[0058] The acquisition module is used to acquire images of bushings inside the substation cabinet, and to annotate the acquired bushing images and CAD terminal diagrams, and divide them into training and validation sets.

[0059] The preprocessing module is used to preprocess the CAD terminal diagrams and sleeve images in the training set to obtain standardized CAD terminal diagrams and enhanced sleeve images.

[0060] The character detection module is used to classify the standardized CAD terminal diagram and the enhanced sleeve image into CAD domain and sleeve domain. Each time, the same number of images are obtained from their respective type domains and input into the character detection and recognition model. The character detection and recognition model includes a shared feature extractor, a domain adaptive module and a three-channel backbone network.

[0061] The shared feature extractor first extracts low-level shared features. The domain adaptation module maps features from different domains to the same vector space through transfer learning and updates the parameters of the shared feature extractor through backpropagation, enabling the shared feature extractor to extract high-level shared features. Meanwhile, in the domain adaptation module, feature classification is performed on the two types of images using real domain labels. If it is a CAD domain feature, it is sent to the CAD terminal diagram character channel of the three-channel backbone network. If it is a sleeve domain feature, it is sent to the sleeve character channel and tilt correction channel of the three-channel backbone network. After feature extraction and feature decoding by the three-channel backbone network, CAD terminal diagram character recognition output and sleeve character recognition output are obtained.

[0062] The corner detection and reconstruction module is used to synchronously input the standardized CAD terminal diagram into the corner detection model for corner detection, obtain the position label of the corner box, and then reconstruct the contextual logic relationship with the character recognition output of the CAD terminal diagram to obtain the formatted CAD terminal diagram character recognition output.

[0063] The sorting module is used to sort the sleeve character recognition output to obtain a formatted sleeve character recognition output;

[0064] The matching module is used to match the formatted CAD terminal diagram character recognition output with the formatted sleeve character recognition output, and output the matching result.

[0065] On the other hand, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the above-described method for character recognition and matching of CAD terminal diagrams and sleeve images.

[0066] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, the at least one instruction being loaded and executed by a processor to implement the above-described method for character recognition and matching of CAD terminal diagrams and sleeve images.

[0067] The beneficial effects of the technical solution provided by this invention include at least the following:

[0068] 1) This invention designs a standardized cutting method to solve the problem of small characters in high-resolution CAD terminal diagrams, and adds a tilt correction channel to enhance the recognition accuracy of characters in sleeve images, which greatly improves the ability to provide targeted solutions to different problems in two types of drawings.

[0069] 2) This invention processes CAD terminal diagrams and sleeve images simultaneously through shared feature extraction. By using transfer learning, it maps features from different domains to the same vector space, eliminates low-level differences between domains, and establishes a bridge from the source domain to the target domain. This allows knowledge learned in the CAD domain to be transferred to the sleeve domain, resulting in higher recognition accuracy for the same model when dealing with two types of images.

[0070] 3. The corner detection model of the present invention utilizes the significant pixel differences between the corners next to the list and the points on other line segments, along with skeletonized image processing, to better identify these corners in the image, providing excellent technical support for reconstructing the context relationship. At the same time, it enables the recognition model to interrupt and reconstruct position information during the recognition process. Attached Figure Description

[0071] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0072] Figure 1 This is a flowchart of a method for character recognition and matching between CAD terminal diagrams and sleeve images provided by an embodiment of the present invention;

[0073] Figure 2 This is a general block diagram of a method for character recognition and matching between CAD terminal diagrams and sleeve images provided in an embodiment of the present invention;

[0074] Figure 3 This is a flowchart of the high-resolution CAD terminal diagram and sleeve image data preprocessing provided in the embodiments of the present invention;

[0075] Figure 4 This is a flowchart of the CAD terminal drawing cutting process provided in an embodiment of the present invention;

[0076] Figure 5 This is a flowchart of the character detection and recognition model processing provided in the embodiments of the present invention;

[0077] Figure 6 This is a flowchart of the shared feature extractor processing provided in an embodiment of the present invention;

[0078] Figure 7 This is a flowchart of the domain adaptive module processing provided in an embodiment of the present invention;

[0079] Figure 8 This is a flowchart of the three-channel backbone network feature extraction provided in an embodiment of the present invention;

[0080] Figure 9This is a flowchart of the tilt correction channel processing provided in an embodiment of the present invention;

[0081] Figure 10 This is a flowchart of the three-channel backbone network feature decoding provided in an embodiment of the present invention;

[0082] Figure 11 This is a flowchart of the corner detection model processing provided in an embodiment of the present invention;

[0083] Figure 12 This is a flowchart of the context logic relationship reconstruction provided in the embodiments of the present invention;

[0084] Figure 13 This is a block diagram of a CAD terminal diagram and sleeve image character recognition and matching system provided in an embodiment of the present invention;

[0085] Figure 14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0086] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0087] This invention provides a method for character recognition and matching between CAD terminal diagrams and sleeve images. This method can be implemented by an electronic device, which can be a terminal or a server. Figure 1 The flowchart of this method is shown below. Figure 2 The diagram shown is an overall block diagram of the method. The processing flow may include the following steps:

[0088] S1. Collect images of bushings inside the substation cabinet, label the collected bushing images and CAD terminal diagrams, and divide them into training and validation sets.

[0089] In this embodiment of the invention, a high-resolution camera is used to acquire images of bushings inside the substation cabinet. The PaddleOCR tool is used to annotate the acquired bushing images and CAD terminal diagrams. The annotation format is PPOCRLabel, and the annotation results (transcription, points) are exported. Here, the transcription represents the character recognition data, and the points represent the position information of the detection box, which respectively represent the x-coordinate and y-coordinate of the four corners of the detection box. The annotation file is stored in txt format.

[0090] The two classes of images are divided into a training set and a validation set in a 4:1 ratio to form a complete dataset. The model is trained on the training set and its performance is evaluated on the validation set. The model with the best detection results on the validation set is selected as the final model for actual inference and application.

[0091] S2. Perform data preprocessing on the CAD terminal diagrams and sleeve images in the training set to obtain standardized CAD terminal diagrams and enhanced sleeve images;

[0092] Optionally, such as Figure 3 As shown, step S2 involves data preprocessing of the CAD terminal diagrams in the training set, specifically including:

[0093] The CAD terminal diagram is converted into a grayscale image, then the image is binarized to separate the bright effective area from the dark background. Then, a morphological erosion operation is applied, which is to erode the positions of all characters in the list in the image, so that the white area becomes smaller and the black area becomes larger.

[0094] Analyze the empty connected regions in the eroded image, statistically analyze the attributes of each region, find all empty connected regions and sort them, and retain the largest empty connected region as the background.

[0095] Find the outlines and hierarchical structure in the background, extract the list outlines, draw the outlines, obtain the bounding boxes of the outlines, save the image extracted from each outline and record the cropping position;

[0096] After cutting out each list individually, sort them from left to right. Detect the length and width of each list, and cut along the grid lines of the shorter side of the longer side, ensuring the cutting does not affect the characters. Label and record the number of each cut list block to ensure that it can be reconstructed from the original image later. Use each cut list block as a standardized CAD terminal diagram, such as... Figure 4 As shown;

[0097] The S2 step involves data preprocessing of the sleeve images in the training set, specifically including:

[0098] The actual acquired sleeve image is input, aligned with the resolution of the CAD terminal diagram list, and perspective distortion is corrected through affine transformation. Then, the sleeve image is grayscaled and binarized to extract a grid structure similar to the list in the CAD terminal diagram. A position mapping table is established based on the cutting coordinates of the CAD terminal diagram to ensure that the position of the sleeve character area corresponds to the character on the CAD terminal diagram, thus obtaining the enhanced sleeve image.

[0099] S3. Classify the standardized CAD terminal diagram and the enhanced sleeve image into CAD domain and sleeve domain (for example, the label of the CAD domain can be set to 0 and the label of the sleeve domain can be set to 1). Each time, take the same number of images from their respective type domains and input them into the character detection and recognition model. The character detection and recognition model includes a shared feature extractor, a domain adaptive module and a three-channel backbone network.

[0100] The shared feature extractor first extracts low-level shared features. The domain adaptation module maps features from different domains to the same vector space through transfer learning and updates the parameters of the shared feature extractor through backpropagation, enabling the shared feature extractor to extract high-level shared features. Simultaneously, in the domain adaptation module, feature classification is performed on the two types of images based on the real domain labels (feature type judgment is based on the real domain label (set), not changing the domain prediction probability obtained by the domain classifier). If it is a CAD domain feature, it is sent to the CAD terminal diagram character channel of the three-channel backbone network; if it is a sleeve domain feature, it is sent to the sleeve character channel and tilt correction channel of the three-channel backbone network. After feature extraction and feature decoding by the three-channel backbone network, CAD terminal diagram character recognition output and sleeve character recognition output are obtained. Figure 5 As shown;

[0101] Optionally, such as Figure 6 As shown, the processing procedure of the shared feature extractor is as follows:

[0102] The standardized CAD terminal diagram and the enhanced sleeve image are first captured by a 7×7 convolutional layer to initially capture global structural features, and then spatial downsampling is performed by max pooling to reduce computation and enhance translation invariance.

[0103] The processed features are fed into a group of residual blocks consisting of four residual blocks: in each residual block, the features are passed through a 1×1 convolutional layer, a 3×3 convolutional layer and a 1×1 convolutional layer in sequence, while the original input is matched for dimension through parallel shortcut connections. The output of the main path is added to the output of the shortcut connections to form the output of each residual block.

[0104] After processing through four residual blocks to avoid gradient vanishing and preserve the original information, the output is then processed by the ReLU activation function and then compressed through a global average pooling layer. The final output feature vector serves as the shared feature.

[0105] Optionally, such as Figure 7 As shown, the processing procedure of the domain adaptation module is as follows:

[0106] The shared features first pass through the gradient reversal layer, which performs the opposite operation in the forward and backward propagation stages. Through adversarial training, the shared feature extractor is induced to undergo parameter updates to feed back training and extract high-level shared features.

[0107] After passing through the gradient reversal layer, the feature is input into the domain classifier: the feature dimension is compressed by global average pooling, the feature is mapped by a fully connected layer, processed by ReLU activation function, the class transformation of the fully connected layer and the probability output is obtained by Softmax normalization. The output is the domain prediction probability of whether the feature comes from the CAD domain or the casing domain.

[0108] When training the domain classifier, because there are more samples in the CAD domain and fewer samples in the tubing domain, transfer learning is used to map the features of different domains to the same vector space, eliminate low-level differences between domains, establish a bridge from the CAD domain to the tubing domain, so that the knowledge learned in the CAD domain can be transferred to the tubing domain, making it difficult for the domain classifier to distinguish which domain the features come from, thereby driving the shared feature extractor to extract high-level shared features.

[0109] The reversed features are then transformed by 1×1 convolution, batch normalization is performed to accelerate convergence, and ReLU activation function is used to process them before entering the three-channel backbone network after feature type determination.

[0110] Optionally, such as Figure 8 As shown, the three-channel backbone network feature extraction includes: CAD terminal diagram character channel, sleeve character channel, and tilt correction channel;

[0111] The processing procedure for the CAD terminal diagram character channel is as follows:

[0112] The CAD domain features are sequentially subjected to three parallel sets of 3×3 dilated convolutions with 1×1 convolution to adjust the channel dimension, 4x upsampling to restore the resolution, and dilation rate rate=1 / 2 / 4 to capture multi-scale features. The final feature concatenation outputs CAD terminal image character features.

[0113] The processing procedure for the sleeve character channel is as follows:

[0114] The sleeve domain features are first processed by an STN spatial transformation network, which automatically learns to correct for small tilt angles in the input image, making subsequent feature extraction more robust. The STN spatial transformation network first predicts a 2×3 affine transformation matrix through a localization network. Then use the affine transformation matrix Geometric correction is performed using the following formula:

[0115]

[0116] in It is a 2×3 affine transformation matrix containing translation, rotation, and scaling transformation parameters; (x, y) are the pixel coordinates in the input image, and (x', y') are the coordinates of the corresponding pixel in the corrected image;

[0117] After spatial transformation, the transformed feature maps are normalized. The normalized feature maps are then fed into four parallel angular convolutional layers. These four angular convolutional layers share the same structure, but their kernel parameters are learned independently. The key point is that these convolutional layers themselves do not physically rotate. Their design philosophy is to allow the network to automatically learn and capture feature patterns related to the 0°, 45°, 90°, and 135° directions, respectively. The feature maps extracted by these four convolutional layers are stitched together to obtain stitched features. Then, the stitched features are aligned with the correction features obtained from the tilt correction channel to obtain the sleeve image character features.

[0118] Optionally, such as Figure 9 As shown, the processing procedure for the tilt correction channel is as follows:

[0119] To correct large tilt angles in the input image, a spatial saliency analysis module is first used: a 1×1 convolution is used for feature transformation to reduce dimensionality, followed by batch normalization to stabilize the training process, and the ReLU activation function is added to enhance feature representation. Then, a single-channel feature map is generated through a 1×1 convolution, and a heatmap is generated through the Sigmoid activation function. The values ​​are all between [0, 1]. The larger the value, the more important the corresponding spatial location is for judging the tilt angle of the image. It can suppress the interference of background or irrelevant areas and focus on the area that can truly reflect the directionality of the characters.

[0120] The heat map is multiplied element-wise with the feature map fed into the tilt correction channel to obtain a weighted feature map;

[0121] The weighted feature map is fed into the angle prediction module: first, global flat pooling is used to generate a global feature vector, then a fully connected layer is used to predict the angle and output the original angle value. The Tanh activation function limits the output value to [-1, 1], and multiplies it by the maximum allowed angle π / 4 to obtain the output predicted rotation angle. ;

[0122] The predicted rotation angle Input geometry correction module: First use the predicted rotation angle Calculate the rotation matrix The formula is as follows:

[0123]

[0124] This rotation matrix Instead of directly multiplying by the image pixel values, it defines a spatial coordinate transformation, generates a virtual spatial coordinate system, and creates a blank grid matrix of the same size as the input feature map. Each element position (i,j) corresponds to the target coordinate system position (x'_d, y'_d). Through inverse coordinate mapping, it utilizes... The source point corresponding to the target point on the weighted feature map is calculated. Bilinear sampling is performed on the floating-point coordinates (x, y) on the weighted feature map to generate feature values ​​of the target position. Finally, the corrected feature is obtained. The corrected feature is then aligned with the spliced ​​feature obtained from the sleeve character channel to obtain the sleeve image character feature.

[0125] Optionally, such as Figure 10 As shown, the three-channel backbone network feature decoding includes:

[0126] The CAD terminal diagram character features are decoded using bidirectional LSTM to obtain timing information, multi-head self-attention to reconstruct key structures, layer normalization to stabilize the decoding process, and dynamic convolution to generate pixel space mapping, ultimately resulting in a CAD terminal diagram character recognition output with coordinates.

[0127] The sleeve image character features are captured by forward LSTM and backward LSTM working together to capture bidirectional context. The features are fused and integrated with the bidirectional decoding results. Dynamic convolution is used to perform spatial coordinate regression and character generation to obtain sleeve character recognition output with coordinates.

[0128] S4. Synchronously input the standardized CAD terminal diagram into the corner detection model to perform corner detection. After obtaining the position label of the corner box, reconstruct the contextual logic relationship with the character recognition output of the CAD terminal diagram to obtain the formatted CAD terminal diagram character recognition output.

[0129] Optionally, such as Figure 11 As shown, the corner detection model uses the edge lines in the list to detect corners next to the list. The processing procedure is as follows:

[0130] The characters in the standardized CAD terminal diagram are painted white, leaving only the grid lines to help detect corners;

[0131] The image after being painted white is binarized to clearly distinguish between grid lines and background;

[0132] The binarized image is then inverted to white, making the grid lines white and the background black. The white grid lines are then skeletonized to reduce their width to a single pixel, accurately representing the central axis of the lines. The image is then inverted again to clearly expose the intersections of the grid lines, which have significant pixel differences from points on other line segments.

[0133] Calculate the centroid coordinates of the small area where these intersections are located, create a small rectangle, add the rectangle to the corner box list, record and output the position coordinates of these corner boxes.

[0134] Optionally, such as Figure 12 As shown, the context logic relationship reconstruction in step S4 specifically includes:

[0135] The center point of each corner frame is considered a key coordinate node. These nodes are naturally distributed at the grid intersections of the list. Although there is no explicit line drawing operation, logically, the center points of adjacent corner frames implicitly define the rows and columns of the grid. This set of center points of corner frames itself forms an implicit coordinate network, providing a benchmark for subsequent spatial relationship judgments. In the corner coordinate network, the character box closest to the center point of a certain corner frame is found: vertically downwards (i.e., in the direction of increasing Y-coordinate), the character box horizontally aligned with the center point of this corner frame is found; horizontally, the character box closest to the center point of this corner frame is found. Vertically aligned character boxes are used, with a small threshold defined in both directions (based on image resolution and grid size). These thresholds define the tolerance range for alignment and are used to determine whether a character is in the row or column defined by the center point of the corner box. Through this search based on the nearest corner and horizontal and vertical thresholds, character boxes are assigned to a specific grid area defined by the center point of the corner box. The character box found in the vertical direction is defined as a cable (generally), the nearest character box found in the horizontal direction is defined as a slot, and the second nearest is defined as a terminal number. The search for slots, cables, and numbers is performed by associating the coordinate network with the corner position.

[0136] After a successful search, the logical relationship between the regions is reconstructed. A simple Excel spreadsheet is created, with the first column being the number column, the second column being the searched cable tray column, and the third column being the searched cable column. The corresponding horizontal and vertical coordinates can be added later.

[0137] To ensure the accuracy of terminal number identification, the terminal numbers are then subject to digital characteristic constraints to strictly guarantee that they are purely numerical and to eliminate misidentification of non-numerical interference items.

[0138] Finally, the formatted CAD terminal diagram character recognition output is obtained.

[0139] S5. Organize the sleeve character recognition output to obtain a formatted sleeve character recognition output;

[0140] The sleeve character recognition output is reordered and organized to form a formatted sleeve character recognition output that corresponds to the formatted CAD terminal diagram character recognition.

[0141] S6. Match the formatted CAD terminal diagram character recognition output with the formatted sleeve character recognition output, and output the matching result.

[0142] The characters in the output formatted sleeve image are compared with the formatted CAD terminal diagram data structure table using feature-level similarity calculation. If the similarity is >0.85 and the sleeve classification confidence is >0.9, it is marked as a strong matching candidate pair, and the matching result is output.

[0143] In this embodiment of the invention, during model training, two types of images are labeled and preprocessed to obtain standardized CAD terminal diagrams and coordinates, and enhanced sleeve images and coordinates. These are then fed into the character detection and recognition model. The domain classification loss is obtained in the adaptive module, and the recognition loss is obtained from the character recognition outputs and coordinates of the CAD terminal diagrams and the sleeve characters. These are combined into a joint loss function, and then backpropagation is used to update the parameters of each module. The joint loss function is as follows:

[0144]

[0145] in, It is about identifying loss. It is a domain classification loss. A value of 0.5 can be used, and the specific losses can be binary cross-entropy losses.

[0146] During the model inference process, only two types of images are preprocessed and fed into the character detection and recognition model to obtain the CAD terminal diagram character recognition output and the sleeve character recognition output. At the same time, the preprocessed standardized CAD terminal diagram is simultaneously subjected to corner detection. After obtaining the position label of the corner box, the contextual logic relationship is reconstructed with the CAD terminal diagram character recognition output to obtain the formatted CAD terminal diagram character recognition output. This output is then matched with the formatted sleeve character recognition output obtained through the same process, and the matching result is output.

[0147] like Figure 13 As shown, this embodiment of the invention also provides a CAD terminal diagram and sleeve image character recognition and matching system, the system comprising:

[0148] The acquisition module 1310 is used to acquire images of bushings inside the substation cabinet, and to annotate the acquired bushing images and CAD terminal diagrams, and divide them into training set and validation set.

[0149] Preprocessing module 1320 is used to preprocess the CAD terminal diagrams and sleeve images in the training set to obtain standardized CAD terminal diagrams and enhanced sleeve images.

[0150] The character detection module 1330 is used to classify the standardized CAD terminal diagram and the enhanced sleeve image into CAD domain and sleeve domain. Each time, the same number of images are obtained from their respective type domains and input into the character detection and recognition model. The character detection and recognition model includes a shared feature extractor, a domain adaptive module and a three-channel backbone network.

[0151] The shared feature extractor first extracts low-level shared features. The domain adaptation module maps features from different domains to the same vector space through transfer learning and updates the parameters of the shared feature extractor through backpropagation, enabling the shared feature extractor to extract high-level shared features. Meanwhile, in the domain adaptation module, feature classification is performed on the two types of images using real domain labels. If it is a CAD domain feature, it is sent to the CAD terminal diagram character channel of the three-channel backbone network. If it is a sleeve domain feature, it is sent to the sleeve character channel and tilt correction channel of the three-channel backbone network. After feature extraction and feature decoding by the three-channel backbone network, CAD terminal diagram character recognition output and sleeve character recognition output are obtained.

[0152] The corner detection and reconstruction module 1340 is used to synchronously input the standardized CAD terminal diagram into the corner detection model for corner detection, obtain the position label of the corner box, and then reconstruct the contextual logic relationship with the character recognition output of the CAD terminal diagram to obtain the formatted CAD terminal diagram character recognition output.

[0153] The sorting module 1350 is used to sort the sleeve character recognition output to obtain a formatted sleeve character recognition output.

[0154] The matching module 1360 is used to match the formatted CAD terminal diagram character recognition output and the formatted sleeve character recognition output, and output the matching result.

[0155] The CAD terminal diagram and sleeve image character recognition and matching system provided in this embodiment of the invention has a functional structure that corresponds to the CAD terminal diagram and sleeve image character recognition and matching method provided in this embodiment of the invention, and will not be described again here.

[0156] Figure 14 This is a schematic diagram of the structure of an electronic device 1400 provided in an embodiment of the present invention. The electronic device 1400 may vary considerably due to different configurations or performance. It may include one or more central processing units (CPUs) 1401 and one or more memories 1402. The memory 1402 stores at least one instruction, which is loaded and executed by the processor 1401 to implement the steps of the above-described method for character recognition and matching of CAD terminal diagrams and sleeve images.

[0157] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including instructions that can be executed by a processor in a terminal to complete the aforementioned method for character recognition and matching of CAD terminal diagrams and sleeve images. For example, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device.

[0158] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0159] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for character recognition and matching between CAD terminal diagrams and sleeve images, characterized in that, The method includes: S1. Collect images of bushings inside the substation cabinet, label the collected bushing images and CAD terminal diagrams, and divide them into training and validation sets. S2. Perform data preprocessing on the CAD terminal diagrams and sleeve images in the training set to obtain standardized CAD terminal diagrams and enhanced sleeve images; S3. Classify the standardized CAD terminal diagram and the enhanced sleeve image into CAD domain and sleeve domain. Each time, obtain the same number of images from their respective type domains and input them into the character detection and recognition model. The character detection and recognition model includes a shared feature extractor, a domain adaptive module, and a three-channel backbone network. The shared feature extractor first extracts low-level shared features. The domain adaptation module maps features from different domains to the same vector space through transfer learning and updates the parameters of the shared feature extractor through backpropagation, enabling the shared feature extractor to extract high-level shared features. Meanwhile, in the domain adaptation module, feature classification is performed on the two types of images using real domain labels. If it is a CAD domain feature, it is sent to the CAD terminal diagram character channel of the three-channel backbone network. If it is a sleeve domain feature, it is sent to the sleeve character channel and tilt correction channel of the three-channel backbone network. After feature extraction and feature decoding by the three-channel backbone network, CAD terminal diagram character recognition output and sleeve character recognition output are obtained. S4. Synchronously input the standardized CAD terminal diagram into the corner detection model to perform corner detection. After obtaining the position label of the corner box, reconstruct the contextual logic relationship with the character recognition output of the CAD terminal diagram to obtain the formatted CAD terminal diagram character recognition output. S5. Organize the sleeve character recognition output to obtain a formatted sleeve character recognition output; S6. Match the formatted CAD terminal diagram character recognition output with the formatted sleeve character recognition output, and output the matching result.

2. The method according to claim 1, characterized in that, The S2 step involves data preprocessing of the CAD terminal diagrams in the training set, specifically including: The CAD terminal diagram is converted into a grayscale image, then the image is binarized to separate the bright effective area from the dark background. Then, a morphological erosion operation is applied, which is to erode the positions of all characters in the list in the image, so that the white area becomes smaller and the black area becomes larger. Analyze the empty connected regions in the eroded image, statistically analyze the attributes of each region, find all empty connected regions and sort them, and retain the largest empty connected region as the background. Find the outlines and hierarchical structure in the background, extract the list outlines, draw the outlines, obtain the bounding boxes of the outlines, save the image extracted from each outline and record the cropping position. After cutting out each list individually, sort them from left to right, detect the length and width of each list, cut the longer side along the grid line of the shorter side to ensure that the cutting does not affect the characters, record the number of each cut list block to ensure that the cut list blocks can be reconstructed into the original image later, and use each cut list block as a standardized CAD terminal diagram. The S2 step involves data preprocessing of the sleeve images in the training set, specifically including: The actual acquired sleeve image is input, aligned with the resolution of the CAD terminal diagram list, and perspective distortion is corrected through affine transformation. Then, the sleeve image is grayscaled and binarized to extract a grid structure similar to the list in the CAD terminal diagram. A position mapping table is established based on the cutting coordinates of the CAD terminal diagram to ensure that the position of the sleeve character area corresponds to the character on the CAD terminal diagram, thus obtaining the enhanced sleeve image.

3. The method according to claim 1, characterized in that, The processing procedure of the shared feature extractor is as follows: The standardized CAD terminal diagram and the enhanced sleeve image are first captured by a 7×7 convolutional layer to initially capture global structural features, and then spatial downsampling is performed by max pooling to reduce computation and enhance translation invariance. The processed features are fed into a group of residual blocks consisting of four residual blocks: in each residual block, the features are passed through a 1×1 convolutional layer, a 3×3 convolutional layer and a 1×1 convolutional layer in sequence, while the original input is matched for dimension through parallel shortcut connections. The output of the main path is added to the output of the shortcut connections to form the output of each residual block. After processing through four residual blocks to avoid gradient vanishing and preserve the original information, the output is then processed by the ReLU activation function and then compressed through a global average pooling layer. The final output feature vector serves as the shared feature.

4. The method according to claim 1, characterized in that, The processing procedure of the domain adaptive module is as follows: The shared features first pass through the gradient reversal layer, which performs the opposite operation in the forward and backward propagation stages. Through adversarial training, the shared feature extractor is induced to undergo parameter updates to feed back training and extract high-level shared features. After passing through the gradient reversal layer, the feature is input into the domain classifier: the feature dimension is compressed by global average pooling, the feature is mapped by a fully connected layer, processed by ReLU activation function, the class transformation of the fully connected layer and the probability output is obtained by Softmax normalization. The output is the domain prediction probability of whether the feature comes from the CAD domain or the casing domain. When training the domain classifier, because there are more samples in the CAD domain and fewer samples in the tubing domain, transfer learning is used to map the features of different domains to the same vector space, eliminate low-level differences between domains, establish a bridge from the CAD domain to the tubing domain, so that the knowledge learned in the CAD domain can be transferred to the tubing domain, making it difficult for the domain classifier to distinguish which domain the features come from, thereby driving the shared feature extractor to extract high-level shared features. The reversed features are then transformed by 1×1 convolution, batch normalization is performed to accelerate convergence, and ReLU activation function is used to process them before entering the three-channel backbone network after feature type determination.

5. The method according to claim 1, characterized in that, The three-channel backbone network feature extraction includes: CAD terminal diagram character channel, sleeve character channel, and tilt correction channel; The processing procedure for the CAD terminal diagram character channel is as follows: The CAD domain features are sequentially subjected to three parallel sets of 3×3 dilated convolutions with 1×1 convolution to adjust the channel dimension, 4x upsampling to restore the resolution, and dilation rate rate=1 / 2 / 4 to capture multi-scale features. The final feature concatenation outputs CAD terminal image character features. The processing procedure for the sleeve character channel is as follows: The sleeve domain features are first processed by an STN spatial transformation network, which automatically learns to correct for small tilt angles in the input image, making subsequent feature extraction more robust. The STN spatial transformation network first predicts a 2×3 affine transformation matrix through a localization network. Then use the affine transformation matrix Geometric correction is performed using the following formula: in It is a 2×3 affine transformation matrix containing translation, rotation, and scaling transformation parameters; (x, y) are the pixel coordinates in the input image, and (x', y') are the coordinates of the corresponding pixel in the corrected image; After spatial transformation, the transformed feature maps are normalized. The normalized feature maps are then fed into four parallel angular convolutional layers. These four angular convolutional layers share the same structure, but their kernel parameters are learned independently. The key point is that these convolutional layers themselves do not physically rotate. Their design philosophy is to allow the network to automatically learn and capture feature patterns related to the 0°, 45°, 90°, and 135° directions, respectively. The feature maps extracted by these four convolutional layers are stitched together to obtain stitched features. Then, the stitched features are aligned with the correction features obtained from the tilt correction channel to obtain the sleeve image character features.

6. The method according to claim 5, characterized in that, The processing procedure for the tilt correction channel is as follows: To correct large tilt angles in the input image, a spatial saliency analysis module is first used: a 1×1 convolution is used for feature transformation to reduce dimensionality, followed by batch normalization to stabilize the training process, and the ReLU activation function is added to enhance feature representation. Then, a single-channel feature map is generated through a 1×1 convolution, and a heatmap is generated through the Sigmoid activation function. The values ​​are all between [0, 1]. The larger the value, the more important the corresponding spatial location is for judging the tilt angle of the image. It can suppress the interference of background or irrelevant areas and focus on the area that can truly reflect the directionality of the characters. The heat map is multiplied element-wise with the feature map fed into the tilt correction channel to obtain a weighted feature map; The weighted feature map is fed into the angle prediction module: first, global flat pooling is used to generate a global feature vector, then a fully connected layer is used to predict the angle and output the original angle value. The Tanh activation function limits the output value to [-1, 1], and multiplies it by the maximum allowed angle π / 4 to obtain the output predicted rotation angle. ; The predicted rotation angle Input geometry correction module: First use the predicted rotation angle Calculate the rotation matrix The formula is as follows: This rotation matrix Instead of directly multiplying by the image pixel values, it defines a spatial coordinate transformation, generates a virtual spatial coordinate system, and creates a blank grid matrix of the same size as the input feature map. Each element position (i, j) corresponds to the target coordinate system position (x'_d, y'_d). Through inverse coordinate mapping, it utilizes... The source point corresponding to the target point on the weighted feature map is calculated. Bilinear sampling is performed on the floating-point coordinates (x, y) on the weighted feature map to generate feature values ​​of the target position. Finally, the corrected feature is obtained. The corrected feature is then aligned with the spliced ​​feature obtained from the sleeve character channel to obtain the sleeve image character feature.

7. The method according to claim 6, characterized in that, The three-channel backbone network feature decoding includes: The CAD terminal diagram character features are decoded using bidirectional LSTM to obtain timing information, multi-head self-attention to reconstruct key structures, layer normalization to stabilize the decoding process, and dynamic convolution to generate pixel space mapping, ultimately resulting in a CAD terminal diagram character recognition output with coordinates. The sleeve image character features are captured by forward LSTM and backward LSTM working together to capture bidirectional context. The features are fused and integrated with the bidirectional decoding results. Dynamic convolution is used to perform spatial coordinate regression and character generation to obtain sleeve character recognition output with coordinates.

8. The method according to claim 1, characterized in that, The corner detection model uses the edge lines in the list to detect corners next to the list. The processing procedure is as follows: The characters in the standardized CAD terminal diagram are painted white, leaving only the grid lines to help detect corners; The image after being painted white is binarized to clearly distinguish between grid lines and background; The binarized image is then inverted to white, making the grid lines white and the background black. The white grid lines are then skeletonized to reduce their width to a single pixel, accurately representing the central axis of the lines. The image is then inverted again to clearly expose the intersections of the grid lines, which have significant pixel differences from points on other line segments. Calculate the centroid coordinates of the small area where these intersections are located, create a small rectangle, add the rectangle to the corner box list, record and output the position coordinates of these corner boxes.

9. The method according to claim 1, characterized in that, The context logic relationship reconstruction in S4 specifically includes: Consider the center point of each corner box as a key coordinate node. These nodes are naturally distributed at the grid intersections of the list. Although there is no explicit line drawing operation, logically, the center points of adjacent corner boxes implicitly define the rows and columns of the grid. This set of center points of corner boxes itself forms an implicit coordinate network, providing a benchmark for subsequent spatial relationship judgments. In the corner coordinate network, find the character box closest to the center point of a certain corner box: vertically downwards, find the character box horizontally aligned with the center point of this corner box; horizontally, find the character box that is closest to the center point of this corner box. The character boxes are vertically aligned with the center point of each corner box, and a small threshold is defined for both directions. These thresholds define the tolerance range for alignment and are used to determine whether a character is in the row or column defined by the center point of the corner box. Through this search based on the nearest corner and horizontal and vertical thresholds, the character boxes are assigned to a specific grid area defined by the center point of the corner box. The character box found in the vertical direction is defined as a cable, the nearest character box found in the horizontal direction is defined as a cable tray, and the second nearest is defined as a terminal number. The search for cable trays, cables, and numbers is performed by associating the coordinate network with the corner position. After a successful search, the logical relationship between the regions is reconstructed. A simple Excel spreadsheet is created, with the first column being the number column, the second column being the searched cable tray column, and the third column being the searched cable column. The corresponding horizontal and vertical coordinates can be added later. To ensure the accuracy of terminal number identification, the terminal numbers are then subject to digital characteristic constraints to strictly guarantee that they are purely numerical and to eliminate misidentification of non-numerical interference items. Finally, the formatted CAD terminal diagram character recognition output is obtained.

10. A character recognition and matching system for CAD terminal diagrams and sleeve images, characterized in that, The system includes: The acquisition module is used to acquire images of bushings inside the substation cabinet, and to annotate the acquired bushing images and CAD terminal diagrams, and divide them into training and validation sets. The preprocessing module is used to preprocess the CAD terminal diagrams and sleeve images in the training set to obtain standardized CAD terminal diagrams and enhanced sleeve images. The character detection module is used to classify the standardized CAD terminal diagram and the enhanced sleeve image into CAD domain and sleeve domain. Each time, the same number of images are obtained from their respective type domains and input into the character detection and recognition model. The character detection and recognition model includes a shared feature extractor, a domain adaptive module and a three-channel backbone network. The shared feature extractor first extracts low-level shared features. The domain adaptation module maps features from different domains to the same vector space through transfer learning and updates the parameters of the shared feature extractor through backpropagation, enabling the shared feature extractor to extract high-level shared features. Meanwhile, in the domain adaptation module, feature classification is performed on the two types of images using real domain labels. If it is a CAD domain feature, it is sent to the CAD terminal diagram character channel of the three-channel backbone network. If it is a sleeve domain feature, it is sent to the sleeve character channel and tilt correction channel of the three-channel backbone network. After feature extraction and feature decoding by the three-channel backbone network, CAD terminal diagram character recognition output and sleeve character recognition output are obtained. The corner detection and reconstruction module is used to synchronously input the standardized CAD terminal diagram into the corner detection model for corner detection, obtain the position label of the corner box, and then reconstruct the contextual logic relationship with the character recognition output of the CAD terminal diagram to obtain the formatted CAD terminal diagram character recognition output. The sorting module is used to sort the sleeve character recognition output to obtain a formatted sleeve character recognition output; The matching module is used to match the formatted CAD terminal diagram character recognition output with the formatted sleeve character recognition output, and output the matching result.

Images