Deep learning-based inclined power signboard identification method and system

Abstract

The invention provides an inclined electric power signboard identification method and system based on deep learning. The method comprises the following steps: taking a to-be-detected electric power signboard image as an input image; carrying out multi-direction and multi-scale filtering to extract key points so as to generate a space mask; extracting multi-scale features from an input image, dynamically weighting and fusing the multi-scale features and associating a spatial mask, and generating a binary segmentation mask; calculating a mass center position of the binarized segmentation mask as an original point position of polar coordinates, performing polar coordinate transformation on the input image, performing multi-direction multi-scale filtering on the obtained polar coordinate domain image, calculating a multi-scale feature descriptor, and further obtaining an RMLG descriptor; and carrying out tilt correction on the input image based on the centroid position and the multi-scale feature descriptor, and carrying out character prediction on the image subjected to tilt correction in combination with the RMLG descriptor to obtain a character prediction sequence corresponding to the input image. According to the invention, identification of the inclined power equipment signboard in a complex scene can be realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing and power equipment detection, and specifically relates to a method and system for recognizing tilted power signboards based on deep learning. Background Art

[0002] Power equipment identification plates record key information such as equipment model, voltage level, and operating parameters, playing a vital role in the safe operation and maintenance of power grids and the rapid location of faults. However, identification of identification plates in complex environments faces numerous practical challenges. First, the installation location of identification plates varies with the equipment layout, and images captured during inspections often suffer from tilt, rotation, and perspective distortion. Second, outdoor lighting conditions are complex, with strong sunlight reflections, shadows, rain, or dust covering the edges of text blurring, severely reducing its readability. Furthermore, the size of identification plates varies significantly during actual inspections, and the multi-scale issues caused by varying distances exacerbate the difficulty of identification.

[0003] Existing sign recognition technologies each have their limitations. Traditional image processing methods such as SIFT and SURF, although they have a certain degree of rotation invariance, are easily affected by lighting and are not suitable for power signboards with regular characters. In recent years, deep learning-based detection methods such as Faster R-CNN and the YOLO series have achieved good results in general target detection, but these methods have obvious shortcomings in detecting rotated targets, especially when the signboard is tilted or the size changes significantly, the recognition accuracy drops significantly. In addition, although the Transformer-based AO²-DETR method has stronger rotation detection capabilities, its high computational complexity makes real-time deployment difficult, and it has high requirements for data labeling, which makes it difficult to meet the needs of actual power inspection scenarios. Therefore, there is an urgent need for an efficient and robust method and system for power signboard recognition under complex backgrounds, tilt rotation and lighting changes. Summary of the Invention

[0004] The purpose of this application is to provide a method and system for recognizing tilted power signboards based on deep learning, which can realize the recognition of tilted power equipment signboards in complex scenarios with rotation, occlusion, multi-scale changes and lighting differences.

[0005] The technical solutions provided in this application are: In a first aspect, the present application provides a method for recognizing tilted electric signage based on deep learning, comprising: Step 1: Obtain an image of the power sign to be detected as an input image; Step 2: Filter the input image using a multi-directional and multi-scale Log-Gabor filter bank to obtain a multi-directional and multi-scale filtered image; extract key points based on the multi-directional and multi-scale filtered response image; and generate a spatial mask based on the position of the key points; Step 3: Extract multi-scale features from the input image based on the feature extraction module, dynamically weight and fuse the multi-scale features based on the adaptive scale fusion ASF module, and associate the spatial mask based on the spatial attention mechanism to obtain a multi-scale fusion feature map. ; Step 4: Use the differentiable binary DB module to fuse feature maps based on multi-scale , generate a binary segmentation mask ; Step 5: Calculate the centroid position of the binary segmentation mask , as the origin of polar coordinates; Step 6: Based on the determined polar coordinate origin position, the input image is polar transformed to obtain a polar coordinate domain image; Step 7: Use multi-directional and multi-scale Log-Gabor filter banks to filter the polar coordinate domain images Perform filtering to obtain filter response images of different directions and scales; calculate multi-scale feature descriptors based on filter response images of different directions and scales ; Multi-scale feature descriptors Perform convolution and dimensionality reduction to obtain the RMLG descriptor ; Step 8: Use the character detection module based on the centroid position and multi-scale feature descriptors Perform tilt correction on the input image to obtain the tilt-corrected image ; Combined with RMLG descriptor for tilt-corrected image Perform character prediction to obtain a character prediction sequence corresponding to the input image.

[0006] In a possible implementation, in step 2, extracting key points based on the multi-directional and multi-scale filtered response image includes: Step 2.1: For each frequency scale filter response image, construct a composite gradient matrix: ; Where, For the first The filter response image of the frequency scale is used to construct the composite gradient matrix, that is, The composite gradient matrix of frequency scales, and Respectively represent the application of the input image The pixel point in the filtered response image obtained by the Log-Gabor filter in the direction The horizontal and vertical deviator responses at ; Represents pixel points Gaussian weights, is a Gaussian weighted window; Step 2.2: Calculate the characteristic point response value matrix for each frequency scale composite gradient matrix: ; Where, Express request The determinant of express traces; For the The characteristic point response value matrix of frequency scale; is the key point response suppression coefficient; Step 2.3: Perform weighted summation on the feature point response value matrices of different frequency scales to obtain the final feature point response value matrix; take the top N feature points in the final feature point response value matrix as key points to obtain the key point position set. .

[0007] In a possible implementation, in step 3, the feature extraction module uses a ResNet neural network; ; in, Represents the neural network from ResNet Feature maps of different scales; Express Perform 1×1 convolution with the number of channels unified to 256; Indicates that through Apply 5×5 dilated convolution to generate spatial attention map; Dynamic weights for Softmax normalization; adaptively assign the importance of multi-scale features through Softmax; represents the Hadamard product.

[0008] In a possible implementation, in step 4, the DB module includes a probability map branch and a threshold map branch; the probability map branch and the threshold map branch are two independent convolutional branch networks, respectively used for Perform convolution and nonlinear activation to obtain a probability map and dynamic threshold maps ; Among them, the probability map Indicates the confidence probability that each pixel belongs to the foreground, dynamic threshold map Indicates the local threshold corresponding to each pixel position; Based on probability graph and dynamic threshold maps , output binary segmentation mask through differentiable binarization , its mathematical expression is as follows: ; in: is the binarization slope factor.

[0009] In a possible implementation, in step 5, the centroid position of the binary segmentation mask The calculation formula is: ; in, is the threshold; Represents the set of all pixels that are judged to belong to the foreground; |Ω| is the set The number of pixels in the image.

[0010] In a possible implementation, in step 7, calculating a multi-scale feature descriptor based on filter response images at different directions and scales includes: For each feature point , the corresponding main direction index calculated : ; in, Is to use the Direction Log-Gabor filter of different scales Polar coordinate domain image Perform filtering to obtain a corresponding filtering response image; Feature points The corresponding main direction index; Indicates that Get the maximum value ; The main direction indexes corresponding to all feature points constitute a multi-scale feature descriptor ; The multi-scale feature descriptor Perform convolution and dimensionality reduction to obtain the RMLG descriptor ,include: ; ; in, Express Perform a periodic one-dimensional convolution operation, Express Perform pooling operations; and Represent the weight and bias of the fully connected layer respectively.

[0011] In a possible implementation, in step 8, the centroid position and multi-scale feature descriptors Perform tilt correction on the input image to obtain the tilt-corrected image ,include: ; ; Where, is an intermediate variable; Image after tilt correction The coordinates of the points in ; It represents the optimal rotation angle estimate of the sign in the input image. The number of times each direction (angle) appears in the MIM is counted, and the direction with the most occurrences is selected as .

[0012] In a possible implementation, the image after tilt correction based on the RMLG descriptor Perform character prediction to obtain a character prediction sequence corresponding to the input image, including: First, the tilt-corrected image Character area recognition is performed along the horizontal direction, and the character area is Split by column, and get the length Character region sequence ,in Indicates the image The feature vector of the character region, is the sequence length, equal to The number of character regions identified in ; Will As the 1st to the The input feature vector of time steps and the RMLG descriptor As the first The input feature vector of time steps is constructed as follows: ,in ; The character recognition input sequence is input into the bidirectional recurrent neural network Bi-RNN for sequence modeling, and the context representation of each time step is obtained, and then projected into the character category space through the fully connected layer, and finally Calculate the probability distribution vector of the character category for each character region: ; in, Is a probability distribution vector, indicating the The probability that a character region belongs to each character category, For the context representation of time steps; The weights of the fully connected layers of the neural network, is bias; For each character region, select the character category corresponding to the maximum probability value in the probability distribution vector of its character category, that is, the preliminary character prediction result of the character region; This gives us a preliminary character prediction sequence , which is composed as follows: ; in, It is Preliminary character prediction results for character regions.

[0013] In a possible implementation, step 8 further includes: obtaining a preliminary character prediction sequence Finally, the geometric mean of the probability values corresponding to the preliminary character prediction results of all character regions is taken as the confidence score of the whole sentence character recognition: ; in, For the The maximum probability value in the probability distribution vector of the character category in the character area, that is, The probability value corresponding to the preliminary character prediction result of the character area, Indicates a continuous multiplication operation; like ≥ , then the preliminary character prediction sequence is considered Reliable, direct output ; like < , the initial character prediction sequence is considered unreliable, and the semantic post-processing module Corrected ( ) is triggered to correct the initial character prediction sequence. Make corrections; in, is the confidence threshold, and is an empirical parameter.

[0014] In a possible implementation, the trigger semantic post-processing module Corrected() corrects the character sequence, including: First Matches against predefined format specifications, checking Does it comply with the common naming conventions in the power industry (such as "voltage level-equipment number")? If so, After regular rule verification, the power equipment database is further used for fuzzy query, that is, fuzzy matching is performed in the power equipment database. If a matching record is found, it is confirmed to be correct and output If no matching record is found, it indicates a suspected error, and the closest record pair is selected. Substitute and get ; The semantically corrected character prediction sequence is the final output.

[0015] like If the regular rule verification fails, an error message will be output.

[0016] In one possible implementation, during the training phase, a sample image is obtained and used as an input image to execute steps 2 to 8 to train the parameters of each module in the present application and obtain optimal parameters for image recognition of the power sign to be detected.

[0017] In a second aspect, the present application provides a system for recognizing tilted electric signage based on deep learning, comprising: a memory and a processor; The memory is used to store computer programs; The processor is configured to call the computer program to execute the method described above.

[0018] In a third aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed on an electronic device, the electronic device implements the method described above.

[0019] In a fourth aspect, the present application provides a computer program product, including a computer program, which, when executed on an electronic device, enables the electronic device to implement the method described above.

[0020] The specific implementation methods of the second to fourth aspects of this application can refer to the implementation method of the first aspect above, and will not be repeated here.

[0021] This application proposes a system and method for recognizing tilted electric equipment signs based on deep learning. Through multi-module collaborative innovation, it solves key problems such as rotation adaptation, multi-scale occlusion, and illumination sensitivity in complex scenes. The MALG detector is used to construct rotation-robust key points, and the ASF module is combined to realize multi-scale dynamic feature fusion, effectively improving the performance of small target detection; the boundary positioning accuracy is optimized through the differentiable binarization of the DB module, and the polar coordinate circular convolution of the RMLG descriptor is used to ensure rotation-invariant matching. The character detection module deeply integrates lightweight CRNN and dynamic post-processing technology to achieve high-precision character recognition. This application can realize the recognition of tilted electric equipment signs in complex scenes with rotation, occlusion, multi-scale changes and illumination differences. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is the overall flow chart of the embodiment of this application. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution of the present application will be further described in detail below in conjunction with the embodiments and drawings of the present application.

[0024] The following will refer to Figure 1 Specific embodiments according to the present application are described.

[0025] Example 1: like Figure 1 As shown, the embodiment of the present application provides a method for recognizing tilted power signboards based on deep learning, comprising the following steps: Step 1: Obtain an image of the power sign to be detected as an input image; In some embodiments, the input image is an RGB image with a size of , which can be expressed as: .

[0026] Step 2: Filter the input image using a multi-directional and multi-scale Log-Gabor filter bank to obtain a multi-directional and multi-scale filtered image; extract key points based on the multi-directional and multi-scale filtered response image; and generate a spatial mask based on the position of the key points; In some embodiments, extracting key points based on the multi-directional and multi-scale filtered response image includes: Step 2.1: For each frequency scale filter response image, construct a composite gradient matrix: ; Where, For the first The filter response image of the frequency scale is used to construct the composite gradient matrix, that is, The composite gradient matrix of frequency scales, and Respectively represent the application of the input image The pixel point in the filtered response image obtained by the Log-Gabor filter in the direction The horizontal and vertical deviator responses at ; Represents pixel points Gaussian weight; Gaussian weight uses the mean , the variance is Gaussian function Calculated, in some embodiments, it can be set 0, =2.0~3.0, such as =2.5, which can balance positioning accuracy and noise immunity; is a Gaussian weighted window, the window size is adjusted according to the input image resolution. In some embodiments, the window size can be set to 9×9 to 15×15, such as 11×11; Step 2.2: Calculate the characteristic point response value matrix for each frequency scale composite gradient matrix: ; Where, Express request The determinant of reflects the curvature information of the local area; express The trace reflects the average gradient strength of the local area; For the The frequency-scale feature point response value matrix is used to screen significant feature points, that is, rotation-robust key points; is the keypoint response suppression coefficient (typical value 0.05), which is used to balance the contribution of the determinant and the trace and suppress the spurious response in the low-contrast area; Step 2.3: Perform weighted summation on the feature point response value matrices of different frequency scales to obtain the final feature point response value matrix; take the top N feature points in the final feature point response value matrix as key points to obtain the key point position set. .

[0027] The generating of the spatial mask based on the position of the key points includes generating the spatial mask based on the key point position set ; Spatial mask The size of is the same as that of the input image, where key points are marked as 1 and the rest of the points are marked as 0.

[0028] In this step, a multi-directional, multi-scale Log-Gabor filter bank serves as a MALG detector (Multi-scale Adaptive Log-Gabor detector). This filter performs multi-frequency and multi-directional filtering on the input image. The horizontal and vertical partial derivatives of the filtered response image are calculated for each scale and direction. These partial derivatives are Gaussian-weighted to produce a corresponding composite gradient matrix. The response strength is measured using determinants and traces, ultimately generating a comprehensive feature point response matrix. This allows for the extraction of direction-sensitive gradient responses. The final feature point response matrix is obtained by fusing the feature point response matrices at different scales. Keypoint selection is achieved by sorting the fused feature point response matrix and selecting the top N feature points with the highest response values to form a keypoint set. This enables the extraction of keypoints using a multi-channel autocorrelation strategy based on the Log-Gabor filter bank. These keypoints effectively represent important regions of the input image that are rotationally invariant and scale-robust.

[0029] The spatial position and intensity distribution of the keypoints selected in this step provide prior information for spatial attention, which guides the subsequent dynamic weighted fusion of multi-scale features. Specifically, the positions of these keypoints are used to generate a spatial mask to guide the ASF module's attention allocation during the feature fusion stage, enabling more effective capture of target area features during the fusion process, especially small objects and occluded areas of tilted signs.

[0030] In some embodiments, N=100.

[0031] In some embodiments, the multiple directions include 6 directions, which are respectively denoted as , where each direction is 30° apart, and the six directions are 0°, 30°, 60°, 90°, 120°, and 150°.

[0032] In some embodiments, the different frequency scales include , the unit is cycles / pixel. =1,2,3,4.

[0033] This step has the following effects: Rotation invariance: the filter bank covers multiple (e.g. 6) evenly spaced directions Covering the full angle range, ensuring feature consistency at different angles; Multi-scale adaptability: This system addresses sign size variations based on multiple (e.g., four) frequency scales, capturing textures on signs of varying sizes. For example, a low-frequency scale (e.g., 0.1 cycles / pixel) captures large-scale structures (e.g., sign borders), while a high-frequency scale (e.g., 0.4 cycles / pixel) captures details (e.g., character edges). Noise immunity: The filter bank is designed to integrate multiple scales and dimensions (e.g., 4 scales × 6 directions) to cover the frequency domain characteristics of typical power scenarios. False responses in low-contrast areas are suppressed using the characteristic point response value matrix calculation formula. Key point screening: After sorting the feature point response value matrix from large to small, retain the first N feature points with high response values (high confidence) as key points, thereby balancing accuracy and computational efficiency.

[0034] Step 3: Extract multi-scale features from the input image based on the feature extraction module, dynamically weight and fuse the multi-scale features based on the Adaptive Scale Fusion module (ASF), and associate the spatial mask based on the spatial attention mechanism to obtain a multi-scale fusion feature map. ; In some embodiments, the feature extraction module uses a ResNet neural network; based on the ResNet neural network, feature maps of different scales are extracted from the input image, respectively representing spatial information at different semantic levels for multi-scale fusion; In some embodiments, It can be expressed as: ; in, Represents the neural network from ResNet feature maps of scales; in some embodiments =3; Express Perform 1×1 convolution with the number of channels unified to 256; Indicates that through Apply 5×5 dilated convolution (dilation rate 3) to generate the spatial attention map; is the dynamic weight of Softmax normalization, which adaptively distributes the importance of multi-scale features through Softmax. =3, 3 Typical values are 0.38, 0.47, and 0.15; represents the Hadamard product.

[0035] This step associates high-dimensional abstract features from the semantic layer of the ResNet neural network with keypoint information through spatial mapping. Specifically, the keypoint spatial location information first generates a spatial mask that reflects keypoint-dense areas in the input image. This spatial mask is then mapped to the spatial attention mechanism of the ASF module to guide weight allocation during feature fusion.

[0036] This step has the following effects: The keypoint position information extracted by the MALG detector serves as one of the input features of the ASF module. The ASF module fuses multi-scale features using dynamic weights, including rich contextual information (such as small object details and the semantics of occluded regions), enhancing the representation of small objects and occluded regions. The keypoint orientation information provides a prior for the ASF attention mechanism, guiding the weight allocation for feature fusion.

[0037] Dynamic weight allocation: The ASF module uses dilated convolution to generate a spatial attention map and a dynamic weight allocation mechanism with softmax normalization to adaptively fuse multi-scale features, significantly improving the detection performance of small objects and occluded scenes. Dilated convolution enhances receptive field: capturing long-distance dependencies without increasing computational effort; Performance improvement: Experiments show that for small objects (32×32 pixels), the recall rate can be improved by 29%, and the average precision can be increased from 67.2% to 86.7%.

[0038] Step 4: Use the Differentiable Binarization (DB) module to fusion the feature map based on the multi-scale , generate a binary segmentation mask; The fused feature map output by ASF is fed into the DB module, where it is binarized to generate a high-precision segmentation mask. The multi-scale features of ASF provide rich contextual information for DB, and the dynamic weighting mechanism optimizes the contribution of features at different scales to binarization.

[0039] The DB module includes a probability map branch and a threshold map branch; the probability map branch and the threshold map branch are two independent convolutional branch networks, respectively used for Perform convolution and nonlinear activation to obtain a probability map and dynamic threshold maps ; Among them, the probability map Indicates the confidence probability that each pixel belongs to the foreground, dynamic threshold map Represents the local threshold corresponding to each pixel position; specifically: The probability map branch uses a single layer of 1×1 convolution followed by a Sigmoid activation function, based on the fused multi-scale fusion feature map , generating a probability map , its mathematical expression is: ; Where σ( ) is the Sigmoid activation function, Indicates the use of Perform a 1×1 convolution operation, is the convolution weight; The threshold map branch adopts a three-layer 1×1 convolution layer structure, each layer is connected to the ReLU (rectified linear unit) activation function, based on the fused multi-scale fusion feature map , generate dynamic threshold map , its mathematical expression is as follows: ; ; ; in, represents the convolution weight of the probability map branch, 、 、 They are three sets of convolution weights for the threshold map branches, and their sets can be uniformly recorded as , . and is an intermediate variable; Fusion feature map The preliminary feature map after the first layer of convolution and nonlinear activation is used to extract local context information; is the intermediate expression feature obtained by further convolution compression, that is The intermediate feature map after the second layer of convolution and nonlinear activation; for The final dynamic threshold map after the third layer of convolution and nonlinear activation. ReLU ( ) is the ReLU activation function, a nonlinear activation function used to constrain the output range to [0, 1].

[0040] The two branch structures of probability map branch and threshold map branch are used to compress the channel dimension and generate single-channel prediction results.

[0041] The convolution weights and During the model training process, learning is performed through an end-to-end back-propagation mechanism. At the beginning of training, the convolution weights are initialized using the Xavier initialization strategy, and then the model output is calculated through forward propagation, and then compared with the segmentation mask real label to calculate the loss function. , where the binary cross entropy loss is .

[0042] Automatically solve the loss function through the back-propagation algorithm The gradient of the weights is calculated and the convolution weights are iteratively updated by the SGD (stochastic gradient descent) optimizer, eventually making and Converging to the optimal solution that minimizes the loss, the convolution weight update formula is as follows: ; ; in, and Respectively represent Hedi The convolution weights in the iteration The value of and Respectively represent Hedi Convolution weights in the iteration The value of is the update rate; Based on probability graph and dynamic threshold maps , a high-precision binary segmentation mask is output through differentiable binarization, and its mathematical expression is as follows: ; in: It is the binary segmentation mask output by the DB module; is a binary slope factor, is an empirical parameter, and in some embodiments, It can be set to a fixed value of 50.

[0043] This step has the following effects: End-to-end training: Jointly optimizes the probability map branch and the threshold map branch to improve boundary localization accuracy; Lightweight design: The threshold map branch uses 3 layers of 1×1 convolution (128→64→1 channels), reducing the computational complexity by 18%; Enhanced robustness: Dynamic thresholding adapts to uneven illumination and blurred boundaries, and the binary segmentation mask is more stable.

[0044] Binarized segmentation mask output by DB module The dynamic threshold optimizes the segmentation boundary, can accurately segment the sign area, can improve the detection capability of complex scenes, can provide accurate center of mass for subsequent polar coordinate transformation, eliminate the influence of rotation on feature distribution, and the precise positioning capability of the segmentation mask ensures feature alignment during polar coordinate sampling, thus ensuring the accuracy of polar coordinate transformation.

[0045] Step 5: Calculate the centroid position of the binary segmentation mask , as the origin of polar coordinates; ; in, Is the threshold value, which is used to determine whether a pixel belongs to the target (foreground) area. , determine the pixel Belongs to the target area, otherwise the pixel is determined Belongs to the background area; Represents the set of all pixels that are judged to belong to the foreground, used to extract the centroid; the quantity |Ω| is the number of pixels in the set, which is the weighted mean factor in the centroid calculation to ensure that the result falls at the geometric center of the foreground area.

[0046] This formula uses the binary segmentation mask output by the DB module Calculate the centroid of the connected domain (signboard area) , as the reference center for polar coordinate transformation.

[0047] Step 6: Based on the determined polar coordinate origin position, the input image is polar transformed to obtain a polar coordinate domain image; In this step, the centroid position obtained in step 5 is used is the extreme point position, from Draw a ray as the polar axis, select a length unit and the positive direction of the angle (usually counterclockwise), establish a polar coordinate system, and perform polar coordinate transformation on the input image, which can be expressed as follows: ; in, is the polar diameter, is the polar angle, is the image after polar coordinate transformation, that is, the polar coordinate domain image; Step 7: Use multi-directional and multi-scale Log-Gabor filter banks to filter the polar coordinate domain images Perform filtering to obtain filter response images of different directions and scales; calculate multi-scale feature descriptors based on filter response images of different directions and scales ; Multi-scale feature descriptors Perform convolution and dimensionality reduction to obtain the RMLG descriptor ; In this step, the English name of RMLG descriptor is Rotation-invariant Multi-scale Log-GaborDescriptor, which is referred to as RMLG descriptor. Among them, using Direction Log-Gabor filter of different scales Polar coordinate domain image Perform filtering to obtain the corresponding filter response image , which can be expressed as: ; In some embodiments, ; ; The polar coordinate domain image is subjected to multi-directional and multi-scale Log-Gabor filtering to achieve rotation invariance. The multi-scale feature descriptor is calculated based on the filter response images of different directions and scales, including: For each feature point , the corresponding main direction index calculated : ; in, Feature points The corresponding main direction index; Indicates that Get the maximum value , that is, feature points In the multi-scale Log-Gabor filter responses in multiple directions, the direction index with the strongest feature point response value is used as the main direction index; In some embodiments, it corresponds to 6 directions (0°, 30°, 60°, 90°, 120°, 150°).

[0048] The main direction indexes corresponding to all feature points constitute a multi-scale feature descriptor .

[0049] In some embodiments, the multi-scale feature descriptor Perform convolution and dimensionality reduction to obtain the RMLG descriptor ,include: ; ; in, Express Perform a periodic one-dimensional convolution operation, The input data is the multi-scale feature descriptor MIM. The convolution kernel will circulate in the direction dimension to maintain the continuity of the direction information to simulate the periodic boundary of the polar coordinate feature. The output data is , that is, The directional feature map is obtained by performing a periodic one-dimensional convolution operation. A rotationally invariant directional feature representation constructed by performing periodic one-dimensional convolution operations; Express Perform a pooling operation to aggregate the directional feature map in the spatial dimension and output it as a vector to achieve feature dimensionality reduction; and Represents the weight and bias of the fully connected layer, which are used to convert the intermediate features after pooling into a fixed-length vector , It is a rotation-invariant feature vector used for subsequent character recognition or feature comparison.

[0050] In some embodiments, is a vector of length 128.

[0051] Step 8: Use the character detection module based on the centroid position and multi-scale feature descriptors Perform tilt correction (affine alignment) on the input image to obtain the tilt-corrected image ; Combined with RMLG descriptor for tilt-corrected image Perform character prediction to obtain a character prediction sequence corresponding to the input image.

[0052] The character detection module is used to complete a lightweight text recognition process that is deeply coupled with a binary segmentation mask.

[0053] In some embodiments, the character detection module is an OCR (Optical Character Recognition) module.

[0054] This step performs tilt correction and character recognition on the binarized sign area. Its input relies entirely on the output of the pre-processor module. A lightweight OCR process is designed for the text characteristics of power signage, achieving high-precision text recognition in complex scenarios through end-to-end optimization.

[0055] In some embodiments, the centroid position and multi-scale feature descriptors Perform tilt correction on the input image to obtain the tilt-corrected image , its mathematical expression is: ; Where, represents the input image; Represents the optimal rotation angle estimate of the sign in the input image. The optimal rotation angle estimate is obtained by Specifically, the number of occurrences of each direction (angle) in the MIM is counted, and the direction with the most occurrences is selected as ; Represents a de-skew operation, which is a geometric transformation that performs rotation and translation on an image. Specifically, it performs the following operations: ; ; Where, is an intermediate variable, which is the translation amount required to rotate the image around the center without shifting the position during tilt correction. The value is the translation value required to set the image center as the rotation center, and is calculated based on the rotation center and angle. Image after tilt correction The coordinates of the points in ; Image after tilt correction In the image, the sign area has been rotated back to the horizontal direction, with a more regular structure and consistent character orientation. This is the input image for the subsequent OCR module.

[0056] In the character prediction module, sequence modeling is used to perform character-by-character recognition.

[0057] In some embodiments, the tilt-corrected image is descripted based on the RMLG descriptor. Perform character prediction to obtain a character prediction sequence corresponding to the input image, including: First, the tilt-corrected image Character area recognition is performed along the horizontal direction, and the character area is Split by column, and get the length Character region sequence ,in Indicates the image The feature vector of the character region, is the sequence length, equal to The number of character regions identified in ; Will As the 1st to the The input feature vector of time steps and the RMLG descriptor As the first The input feature vector of the time step is the global direction guide vector, which guides the context modeling process of the subsequent sequence. The corresponding output does not participate in character prediction and confidence calculation. Therefore, the final constructed character recognition input sequence is: ,in ; The character recognition input sequence is input into the Bidirectional Recurrent Neural Network (Bi-RNN) for sequence modeling to obtain the context representation of each time step, and then projected into the character category space through the fully connected layer, and finally Calculate the probability distribution vector of the character category for each character region: ; in, Is a probability distribution vector, indicating the The probability that a character region belongs to each character category, For the context representation of time steps; The weights of the fully connected layers of the neural network, is bias; Among them, Bi-RNN is an enhanced recurrent neural network (RNN) structure that considers both the forward and backward information of the input sequence, thereby capturing the completeness of the context and improving the ability to process sequence data; For each character region, select the character category corresponding to the maximum probability value in the probability distribution vector of its character category, that is, the preliminary character prediction result of the character region; This gives us a preliminary character prediction sequence , which is composed as follows: ; in, It is Preliminary character prediction results for character regions.

[0058] In some embodiments, step 8 further includes: obtaining a preliminary character prediction sequence Finally, the geometric mean of the probability values corresponding to the preliminary character prediction results of all character regions is taken as the confidence score of the whole sentence character recognition: ; in, For the The maximum probability value in the probability distribution vector of the character category in the character area, that is, The probability value corresponding to the preliminary character prediction result of the character area, Indicates a continuous multiplication operation; In the actual reasoning process, the system will score the confidence Used as a credibility evaluation indicator for character recognition results. If the confidence level is higher than the set threshold, the initial character prediction sequence is directly output; otherwise, the initial character prediction sequence is corrected through regular rule verification and fuzzy matching with the power equipment database, or the current result is directly discarded to avoid misrecognition when the confidence level is extremely low. like ≥ , then the preliminary character prediction sequence is considered Reliable, direct output ; like < , the initial character prediction sequence is considered unreliable, and the semantic post-processing module Corrected ( ) is triggered to correct the initial character prediction sequence. Make corrections; in, is the confidence threshold, and is an empirical parameter.

[0059] In a possible implementation, the trigger semantic post-processing module Corrected() corrects the character sequence, including: Combining domain prior information, regular rule matching and fuzzy query of the power equipment database (equipment ledger database) are used to perform semantic correction on the character sequence, and finally output the structured recognition result. The following discrimination mechanism is established: ; in, Indicates the comparison between the regular rule Regex and the power equipment database Perform error correction operations, including: Matches against predefined format specifications, checking Does it comply with the common naming conventions in the power industry (such as "voltage level-equipment number")? If so, After regular rule verification, the power equipment database is further used for fuzzy query, that is, fuzzy matching is performed in the power equipment database. If a matching record is found, it is confirmed to be correct and output If no matching record is found, it indicates a suspected error, and the closest record pair is selected. Substitute and get ; The semantically corrected character prediction sequence is the final output.

[0060] like If the regular rule verification fails, an error message will be output.

[0061] This step has technical advantages: End-to-end lightweight: CRNN parameters are compressed to less than 50KB, meeting embedded deployment requirements; Multimodal fusion: combining image features with language models to improve recognition robustness; Real-time optimization: Experiments show that the entire OCR process takes only 4.9ms and has an accuracy rate of 92.5%.

[0062] In some embodiments, during the training phase, a sample image is obtained and used as an input image to execute steps 2 to 8 to train the parameters of each module in the present application and obtain optimal parameters for image recognition of the power sign to be detected.

[0063] It should be understood that the above step numbers are only used to distinguish and facilitate the expression of different steps, and do not necessarily constitute a restriction on the order in which the steps are executed. The order in which the steps are executed should be determined by their functions and internal logic.

[0064] The embodiments of this application have the following features: 1) Rotation-robust keypoints are constructed based on a Log-Gabor filter bank (MALG detector), combined with polar coordinate circular convolution (RMLG descriptor) to achieve rotation-invariant matching; 2) A dynamic weighted fusion of multi-scale features based on the Adaptive Scale Fusion (ASF) module is designed, and the feature representation of small objects is enhanced through dilated convolution; 3) A differentiable binarization (DB) module and a lightweight convolutional recurrent neural network (CRNN) are introduced; all modules are jointly trained to achieve end-to-end optimization. For example, the threshold map branch of the DB module shares gradients with the feature map of the ASF module, and the rotation-invariance constraints of the RMLG descriptor are back-propagated to the MALG detector, improving overall rotation robustness and achieving end-to-end optimization and embedded deployment.

[0065] Therefore, the embodiments of the present application have the following advantages: constructing rotation-invariant feature primitives through the Log-Gabor filter group to solve the problem of geometric deformation adaptability; designing a dynamic weight multi-scale fusion mechanism to enhance the feature discrimination ability of the algorithm in lighting interference and occlusion scenarios; using differentiable binarization and polar coordinate descriptor collaborative optimization to reduce the computational complexity while maintaining accuracy, thereby achieving deep matching of algorithm performance and hardware resources.

[0066] Experimental verification: (1) Hardware deployment is: Platform: NVIDIA Jetson AGX Xavier; Input resolution: 640×640 RGB; Real-time performance: 35 FPS (TensorRT acceleration, FP16 quantization); Memory usage: <1.2GB.

[0067] (2) Algorithm flow

[0068] First, the original input image is preprocessed by normalizing its pixels to [0, 1] and randomly rotating it (e.g., by ±45°). This random rotation simulates text at different tilt angles, enhancing the model's generalization ability to rotational changes and motion blur enhancement. The input image is processed based on the solution in the above embodiment, and the performance indicators obtained are as follows: .

[0069] Experiments have shown that the technical solution of the embodiment of this application can increase the recognition rate of tilted samples to 94.3% (26.5% higher than the traditional method), and the accuracy rate under strong light interference reaches 89.7%. The overall OCR process takes only 4.9ms, which is significantly better than the existing solution.

[0070] It can be seen that the method for recognizing tilted electric equipment signs based on deep learning proposed in this application solves key problems such as rotation adaptation, multi-scale occlusion, and light sensitivity in complex scenes through multi-module collaborative innovation. The MALG detector is used to construct rotation-robust key points, and the ASF module is combined to realize multi-scale dynamic feature fusion, effectively improving the performance of small target detection; the boundary positioning accuracy is optimized through the differentiable binarization of the DB module, and the polar coordinate circular convolution of the RMLG descriptor is used to ensure rotation invariant matching. The character detection module deeply integrates lightweight CRNN and dynamic post-processing technology to achieve high-precision character recognition. Through end-to-end optimization and multi-modal feature fusion, this application provides an efficient and robust recognition solution for complex power scenarios.

[0071] Example 2: This embodiment provides a system for recognizing tilted electric power signs based on deep learning, including: a memory and a processor; The memory is used to store computer programs; The processor is configured to call the computer program to execute the method described in the first embodiment.

[0072] Example 3: This embodiment provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed on an electronic device, the electronic device implements the method according to the first embodiment.

[0073] Example 4: This embodiment provides a computer program product, including a computer program. When the computer program is run on an electronic device, the electronic device implements the method described in the first embodiment.

[0074] The specific implementation methods of a system, electronic device, computer-readable storage medium, and computer program product provided in the embodiments of the present application can refer to the specific embodiments of the above-mentioned method and will not be repeated here.

[0075] Obviously, those skilled in the art should understand that the above-mentioned units or steps of the present application can be implemented using a general-purpose computing device. They can be concentrated on a single computing device or distributed across a network composed of multiple computing devices. Alternatively, they can be implemented using program codes executable by the computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0076] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A method for recognizing tilted electric power signs based on deep learning, characterized in that: include: Step 1: Obtain an image of the power sign to be detected as an input image; Step 2: Filter the input image using a multi-directional and multi-scale Log-Gabor filter bank to obtain a multi-directional and multi-scale filtered image; extract key points based on the multi-directional and multi-scale filtered response image; Generate spatial masks based on the locations of key points; Step 3: Extract multi-scale features from the input image based on the feature extraction module, dynamically weight and fuse the multi-scale features based on the adaptive scale fusion ASF module, and associate the spatial mask based on the spatial attention mechanism to obtain a multi-scale fusion feature map. ; Step 4: Use the differentiable binary DB module to fuse the feature map based on the multi-scale , generate a binary segmentation mask ; Step 5: Calculate the binary segmentation mask The center of mass position , as the origin of polar coordinates; Step 6: Based on the determined polar coordinate origin position, the input image is polar transformed to obtain a polar coordinate domain image; Step 7: Use multi-directional and multi-scale Log-Gabor filter banks to filter the polar coordinate domain images Perform filtering to obtain filter response images of different directions and scales; calculate multi-scale feature descriptors based on filter response images of different directions and scales ; Multi-scale feature descriptors Perform convolution and dimensionality reduction to obtain the RMLG descriptor ; Step 8: Use the character detection module based on the centroid position and multi-scale feature descriptors Perform tilt correction on the input image to obtain the tilt-corrected image ; Combined with RMLG descriptor for tilt-corrected image Perform character prediction to obtain a character prediction sequence corresponding to the input image.

2. The method according to claim 1, characterized in that In step 2, extracting key points based on the multi-directional and multi-scale filtered response image includes: Step 2.1: For each frequency scale filter response image, construct a composite gradient matrix: ； Where, For the first The filter response image of the frequency scale is used to construct the composite gradient matrix, that is, The composite gradient matrix of frequency scales, and Respectively represent the application of the input image The pixel point in the filtered response image obtained by the Log-Gabor filter in the direction The horizontal and vertical deviator responses at ; Represents pixel points Gaussian weights, is a Gaussian weighted window; Step 2.2: Calculate the characteristic point response value matrix for each frequency scale composite gradient matrix: ； Where, Express request The determinant of express traces; For the The characteristic point response value matrix of frequency scale; is the key point response suppression coefficient; Step 2.3: Perform weighted summation on the feature point response value matrices of different frequency scales to obtain the final feature point response value matrix; take the top N feature points in the final feature point response value matrix as key points to obtain the key point position set. .

3. The method according to claim 1, characterized in that In step 3, the feature extraction module uses a ResNet neural network; ； in, Represents the neural network from ResNet Feature maps of different scales; Express Perform 1×1 convolution with the number of channels unified to 256; Indicates that through Apply 5×5 dilated convolution to generate spatial attention map; Dynamic weights for Softmax normalization; adaptively assign the importance of multi-scale features through Softmax; represents the Hadamard product.

4. The method according to claim 1, wherein In step 4, the DB module includes a probability map branch and a threshold map branch; the probability map branch and the threshold map branch are two independent convolutional branch networks, respectively used for Perform convolution and nonlinear activation to obtain a probability map and dynamic threshold maps ; Among them, the probability map Indicates the confidence probability that each pixel belongs to the foreground, dynamic threshold map Indicates the local threshold corresponding to each pixel position; Based on probability graph and dynamic threshold maps , output binary segmentation mask through differentiable binarization , its mathematical expression is as follows: ； in: is the binarization slope factor.

5. The method according to claim 1, wherein In step 5, the centroid position of the binary segmentation mask The calculation formula is: ； in, is the threshold; Represents the set of all pixels that are judged to belong to the foreground; |Ω| is the set The number of pixels in the image.

6. The method according to claim 5, characterized in that In step 7, the multi-scale feature descriptor is calculated based on the filter response images of different directions and scales, including: For each feature point , the corresponding main direction index calculated : ； in, Is to use the Direction Log-Gabor filter of different scales Polar coordinate domain image Perform filtering to obtain a corresponding filtering response image; Feature points The corresponding main direction index; Indicates that Get the maximum value ; The main direction indexes corresponding to all feature points constitute a multi-scale feature descriptor ; The multi-scale feature descriptor Perform convolution and dimensionality reduction to obtain the RMLG descriptor ,include: ； ； in, Express Perform a periodic one-dimensional convolution operation, Express Perform pooling operations; and denote the weight and bias of the fully connected layer respectively.

7. The method according to claim 6, characterized in that In the step 8, the centroid position and multi-scale feature descriptors Perform tilt correction on the input image to obtain the tilt-corrected image ,include: ； ； Where, is an intermediate variable; The image after tilt correction The coordinates of the points in ; represents the optimal rotation angle estimate of the sign in the input image, counts the number of times each direction appears in the MIM, and selects the direction with the most occurrences as .

8. A system for recognizing tilted electric signage based on deep learning, characterized in that: include: memory and processor; The memory is used to store computer programs; The processor is configured to call the computer program to execute the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed on an electronic device, the electronic device implements the method according to any one of claims 1 to 7.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed on an electronic device, the electronic device implements the method according to any one of claims 1 to 7.

Images