An image feature enhancement method, object recognition method and system

Abstract

The application discloses an image feature enhancement method, an object recognition method and a system. The image feature enhancement method fuses three types of key information by adding a multi-source context reasoning graph convolution network. The first type is spatial detail information captured by multi-scale global visual features of an image. The second type is local image features and semantic embedding information of various categories, which injects advanced semantic prior into visual representation, strengthens feature discrimination and context perception. The third type is co-occurrence relationship information in a data set, which explicitly models the spatial dependence between categories and improves the understanding ability of the model to complex scenes. Through the image feature enhancement method, the collaborative reasoning of visual, semantic and statistical information is finally realized, the robustness and category discrimination ability of the model are improved in complex scenes, and the feature confusion problem caused by insufficient single visual representation ability and intra-class difference is alleviated.

Description

Technical Field

[0001] This invention relates to the field of image processing, and specifically to an image feature enhancement method, an object recognition method, and a system. Background Technology

[0002] By using drones to inspect power transmission lines and performing object recognition on the power inspection images captured by the drones, real-time defects in the power transmission lines can be quickly located.

[0003] However, in practice, it has been found that aerial power line inspection images have complex backgrounds, often including vegetation, buildings, and other elements, resulting in diverse and complex textures. When the background and objects are highly similar in color or texture, the object's boundaries are easily confused with the background, making it difficult for detection algorithms to accurately identify and locate objects. Furthermore, aerial power line inspection images contain numerous multi-scale targets with significant morphological differences. At the acquisition level, due to the limited shooting distance of drones, equipment and defects in the images are generally small, lack detail, and are easily affected by background interference. At the sample level, objects such as utility poles are often long and narrow with significant aspect ratio differences, further increasing the difficulty of detection. Aerial power line inspection images also exhibit large differences in intra-class features. Similar components show significant differences in features due to factors such as the shooting angle, dense equipment occlusion, and material differences, easily leading to feature confusion and making it difficult for detection models to achieve stable recognition. Therefore, to meet the needs of specific application scenarios, a new image feature enhancement method is urgently needed. Summary of the Invention

[0004] This invention provides an image feature enhancement method, an object recognition method, and a system, which have excellent image feature enhancement capabilities in certain applications, thereby providing good technical support for object recognition and ensuring the accuracy of object recognition.

[0005] Accordingly, the present invention provides an image feature enhancement method, comprising:

[0006] Obtain the feature tensor of the input image , For batch size, For the number of channels, Spatial resolution;

[0007] Obtain the pre-constructed category node feature matrix and normalized class adjacency matrix , The total number of categories, Embed dimensions for categories. , for Co-occurrence matrix of each category To add a self-loop to the adjacency matrix, for The degree matrix;

[0008] For the image feature tensor Perform adaptive max pooling and flatten to obtain the global image feature tensor. ;

[0009] The global image feature tensor The input is fed into a two-layer graph convolutional neural network to obtain the category prototype matrix.

[0010] , and These are the first learnable weight matrix and the second learnable weight matrix, respectively.

[0011] For the category prototype matrix By performing transpose alignment, we obtain the transpose category prototype matrix. ;

[0012] Calculate the semantic response matrix ;

[0013] The semantic response matrix Passing sequentially through the first fully connected layer and the output dimension is After the second fully connected layer, channel attention weights are generated using the sigmoid function. And reshaped into a dimension of Reshaping channel attention weights ;

[0014] Reshape the channel attention weights With image feature tensor Multiplying by each channel yields semantically weighted features. ;

[0015] The semantically weighted features Perform channel-wise dimensionality reduction and compare it with the image feature tensor after channel-wise dimensionality reduction. By stitching along the channel dimension, an enhanced image feature tensor is generated. .

[0016] Accordingly, the present invention also provides an object recognition method, comprising:

[0017] Based on the Backbone module, extract the P3, P4 and P5 feature maps of the target image P;

[0018] The image feature enhancement method described above is used to process the P3 feature map, P4 feature map and P5 feature map respectively to obtain the corresponding P3 reconstructed feature map, P4 reconstructed feature map and P5 reconstructed feature map;

[0019] After multi-scale information fusion processing of P3 recombination feature map, P4 recombination feature map and P5 recombination feature map based on Neck module, the first multi-scale feature map, the second multi-scale feature map and the third multi-scale feature map are obtained.

[0020] After decoupling the first multi-scale feature map, the second multi-scale feature map, and the third multi-scale feature map based on the Head module, the first decoupling file, the second decoupling file, and the third decoupling file are obtained.

[0021] The post-processing module performs joint processing on the first decoupled file, the second decoupled file, and the third decoupled file to generate a prediction detection box.

[0022] Accordingly, the present invention also provides an object recognition system, characterized in that, for implementing the object recognition method, it includes a Backbone module, an image feature enhancement module, a Neck module, a Head module and a post-processing module connected in sequence;

[0023] The image feature enhancement module is used to process the P3 feature map, P4 feature map and P5 feature map respectively through image feature enhancement methods to obtain the corresponding P3 recombined feature map, P4 recombined feature map and P5 recombined feature map.

[0024] In summary, this invention provides an image feature enhancement method, an object recognition method, and a system. This image feature enhancement method incorporates a graph convolutional network with multi-source contextual reasoning, fusing three types of key information: the first type is spatial detail information captured using multi-scale global visual features of the image; the second type is local image features and semantic embedding information of each category, injecting high-level semantic priors into visual representations and strengthening feature discrimination and context awareness; the third type is co-occurrence relationship information in the dataset, explicitly modeling spatial dependencies between categories and improving the model's understanding of complex scenes. Through this image feature enhancement method, collaborative reasoning of visual, semantic, and statistical information is ultimately achieved, improving the model's robustness and category discrimination ability in complex scenes, and alleviating the problem of insufficient single visual representation capabilities and feature confusion caused by intra-class differences. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the image feature enhancement system module structure according to an embodiment of the present invention.

[0026] Figure 2 This is a schematic diagram of the Backbone module structure according to an embodiment of the present invention.

[0027] Figure 3 This is a schematic diagram of the improved Neck module structure according to an embodiment of the present invention.

[0028] Figure 4This is a schematic diagram showing the distribution of objects in the TLCAD dataset.

[0029] Figure 5 This is a schematic diagram of the category morphology and intra-category differences of objects in the TLCAD dataset.

[0030] Figure 6 This is a schematic diagram showing the distribution of category label sizes for objects in the TLCAD dataset.

[0031] Figure 7 This is a schematic diagram illustrating the aspect ratio distribution characteristics of objects in the TLCAD dataset.

[0032] Figure 8 This is a schematic diagram illustrating the aspect ratio distribution characteristics of objects in the TLCAD dataset.

[0033] Figure 9 This is a schematic diagram of the object recognition results of the object recognition system for a sample image according to an embodiment of the present invention.

[0034] Figure 10 This is a schematic diagram of the Neck module under different branch structures during the ablation experiment. Detailed Implementation

[0035] To further illustrate the technical means and effects adopted by this application to achieve its intended purpose, the specific implementation methods, structures, features, and effects according to this application are described in detail below with reference to the accompanying drawings and preferred embodiments. In the following description, different "an embodiment" or "an embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0036] The image feature enhancement method, object recognition method, and system provided in this embodiment of the invention should be noted as follows: the purpose of the image feature enhancement method is to enhance the image feature tensor so as to better utilize the object recognition method and system to identify objects in the image. Therefore, the image feature enhancement method can be considered as an independently operating technical method or as a processing procedure in the object recognition method. To simplify the description, this embodiment of the invention will describe the complete execution flow of the object recognition method, which will include the relevant content of the image feature enhancement method.

[0037] Figure 1 This is a schematic diagram of the image feature enhancement system module structure according to an embodiment of the present invention.

[0038] Specifically, this invention discloses an image feature enhancement method. The algorithmic basis of this image feature enhancement method is the YOLOv11 model. For some of the specialized terminology involved, please refer to the relevant materials disclosed regarding the YOLOv11 model. This invention will only provide a detailed description of the improvements involved. The image feature enhancement system corresponding to the image feature enhancement method includes a preprocessing module, a backbone module, an image feature enhancement module, an improved neck module, a head module, and a post-processing module. The structure and function of each module are briefly described below. Specifically, the functional descriptions correspond to the method steps.

[0039] Preprocessing module

[0040] The preprocessing module is mainly used to resize images to a preset size and fill them with Letterboxes to maintain aspect ratio and avoid image distortion. Depending on the actual needs, it can also perform operations such as color enhancement and geometric transformation on the images.

[0041] Generally, based on the model's recommendation, the images will be formatted as H640xW640xC3 for output.

[0042] To facilitate differentiation, the image output from the preprocessing module is set as the target image P.

[0043] Backbone module: Figure 2 This is a schematic diagram of the Backbone module structure according to an embodiment of the present invention. In each module structure box of the schematic diagram, for clarity, the first line indicates the name of the module, the second line indicates the main parameters and main structure of the module, and the third line indicates the output format of the module.

[0044] According to the official YOLO11 code and technical documentation, the Backbone adopts a hierarchical convolution-C3k2-SPPF-C2PSA concatenated architecture. Taking YOLO11n (nano version) as an example, the Backbone module includes downsampling convolutional layers, C3k2 modules, C2PSA, SPPF and other unit structures.

[0045] Specifically, unlike earlier simple convolutions, YOLO11 uses optimized convolutional layers for downsampling. With each downsampling convolutional layer, the image resolution (H, W) is halved, while the number of channels (C) increases. This helps the model capture more complex patterns while reducing computational cost.

[0046] The C3k2 (Cross-Stage Partial Network with 3 Blocks and 2 Kernel Sizes) module is an improved structure in YOLO11 compared to the C2f module in YOLOv8. Based on the CSP (Cross-Stage Partial) concept, it consists of multiple stacked C3k bottleneck units. By optimizing the convolutional kernel configuration and feature splitting method, it further reduces the model's computational complexity while maintaining feature representation capabilities. In lightweight models such as YOLO11n, C3k units typically enable residual connections (shortcut=True) and use a smaller number of channels to enhance gradient propagation capabilities in deep networks and avoid performance degradation.

[0047] The C2PSA (Cross Stage Partial with Spatial Attention) module is a novel structure introduced at the backbone of the YOLO11n model. This module introduces the PSA (Position-Sensitive Attention) mechanism, which enhances the model's attention to important regions in the image (such as small or partially occluded objects) by emphasizing the spatial correlation in the feature maps.

[0048] The SPPF (Spatial Pyramid Pooling-Fast) module follows the traditional design, efficiently expanding the network's receptive field through serially stacked multi-scale max pooling operations (usually 5×5 pooling kernels), and enhancing the multi-scale feature fusion capability while maintaining low computational overhead, enabling the network to adapt to the detection needs of targets at different scales.

[0049] Unlike the existing YOLO11 application (which extracts three feature maps, P3, P4, and P5, from the Backbone module), this embodiment of the invention improves the Neck module. Therefore, in this embodiment, it is necessary to extract four feature maps, P2, P3, P4, and P5, from the Backbone module.

[0050] Because YOLO11n (nano version) is used as the base model for improvement, taking the input image P (H640×W640×C3) as an example, the corresponding output formats for the P2 feature map are (160×160×64), P3 feature map (40×40×128), P4 feature map (20×20×128), and P5 feature map (20×20×256). Compared to the non-nano version, the number of channels is reduced by 0.25 times.

[0051] In this embodiment of the invention, the structure of the Backbone module is not substantially different from that of the Backbone module in the existing YOLO11 model. The main difference lies in the fact that the P2 feature map is required in subsequent process steps. Therefore, the output data of the Backbone module includes the P2 feature map.

[0052] Image feature enhancement module: Structurally, the image feature enhancement module essentially includes three image feature enhancement modules, namely P3 image feature enhancement module, P4 image feature enhancement module, and P5 image feature enhancement module. The three image feature enhancement modules have essentially the same form and structure. However, in order to adapt to the dimensions of P3 feature map, P4 feature map, and P5 feature map, there are certain differences in the relevant content of data format in practical applications. The following describes the image feature enhancement module of the embodiment of the present invention.

[0053] Specifically, the image feature enhancement module processes data using an image feature enhancement method, which includes:

[0054] S101: Obtain the feature tensor of the input image ;

[0055] For batch size, For the number of channels, Spatial resolution;

[0056] It should be noted that, in the embodiments of the present invention The parameters do not refer to specific objects; they are only used to express the size format of tensors / matrices.

[0057] S102: Obtain the pre-constructed category node feature matrix and normalized class adjacency matrix ;

[0058] Specifically, The total number of categories, Embed dimensions for categories. , for Co-occurrence matrix of each category To add a self-loop to the adjacency matrix, for The degree matrix;

[0059] Specifically, the total number of categories is determined for a specific application scenario and represents the number of objects that need to be identified and classified in object recognition. The image feature enhancement method in this embodiment of the invention can be understood as incorporating prior knowledge about categories to enhance some category-related features in the image; the category embedding dimension can be understood as how many dimensions are used to describe each category.

[0060] Specifically, the category node feature matrix and the normalized category adjacency matrix are prepared for its use in graph convolutional neural networks.

[0061] For the normalized class adjacency matrix, specifically, in the training set, the number of times any two classes appear simultaneously in the same image is counted to obtain the co-occurrence matrix. , and represent as

[0062] ;

[0063] in, Indicates category With category The number of times they appear together in the same image , .

[0064] To prevent uneven weighting caused by differences in node degree during information propagation, and to enable nodes to retain their own characteristic information, self-loops are added to the co-occurrence matrix and degree normalization is performed.

[0065] The adjacency matrix after adding self-loops is defined as follows:

[0066] ;

[0067] for The identity matrix and the corresponding degree matrix are:

[0068] ;

[0069] Finally, the normalized adjacency matrix is ​​expressed as:

[0070] ;

[0071] Normalization can effectively mitigate the impact of node degree differences on feature propagation, balance the weight distribution of each node in feature aggregation, and improve the training stability and representation consistency of the model.

[0072] For the category node feature matrix, specifically, the category node feature matrix of this embodiment of the invention. From word vector matrix and instance-level feature matrix The result is obtained by combining the two.

[0073] Specifically, word vector matrix Generated based on CLE (Category-Label Embeddings), but can actually be generated using the GloVe model. The specific expression is as follows: .

[0074] Specifically, instance-level feature matrix Based on IFR (Instance-level Feature Representation), specifically, after cropping the RoI regions of the image, instance-level features are extracted using the ResNet34 module. To align with the word vector matrix, an encoding layer is typically added to the ResNet34 module; for example, a 1×d encoding layer is added in this embodiment to obtain the instance-level feature matrix. .

[0075] S103: For the image feature tensor Perform adaptive max pooling and flatten to obtain the global image feature tensor. ;

[0076] Specifically, first consider the image feature tensor. Perform global adaptive max pooling processing ,get , The output size is 1×1, and this step compresses the spatial dimension of each sample to 1×1. ,get ;

[0077] Flatten Spatial Dimensions ,Will Spatial dimension from Transform into a global image feature tensor of .

[0078] S104: Convert the global image feature tensor The input is fed into a two-layer graph convolutional neural network to obtain the category prototype matrix;

[0079] Specifically, in this embodiment of the invention, the graph convolutional neural network adopts a two-layer GCN structure, namely...

[0080] and ;in, It is a non-linear activation function. For the normalized class adjacency matrix, For the category node feature matrix, and These are the first and second learnable weight matrices, respectively.

[0081] In the first-layer GCN, In the second-layer GCN, .

[0082] Specifically, in the first-layer GCN, the relationships between categories are utilized. Propagate information, expand semantic representation, and input node feature matrix The number of output channels can be expanded to 1024*0.25; in the second-layer GCN, Number of channels and number of channels in the image feature tensor Alignment.

[0083] S105: On the category prototype matrix By performing transpose alignment, we obtain the transpose category prototype matrix. ;

[0084] Specifically, for Perform transpose alignment dimension, let ,Will Dimensions from Transpose ;

[0085] S106: Calculate the semantic response matrix ;

[0086] Specifically, matrix multiplication is used to calculate the semantic similarity of image categories. Based on the format in the two matrices, the final image category semantic similarity is calculated. The matrix dimension is In this step, a "semantic activation" score for each sample for each category is given.

[0087] S107: Based on the semantic response matrix Obtain the reshaped channel attention weights ;

[0088] Image category semantic similarity Mapping to a higher dimension for intermediate representation yields... Dimensions adjusted to ;

[0089] The intermediate representation is then mapped back to the channel dimension to obtain the final attention matrix. Dimensions adjusted to ;

[0090] Generate channel attention weights based on the attention matrix. ;

[0091] Reshape channel attention weights Dimensions from Adjusted to ;

[0092] S108: Reshape the channel attention weights With image feature tensor Multiplying by each channel yields semantically weighted features. ;

[0093] Specifically, in this embodiment of the invention, the original feature map is semantically channel weighted to obtain a first reconstructed feature map. , This indicates broadcasting.

[0094] The Squeeze-and-Excitation (SE) module is used to enhance the first recombination feature map. The SE module will model channel dependencies again, which generally includes global average pooling, two layers of MLP and one layer of sigmoid;

[0095] S109: Weight the semantic features Perform channel dimensionality reduction;

[0096] The second recombined feature map is obtained by dimensionality reduction projection of the first recombined feature map. , Dimensions ;

[0097] S110: Image feature tensor Perform channel dimensionality reduction;

[0098] Similarly, the original input is also subjected to the same dimensionality reduction projection to obtain the residual branch graph. , Dimensions ;

[0099] S111: Segmenting residual branch diagrams along the channel dimension Second recombination feature map Generate enhanced image feature tensors .

[0100] Finally, the weighted feature output map is obtained by concatenating along the channel dimension. The dimension is consistent with the input image feature tensor. .

[0101] It should be noted that some of the symbols used in the data processing process of steps S101 to S111 are function names of the PyTorch framework. The embodiments of the present invention have provided a brief description based on the functions involved. In specific applications, other computer code can be selected to implement the functions as needed.

[0102] In this image feature enhancement module, the feature matrix of the category nodes is first embedded and propagated on the normalized category adjacency matrix through a graph convolutional neural network (GCN) to generate a category prototype matrix aligned with the channels of the image feature tensor. Then, the semantic similarity between the global image feature tensor and the category prototype matrix is ​​used to map the matrix into channel attention weights through a multilayer perceptron. Finally, the channel attention weights are used to perform semantic modulation on the original features and are concatenated and fused with the dimensionality-reduced original features to output the enhanced image feature tensor.

[0103] For the data itself, the original image feature tensor It only includes visual patterns (edges, textures, parts, etc.), without explicit category language, and enhances the image feature tensor. Prior relationships between categories are injected through category prototypes propagated in graph convolutional neural networks. During data processing, irrelevant channels are dynamically suppressed and relevant channels are enhanced based on the matching degree between the current image content and category semantics, resulting in a higher signal-to-noise ratio for the output features. Redundant or interfering channels are weakened, achieving semantic-driven channel attention modulation. This also strengthens the image feature tensor. Through the second recombination feature map after dimensionality reduction and the residual branch graph after dimensionality reduction The splicing process avoids information loss due to attention misjudgment and introduces new semantic guidance clues, preserving the original information while injecting enhanced signals. In zero-sample or few-sample scenarios, because the feature and category embeddings are aligned, it can generalize to unseen categories, making the feature distribution more compact within classes and separated between classes more obvious, thus improving the discriminativeness and generalization ability of features.

[0104] Correspondingly, in the image feature enhancement module, the P3 feature map is transformed into the Q3 reconstructed feature map after being processed by the P3 image feature enhancement module, the P4 feature map is transformed into the Q4 reconstructed feature map after being processed by the P4 image feature enhancement module, and the P5 feature map is transformed into the Q5 reconstructed feature map after being processed by the P5 image feature enhancement module.

[0105] Improved Neck module: Figure 3 This is a schematic diagram of the improved Neck module structure according to an embodiment of the present invention. Specifically, compared to the PAFPN structure introduced in the original YOLOv11 model's Neck module, the improved Neck module used in this embodiment of the present invention has undergone the following optimizations:

[0106] In the Neck module, a P2-scale branch of the shallow network is added, which introduces the P2 (1 / 4 scale) feature map with higher spatial resolution output by the Backbone. This more fully preserves fine-grained information such as the edges and textures of small objects, and improves the utilization of low-level fine-grained features by participating in cross-scale fusion.

[0107] By increasing the depth of the Neck module, unlike the traditional single downsampling and upsampling FPN / PAN structure, this embodiment of the invention achieves multiple interactive fusions of global and local features in a two-layer structure through alternating downlink and uplink paths;

[0108] The branch that independently extracts features at the P5 scale in the Neck module is removed. Specifically, since the P5 branch is located in the deep layers of the network, it is mainly used to extract more complex and abstract high-level features. After multiple convolutions and pooling, it focuses more on global semantics, and the spatial sensitivity decreases accordingly. It is difficult to retain the positional information and boundary details of small targets. By removing the P5 branch, we can avoid its excessively large receptive field from drowning local details in global context noise, while reducing the complexity and parameter overhead of the model.

[0109] The improved Neck module of this invention fully utilizes feature information by designing information jump interaction and fusion between multiple layers, making up for the loss during feature transmission and sampling, and enhancing the retention of key feature information.

[0110] Accordingly, the structure of the improved Neck module in this embodiment of the invention includes:

[0111] The first link includes a first 1×1 convolution kernel module, a first Concat module, a first C3k2 module, a second Concat module, a second C3k2 module, a third Concat module, and a third C3k2 module connected in sequence. The output of the first 1×1 convolution kernel module is also connected to the second C3k2 module, and the output of the first C3k2 module is also connected to the third Concat module.

[0112] The second link includes a second 1×1 convolution kernel module, a fourth Concat module, a fourth C3k2 module, a fifth Concat module, a fifth C3k2 module, a sixth Concat module, and a sixth C3k2 module connected in sequence. The output of the second 1×1 convolution kernel module is also connected to the fifth C3k2 module, and the output of the fourth C3k2 module is also connected to the sixth Concat module.

[0113] The third link includes a third 1×1 convolution kernel module and a first upsampling module connected in sequence. The output of the first upsampling module is connected to the fourth Concat module, the fifth Concat module and the sixth Concat module respectively.

[0114] The fourth link includes a first convolutional module, the output of which is connected to the second Concat module;

[0115] The upload link includes a second upsampling module, a third upsampling module, and a fourth upsampling module. The output of the fourth C3k2 module is connected to the input of the second upsampling module. The output of the second upsampling module is connected to the first Concat module and the second Concat module, respectively. The output of the fifth C3k2 module is connected to the input of the third upsampling module. The output of the sixth C3k2 module is connected to the input of the fourth upsampling module. The outputs of the third upsampling module and the fourth upsampling module are connected to the third Concat module, respectively.

[0116] The downlink includes a second convolutional module and a third convolutional module. The input of the second convolutional module is connected to the output of the first C3k2 module, and the output of the second convolutional module is connected to the fifth Concat module. The input of the third convolutional module is connected to the output of the second C3k2 module, and the output of the third convolutional module is connected to the fifth Concat module and the sixth Concat module, respectively.

[0117] Wherein, the input end of the first 1×1 convolution kernel module is the input interface of the P3 reconstructed feature map, the input end of the second 1×1 convolution kernel module is the input interface of the P4 reconstructed feature map, the input end of the third 1×1 convolution kernel module is the input interface of the P5 reconstructed feature map, and the input end of the first convolution module is the input interface of the P2 feature map.

[0118] The output of the second C3k2 module is the first multi-scale feature map. The output interface of the third C3k2 module is the second multi-scale feature map. The output interface of the sixth C3k2 module is the third multi-scale feature map. The output interface.

[0119] Specifically, in the above structural description of the improved Neck module, to avoid ambiguity, modules with the same structure but different locations are prefixed for distinction. In practical applications, modules with the same name, such as the Concat module and the C3k2 module, have the same function. Figure 3In the illustrated improved Neck module structure, for the sake of simplicity, each module is labeled with its basic module name or basic symbol structure. For details, the content of each basic module name or basic symbol structure and the specific implementation parameters can be found in Table 1.

[0120]

[0121] Table 1

[0122] It should be noted that the 1x1 convolutional kernel module performs channel compression, reducing computational load by compressing the number of channels in the input data. Correspondingly, in the first and second links, the adjustment of the number of channels only occurs within the 1x1 convolutional kernel module; the tensor resolution remains unchanged. Correspondingly, the first multi-scale feature map... Second multi-scale feature map The resolution is the same as Q3, first multi-scale feature map Second multi-scale feature map The number of channels is controlled by the first 1x1 convolutional kernel module; the third multi-scale feature map The resolution is the same as Q4, and the number of channels is controlled by the second 1x1 convolutional kernel module.

[0123] Head Module: Specifically, the Head module includes three decoupling modules corresponding to the first multi-scale feature map, the second multi-scale feature map, and the third multi-scale feature map, respectively named the first decoupling module, the second decoupling module, and the third decoupling module.

[0124] After the multi-scale feature map is input into the corresponding decoupling module, the decoupling module will output a data file containing detection box information, confidence information, and class probability information, which will be named the first decoupling file, the second decoupling file, and the third decoupling file, respectively.

[0125] The functions of the post-processing module include:

[0126] After flattening and concatenating the first, second, and third decoupled files into a tensor, the detection box information is decoded, and threshold filtering is performed on all detection box information using confidence information. Finally, the predicted detection boxes are output after non-maximum suppression.

[0127] The final detection box data, along with the corresponding confidence scores and category probability information, can be overlaid onto the input graph P.

[0128] It should be noted that the structure and content of the Head module and post-processing module are existing technologies and can be designed according to actual needs.

[0129] Loss Function: The entire object recognition system involves the application of neural networks. Therefore, in practical applications, the object recognition system needs to be trained first to achieve efficient object recognition results. The model structure has been explained above. During training, parameters need to be adjusted according to the loss function. In this embodiment of the invention, a novel method for calculating the loss function is provided for reference.

[0130] Specifically, the training loss function of the object recognition system is: ;

[0131] Center distance loss , To predict the coordinates of the center point of the detection box, The coordinates of the center point of the actual detection box. For Euclidean distance, Let be the diagonal length of the smallest closure region containing both the predicted and ground truth boxes. The width of the minimum closure region. The height of the minimum closure region;

[0132] Width and height loss , and These represent the width and height of the actual bounding box, respectively. and This indicates the width and height of the prediction box;

[0133] Crossover and comparison loss , This is the scaled predicted detection box. This is the scaled-down version of the actual detection bounding box, specifically...

[0134]

[0135]

[0136]

[0137]

[0138]

[0139]

[0140] in, This represents the area of ​​the intersection between the scaled predicted detection box and the ground truth detection box. This represents the area of ​​the union of the predicted detection box and the actual detection box after scaling. and These represent the left boundaries of the predicted and ground truth bounding boxes, respectively. and These represent the right boundaries of the predicted and ground truth bounding boxes, respectively. and These represent the upper boundaries of the predicted and ground truth bounding boxes, respectively. and These represent the lower boundaries of the predicted and ground truth bounding boxes, respectively.

[0141] The ratio represents the scaling factor variable. When ratio < 1, it indicates shrinkage; when ratio > 1, it indicates expansion. The scaling factor variable is a configurable quantity that needs to be precisely set to ensure the evaluation effect of the loss function.

[0142] Based on the foregoing description of the object recognition method and system of the present invention, the present invention provides a specific practical example for reference.

[0143] Specifically, drones were first used to collect images of the main and branch lines of a 10 kV transmission line in a certain area, covering approximately 700 towers. Two to four multi-angle photos were taken of each tower according to standard operating procedures, totaling 2,312 images, covering equipment and defect samples in the actual line. The image resolution of each photo varied from batch to batch due to differences in the collection process, mainly around 4000×3000 pixels. Finally, the images were compiled into the TLCAD dataset.

[0144] Figure 4 This is a schematic diagram showing the distribution of objects in the TLCAD dataset.

[0145] Based on the actual facilities and existing faults and foreign objects in the collected lines, all objects in the TLCAD dataset are divided into 9 types to be detected. Among them, suspension insulator (SI), composite insulator (CI), cross-arm insulator (CaI), pillar insulator (PilI), and pin insulator (PinI) are the detection of the five types of insulators that actually exist in the lines. pole is the detection of the pole body. Insulator defect (defect) is the damage on the insulator. nest is the bird nest and hole is the hole on the top of the pole.

[0146] Device components and defects were manually labeled using LabelImg. While ensuring that the number of images and labels were approximately 7:2:1, the dataset was divided into a training set (1629 images), a validation set (462 images), and a test set (221 images) to maintain a balanced distribution of samples in the dataset.

[0147] To comprehensively characterize the feature distribution and sample differences of the TLCAD dataset, a systematic analysis of the data was conducted from three aspects: visual representation, target size, and geometric morphology, which more intuitively reveals the diversity of different categories of samples in terms of appearance, scale, and shape.

[0148] Figure 5 This is a diagram illustrating the category morphology and intra-category difference morphology of objects in the TLCAD dataset, combined with... Figure 5 The illustration shows that objects of different categories have differences, but also have intuitive visual similarities, while different objects of the same category have certain differences while having similarities.

[0149] Figure 6 This diagram illustrates the size distribution of object category labels in the TLCAD dataset. Overall, small, medium, and large objects account for 24.00%, 48.60%, and 27.40% respectively. Small objects occupy less than 0.56% of the entire image area, medium objects range from 0.56% to 2.25%, and large objects exceed 2.25%. Large objects are mainly distributed in the categories of cross-arm insulator, pole, and nest, while the remaining categories are dominated by small and medium-sized objects. Among small objects, the defect category has the highest proportion.

[0150] Figure 7 This is a schematic diagram illustrating the aspect ratio distribution characteristics of objects in the TLCAD dataset.

[0151] Figure 8 This is a schematic diagram illustrating the aspect ratio distribution characteristics of objects in the TLCAD dataset.

[0152] Among them, the maximum width-to-height ratio (Width / Height) of the suspension insulator is 4.73, and the minimum is as low as 0.18; the maximum width-to-height ratio of the defect is 3.72, and the minimum is only 0.14; the maximum height-to-width ratio (Height-to-Width) of the pole is 30.01, and the minimum is only 0.41; the maximum height-to-width ratio of the pillar insulator is 14.37, and the minimum is 0.40. Specifically, the width-to-height ratio is used for targets that extend laterally, and the height-to-width ratio is used for targets that extend vertically, to more intuitively show the differences in geometric features of each type of target. The span from the minimum to the maximum value shows that different types of targets vary significantly in shape, ranging from small targets that are close to square to tall, extended pole targets, meaning that the model needs to adapt to different shapes within the same category.

[0153] To verify the performance of the object recognition system in this embodiment of the invention, the object recognition system is compared and analyzed with current mainstream target detection algorithms. The target detection algorithms used for comparison include YOLOv8 to YOLOv13, FCOS, Dynamic R-CNN, RT-DETR, Mamba-YOLO, LW-DETR, and Faster R-CNN. The comparison metrics include detection accuracy, inference speed, number of model parameters, and detection visual effect.

[0154] Detection accuracy:

[0155] The object recognition system (Ours) of this invention achieves a detection accuracy of mAP. 50 88%, mAP 50:95 The accuracy rate was 59.9%. The detection accuracy data of the other target detection algorithms participating in the comparison are shown in Table 2, where the bolded data is the best item in that column.

[0156]

[0157] Table 2

[0158] The object recognition system of this invention performs best among all comparison methods, outperforming YOLOv11n (baseline) by 12% and 8.3% respectively; mAP 50 It outperformed the second-ranked Faster R-CNN by 4.4% in mAP. 50:95 It is 5.9% higher than Mamba-YOLO, which ranks second.

[0159] The AP of the object recognition system in this embodiment of the invention 50 Among all comparison methods, CI, PinI, pole, defect, and nested classes showed the best accuracy. Compared to YOLOv11n (baseline), AP across nine classes was significantly higher. 50 All showed improvement, with the following specific increases: 4.2% (SI), 2.5% (CI), 2.4% (CaI), 3.5% (PilI), 3.7% (PinI), 0.7% (pole), 47.09% (defect), 37.2% (nest), and 6.2% (hole).

[0160] As shown in Table 2, the object recognition system of this invention demonstrates superior performance when handling small targets, targets with large differences in geometric proportions and intra-class features, and targets with severe background interference. This verifies its comprehensive capabilities in the detection of power transmission line components and abnormal conditions, provides effective technical support for the safety inspection and intelligent monitoring of power systems, and has good application potential.

[0161] Model inference speed, number of parameters, and computational complexity

[0162] The inference speed, number of parameters, and computational complexity of different models are summarized in Table 3.

[0163]

[0164] Table 3

[0165] The object recognition system of this invention is slightly slower than YOLOv11n (baseline), but while the accuracy is greatly improved, the detection speed can still reach 193.0 FPS, remaining above 100 FPS, which has high practical value and deployment potential.

[0166] Table 4 is a performance comparison diagram between the object recognition system of this invention and various versions of YOLO models.

[0167]

[0168] Table 4

[0169] Compared with the performance of various versions of YOLO models, the object recognition system of this invention not only maintains a leading position in detection accuracy, but also has advantages in parameter scale and computational complexity, demonstrating superior overall performance.

[0170] Visualization of test results: Figure 9 This diagram illustrates the object recognition results of the object recognition system for sample images according to an embodiment of the present invention. In the diagram, each predicted bounding box is labeled with the target location and its corresponding category. Compared to the detection performance of other models, the object recognition system of this embodiment offers advantages for different sample images, including:

[0171] In the detection of (a), the Mamba-YOLO, YOLOv12n and Faster R-CNN models all missed detections in the recognition of composite insulators with Width / Height differences and easy background confusion, and YOLOv10n showed duplicate detection of cross-arm insulators.

[0172] In the detection of (b), YOLOv10n and YOLOv13n both misidentified the background as pole, while Faster R-CNN missed detecting the pillar insulator.

[0173] In the detection of (c), only Mamba-YOLO, YOLOv13n and CFG-YOLO identified the insulator defect. However, Mamba-YOLO showed false positives for the pillar insulator and YOLOv13n showed repeated detections of the pillar insulator.

[0174] In the detection of (d) and (e), only CFG-YOLO captured the defect without error, and YOLOv13n missed the detection of the suspension insulator. Faster R-CNN missed and falsely detected the composite insulator.

[0175] In the detection of (f), only YOLOv10n and CFG-YOLO can achieve good recognition results for nest, which is easily confused with the background.

[0176] Based on the characteristics of the TDLCD dataset in this embodiment of the invention, which contains samples with complex backgrounds, small targets, and significant differences in length, width, and intra-class features, the object recognition system of this embodiment of the invention can still accurately detect various devices on this data, fully demonstrating the effectiveness and robustness of the proposed model in complex environments.

[0177] To further verify the effectiveness of the object recognition system in this embodiment of the invention, we designed an ablation experiment based on YOLOv11n (baseline) to verify the effectiveness and performance improvement of the image feature enhancement module, the improved Neck module, and the specially designed loss function for power equipment image detection.

[0178] Image Feature Enhancement Module: To verify the effectiveness of different modalities and granularities of features in the image feature enhancement module, ablation experiments were conducted on the image feature enhancement module in this embodiment of the invention. While maintaining the multi-scale global feature representation as the baseline, instance-level feature representation and word embedding of category labels were introduced. The experimental results are shown in Table 5. Introducing feature representations of any category can improve model performance; when introduced simultaneously, the image feature enhancement module achieved the best performance.

[0179]

[0180] Table 5

[0181] Improved Neck Module: In the Neck module, the number and location of skip-interaction fusion branches can affect detection accuracy. In the ablation experiments of this embodiment, various adjustments were made to the branch positions of the Neck module. Figure 10The diagram shows the structure of the Neck module under different branch structures in the ablation experiment. The dashed lines represent the branches that were deleted or modified in the experiment, and are numbered from R0 to R4. Multiple ablation experiments were carried out on the Neck module in the enhanced model CFG-YOLO, and the results are shown in Table 6. The Methods column represents the different combinations of the deleted branches.

[0182]

[0183] Table 6

[0184] When only branch R4 is deleted, mAP 50 and mAP 50:95 The decreases of 2.8% and 3.3% respectively indicate the important role of fine-grained P2 information in overall recognition. Further deletion of intermediate layer branches R3 and R2 further reduces mAP. 50:95 The further reduction indicates that the interaction between features across multiple layers is crucial for the full utilization of features. After removing branches R4 to R0, mAP... 50 and mAP 50:95 The accuracy dropped to 80.9% and 53.1% respectively, indicating a significant decrease in detection precision. Conversely, when all branches were retained (i.e., the improved Neck module of this embodiment), the model detection precision reached its optimal level, demonstrating that the improved Neck module of this embodiment can effectively enhance the information flow and utilization between multi-scale features.

[0185] Loss function: To determine the optimal scaling factor ratio parameter value of the loss function in the embodiments of the present invention, multiple ablation experiments were conducted in the embodiments of the present invention.

[0186] The scaling hyperparameter varies between 0.1 and 0.9 and is used to adjust the geometry and position of the auxiliary bounding box.

[0187] The results of the loss function ablation experiment are shown in Table 7. When the ratio is in the range of [0.2, 0.4], the detection accuracy only improves slightly. In contrast, when the scaling factor ratio is between [0.6, 0.9], the detection performance is significantly improved, with the best performance achieved at 0.7. Therefore, in this embodiment of the invention, a scaling factor of 0.7 is ultimately chosen as the bounding box regression loss parameter for the loss function.

[0188]

[0189] Table 7

[0190] Comprehensive ablation experiment:

[0191] This section is used to verify the effectiveness of the image feature enhancement module (X1), the improved Neck module (X2), and the loss function (X3) in the CFG-YOLO model. The results are shown in Table 8.

[0192]

[0193] Table 8

[0194] (a) indicates YOLOv11n (baseline) without any modifications to the network modules.

[0195] (b) indicates the addition of an image feature enhancement module, mAP, to YOLOv11n (baseline). 50 and mAP 50:95 The improvements of 6.4% and 2.7% respectively demonstrate that the image feature enhancement module effectively strengthens feature representation, enhances category discrimination ability, and alleviates feature confusion.

[0196] (c) indicates that the Neck module in (b) is replaced with an improved Neck module, mAP 50 and mAP 50:95 The improvements of 4.2% and 2.9% respectively indicate that the proposed improved Neck module structure enhances the utilization of fine-grained features and strengthens feature fusion between different scales and levels, thus enriching the feature space.

[0197] (d) Replace the loss function in (c) with the loss function of this embodiment, mAP 50 and mAP 50:95 The improvements of 1.4% and 2.7% respectively demonstrate that the loss function of this invention can optimize the regression and classification of the model, guide the model to capture target features and spatial location more effectively, and enhance the recognition and localization capabilities.

[0198] Based on the above descriptions of actual embodiments and the ablation experimental data of various projects, the image feature enhancement module, improved Neck module, and loss function proposed in the embodiments of the present invention all contribute to improving object detection performance.

[0199] In summary, this invention provides an image feature enhancement method, an object recognition method, and a system. The image feature enhancement method incorporates a graph convolutional network with multi-source contextual reasoning, fusing three types of key information: the first type is spatial detail information captured using multi-scale global visual features of the image; the second type is local image features and semantic embedding information of each category, injecting high-level semantic priors into visual representations and strengthening feature discrimination and context awareness; the third type is co-occurrence relationship information in the dataset, explicitly modeling spatial dependencies between categories and improving the model's understanding of complex scenes. Through this image feature enhancement method, collaborative reasoning of visual, semantic, and statistical information is ultimately achieved, improving the model's robustness and category discrimination ability in complex scenes, and alleviating the problems of insufficient single visual representation capabilities and feature confusion caused by intra-class differences. Based on this, an image recognition method is constructed. The proposed method and system, through an improved Neck module, introduces a P2 branch to compensate for feature detail information and employs a leapfrog cross-scale interaction mechanism to achieve multi-layer feature fusion. Furthermore, the simplification of the P5 branch structure and direct information transmission alleviate information redundancy and loss, ultimately enhancing the model's information integration capabilities and improving recognition accuracy. Further, EIoU is used in bounding box regression to introduce direct geometric constraints on the width and height of the predicted box, replacing CIoU's indirect constraint method based solely on aspect ratio. This enhances the model's ability to perceive target shapes, enabling refined modeling of bounding box shapes. Based on these geometric constraints, an Inner-IoU region-aware mechanism is introduced to achieve collaborative modeling of the internal overlap and external geometry of the bounding box, aiming to achieve accurate multi-scale target localization in complex aerial photography environments.

[0200] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An image feature enhancement method, characterized in that, include: Obtain the feature tensor of the input image , For batch size, For the number of channels, Spatial resolution; Obtain the pre-constructed category node feature matrix and normalized class adjacency matrix , The total number of categories, Embed dimensions for categories, , for Co-occurrence matrix of each category To add a self-loop to the adjacency matrix, for degree matrix, for The identity matrix; For the image feature tensor Perform adaptive max pooling and flatten to obtain the global image feature tensor. This includes first processing the image feature tensor. Global adaptive max pooling is performed to obtain flatten The spatial dimension will Spatial dimension from Transform into a global image feature tensor of ; The global image feature tensor The input is fed into a two-layer graph convolutional neural network (GCN) to obtain the category prototype matrix. The GCN employs a two-layer structure, namely... and ; Utilizing inter-category relationships in the first-level GCN Propagate information, expand semantic representation, and input node feature matrix In the second layer GCN, Number of channels and number of channels in the image feature tensor Alignment , , It is a non-linear activation function. For the normalized class adjacency matrix, For the category node feature matrix, and These are the first learnable weight matrix and the second learnable weight matrix, respectively. For the category prototype matrix By performing transpose alignment, we obtain the transpose category prototype matrix. ; Calculate the semantic response matrix ; The semantic response matrix Passing sequentially through the first fully connected layer and the output dimension is After the second fully connected layer, via Function generates channel attention weights And reshaped into a dimension of Reshaping channel attention weights This includes: the semantic response matrix The intermediate representation is mapped to a higher dimension; then it is mapped back to the channel dimension to obtain the final attention matrix, with the dimensions adjusted accordingly. ; Generate channel attention weights based on the attention matrix. Reshape channel attention weights Dimensions from Adjusted to ; Reshape the channel attention weights With image feature tensor Multiplying by each channel yields semantically weighted features. ; The semantically weighted features Perform channel-wise dimensionality reduction and compare it with the image feature tensor after channel-wise dimensionality reduction. By stitching along the channel dimension, an enhanced image feature tensor is generated. .

2. The image feature enhancement method as described in claim 1, characterized in that, The category node feature matrix ,in, For word vector matrix, This is the instance-level feature matrix.

3. The image feature enhancement method as described in claim 2, characterized in that, The word vector matrix Generated using the GloVe model.

4. The image feature enhancement method as described in claim 2, characterized in that, The instance-level feature matrix Generated based on the ResNet34 module.

5. The image feature enhancement method as described in claim 1, characterized in that, In the semantic weighted features Before channel dimensionality reduction, the semantically weighted features are processed using the Squeeze-and-Excitation module. Perform a second channel recalibration.

6. The image feature enhancement method as described in claim 1, characterized in that, The channel dimensionality reduction is achieved through a 1×1 convolution module.

7. An object recognition method, characterized in that, include: Based on the Backbone module, extract the P3, P4 and P5 feature maps of the target image P; The image feature enhancement method according to any one of claims 1 to 6 is used to process the P3 feature map, P4 feature map and P5 feature map respectively to obtain the corresponding P3 reconstructed feature map, P4 reconstructed feature map and P5 reconstructed feature map; After multi-scale information fusion processing of P3 recombination feature map, P4 recombination feature map and P5 recombination feature map based on Neck module, the first multi-scale feature map, the second multi-scale feature map and the third multi-scale feature map are obtained. After decoupling the first multi-scale feature map, the second multi-scale feature map, and the third multi-scale feature map based on the Head module, the first decoupling file, the second decoupling file, and the third decoupling file are obtained. The post-processing module performs joint processing on the first decoupled file, the second decoupled file, and the third decoupled file to generate a prediction detection box.

8. The object recognition method as described in claim 7, characterized in that, It also includes the P2 feature map of the target image P extracted based on the Backbone module; The multi-scale information fusion processing of the P3 recombination feature map, P4 recombination feature map, and P5 recombination feature map based on the Neck module includes: the multi-scale information fusion processing of the P2 feature map, P3 recombination feature map, P4 recombination feature map, and P5 recombination feature map based on the Neck module.

9. An object recognition system, characterized in that, The method for implementing the object recognition method of claim 7 or 8 includes a Backbone module, an image feature enhancement module, a Neck module, a Head module, and a post-processing module connected in sequence. The image feature enhancement module is used to process the P3 feature map, P4 feature map and P5 feature map respectively through image feature enhancement methods to obtain the corresponding P3 reconstructed feature map, P4 reconstructed feature map and P5 reconstructed feature map.

10. The object recognition system as described in claim 9, characterized in that, The training loss function of the object recognition system is: ; Center distance loss , To predict the coordinates of the center point of the detection box, The coordinates of the center point of the actual detection box. For Euclidean distance, Let be the diagonal length of the smallest closure region containing both the predicted and ground truth boxes. The width of the minimum closure region. The height of the minimum closure region; Width and height loss , and These represent the width and height of the actual bounding box, respectively. and This indicates the width and height of the prediction box; Crossover and comparison loss , This is the scaled predicted detection box. This is the scaled-down version of the actual detection bounding box.

Images