An open vocabulary small object detection method based on cascaded multi-level refinement

Abstract

The present invention relates to the technical field of open vocabulary small target detection, and discloses an open vocabulary small target detection method based on cascaded multi-level refinement, including obtaining power grid engineering drawings and all potential detection category sets of power grid infrastructure projects; using a multi-level refinement region proposal network to extract initial feature maps for the power grid engineering drawings, and generating candidate regions through a non-sampling anchor point strategy; using a cascade feature alignment network to align the candidate regions with regions of interest, and then sequentially performing coarse-grained feature alignment and fine-grained feature alignment with text features converted from all potential detection category sets to generate detection results for each candidate region. The present invention not only improves the accuracy, reliability and generalization ability of open vocabulary small target detection, but also improves the automation level, accuracy and efficiency of drawing verification, and effectively solves the problem of time-consuming, labor-intensive and error-prone manual verification.

Description

Technical Field

[0001] The present invention relates to the technical field of open vocabulary small target detection, and in particular to an open vocabulary small target detection method based on cascade multi-level refinement. Background Art

[0002] In the field of open-vocabulary object detection (OVD), many research methods have emerged in recent years, which attempt to expand the vocabulary of detectors through visual-language pre-training models (such as CLIP) so that they can recognize object categories that have not been seen during training during the test phase. Although these methods have made progress, they still face some significant technical challenges and limitations when dealing with small objects and open vocabulary scenarios.

[0003] Small target detection is particularly important in application scenarios such as power grid engineering drawings. In power grid engineering drawings, common small targets include various equipment identifiers, switches, transformer symbols, line connection points, etc. These small targets are not only small in size and occupy few pixels, but also may appear in different forms in different drawings. Since power grid engineering involves a large number of complex symbol systems and structured data, traditional closed vocabulary detectors have difficulty dealing with unseen small targets, which in turn affects the efficiency of automated drawing analysis and fault detection. Therefore, how to effectively detect small targets in power grid engineering drawings in open vocabulary scenarios has become an urgent problem to be solved.

[0004] At present, the technical solutions similar to the present invention "open vocabulary small target detection method based on multi-level refinement region proposal and cascade alignment" mainly include the following:

[0005] 1. RegionCLIP: "RegionCLIP: Region-based Language-Image Pretraining" attempts to leverage the knowledge of visual-language pre-trained models for open vocabulary object detection by applying the pre-trained region-text semantic alignment strategy of visual-language models (such as CLIP) to the detection task. This method leverages large-scale visual-language data to enhance the model's ability to recognize new categories, but RegionCLIP's performance is limited when dealing with small objects. This is mainly because the method relies on coarse-grained image-level feature alignment and ignores the fine-grained feature expression of small objects. Since many small objects in power grid engineering drawings occupy fewer pixels, their features are easily overwhelmed by the information of larger structures, resulting in a higher missed detection rate.

[0006] 2. OV-DETR: "Open-Vocabulary DETR with Conditional Matching" proposes a transformer-based open vocabulary object detection method that uses cross-modal interactions between visual encoders and text encoders to identify new categories of objects. Although OV-DETR has achieved good results in large object detection, its performance in small object detection is still poor. The small symbols in power grid engineering drawings are usually small in size. OV-DETR relies on a global self-attention mechanism, which makes it difficult to accurately capture these local features in high-resolution drawings, resulting in poor detection results.

[0007] 3. Detpro: “Learning to Prompt for Open-Vocabulary Object Detectionwith Vision-Language Model” proposes a method for open-vocabulary object detection using a vision-language model, which improves the detection capability of the model by enhancing the feature expression of specific targets during training. However, DetPro’s feature enhancement strategy is mainly aimed at regular-sized targets, and the enhancement effect is limited when dealing with small targets in power grid engineering drawings. In addition, this method has insufficient generalization ability when detecting small targets of unseen categories.

[0008] 4. PromptDet: "Towards Open-vocabulary Detection using Uncurated Images" proposes to use natural language prompts (Prompt) to enhance the ability of open-vocabulary object detection. It guides the detector to identify objects by generating natural language prompts during training. However, small symbols in power grid engineering drawings may be expressed in many ways. When PromptDet processes these symbols, the accuracy of prompt generation and matching is limited, resulting in missed detection and false detection of small objects.

[0009] 5. VLDet: "Learning Object-Language Alignments for Open-VocabularyObject Detection" uses visual-language alignment technology to improve open-vocabulary object detection. By formulating the task as a binary matching problem, the open-vocabulary object detector is trained directly from the image-text pair. However, the limitation of VLDet is that it is not able to capture the visual features of small objects, especially the symbol detection in power grid engineering drawings. Since small objects occupy fewer pixels in the image, their expression in the visual feature space is not sufficient. VLDet is prone to false detection and missed detection when detecting power grid engineering drawings, especially in the open vocabulary scenario, and its generalization ability for new categories of small objects is insufficient.

[0010] 6.GA-RPN and Cascade RPN: GA-RPN (Guided Anchoring Region Proposal Network) and Cascade RPN (Delving into high-quality region proposal network with adaptive convolution) are two common region proposal network (RPN) methods. These methods enhance detection performance by improving anchor generation strategies and optimizing the feature expression of candidate regions step by step. Among these methods, GA-RPN uses a guided anchor generation strategy to improve the quality of candidate regions, while Cascade RPN uses a multi-level feature refinement mechanism to gradually improve the accuracy of candidate regions.

[0011] 7. C-RPN: “Small Object Detection via Coarse-to-fine Proposal Generation and Imitation Learning” proposes an improved method for small object detection, which improves the accuracy of small object detection by adopting a coarse-to-fine candidate region generation strategy. This method uses a multi-level anchor selection mechanism in the region proposal process to ensure high coverage of small objects.

[0012] The above existing technical solutions have the following main disadvantages in the open vocabulary small object detection technology:

[0013] 1. Insufficient candidate region generation: When processing small targets in power grid engineering drawings, existing RPN methods (such as GA-RPN and Cascade RPN) generate insufficient number and quality of small target candidate regions. Especially for unseen small symbols, their candidate region proposal capability is limited, resulting in a high missed detection rate.

[0014] 2. Limited feature expression ability: In open vocabulary object detection, the visual features of small targets are often weak, and existing detection methods based on visual-language models (such as RegionCLIP and OV-DETR) are difficult to capture the fine-grained features of small targets. These methods mostly rely on coarse-grained feature alignment strategies, ignoring the feature expression of small targets in the image, especially the targets represented by complex small symbols in power grid engineering drawings, resulting in high false detection and missed detection rates.

[0015] 3. Lack of generalization ability for open vocabulary scenarios: Most existing methods rely on training data from closed vocabulary sets and lack the ability to generalize in open vocabulary scenarios. In particular, it is difficult to accurately identify and detect small symbols that have never been seen in power grid projects, showing significant limitations.

[0016] 4. Over-reliance on sampling strategies: Methods such as CRPN usually adopt sampling strategies to optimize computing resources and sample balance during training. However, this strategy will cause a large number of valid samples to be discarded in open vocabulary small target detection, resulting in many low-confidence and rare component symbol samples being ignored, limiting the model's ability to generalize and learn new targets in power grid drawings.

[0017] Based on the above analysis, it can be seen that most of the existing open vocabulary small target detection methods rely on existing feature extraction and matching mechanisms, and fail to fully consider the particularity of small targets in power grid engineering drawings in open vocabulary scenarios. Small targets in power grid engineering drawings, such as switches, transformer symbols, connection points, etc., are usually very sparse in visual features and appear small and complex compared to large background elements. Existing methods have significant deficiencies in generating candidate regions and feature alignment, making it difficult to accurately capture and identify these new categories of small targets. In addition, these methods have limited generalization capabilities for new categories of small targets, making it difficult to adapt to the open environment detection needs of diverse symbols and structures in power grid engineering drawings, thus significantly limiting the improvement of overall detection performance.

[0018] The existing open vocabulary small target detection technology has the following major defects in the application of power grid engineering drawings:

[0019] 1. Insufficient generation of candidate regions for small objects: Existing methods (such as RPN, GA-RPN and Cascade RPN) cannot generate sufficient and high-quality candidate regions when processing small objects (such as equipment identifiers, line connection points, etc.) in power grid drawings, resulting in a high rate of missed detection of small objects. These small objects are usually small in size and have large differences from surrounding elements, making it difficult for existing fixed anchor point strategies to effectively deal with them.

[0020] 2. Insufficient matching between visual features and text features: Most open vocabulary detection methods (such as RegionCLIP, OV-DETR, etc.) find it difficult to effectively align the visual features and text features of small targets in power grid engineering drawings. Especially when these symbols are small in size, it is difficult for the model to accurately capture their fine-grained features, which is prone to false detection and missed detection.

[0021] 3. Poor generalization ability for small objects of new categories: Existing technologies have limited performance in open vocabulary scenarios for small symbols and new categories that have not been seen in power grid engineering drawings. There is a lack of specialized optimization strategies for these small objects of new categories, resulting in low detection accuracy, especially for unique or rare symbols in drawings.

[0022] 4. Low sample utilization: The training strategy of existing methods leads to the discarding of many valid small target samples in power grid engineering drawings, especially target symbols of rare categories and low confidence. This limits the model's ability to learn these special symbols and also weakens the model's generalization ability and overall detection accuracy. Summary of the invention

[0023] In view of the above-mentioned deficiencies in the prior art, the present invention provides an open vocabulary small target detection method based on cascaded multi-level refinement to improve the accuracy, reliability and generalization ability of open vocabulary small target detection in power grid drawing verification.

[0024] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:

[0025] An open vocabulary small object detection method based on cascaded multi-level refinement, comprising the following steps:

[0026] Obtain power grid engineering drawings and all potential detection category sets of power grid infrastructure projects;

[0027] A multi-level refined region proposal network is used to extract initial feature maps from power grid engineering drawings, and candidate regions are generated through a non-sampling anchor strategy.

[0028] After aligning the candidate regions with the regions of interest using the cascade feature alignment network, the coarse-grained feature alignment and fine-grained feature alignment are performed in sequence with the text features converted from the set of all potential detection categories to generate the detection results for each candidate region.

[0029] Preferably, the multi-level refined region proposal network includes:

[0030] Feature extraction module, non-sampling anchor generation module, first convolution layer, second convolution layer, feature conversion module and proposal decoding module;

[0031] The feature extraction module is used to extract an initial feature graph from a power grid engineering drawing;

[0032] The non-sampling anchor point generation module is used to generate candidate areas for power grid engineering drawings using a non-sampling anchor point strategy;

[0033] The first convolutional layer is used to calculate the target score of each candidate region according to the initial feature map and the candidate region;

[0034] The second convolutional layer is used to calculate the bounding box deviation based on the initial feature map and the candidate region;

[0035] The feature conversion module is used to generate an updated feature map and an adjusted anchor point according to the target score and the bounding box deviation of each candidate region;

[0036] The proposal decoding module is used to decode the updated feature map and the adjusted anchor point to obtain a detection proposal including a category prediction and a position adjustment result for each candidate region.

[0037] Preferably, the non-sampling anchor point generation module adopts the non-sampling anchor point strategy to generate the region proposal network loss function of the candidate region for the power grid engineering drawing:

[0038]

[0039] in, is the region proposal network loss function, is the category label, is the probability of the predicted positive sample, is a balance item, is the real regional coordinate parameter, is the predicted regional coordinate parameter, i is the candidate region number, M is the number of candidate regions.

[0040] Preferably, the cascade feature alignment network comprises:

[0041] Region of Interest Alignment Module, Visual Encoder, Text Encoder, Coarse-Grained Feature Alignment Module, Proposal Transformation Module, and Fine-Grained Feature Alignment Module;

[0042] The region of interest alignment module is used to align the candidate region to a feature map of a fixed size;

[0043] The visual encoder is used to convert the feature map of the candidate region alignment into a visual feature;

[0044] The text encoder is used to convert all potential detection category sets into text features;

[0045] The coarse-grained feature alignment module is used to calculate the similarity matrix between visual features and text features;

[0046] The proposal conversion module is used to take candidate regions and category pairs with similarities higher than a set threshold as preliminary detection results, extract the bounding box of the preliminary detection results and convert it into a new proposal, and perform an alignment operation on the region of interest;

[0047] The fine-grained feature alignment module is used to align cross-region and cross-text fine features.

[0048] Preferably, the similarity matrix between the visual features and the text features is:

[0049]

[0050] in, is the similarity matrix between visual features and text features, For the i The visual features of the candidate regions, For the j The text features of potential detection categories.

[0051] Preferably, the proposal conversion module further includes taking candidate region and category pairs with similarity higher than a set threshold as preliminary detection results:

[0052] The non-maximum suppression strategy was used to screen the preliminary detection results.

[0053] Preferably, the screening of the preliminary detection results by using a non-maximum suppression strategy includes:

[0054] Arrange all candidate regions in descending order according to their confidence scores;

[0055] Select the candidate area with the highest confidence score as the current detection box and put it into the final detection result set;

[0056] Calculate the overlap ratio between the current detection box and other candidate regions, and consider candidate regions whose overlap ratio exceeds a preset threshold as redundant detections and delete them;

[0057] Repeat the above steps until all candidate regions have been processed.

[0058] Preferably, the fine-grained feature alignment module uses the feature alignment loss function of aligning fine features across regions and across texts as follows:

[0059]

[0060] in, is the feature alignment loss function, For the i candidate regions and j matching labels for potential detection categories, is the softmax function, is the similarity matrix between visual features and text features, M is the number of candidate regions, N is the number of potential detection categories.

[0061] Preferably, during the training process, a multi-task learning strategy is adopted to simultaneously optimize the candidate region generation task, the visual feature and text feature alignment task, and the classification task. The total loss function is:

[0062]

[0063] in, is the total loss function, is the region proposal network loss function, is the feature alignment loss function, is the classification loss function, is the weight hyperparameter.

[0064] Preferably, the detection result of the candidate area includes:

[0065] The category label, detection box coordinates and corresponding confidence score of the candidate region.

[0066] The present invention has the following beneficial effects:

[0067] 1. Improved accuracy and recall of small target detection: The present invention uses a multi-level refined region proposal network (MR-RPN) and adopts a non-sampling anchor strategy to generate candidate regions. Unlike existing methods (such as RPN, GA-RPN, etc.) that rely on fixed anchor generation strategies, the MR-RPN of the present invention can more fully cover small target areas and effectively reduce the missed detection rate of small targets. In addition, through the application of feature conversion modules (such as deformable convolutional networks), the present invention enhances the granularity and diversity of feature expression in candidate regions, significantly improving the detection accuracy of small targets.

[0068] 2. Optimized the alignment of visual features and text features: Existing open vocabulary object detection methods (such as RegionCLIP, OV-DETR, etc.) have the problem of coarse-grained matching in visual-text feature alignment, resulting in low detection accuracy. The present invention designs a cascade alignment mechanism to perform a multi-stage feature matching process of primary, intermediate and advanced alignment, and gradually optimizes the matching relationship between visual features and text features. This mechanism effectively solves the semantic gap problem in the prior art, improves the recognition ability of small objects of new categories, and reduces false detections and missed detections.

[0069] 3. Enhanced generalization capability in open vocabulary scenarios: By combining a multi-level feature optimization strategy and an open vocabulary-driven detection framework, the present invention significantly improves the small target detection performance in a variety of scenarios. In particular, in the verification of power grid drawings, the present invention optimizes the power grid engineering drawings, ensures accurate detection of small targets such as power grid device categories, and enhances the applicability and reliability of the model in practical applications.

[0070] 4. Improved utilization of effective samples: In the training process, existing methods usually use sampling strategies to optimize computing resources and sample balance, but they tend to discard a large number of effective samples. The present invention uses a non-sampling anchor strategy to ensure that all potential effective samples, especially rare categories and small target samples with low confidence, can participate in the training process, thereby improving the model's learning ability and detection performance for new categories of small targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 It is a flowchart of an open vocabulary small object detection method based on cascaded multi-level refinement;

[0072] Figure 2 A schematic diagram of the principle of an open vocabulary small object detection method based on cascaded multi-level refinement;

[0073] Figure 3 Schematic diagram of network structure proposed for multi-level refinement region;

[0074] Figure 4 Schematic diagram of the cascade feature alignment network structure. DETAILED DESCRIPTION

[0075] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0076] This paper proposes a new Cascade Open-Vocabulary Small Object Detector (Cas-OVD) method, which is specifically designed to solve the problem of small object detection in complex scenes. This method effectively improves the accuracy and robustness of open-vocabulary small object detection by introducing a multi-level refined region proposal network (MR-RPN) and a cascade feature alignment mechanism.

[0077] like Figure 1 and Figure 2As shown, an open vocabulary small target detection method based on cascade multi-level refinement provided by an embodiment of the present invention includes the following steps S1 to S3:

[0078] S1. Obtain the power grid engineering drawings and all potential detection category sets of the power grid infrastructure project;

[0079] In an optional embodiment of the present invention, step S1 obtains an input image I and category sets C ,in I Indicates the power grid engineering drawings to be tested, C ={c 1 ,c 2, ...,c N} is the set of all potential detection categories, and N is the number of potential detection categories. These inputs will be processed by subsequent modules to generate detection results for each small target.

[0080] S2, using a multi-level refined region proposal network to extract the initial feature map of the power grid engineering drawings, and generate candidate regions through a non-sampling anchor strategy;

[0081] In an optional embodiment of the present invention, step S2 receives an input image through a multi-level refinement region proposal network (MR-RPN). I , and use a convolutional neural network (CNN) to extract the initial feature map F Then, a high-quality small target candidate region is generated through a non-sampling anchor strategy, which is defined as R ={r 1 ,r 2 ,...,r M}, where each candidate region r i Include possible small goals.

[0082] Multi-level Refinement Region Proposal Network (MR-RPN) first generates initial candidate region proposals through the Multi-level Refinement Region Proposal Network (MR-RPN). MR-RPN generates a large number of candidate regions based on the input image. R , these regions are passed to subsequent modules as potential target regions.

[0083] like Figure 3 As shown, the multi-level refined region proposal network includes:

[0084] Feature extraction module, non-sampling anchor generation module, first convolution layer, second convolution layer, feature conversion module and proposal decoding module;

[0085] The feature extraction module is used to extract the initial feature map from the power grid engineering drawing; specifically, the feature extraction module inputs the image IExtract feature maps through convolutional neural network (CNN) ,in H and W Represent the height and width of the feature map respectively, D Represents the depth of the feature map:

[0086] F=CNN( I )

[0087] The non-sampling anchor point generation module is used to generate candidate regions for power grid engineering drawings using a non-sampling anchor point strategy; specifically, a multi-level refinement region proposal network (MR-RPN) generates a large number of candidate regions using a non-sampling anchor point strategy. R ={r 1 ,r 2 ,...,r M}, does not rely on traditional sampling strategies, ensuring that all possible small targets can be fully covered.

[0088] The non-sampling anchor point generation module adopts the non-sampling anchor point strategy to generate the region proposal network loss function of the candidate region for the power grid engineering drawing:

[0089]

[0090] in, is the region proposal network loss function, is the category label, is the probability of the predicted positive sample, is a balance item, is the real regional coordinate parameter, is the predicted area coordinate parameter, which is used to optimize the position of the coordinate frame. i is the candidate region number, M is the number of candidate regions.

[0091] The first convolutional layer is used to calculate the target score of each candidate region according to the initial feature map and the candidate region;

[0092] The second convolutional layer is used to calculate the bounding box deviation based on the initial feature map and the candidate region; specifically, the multi-level refinement region proposal network (MR-RPN) extracts the feature map F The input is fed into two parallel 1x1 convolutional layers, one for calculating the objectness logits of each candidate region and the other for calculating the bounding-box deltas. The objectness logits are used to measure whether the candidate region contains an object, while the bounding-box deltas are used to further adjust the position of the candidate region.

[0093] The feature conversion module is used to generate an updated feature map and adjusted anchor points according to the target score and bounding box deviation of each candidate region; specifically, the feature conversion module uses the calculated target score and bounding box deviation as input, and uses a deformable convolution network (DCNv2) to enhance the expressiveness of features in the candidate region. The deformable convolution network can more accurately capture the local features of small targets by flexibly adjusting the position of the convolution kernel, thereby improving the detection accuracy of small targets. The feature conversion module generates an updated feature map and adjusted anchor points by inputting the target score and bounding box deviation. The updated feature map highlights the important features in the candidate region, and the adjusted anchor points locate the potential target region more accurately.

[0094] The proposal decoding module is used to decode the updated feature map and the adjusted anchor point to obtain a detection proposal containing the category prediction and position adjustment result of each candidate region. Specifically, the proposal decoding module takes the updated feature map and anchor point generated by the feature conversion module as input, decodes them, and outputs the final detection proposal. These proposals contain the category prediction and position adjustment result of each candidate region.

[0095] The multi-level refined region proposal network (MR-RPN) designed in this paper adopts a non-sampling anchor strategy, fully utilizes all anchor sample information to generate candidate regions, and is especially optimized for the detection of small objects. The network can improve the coverage and feature expression ability of small objects in the candidate region generation stage, and significantly reduce the missed detection rate.

[0096] S3. After aligning the candidate regions with the regions of interest using the cascade feature alignment network, coarse-grained feature alignment and fine-grained feature alignment are performed in turn with the text features converted from the set of all potential detection categories to generate the detection results for each candidate region.

[0097] In an optional embodiment of the present invention, in order to improve the accuracy of detection, step S3 uses a cascade feature alignment module to optimize the matching of image features and text features. First, the image features are mapped to the feature space V(F) through the visual encoder V( ), and then the text description of the category set C is converted into text features T(C) through the text encoder T( ). This module performs an alignment process of two stages, coarse-grained matching and fine-grained matching, to gradually optimize the semantic matching between the candidate regions and the category descriptions. In the cascade alignment process, the detection results of the primary matching stage will be converted into new proposals. Specifically, the detection results obtained by the initial matching will be adjusted to new proposals to be used as input for the second stage, and input into the fine-grained matching stage for further training and optimization. This process ensures that the candidate regions can be gradually filtered and concentrated on more likely small target locations, thereby improving the accuracy of the final detection. The detection results of each candidate region obtained, including the category c i ∈ C and the corresponding confidence s i ∈[0,1], and merge the detection box and the category label to generate the final detection result.

[0098] The cascade feature alignment module consists of two stages: coarse-grained matching and fine-grained matching. Figure 4 As shown, the alignment of image and text features is gradually optimized.

[0099] The cascade feature alignment network includes:

[0100] Region of Interest Alignment Module, Visual Encoder, Text Encoder, Coarse-Grained Feature Alignment Module, Proposal Transformation Module, and Fine-Grained Feature Alignment Module;

[0101] The ROI alignment module is used to align the candidate regions to a fixed-size feature map; specifically, the ROI alignment module passes the candidate regions from MR-RPN through the ROI Align step, and the candidate regions accurately extract features from the original feature map to eliminate the boundary alignment error caused by quantization. The ROI Align operation aligns these regions to a fixed-size feature map to ensure that subsequent feature matching is more accurate.

[0102] The visual encoder is used to convert the feature map of the candidate region alignment into visual features; specifically, the visual encoder (ResNet50) processes the feature map aligned by ROI Align and converts the visual features of the candidate region into F Transform to feature space V(F) ,These visual features represent the characteristic information of each candidate region and are used in the subsequent text-visual matching.

[0103] The text encoder is used to convert all potential detection category sets into text features; specifically, the category label or the text description of the category is converted into text features through a text encoder (Transformer) T(C) The text encoder processes the input category description information and generates the corresponding text feature vector.

[0104] The coarse-grained feature alignment module is used to calculate the similarity matrix between visual features and text features; specifically, the candidate region features are aligned by the visual encoder V( ) F Convert to visual features V( F ), and the category description is transformed into C Converted to text features T( C ). In this stage, the similarity matrix between visual features and text features is calculated using cosine similarity S :

[0105]

[0106] in, is the similarity matrix between visual features and text features, For the i The visual features of the candidate regions, For the j The text features of potential detection categories.

[0107] The proposal conversion module is used to take candidate regions and category pairs with similarity higher than a set threshold as preliminary detection results, extract the bounding boxes of the preliminary detection results and convert them into new proposals, and perform region of interest alignment operations; specifically, based on the results of the coarse-grained feature alignment stage, candidate regions and category pairs with higher similarity are regarded as preliminary detection results. Then, the bounding boxes of these preliminary results are extracted and converted into new proposals, and the corresponding ROI Align operations are performed so as to be sent to the next stage for further optimization.

[0108] The fine-grained feature alignment module is used to align fine features across regions and texts. Specifically, the detection performance is further improved by aligning fine features across regions and texts in the fine-grained matching stage. The feature alignment loss function of the self-attention mechanism is used to optimize the matching relationship between visual and text features:

[0109]

[0110] in, is the feature alignment loss function, For the i candidate regions and j matching labels for potential detection categories, is the softmax function, is the similarity matrix between visual features and text features, M is the number of candidate regions, N is the number of potential detection categories.

[0111] The visual encoder is then trained using a back-propagation algorithm to improve the accuracy and robustness of the model in matching visual and corresponding text features.

[0112] The cascade alignment mechanism proposed in this paper gradually optimizes the matching relationship between visual features and text features through a multi-stage process of primary, intermediate and advanced alignment. This multi-level alignment strategy enhances the detection model's ability to recognize small objects of new categories in open vocabulary scenarios and reduces false detections and missed detections.

[0113] The present invention constructs an open vocabulary object detection system by introducing a multi-level refined region proposal network (MR-RPN) and a cascade alignment mechanism, so that small objects can be effectively detected in open vocabulary scenarios.

[0114] During the training phase, adaptive training was performed on power grid engineering drawings. The model learned the features of small targets in different scenarios and improved its adaptability and generalization ability to various power grid engineering drawings.

[0115] During the training process, the system adopts a multi-task learning strategy to optimize the candidate region generation task, the visual feature and text feature alignment task, and the classification task at the same time. The total loss function is:

[0116]

[0117] in, is the total loss function, is the region proposal network loss function, is the feature alignment loss function, is the classification loss function, is the weight hyperparameter.

[0118] During the test, the system first uses MR-RPN to generate candidate regions, then optimizes the match between visual features and category descriptions through a cascaded feature alignment module, calculates the similarity matrix S, and generates detection results.

[0119] Based on the similarity matrix, the system processes the detection results of each candidate region in the following steps:

[0120] (1) Similarity calculation and preliminary screening:

[0121] For each candidate region r iand each category label c j ∈ C , similarity matrix S ij Represents the candidate region r i With each category label c j The system first calculates the similarity scores between all candidate regions and each category label. S ij , and sort these scores, selecting the category labels with the highest similarity scores as the preliminary detection results. This step ensures that the system can give priority to the most likely category labels.

[0122] (2) Category label assignment:

[0123] Based on the similarity score, the system r i Assign a class label c j ∈ C , which corresponds to r i The category with the highest similarity. Correspondingly, the confidence score s i The similarity score is also assigned, and the normalized similarity score is usually used to represent the confidence. s i ∈[0,1].

[0124] (3) Non-maximum suppression (NMS):

[0125] In order to eliminate redundant detection results and improve detection accuracy, the system applies the non-maximum suppression (NMS) strategy to the preliminary detection results. The specific process is as follows:

[0126] For all candidate regions, sort by confidence score s i Sort in descending order.

[0127] Select the candidate region with the highest confidence score r i As the current detection box, and put it into the final detection result set.

[0128] Calculate the overlap ratio between the current detection box and other candidate regions (usually using Intersection over Union, IoU). For candidate regions whose overlap ratio exceeds a preset threshold (e.g. 0.5), the system considers them as redundant detections and deletes them.

[0129] Repeat the above steps until all candidate regions have been processed.

[0130] (4) Optimize the final output:

[0131] After non-maximum suppression (NMS) processing, the system generates the final detection result, which contains the category label of each valid candidate area. c i , detection box coordinates and corresponding confidence scores s i These results will serve as the final output for further application or decision support.

[0132] Through the above technical solution, the present invention solves many deficiencies of the prior art in open vocabulary small object detection and has the following significant technical effects:

[0133] Improve detection accuracy and recall rate: Through the multi-level refined region proposal network (MR-RPN) and feature conversion module, higher quality candidate regions are generated, which improves the accuracy and recall rate of small target detection.

[0134] Optimize visual-text feature alignment: Adopt a cascade alignment mechanism to gradually optimize the matching of visual and text features, effectively reduce false detections and missed detections, and improve the recognition ability of small objects in new categories.

[0135] Enhance the generalization ability of the model: Adaptive training was carried out on power grid engineering drawings to improve the adaptability and robustness of the model in complex and open environments.

[0136] The present invention optimizes a set of power grid engineering drawings to ensure the detection accuracy of specific small targets in the power grid engineering drawings, and enhances the applicability and reliability of the model in practical applications such as power grid component category identification.

[0137] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0138] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0139] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0140] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

[0141] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. An open vocabulary small object detection method based on cascaded multi-level refinement, characterized in that: The following steps are involved: Obtain power grid engineering drawings and all potential detection category sets of power grid infrastructure projects; A multi-level refined region proposal network is used to extract the initial feature map of the power grid engineering drawings, and a non-sampling anchor strategy is used to generate candidate regions. The multi-level refined region proposal network includes: Feature extraction module, non-sampling anchor generation module, first convolution layer, second convolution layer, feature conversion module and proposal decoding module; The feature extraction module is used to extract an initial feature graph from a power grid engineering drawing; The non-sampling anchor point generation module is used to generate candidate areas for power grid engineering drawings using a non-sampling anchor point strategy; The first convolutional layer is used to calculate the target score of each candidate region according to the initial feature map and the candidate region; The second convolutional layer is used to calculate the bounding box deviation based on the initial feature map and the candidate region; The feature conversion module is used to generate an updated feature map and an adjusted anchor point according to the target score and the bounding box deviation of each candidate region; The proposal decoding module is used to decode the updated feature map and the adjusted anchor point to obtain a detection proposal including a category prediction and a position adjustment result of each candidate region; After aligning the candidate regions with the regions of interest using the cascade feature alignment network, the coarse-grained feature alignment and fine-grained feature alignment are performed in sequence with the text features converted from the set of all potential detection categories to generate the detection results for each candidate region.

2. The open vocabulary small object detection method based on cascaded multi-level refinement according to claim 1, characterized in that: The non-sampling anchor point generation module adopts the non-sampling anchor point strategy to generate the region proposal network loss function of the candidate region for the power grid engineering drawing: in, is the region proposal network loss function, is the category label, is the probability of the predicted positive sample, is a balance item, is the real regional coordinate parameter, is the predicted regional coordinate parameter, i is the candidate region number, M is the number of candidate regions.

3. The open vocabulary small object detection method based on cascaded multi-level refinement according to claim 1, characterized in that: The cascade feature alignment network includes: Region of Interest Alignment Module, Visual Encoder, Text Encoder, Coarse-Grained Feature Alignment Module, Proposal Transformation Module, and Fine-Grained Feature Alignment Module; The region of interest alignment module is used to align the candidate region to a feature map of a fixed size; The visual encoder is used to convert the feature map of the candidate region alignment into a visual feature; The text encoder is used to convert all potential detection category sets into text features; The coarse-grained feature alignment module is used to calculate the similarity matrix between visual features and text features; The proposal conversion module is used to take candidate regions and category pairs with similarities higher than a set threshold as preliminary detection results, extract the bounding box of the preliminary detection results and convert it into a new proposal, and perform an alignment operation on the region of interest; The fine-grained feature alignment module is used to align cross-region and cross-text fine features.

4. The open vocabulary small object detection method based on cascaded multi-level refinement according to claim 3, characterized in that: The similarity matrix between visual features and text features is: in, is the similarity matrix between visual features and text features, For the i The visual features of the candidate regions, For the j The text features of potential detection categories.

5. The open vocabulary small object detection method based on cascaded multi-level refinement according to claim 3, characterized in that: The proposal conversion module takes the candidate region and category pairs whose similarity is higher than the set threshold as preliminary detection results and further includes: The non-maximum suppression strategy was used to screen the preliminary detection results.

6. The open vocabulary small object detection method based on cascaded multi-level refinement according to claim 5, characterized in that: The use of a non-maximum suppression strategy to screen the preliminary detection results includes: Arrange all candidate regions in descending order according to their confidence scores; Select the candidate area with the highest confidence score as the current detection box and put it into the final detection result set; Calculate the overlap ratio between the current detection box and other candidate regions, and consider candidate regions whose overlap ratio exceeds a preset threshold as redundant detections and delete them; Repeat the above steps until all candidate regions have been processed.

7. The open vocabulary small object detection method based on cascaded multi-level refinement according to claim 3, characterized in that: The fine-grained feature alignment module aligns fine features across regions and across texts. The feature alignment loss function is: in, is the feature alignment loss function, For the i candidate regions and j matching labels for potential detection categories, is the softmax function, is the similarity matrix between visual features and text features, M is the number of candidate regions, N is the number of potential detection categories.

8. The open vocabulary small object detection method based on cascaded multi-level refinement according to claim 1, characterized in that: During the training process, a multi-task learning strategy is adopted to optimize the candidate region generation task, the visual feature and text feature alignment task, and the classification task at the same time. The total loss function is: in, is the total loss function, is the region proposal network loss function, is the feature alignment loss function, is the classification loss function, is the weight hyperparameter.

9. The open vocabulary small object detection method based on cascaded multi-level refinement according to claim 1, characterized in that: The detection results of the candidate regions include: The category label, detection box coordinates and corresponding confidence score of the candidate region.

Images