Simplified catenary defect detection method based on defect-generated picture and prototype attention weighting
By generating high-quality defect images and using a prototype attention weighting method to enhance feature maps, a simplified and efficient method for detecting contact wire defects is achieved, solving the problems of insufficient detection efficiency and accuracy in existing technologies.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANJING UNIV OF SCI & TECH
- Filing Date
- 2026-01-30
- Publication Date
- 2026-06-05
AI Technical Summary
Existing technologies for simplified contact network defect detection suffer from problems such as long response time, reliance on human experience, and susceptibility to false positives and false negatives. Furthermore, they lack effective labeled data and detection methods for different defects, making it difficult to guarantee detection efficiency and accuracy.
We employ a defect-based image generation and prototype attention weighting approach to generate high-quality defect images using a large language model. We also enhance the feature maps through multi-head self-attention and learnable parameter weighting, and combine this with a backbone network for defect detection.
By generating a sufficient number of high-quality defect images and enhancing features, the accuracy and robustness of simplified contact network defect detection are improved, solving the problems of data scarcity and poor detection results.
Smart Images
Figure CN122156067A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to a simplified method for detecting defects in overhead contact lines based on defect-generated images and prototype attention weighting. Background Technology
[0002] With the continuous advancement of modern technology and the rapid development of high-speed rail construction, the simplified overhead contact system, a key component of the high-speed electrified railway system, has gradually replaced the traditional overhead contact system. However, with the continuous expansion of high-speed rail lines, their complexity is also increasing, placing higher demands on maintenance work. Defects and malfunctions that may exist during high-speed rail operation, such as bird nests, foreign objects, and missing parts, have a serious impact on the safe operation of high-speed rail. Therefore, timely detection and location of overhead contact system defects has become an essential task.
[0003] Currently, defect detection in simplified overhead contact lines typically involves acquiring images using camera equipment, followed by manual inspection by analysts. However, this method suffers from long response times, reliance on human experience, and a high risk of false positives and false negatives, making it difficult to guarantee efficiency and accuracy. Although some research has explored defect detection in simplified overhead contact lines, existing methods often face challenges due to a lack of effective labeled data and detection methods tailored to different defects, as well as the variable orientation of defect locations.
[0004] Object detection, a crucial task in computer vision, aims to accurately identify and locate multiple targets from images or videos. Traditional object detection methods suffer from high demands for large-scale labeled data and high labeling costs, requiring a large amount of accurate labeled data to train effective models. Current research has attempted few-shot object detection using defective image generation datasets, but these efforts often fail due to insufficient defective image data and low-quality prototype generation, resulting in models struggling to achieve accurate detection results. Summary of the Invention
[0005] The purpose of this invention is to provide a simplified method for detecting defects in overhead contact lines based on defect-generated images and prototype attention weighting.
[0006] The technical solution for achieving the objective of this invention is as follows: Firstly, this invention provides a simplified contact wire defect detection method based on defect-generated images and prototype attention weighting, comprising:
[0007] Step S1: Read the simplified normal image of the catenary, customize the basic abnormal prompts, and use the large language model to generate detailed prompts. Input the three into the defect image generation model to generate a simplified catenary defect image.
[0008] Step S2: Read the generated defect images and defect images in the dataset as input data for the pre-trained backbone network. After passing through the feature extraction network, the feature maps undergo multi-head self-attention operation to obtain enhanced feature maps. After aggregating the feature maps to obtain multiple prototypes, the prototypes are weighted and averaged using learnable parameters to obtain category prototypes and save them.
[0009] Step S3: Read the target image to be detected as input data for the pre-trained backbone network. After passing through the feature extraction network, the feature map is input into the pre-trained region proposal network to obtain candidate target regions. Then, the RoI alignment operation is used to convert regions of different sizes into feature maps of a fixed size.
[0010] Step S4: Input the feature map and the saved category prototype into the defect detection network to obtain the defect location and category.
[0011] In a second aspect, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described in the first aspect.
[0012] Thirdly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.
[0013] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in the first aspect.
[0014] This invention fully utilizes image generation models, largely solving the problem of insufficient defect sample quantity and expanding the number of defect image samples. This invention designs a prototype self-attention weighting module to overcome the difficulties of insufficient defect sample data, low quality and lack of robustness in defect prototype generation, and poor defect detection results. Combined with an image generation model, it yields a simplified catenary defect detection network based on defect-generated images and prototype attention weighting. The feature maps generated by the backbone network undergo attention enhancement, are aggregated, and then weighted with learnable parameters to obtain high-quality prototypes. Subsequently, the target image is input into the backbone network to obtain feature maps, which are then input together with the high-quality prototypes into the subsequent detection network to obtain the final defect detection results. This method is suitable for simplified catenary defect detection. Compared with existing technologies, this simplified catenary defect detection method based on defect-generated images and prototype attention weighting has the following advantages:
[0015] (1) By generating a model from defect images, normal images are used to provide background knowledge, basic prompts provide overall defect category information, and detailed prompts enrich the details, generating a sufficient number of defect images and solving the problem of insufficient defect sample quantity.
[0016] (2) By using multi-head self-attention and prototype learnable parameter weighting, the prototype is weighted and averaged after feature enhancement, resulting in a high-quality prototype. Instead of performing simple prototype averaging without feature enhancement, this method provides higher quality and more comprehensive information, thus enhancing the detection effect. Attached Figure Description
[0017] Figure 1 This is a flowchart of the simplified contact wire defect detection method based on defect-generated images and prototype attention weighting according to the present invention. Detailed Implementation
[0018] This invention proposes a simplified catenary defect detection method based on defect-generated images and prototype attention weighting. The method includes: First, inputting a simplified normal catenary image, basic anomaly warnings, and detailed warnings into an existing defect image generation model to generate the required catenary defect image and defect location annotation information. Second, using a backbone network to extract features from the dataset and the generated defect images, multi-head self-attention is used for feature enhancement. After aggregating the features to obtain multiple prototypes, a weighted average of the prototypes is performed using learnable parameters to obtain the category prototypes. Then, using the same backbone network, feature maps of the target image are extracted. These feature maps are input into a region proposal network to obtain candidate target regions, and RoI alignment is used to convert regions of different sizes into fixed-size feature maps. Finally, the converted feature maps are input into a bounding box regression network, and the category prototypes and feature maps are input into a classification network for defect detection, thus obtaining the final defect detection result. This result can be fed back to workers for maintenance. This invention utilizes a defect image generation model to obtain a sufficient number of simplified contact network defect images, solving the problem of insufficient defect data. At the same time, it uses multi-head self-attention to enhance features and weighted averages to obtain a more robust prototype, which helps to improve defect detection accuracy.
[0019] To enable those skilled in the art to understand the technical solution of the present invention more clearly, the present invention will be described in further detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.
[0020] like Figure 1 As shown, the simplified catenary defect detection method based on defect-generated images and prototype attention weighting provided in this embodiment includes:
[0021] Step S1: Read the simplified normal image of the catenary, customize the basic abnormal prompts, and use the large language model to generate detailed prompts. Input the three into the defect image generation model to generate a simplified catenary defect image.
[0022] Step S2: Read the generated defect images and defect images in the dataset as input data for the pre-trained backbone network. After passing through the feature extraction network, the feature maps undergo multi-head self-attention operation to obtain enhanced feature maps. After aggregating the feature maps to obtain multiple prototypes, the prototypes are weighted and averaged using learnable parameters to obtain category prototypes and save them.
[0023] Step S3: Read the target image to be detected as input data for the pre-trained backbone network. After passing through the feature extraction network, the feature map is input into the pre-trained region proposal network to obtain candidate target regions. Then, the RoI alignment operation is used to convert regions of different sizes into feature maps of a fixed size.
[0024] Step S4: Input the feature map and the saved category prototype into the defect detection network to obtain the defect location and category.
[0025] Furthermore, using normal images, basic anomaly warnings, and detailed warnings, a sufficient number of defect images are generated using the defect image generation model, including:
[0026] Step S1.1: Observe the existing dataset, check the defect types, and prepare basic exception message strings according to the defect types;
[0027] Step S1.2: Provide the basic anomaly message string and defect type to a large language model similar to GPT4 to generate the required detailed defect message string. Input the normal image, basic anomaly, and detailed message string into the defect image generation model simultaneously to generate a simplified catenary defect image.
[0028] Furthermore, in step S1, the defect image generation uses a variant of Stable Diffusion, providing three types of information for high-quality, refined generation, including:
[0029] Prepare sufficient information for the Stable Diffusion variant, including normal images, basic anomaly warnings, and detailed defect warnings. The normal images should be normal images of the simplified catenary without defects in the existing dataset. These images provide sufficient background information for the image generation method so that the images generated by the Stable Diffusion variant will not have a background that is different from the simplified catenary.
[0030] The basic anomaly indication should be the defect category present in the simplified catenary in the dataset. This basic indication provides the variant with overall defect category information so that the Stable Diffusion variant can generate images with high defect relevance based on this basic indication information.
[0031] There are two methods for generating detailed defect information. The first is manual generation, where the user manually considers the defect's details, allowing for as much detail as possible. The second method uses a large language model similar to GPT4, providing the model with basic anomaly hints and image information to generate detailed and accurate descriptive information. This detailed defect information can provide the Stable Diffusion variant with more detailed defect information, enabling the variant to generate more realistic details based on this information.
[0032] By providing normal images, basic anomaly warnings, and detailed defect warnings to the Stable Diffusion variant, sufficient, high-quality, and realistically detailed defect images can be generated, which can solve the problems of insufficient defect images in the dataset and low robustness of category prototypes.
[0033] Furthermore, step S2 specifically includes the following steps:
[0034] Step S2.1: Input the generated image along with the defect images and annotation information from the dataset into the backbone network, and obtain the defect features through the feature extraction network. ;
[0035] Step S2.2, use multi-head self-attention operation on features For feature enhancement, the specific calculation formula for the attention module is as follows:
[0036]
[0037]
[0038]
[0039]
[0040]
[0041]
[0042] Where Flatten represents the tensor flattening operation. This indicates that the flattening operation starts from the second dimension, flattening the 23 dimensions into one dimension. To obtain a learnable weight matrix, the features are converted into the required QKV (Quick Key Value) for subsequent attention computation. This represents the dimension of each attention head. This indicates dimensional partitioning, and the resulting channel dimensions are... Concat represents a tensor concatenation operation, which concatenates the attention results of each head to form the final attention result. Reshape is a tensor dimension adjustment operation, which adjusts the dimensions of the attention result for subsequent prototyping operations.
[0043] Step S2.3, perform aggregation operation on... Feature aggregation is performed to obtain prototypes from different images with the same dimensions. A weighted average of these prototypes is then applied using learnable parameters to obtain the desired robust category prototype. .
[0044] Furthermore, in step S2, multi-head self-attention is used to apply the features extracted from the backbone network. Enhancement was performed, and then learnable parameters were used to refine the features. By performing a weighted average, the final category prototype is obtained, which has the following characteristics:
[0045] The backbone network is a mature feature extraction network pre-trained on large-scale data. During training, the network's parameters are not learned. The extracted features are enhanced through multi-head self-attention operations, making them more discriminative and resulting in higher-quality prototypes. The self-attention parameters are learnable, and their effectiveness is improved through backpropagation. The learnable parameters for weighted averaging of prototypes are also learnable. These parameters, through learning, can weighted average defect prototypes from different images to obtain high-quality, robust defect category prototypes.
[0046] Furthermore, step S3 specifically includes the following steps:
[0047] Step S3.1: Input the target image to be detected into the backbone network, and obtain the features of the target image through the feature extraction network. ;
[0048] Step S3.2, feature The input is fed into a pre-trained region proposal network to obtain candidate target regions. The target regions are then fed into a RoI alignment network to obtain fixed-size feature maps. .
[0049] Furthermore, step S4 specifically includes the following steps:
[0050] a fixed-size feature map and the category prototype obtained in step 2 The input is fed into a defect category detection network to obtain defect categories, and the feature maps are then processed. The data is input into the defect location detection network to obtain the defect location, and finally, the defect detection result is obtained.
[0051] The resulting candidate target regions are obtained by region loss. In supervised learning, the final detection result's category and bounding box are determined by the category loss. and bounding box loss Supervised learning is performed. The formulas for these loss functions are:
[0052]
[0053]
[0054]
[0055]
[0056]
[0057]
[0058] in, This represents the overall loss function of the detection method, including class loss. Bounding box loss and regional losses .
[0059] In category loss middle, These are hyperparameters used to adjust the class balance weights. This represents the model's predicted probability for the correct category. It is a hyperparameter used to adjust the weights of easy and difficult samples.
[0060] Loss in the bounding box middle, This represents the actual bounding box location. is the predicted bounding box location, and n represents the number of samples.
[0061] Regional losses In the BCE loss Indicates the true label, is the probability of the proposed region, where n represents the number of samples. In Dice loss, p represents the probability map output by the model, and g represents the true label.
[0062] This invention fully utilizes image generation models, largely solving the problem of insufficient defect sample quantity and expanding the number of defect image samples. This invention designs a prototype self-attention weighting module to overcome the difficulties of limited defect sample data, low quality and lack of robustness in defect prototype generation, and poor defect detection results. Combined with an image generation model, it yields a simplified catenary defect detection network based on defect-generated images and prototype attention weighting. The feature maps generated by the backbone network undergo attention enhancement, are aggregated, and then weighted with learnable parameters to obtain high-quality prototypes. Subsequently, the target image is input into the backbone network to obtain feature maps, which are then input together with the high-quality prototypes into the subsequent detection network to obtain the final defect detection results. This invention is suitable for simplified catenary defect detection and has good application prospects.
[0063] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A simplified catenary defect detection method based on defect-generated images and prototype attention weighting, characterized in that, include: Step S1: Read the simplified normal image of the catenary, customize the basic abnormal prompts, and use the large language model to generate detailed prompts. Input the three into the defect image generation model to generate a simplified catenary defect image. Step S2: Read the generated defect images and defect images in the dataset as input data for the pre-trained backbone network. After passing through the feature extraction network, the feature maps undergo multi-head self-attention operation to obtain enhanced feature maps. After aggregating the feature maps to obtain multiple prototypes, the prototypes are weighted and averaged using learnable parameters to obtain category prototypes and save them. Step S3: Read the target image to be detected as input data for the pre-trained backbone network. After passing through the feature extraction network, the feature map is input into the pre-trained region proposal network to obtain candidate target regions. Then, the RoI alignment operation is used to convert regions of different sizes into feature maps of a fixed size. Step S4: Input the feature map and the saved category prototype into the defect detection network to obtain the defect location and category.
2. The simplified contact wire defect detection method based on defect-generated images and prototype attention weighting as described in claim 1, characterized in that, Using normal images, basic anomaly warnings, and detailed warnings, a defect image generation model is used to generate defect images, including: Step S1.1: Observe the existing dataset, check the defect types, and prepare basic exception message strings according to the defect types; Step S1.2: Provide the basic anomaly message string and defect type to the large language model to generate the required detailed defect message string. Input the normal image, basic anomaly, and detailed message string into the defect image generation model at the same time to generate a simplified catenary defect image.
3. The simplified contact wire defect detection method based on defect-generated images and prototype attention weighting according to claim 2, characterized in that, In step S1, the defect image is generated using the Stable Diffusion variant. Information is prepared for the Stable Diffusion variant, including a normal image, basic anomaly indication, and detailed defect indication information. The normal image should be a normal image of the simplified catenary without defects in the existing dataset. This image provides background information for the image generation method. The basic anomaly indication should be the defect category present in the simplified catenary in the dataset. This basic indication provides the variant with overall defect category information so that the Stable Diffusion variant can generate images with high defect relevance based on this basic indication information. There are two ways to generate detailed defect information: one is manual generation, and the other is generation through a large language model. By providing the large oracle model with basic anomaly information and image information, detailed and accurate descriptive information can be generated.
4. The simplified contact wire defect detection method based on defect-generated images and prototype attention weighting according to claim 2, characterized in that, Step S2 specifically includes the following steps: Step S2.1: Input the generated image along with the defect images and annotation information from the dataset into the backbone network, and obtain the defect features through the feature extraction network. ; Step S2.2, use multi-head self-attention operation on features For feature enhancement, the specific calculation formula for the attention module is as follows: ； ； ； ； ； ； Where Flatten represents the tensor flattening operation. This indicates that the flattening operation starts from the second dimension, flattening the 23 dimensions into one dimension. To obtain a learnable weight matrix, the features are converted into the required QKV (Quick Key Value) for subsequent attention computation. This indicates dimensional partitioning, and the resulting channel dimensions are... Concat represents a tensor concatenation operation, which concatenates the attention results of each head to form the final attention result. Reshape is a tensor dimension adjustment operation, which adjusts the dimensions of the attention result for subsequent prototyping operations. Step S2.3, perform aggregation operation on... Feature aggregation is performed to obtain prototypes from different images with the same dimensions. A weighted average of these prototypes is then applied using learnable parameters to obtain the desired robust category prototype. .
5. The simplified contact wire defect detection method based on defect-generated images and prototype attention weighting according to claim 4, characterized in that, Step S3 specifically includes the following steps: Step S3.1: Input the target image to be detected into the backbone network, and obtain the features of the target image through the feature extraction network. ; Step S3.2, feature The input is fed into a pre-trained region proposal network to obtain candidate target regions. The target regions are then fed into a RoI alignment network to obtain fixed-size feature maps. .
6. The simplified contact wire defect detection method based on defect-generated images and prototype attention weighting according to claim 5, characterized in that, Step S4 specifically includes: a fixed-size feature map and the category prototype obtained in step S2 The input is fed into a defect category detection network to obtain defect categories, and the feature maps are then processed. The data is input into the defect location detection network to obtain the defect location, and finally, the defect detection result is obtained.
7. The simplified contact wire defect detection method based on defect-generated images and prototype attention weighting according to claim 6, characterized in that, The obtained candidate target regions are obtained by region loss. In supervised learning, the final detection result's category and bounding box are determined by the category loss. and bounding box loss Conduct supervised learning; The formula for the loss function is: ； ； ； ； ； ； in, This represents the overall loss function of the detection method, including class loss. Bounding box loss and regional losses ; In category loss middle, These are hyperparameters used to adjust the class balance weights. This represents the model's predicted probability for the correct category. These are hyperparameters used to adjust the weights of easy and difficult samples; Loss in the bounding box middle, This represents the actual bounding box location. This represents the predicted bounding box location, and n represents the number of samples. Regional losses In the BCE loss Indicates the true label, is the probability of the proposed region, where n represents the number of samples; in the Dice loss, p represents the probability map output by the model, and g represents the true label.
8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.
9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method described in any one of claims 1-7.