Contact network 2C detection analysis method based on general foreign matter detection model

Abstract

The invention discloses a contact network 2C detection analysis method based on a general foreign matter detection model, and belongs to the technical field of machine vision and target detection, and the method comprises the steps: carrying out the detection based on a pre-trained general foreign matter detection model according to an obtained picture of a to-be-detected target category, outputting detection results of a known category target and an unknown category target; the general foreign matter detection model performs target classification and bounding box regression on the candidate region features by using a known category target detection module and a general category target detection module to obtain detection results of a known category target and an unknown category target; wherein the general category target detection module uses the energy suppression module to distinguish a known category target from an unknown category target, and performs target frame optimization on the unknown category target based on the density non-maximum suppression module to obtain a detection result of the unknown category target. The method solves the problem of contact network foreign matter leak detection caused by the fact that a traditional method is difficult to cover all foreign matter types.

Description

Technical Field

[0001] The invention relates to a contact network 2C detection and analysis method based on a universal foreign body detection model, and belongs to the technical field of machine vision and target detection. Background Art

[0002] The power contact network is an important part of the high-speed railway power supply system, and its safety and stability are directly related to the normal operation of the train. However, during operation, the contact network is often disturbed by foreign objects, such as branches, plastic bags, kite lines and other natural or man-made objects, which may cause short circuits in the contact network, damage equipment, and even threaten driving safety. Therefore, it is of great significance to quickly and accurately detect foreign objects in the contact network.

[0003] At present, the detection of foreign objects in the contact network mainly relies on traditional target detection technology. These methods are usually based on the training of samples of known categories, and the recognition ability of targets depends on the comprehensiveness of the training data. However, the types and forms of foreign objects in the contact network are diverse, and some foreign objects may not appear in the training data, which makes it difficult for traditional methods to cover all foreign object categories, and the detection effect of foreign objects of unknown categories is poor, such as Figure 1 In addition, complex detection environments such as changing weather and light conditions further increase the difficulty of detection.

[0004] In recent years, the development of deep learning technology has brought significant breakthroughs in target detection. Target detection frameworks such as YOLO and Faster R-CNN have demonstrated excellent performance in specific fields. However, these methods are generally based on the closed set assumption and can only handle targets of predefined categories. It is difficult to identify targets of unknown categories in an open environment. In view of the special needs of contact network foreign body detection, an open set target detection method is needed that can take into account both the detection performance of known categories and the recognition ability of unknown categories, so as to improve the generalization ability and adaptability of the system. Summary of the invention

[0005] The purpose of the present invention is to provide a contact network 2C detection and analysis method based on a universal foreign object detection model. By utilizing a known category target detection module OD and a universal category target detection module GOD to perform target classification and bounding box regression on candidate area features, detection results of known category targets and unknown category targets are obtained to solve the problem of missed detection of foreign objects in the contact network caused by the difficulty of traditional methods in covering all foreign object categories.

[0006] In order to solve the above technical problems, the present invention is implemented by adopting the following technical solutions.

[0007] The present invention provides a contact network 2C detection and analysis method based on a universal foreign body detection model, comprising:

[0008] Get the image of the target category to be detected;

[0009] Perform detection based on the image of the target category to be detected based on a pre-trained general foreign body detection model, and output detection results of known category targets and unknown category targets;

[0010] The universal foreign body detection model uses the known category target detection module OD and the universal category target detection module GOD to perform target classification and bounding box regression on the candidate region features to obtain detection results of known category targets and unknown category targets;

[0011] Among them, the general category target detection module GOD uses the energy suppression module EBS to distinguish between known category targets and unknown category targets, and optimizes the target box of unknown category targets based on the density non-maximum suppression module DS-NMS to obtain the detection results of unknown category targets.

[0012] Furthermore, the universal foreign body detection model uses the known category target detection module OD and the universal category target detection module GOD to perform target classification and bounding box regression on the candidate region features to obtain detection results of known category targets and unknown category targets, including:

[0013] Extracting multi-scale features of the images in the data set as backbone features using a backbone feature extraction network;

[0014] The backbone features are input into the recurrent RPN module, and the confidence and target box of the potential target are predicted using the convolutional layer and the fully connected layer to obtain the output features;

[0015] The output features are input into the ROI Pooling module to extract the fixed-size candidate region features and input into the fully connected layer;

[0016] Use the known category target detection module OD to perform target classification and bounding box regression on the candidate region features to obtain the detection results of the known category targets;

[0017] The general category target detection module GOD uses the energy suppression module EBS to process the candidate region features to distinguish known category targets from unknown category targets, and the density-based non-maximum suppression module DS-NMS is used to optimize the target box of the unknown category target to obtain the detection result of the unknown category target.

[0018] Furthermore, when training the universal foreign body detection model, when the number of iterations exceeds a first set value, a label expansion strategy is used to dynamically filter pseudo labels, and the filtered pseudo labels are used to update the loss function of the cyclic RPN module and calculate the loss of the cyclic RPN module.

[0019] Furthermore, the loss function of the cyclic RPN module includes a classification loss function and a regression loss function;

[0020] Among them, the loss function of the cyclic RPN module is expressed as:

[0021] ;

[0022] In the formula, represents the total loss of the cyclic RPN module, and Respectively represent The probability that the position coordinates contain potential targets and the regression parameters used to determine the prediction box, and Respectively represent The probability that the position coordinates contain potential targets and the true label used to determine the regression parameters of the predicted box, is a constant used to balance the loss function, and They represent the number of samples involved in calculating the classification loss and the number of samples involved in calculating the regression loss, respectively. represents the classification loss of the cyclic RPN module, represents the regression loss predicted by the cyclic RPN module;

[0023] Wherein, the classification loss function is expressed as:

[0024] ;

[0025] In the formula, Indicates the number of samples involved in calculating classification loss and regression loss;

[0026] Wherein, the regression loss function is expressed as:

[0027] ;

[0028] In the formula, , , and They represent the regression parameters of the horizontal coordinate of the center point, the regression parameters of the vertical coordinate, the regression parameters of the width of the prediction box, and the regression parameters of the height of the prediction box, respectively. Represents a loss function in machine learning;

[0029] in, The calculation formula is:

[0030] ;

[0031] In the formula, Represents a judgment statement. If If the following conditions are met, the current operation is executed. Indicates when When the following condition is not met, another operation is performed. express The absolute value of .

[0032] Furthermore, a method for dynamically screening pseudo labels using a label expansion strategy includes:

[0033] Filtering prediction boxes with foreground scores higher than a first threshold from the output features as candidate pseudo labels;

[0034] Calculate the predicted box sum of the candidate pseudo-labels, remove the pseudo-labels whose IoU with the predicted box of the candidate pseudo-label is less than the second set value, and obtain the filtered pseudo-labels;

[0035] Apply non-maximum suppression (NMS) to the filtered pseudo-labels to extract the optimal pseudo-labels;

[0036] The optimal pseudo-label is merged with the true label to update the loss function of the cyclic RPN and calculate the loss of the cyclic RPN module.

[0037] Furthermore, the general category target detection module GOD includes two fully connected layers, an energy suppression module EBS, and a density-based non-maximum suppression module DS-NMS;

[0038] The fully connected layer is used to receive and calculate the confidence scores of the candidate region features as known category targets and unknown category targets as foreign objects;

[0039] An energy suppression module EBS is used to receive and generate corresponding energy scores to distinguish between known and unknown foreign objects based on confidence scores of known and unknown foreign objects.

[0040] The density-based non-maximum suppression module DS-NMS is used to optimize the target box based on the density of the target box and the density clustering method for unknown foreign objects, and determine the number of categories of unknown foreign objects.

[0041] Furthermore, it also includes defining sample rules to train the universal foreign body detection model; the sample rules are:

[0042] When training the general foreign body detection model, given a pair of image labels ,in, Represents a group of pictures, Represents the combination of the true labels corresponding to each picture in the picture group, Represents the first elements;

[0043] Based on the image label , using the cyclic RPN module to generate a large number of region proposals B n = b i |i∈[1,M] ,in, Indicates area recommendations No. elements, Indicates area recommendations The total number of elements;

[0044] Define two sample collection criteria and , among which, the first collection basis And the second collection basis Respectively expressed as:

[0045] ;

[0046] ;

[0047] In the formula, Indicates area recommendations No. Elements and the true label group No. Elements The first basis for collection is Indicates area recommendations No. Elements and the true label group No. Elements The second basis for collection is express The Elements Indicates area recommendations No. Elements and the true label group No. Elements The intersection area of ​​the corresponding target box and the real box, and Respectively represent regional recommendations No. Elements The corresponding target box area and the true label group No. Elements The corresponding real frame area;

[0048] The region proposals containing the same foreign object target are grouped into the same group to obtain K groups of region proposals. , ,..., ;

[0049] Divide the samples in K groups of region proposals into 、Some target suggestions , Oversized Target Recommendations and no target recommendations ;

[0050] The complete target suggestion 、Some target suggestions , Oversized Target Suggestions and no target recommendations Respectively expressed as:

[0051] ;

[0052] ;

[0053] ;

[0054] ;

[0055] In the formula, Indicates Elements and the true label group No. Elements The intersection ratio of , and Represent the first constant, the second constant and the third constant respectively. The first constant is used to filter out targetless suggestions. The second constant is used to filter complete target proposals Samples of the three constants work simultaneously to filter partial target proposals and oversized target suggestions of the sample.

[0056] Furthermore, the loss function of the fully connected layer includes a complete target sample loss function, a partial target sample loss function, and a comparison loss function of the complete target sample and the partial target sample. The loss function of the fully connected layer is expressed as:

[0057] ;

[0058] In the formula, represents the total loss of the fully connected layer, represents the complete target sample loss function, represents the loss function of some target samples, represents the contrast loss function between the complete target sample and the partial target sample,

[0059] Among them, the complete target sample loss function It is expressed as:

[0060] L POS = 1 K ∑ k ∈ [1,K] 1 | B n k,c | ∑ b i ∈ B n k,c ( ∅ ( f i ) - 1 ) 2 ;

[0061] In the formula, Indicates the region recommended Elements The corresponding features, express The corresponding confidence score in the full connection;

[0062] Among them, the loss function of some target samples is It is expressed as:

[0063] L neg = 1 K ∑ k ∈ [1,K] 1 | B n k,po | ∑ b i ∈ B n k,po max(0, ∅ ( f i ) - σ) ;

[0064] In the formula, express and The union of Indicates taking the maximum value, Represents the fourth constant, used to control scope;

[0065] Among them, the contrast loss function of the complete target sample and the partial target sample is It is expressed as:

[0066] L con = 1 K ∑ k ∈ [1,K] 1 | B n k,c | ∑ b i , b j ∈ B n k,c max(0, ∅ ( f i ) -∅ ( f j ) α +τ) ;

[0067] In the formula, express The recommended Elements The corresponding features, express The corresponding confidence score in the full connection, where , Indicates the leveling weight, when The value is 1, otherwise it is -1. Represents a tiny constant with a value of 0.01, used to adjust The value of .

[0068] Furthermore, the loss function of the energy suppression module EBS is expressed as:

[0069] L Energy = 1 T ∑ i∈[1,T] max(0,E( b i )) ;

[0070] In the formula, represents the total loss of the energy suppression module EBS, Indicates the region recommended Elements The corresponding energy score after the energy suppression module EBS, where: E( b i )=-log ∑ c∈[1,C] W C *ex p f c , represents the logarithmic function, Represents the fifth constant, used to constrain Minimum generated Results Represents the category in the classification head in the known category target detection module OD The logarithmic output of represents the number of known foreign body categories, represents a learnable parameter for alleviating class imbalance, Represents an exponential function.

[0071] Furthermore, the number of categories of the unknown category foreign objects is determined based on the density-based K-means clustering method, including:

[0072] The initial number of categories of unknown foreign objects is set to 1;

[0073] Adopting an iterative method, until the intra-class distances of all categories are less than a second threshold, the final number of categories is determined as the number of categories of foreign objects of unknown categories;

[0074] Among them, the iterative method includes:

[0075] Perform K-means clustering on the target frame according to the number of categories of the current unknown category foreign objects, and calculate the intra-category distance of each category;

[0076] If the intra-class distances of all categories are less than the preset second threshold, the number of categories of the current unknown category foreign objects is used as the final number of categories, otherwise the number of categories is increased and K-means clustering is performed again on the target box.

[0077] Compared with the prior art, the present invention has the following beneficial effects:

[0078] 1. The present invention realizes comprehensive detection of known and unknown category targets in the overhead line 2C image by introducing a universal foreign object detection model and combining the known category target detection module OD and the universal category target detection module GOD. In particular, the energy suppression module EBS in the universal category target detection module GOD can effectively distinguish known and unknown category targets and reduce the false detection rate; at the same time, the density-based non-maximum suppression module DS-NMS further optimizes the bounding box of the unknown category target, improves the accuracy and robustness of the detection, and not only improves the efficiency of the overhead line 2C detection, but also enhances the adaptability of the universal foreign object detection model to unknown category targets, providing more reliable data support for subsequent foreign object identification and processing.

[0079] 2. The present invention uses a backbone feature extraction network to extract multi-scale features of an image, and predicts the confidence and target box of a potential target through a cyclic RPN module, thereby effectively improving detection efficiency.

[0080] 3. When training the universal foreign object detection model, the present invention adopts a label expansion strategy to dynamically screen pseudo labels, and uses the screened pseudo labels to update the loss function of the cyclic RPN module, which not only helps the universal foreign object detection model to better learn the characteristics of unknown category targets, but also gradually expands the recognition range of the universal foreign object detection model during the iteration process, thereby improving the generalization ability of the model. By dynamically adjusting the screening criteria of pseudo labels and updating the loss function of the cyclic RPN module, the universal foreign object detection model can gradually adapt to more unknown categories of foreign objects, providing more comprehensive protection for the safe operation of the contact network.

[0081] 4. The present invention uses a density clustering method to optimize the target frame of unknown category targets. The final number of categories of unknown category targets is determined by an iterative method, and the clustering effect is judged based on the intra-class distance, thereby achieving accurate classification and bounding box optimization of unknown category foreign objects, which not only improves the detection accuracy of unknown category foreign objects, but also provides more reliable data support for subsequent foreign object identification and processing. At the same time, the density-based clustering method can also effectively handle foreign object detection problems in complex scenarios, providing more advanced technical means for intelligent monitoring and maintenance of contact networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1 Shown is a schematic diagram of the detection results of a traditional foreign body detector provided by an embodiment of the present invention;

[0083] Figure 2 The figure is a schematic diagram of the network structure of a universal foreign body detection model provided by an embodiment of the present invention;

[0084] Figure 3 FIG. 1 is a schematic diagram of the workflow of the general category target detection module GOD provided in an embodiment of the present invention.

[0085] Figure 4 The figure is a schematic diagram of a flow chart of a label expansion strategy provided by an embodiment of the present invention;

[0086] Figure 5 Shown is a schematic diagram of characteristic distribution of different types of foreign matter provided by an embodiment of the present invention;

[0087] Figure 6 FIG. 1 is a schematic diagram showing an example of training samples of a general category target detection module GOD provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0088] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention. The embodiments of the present invention and the technical features in the embodiments may be combined with each other unless there is a conflict.

[0089] The term "and / or" is only a description of the association relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " generally indicates that the related objects are in an "or" relationship.

[0090] Example 1

[0091] This embodiment introduces a contact network 2C detection and analysis method based on a universal foreign object detection model, including:

[0092] Step 1: Get the image of the target category to be detected.

[0093] The images for obtaining the target categories to be detected provided by the present invention include both targets of known categories, such as natural or man-made objects such as tree branches, plastic bags, and kite strings, and targets of unknown categories, that is, categories that are not clearly marked or defined in the training set.

[0094] Step 2: According to the image of the target category to be detected, detection is performed based on a pre-trained general foreign body detection model, and detection results of known category targets and unknown category targets are output.

[0095] The present invention utilizes a universal foreign body detection model pre-trained on a large number of data sets containing target images of unknown categories to detect images of the target category to be detected, which can save training time and utilize existing knowledge to improve the performance of new tasks.

[0096] The universal foreign body detection model uses the known category target detection module OD and the universal category target detection module GOD to perform target classification and bounding box regression on the candidate region features to obtain detection results of known category targets and unknown category targets.

[0097] The present invention uses a trained universal foreign object detection model to traverse candidate areas in a picture containing targets of unknown categories, classifies each area and performs bounding box regression to determine the existence, category and location of foreign objects, thereby achieving rapid identification of known and unknown category targets in the picture.

[0098] Among them, the general category target detection module GOD uses the energy suppression module EBS to distinguish between known category targets and unknown category targets, and optimizes the target box of unknown category targets based on the density non-maximum suppression module DS-NMS to obtain the detection results of unknown category targets.

[0099] The energy suppression module EBS uses energy functions to distinguish between known category targets and unknown category targets, and enhances the detection capability of unknown category targets by suppressing the characteristic responses of known category targets.

[0100] The density-based non-maximum suppression module DS-NMS further optimizes the bounding box of the detected unknown category targets, improves the detection accuracy and robustness, reduces the number of overlapping detection boxes, effectively distinguishes and detects unknown category targets, improves the accuracy of the bounding box of the unknown category targets, and reduces false detections and missed detections.

[0101] Example 2

[0102] Based on the same inventive concept as Example 1, this example introduces a contact network 2C detection and analysis method based on a universal foreign body detection model, such as Figure 2 As shown, including:

[0103] Step 1: Data preprocessing and enhancement.

[0104] In this embodiment, a data set of contact network 2C images containing targets of unknown categories is obtained, and the contact network 2C images containing targets of unknown categories are uniformly normalized and the size is set to 1024×1024 pixels. In order to increase data diversity, standard horizontal flipping and four-image stitching methods are randomly used for data enhancement to obtain a data set after data preprocessing and enhancement. The universal foreign object detection model is trained by the data set after data preprocessing and enhancement, which can improve the generalization ability of the universal foreign object detection model in different environments.

[0105] Step 2: Use a backbone feature extraction network to extract multi-scale features of the images in the dataset as backbone features.

[0106] In this embodiment, the backbone feature extraction network ResNet50 is used to extract features of the input image in the data set after data preprocessing and enhancement, and the multi-scale features of the image in the data set are obtained as backbone features, and the backbone features are input into the cyclic RPN module.

[0107] Step 3: Obtain the backbone features output by the backbone feature extraction network ResNet50 to generate prediction results for potential target areas.

[0108] This embodiment uses a 3×3 convolutional layer to generate a new feature map based on the backbone features, and predicts the confidence of the potential target and the regression parameters of the target box through two fully connected layers to obtain the output features.

[0109] The loss function of the cyclic RPN module includes a classification loss function and a regression loss function;

[0110] Among them, the loss function of the cyclic RPN module is expressed as:

[0111] ;

[0112] In the formula, represents the total loss of the cyclic RPN module, and Respectively represent The probability that the position coordinates contain potential targets and the regression parameters used to determine the prediction box, and Respectively represent The probability that the position coordinates contain potential targets and the true label used to determine the regression parameters of the predicted box, is a constant used to balance the loss function, and They represent the number of samples involved in calculating the classification loss and the number of samples involved in calculating the regression loss, respectively. represents the classification loss of the cyclic RPN module, represents the regression loss predicted by the cyclic RPN module;

[0113] Wherein, the classification loss function is expressed as:

[0114] ;

[0115] In the formula, Indicates the number of samples involved in calculating classification loss and regression loss;

[0116] Wherein, the regression loss function is expressed as:

[0117] ;

[0118] In the formula, , , and They represent the regression parameters of the horizontal coordinate of the center point, the regression parameters of the vertical coordinate, the regression parameters of the width of the prediction box, and the regression parameters of the height of the prediction box, respectively. Represents a loss function in machine learning;

[0119] in, The calculation formula is:

[0120] ;

[0121] In the formula, Represents a judgment statement. If If the following conditions are met, the current operation is executed. Indicates when When the following condition is not met, another operation is performed. express The absolute value of .

[0122] In this embodiment, when training the universal foreign body detection model, when the number of iterations exceeds the first set value, a label expansion strategy is used to dynamically filter pseudo labels, and the filtered pseudo labels are used to update the loss function of the cyclic RPN module and calculate the loss of the cyclic RPN module.

[0123] In this embodiment, a method of dynamically screening pseudo labels using a label expansion strategy is adopted, such as Figure 4 As shown, including:

[0124] Filtering prediction boxes whose foreground scores are higher than a first threshold G from the output features as candidate pseudo labels;

[0125] Calculate the prediction box and GT of the candidate pseudo-label, remove the pseudo-label whose IoU with the prediction box GT of the candidate pseudo-label is less than the second set value s, and obtain the filtered pseudo-label;

[0126] Apply non-maximum suppression (NMS) to the filtered pseudo-labels to extract the optimal pseudo-labels;

[0127] The optimal pseudo-label is merged with the true label to update the loss function of the cyclic RPN and calculate the loss of the cyclic RPN module.

[0128] Step 4: Get the output features of the cyclic RPN module, input them into the ROI Pooling module to extract the candidate region features of fixed size and input them into the fully connected layer.

[0129] In this embodiment, the input of the ROI Pooling module is divided into two parts: a feature map and a ROI candidate region, wherein the feature map is extracted by a convolutional neural network and has a size of H×W×C, wherein H is the height of the feature map, W is the width of the feature map, and C is the number of channels of the feature map; the ROI candidate region is the output feature of the cyclic RPN module, indicating an area that may contain a target, and is presented in the form of coordinates, expressed as [ ],in, is the lower left corner horizontal coordinate of the candidate area, is the lower left corner ordinate of the candidate region, is the horizontal coordinate of the upper right corner of the candidate area, is the upper right corner ordinate of the candidate region.

[0130] In this embodiment, the workflow of the ROI Pooling module is:

[0131] Map the coordinates of the ROI candidate region to the size of the feature map. Usually, the coordinates of the ROI candidate region are in the original image space. The ROI Pooling module needs to normalize the coordinates of the ROI candidate region to the space of the feature map. For example, the original image size is 500×500, and the feature map size is 20×20. Scale the ROI candidate region according to the ratio.

[0132] The ROI candidate region is divided into a fixed number of sub-regions. In this embodiment, each ROI region is divided into 7×7 sub-regions, and the size of each sub-region is determined by calculating the width and height of the ROI candidate region.

[0133] Perform a maximum pooling operation on each sub-region, select the maximum value in the sub-region and fill it in the corresponding position during pooling. For example, the size of the ROI candidate region of the feature map is , then the area size corresponding to each sub-area is , perform maximum pooling on each sub-region and take the maximum value in the sub-region as the candidate region feature;

[0134] The candidate region feature is a feature map of a fixed size. In this embodiment, the fixed size of the candidate region feature is 7×7×C, where C represents the number of channels of the feature map.

[0135] Step 5: Use the known category target detection module OD and the general category target detection module GOD to perform target classification and bounding box regression on the candidate region features to obtain the detection results of known category targets and unknown category targets.

[0136] Use the known category target detection module OD to perform target classification and bounding box regression on the candidate region features to obtain the detection results of the known category targets;

[0137] The candidate region features of the fully connected layer are obtained to generate feature embedding, and the feature embedding is input into the known category target detection module OD and the general category target detection module GOD. The known category target detection module OD is used to perform target classification and bounding box regression on the candidate region features to obtain the detection results of known category targets.

[0138] The general category target detection module GOD uses the energy suppression module EBS to process the candidate region features to distinguish known category targets from unknown category targets, and the density-based non-maximum suppression module DS-NMS is used to optimize the target box of the unknown category target to obtain the detection result of the unknown category target.

[0139] The general category target detection module GOD uses the energy suppression module EBS to distinguish between known category targets and unknown category targets, and optimizes the target box of unknown category targets based on the density non-maximum suppression module DS-NMS to obtain the detection results of unknown category targets.

[0140] In this embodiment, the general category target detection module GOD includes two fully connected layers, an energy suppression module EBS, and a density-based non-maximum suppression module DS-NMS. The workflow of the general category target detection module GOD is as follows: Figure 3 As shown;

[0141] The fully connected layer is used to receive and calculate the confidence scores of the candidate region features as known category targets and unknown category targets as foreign objects. The distribution diagram of different categories of foreign object features is as follows: Figure 5 As shown;

[0142] An energy suppression module EBS is used to receive and generate corresponding energy scores to distinguish between known and unknown foreign objects based on confidence scores of known and unknown foreign objects.

[0143] The density-based non-maximum suppression module DS-NMS is used to optimize the target box based on the density of the target box and the density clustering method for unknown foreign objects, and determine the number of categories of unknown foreign objects.

[0144] In this embodiment, the loss function of the general category object detection module GOD is expressed as:

[0145] ;

[0146] In the formula, represents the total loss of the general category object detection module GOD, represents the loss function for training the fully connected layer, Represents the loss function for training the energy suppression module EBS.

[0147] In this embodiment, the sample examples in the above-mentioned fully connected layer training process are as follows: Figure 5 shown.

[0148] This embodiment also includes defining sample rules to train the universal foreign body detection model; the sample rules are:

[0149] When training the general foreign body detection model, given a pair of image labels ,in, Represents a group of pictures, Represents the combination of the true labels corresponding to each picture in the picture group, Represents the first elements;

[0150] Based on the image label , using the cyclic RPN module to generate a large number of region proposals B n = b i |i∈[1,M] ,in, Indicates area recommendations No. elements, Indicates area recommendations The total number of elements;

[0151] Define two sample collection criteria and , among which, the first collection basis And the second collection basis Respectively expressed as:

[0152] ;

[0153] ;

[0154] In the formula, Indicates the region recommended Elements and the true label group No. Elements The first basis for collection is Indicates the region recommended Elements and the true label group No. Elements The second basis for collection is express The Elements Indicates the region recommended Elements and the true label group No. Elements The intersection area of ​​the corresponding target box and the real box, and They represent the first Elements The corresponding target box area and the true label group No. Elements The corresponding real frame area;

[0155] The region proposals containing the same foreign object target are grouped into the same group to obtain K groups of region proposals. , ,..., ;

[0156] Divide samples in K groups of region proposals into complete object proposals 、Some target suggestions , Oversized Target Suggestions and no target recommendations ;

[0157] The complete target suggestion 、Some target suggestions , Oversized Target Suggestions and no target recommendations Respectively expressed as:

[0158] ;

[0159] ;

[0160] ;

[0161] ;

[0162] In the formula, Indicates Elements and the true label group No. Elements The intersection ratio of , and Represent the first constant, the second constant and the third constant respectively, which are used for screening thresholds.

[0163] The loss function of the fully connected layer includes a complete target sample loss function, a partial target sample loss function, and a comparison loss function of the complete target sample and the partial target sample. The loss function of the fully connected layer is expressed as:

[0164] ;

[0165] In the formula, represents the total loss of the fully connected layer, represents the complete target sample loss function, represents the loss function of some target samples, represents the contrast loss function between the complete target sample and the partial target sample,

[0166] Among them, the complete target sample loss function It is expressed as:

[0167] L POS = 1 K ∑ k ∈ [1,K] 1 | B n k,c | ∑ b i ∈ B n k,c ( ∅ ( f i ) - 1 ) 2 ;

[0168] In the formula, Indicates the region recommended Elements The corresponding features, express The corresponding confidence score in the full connection;

[0169] Among them, the loss function of some target samples is It is expressed as:

[0170] L neg = 1 K ∑ k ∈ [1,K] 1 | B n k,po | ∑ b i ∈ B n k,po max(0, ∅ ( f i ) - σ) ;

[0171] In the formula, express and The union of Indicates taking the maximum value, Represents the fourth constant, used to control scope;

[0172] Among them, the contrast loss function of the complete target sample and the partial target sample is It is expressed as:

[0173] L con = 1 K ∑ k ∈ [1,K] 1 | B n k,c | ∑ b i , b j ∈ B n k,c max(0, ∅ ( f i ) -∅ ( f j ) α +τ) ;

[0174] In the formula, express The recommended Elements The corresponding features, express The corresponding confidence score in the full connection, where , Indicates the leveling weight, when The value is 1, otherwise it is -1. Represents a tiny constant with a value of 0.01, used to adjust The value of .

[0175] In this embodiment, the loss function of the energy suppression module EBS is expressed as:

[0176] L Energy = 1 T ∑ i∈[1,T] max(0,E( b i )) ;

[0177] In the formula, represents the total loss of the energy suppression module EBS, Indicates the region recommended Elements The corresponding energy score after the energy suppression module EBS, where: E( b i )=-log ∑ c∈[1,C] W C *ex p f c , represents the logarithmic function, Represents the fifth constant, used to constrain Minimum generated Results Represents the category in the classification head in the known category target detection module OD The logarithmic output of represents the number of known foreign body categories, represents a learnable parameter for alleviating class imbalance, Represents an exponential function.

[0178] Since the traditional clustering method requires the number of categories to be determined in advance, and it is difficult to determine the number of categories in the target frame clustering process of unknown foreign objects, this embodiment proposes a density-based K-means clustering method to determine the number of categories of the unknown foreign objects, including:

[0179] The initial number of categories of unknown foreign objects is set to 1;

[0180] Adopting an iterative method, until the intra-class distances of all categories are less than a second threshold, the final number of categories is determined as the number of categories of foreign objects of unknown categories;

[0181] Among them, the iterative method includes:

[0182] Perform K-means clustering on the target frame according to the number of categories of the current unknown category foreign objects, and calculate the intra-category distance of each category;

[0183] If the intra-class distances of all categories are less than the preset second threshold, the number of categories of the current unknown category foreign objects is used as the final number of categories, otherwise the number of categories is increased and K-means clustering is performed again on the target box.

[0184] Step 6: Use the general foreign body detection model to detect the image of the target category to be detected, and output the detection results of known category targets and unknown category targets, including:

[0185] Get the image of the target category to be detected;

[0186] According to the picture of the target category to be detected, detection is performed based on a pre-trained universal foreign object detection model, and detection results of known category targets and unknown category targets are output.

[0187] The detection results of known and unknown category targets are combined to generate a complete foreign body detection report, including the detection location and classification information of known and unknown category targets.

[0188] The present invention relates to the technical field of machine vision and target detection, and specifically discloses a contact network 2C intelligent detection and analysis method based on a universal foreign object detection model, which overcomes the limitation that traditional target detection is difficult to cover all categories, has high detection accuracy and robustness, and shows excellent performance in the detection of foreign objects of unknown categories. The method can be widely used in the intelligent detection and maintenance of high-speed railway contact networks.

[0189] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0190] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0191] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0192] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0193] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A contact network 2C detection and analysis method based on a universal foreign body detection model, characterized in that: include: Get the image of the target category to be detected; Perform detection based on the image of the target category to be detected based on a pre-trained general foreign body detection model, and output detection results of known category targets and unknown category targets; The universal foreign body detection model uses the known category target detection module OD and the universal category target detection module GOD to perform target classification and bounding box regression on the candidate region features to obtain detection results of known category targets and unknown category targets; Among them, the general category target detection module GOD uses the energy suppression module EBS to distinguish between known category targets and unknown category targets, and optimizes the target box of unknown category targets based on the density non-maximum suppression module DS-NMS to obtain the detection results of unknown category targets.

2. The contact network 2C detection and analysis method based on the universal foreign body detection model according to claim 1 is characterized in that: The universal foreign body detection model uses the known category target detection module OD and the universal category target detection module GOD to perform target classification and bounding box regression on the candidate region features to obtain the detection results of known category targets and unknown category targets, including: Extracting multi-scale features of the images in the data set as backbone features using a backbone feature extraction network; The backbone features are input into the recurrent RPN module, and the confidence and target box of the potential target are predicted using the convolutional layer and the fully connected layer to obtain the output features; The output features are input into the ROI Pooling module to extract the fixed-size candidate region features and input into the fully connected layer; Use the known category target detection module OD to perform target classification and bounding box regression on the candidate region features to obtain the detection results of the known category targets; The general category target detection module GOD uses the energy suppression module EBS to process the candidate region features to distinguish known category targets from unknown category targets, and the density-based non-maximum suppression module DS-NMS is used to optimize the target box of the unknown category target to obtain the detection result of the unknown category target.

3. The contact network 2C detection and analysis method based on the universal foreign body detection model according to claim 2 is characterized in that: When training the universal foreign body detection model, when the number of iterations exceeds a first set value, a label expansion strategy is used to dynamically filter pseudo labels, and the filtered pseudo labels are used to update the loss function of the cyclic RPN module and calculate the loss of the cyclic RPN module.

4. The contact network 2C detection and analysis method based on the universal foreign body detection model according to claim 3 is characterized in that: The loss function of the cyclic RPN module includes a classification loss function and a regression loss function; Among them, the loss function of the cyclic RPN module is expressed as: ； In the formula, represents the total loss of the cyclic RPN module, and Respectively represent The probability that the position coordinates contain potential targets and the regression parameters used to determine the prediction box, and Respectively represent The probability that the position coordinates contain potential targets and the true label used to determine the regression parameters of the predicted box, is a constant used to balance the loss function, and They represent the number of samples involved in calculating the classification loss and the number of samples involved in calculating the regression loss, respectively. represents the classification loss of the cyclic RPN module, represents the regression loss predicted by the cyclic RPN module; Wherein, the classification loss function is expressed as: ； In the formula, Indicates the number of samples involved in calculating classification loss and regression loss; Among them, the regression loss function is expressed as: ； In the formula, , , and They represent the regression parameters of the horizontal coordinate of the center point, the regression parameters of the vertical coordinate, the regression parameters of the width of the prediction box, and the regression parameters of the height of the prediction box, respectively. Represents a loss function in machine learning; in, The calculation formula is: ； In the formula, Indicates a judgment statement. If If the following conditions are met, the current operation is executed. Indicates when When the following condition is not met, another operation is performed. express The absolute value of .

5. The contact network 2C detection and analysis method based on the universal foreign body detection model according to claim 3 is characterized in that: The method of dynamically screening pseudo labels using label expansion strategy includes: Filtering prediction boxes with foreground scores higher than a first threshold from the output features as candidate pseudo labels; Calculate the predicted box sum of the candidate pseudo-labels, remove the pseudo-labels whose IoU with the predicted box of the candidate pseudo-label is less than the second set value, and obtain the filtered pseudo-labels; Apply non-maximum suppression (NMS) to the filtered pseudo-labels to extract the optimal pseudo-labels; The optimal pseudo-label is merged with the true label to update the loss function of the cyclic RPN and calculate the loss of the cyclic RPN module.

6. The contact network 2C detection and analysis method based on the universal foreign body detection model according to claim 1 is characterized in that: The general category target detection module GOD includes two fully connected layers, an energy suppression module EBS, and a density-based non-maximum suppression module DS-NMS; The fully connected layer is used to receive and calculate the confidence scores of the candidate region features as known category targets and unknown category targets as foreign objects; An energy suppression module EBS is used to receive and generate corresponding energy scores to distinguish between known and unknown foreign objects based on confidence scores of known and unknown foreign objects. The density-based non-maximum suppression module DS-NMS is used to optimize the target box based on the density of the target box and the density clustering method for unknown foreign objects, and determine the number of categories of unknown foreign objects.

7. The contact network 2C detection and analysis method based on the universal foreign body detection model according to claim 6 is characterized in that: The method also includes defining sample rules to train the universal foreign body detection model; the sample rules are: When training the general foreign body detection model, given a pair of image labels ,in, Represents a group of pictures, Represents the combination of the true labels corresponding to each picture in the picture group, Represents the first elements; Based on the image label , using the cyclic RPN module to generate a large number of region proposals ,in, Indicates area recommendations No. elements, Indicates area recommendations The total number of elements; Define two sample collection criteria and , among which, the first collection basis And the second collection basis Respectively expressed as: ； ； In the formula, Indicates area recommendations No. Elements and the true label group No. Elements The first basis for collection is Indicates area recommendations No. Elements and the true label group No. Elements The second basis for collection is express The Elements Indicates area recommendations No. Elements and the true label group No. Elements The intersection area of ​​the corresponding target box and the real box, and Respectively represent regional recommendations No. Elements The corresponding target box area and the true label group No. Elements The corresponding real frame area; The region proposals containing the same foreign object target are grouped into the same group to obtain K groups of region proposals. , ,..., ; Divide the samples in K groups of region proposals into 、Some target suggestions , Oversized Target Recommendations and no target recommendations ; The complete target suggestion 、Some target suggestions , Oversized Target Recommendations and no target recommendations Respectively expressed as: ； ； ； ； In the formula, Indicates Elements and the true label group No. Elements The intersection ratio of , and Represent the first constant, the second constant and the third constant respectively. The first constant is used to filter out targetless suggestions. The second constant is used to filter complete target proposals Samples of the three constants work simultaneously to filter partial target proposals and oversized target suggestions of the sample.

8. The contact network 2C detection and analysis method based on the universal foreign body detection model according to claim 7 is characterized in that: The loss function of the fully connected layer includes a complete target sample loss function, a partial target sample loss function, and a comparison loss function of the complete target sample and the partial target sample. The loss function of the fully connected layer is expressed as: ； In the formula, represents the total loss of the fully connected layer, represents the complete target sample loss function, represents the loss function of some target samples, represents the contrast loss function between the complete target sample and the partial target sample, Among them, the complete target sample loss function It is expressed as: ； In the formula, Indicates the region recommended Elements The corresponding features, express The corresponding confidence score in the full connection; Among them, the loss function of some target samples is It is expressed as: ； In the formula, express and The union of Indicates taking the maximum value, Represents the fourth constant, used to control scope; Among them, the contrast loss function of the complete target sample and the partial target sample is It is expressed as: ； In the formula, express The recommended Elements The corresponding features, express The corresponding confidence score in the full connection, where , Indicates the leveling weight, when The value is 1, otherwise it is -1. Represents a tiny constant with a value of 0.01, used to adjust The value of .

9. The contact network 2C detection and analysis method based on the universal foreign body detection model according to claim 6 is characterized in that: The loss function of the energy suppression module EBS is expressed as: ； In the formula, represents the total loss of the energy suppression module EBS, Indicates the region recommended Elements The corresponding energy score after the energy suppression module EBS, where: , represents the logarithmic function, Represents the fifth constant, used to constrain Minimum generated Results Represents the category in the classification head in the known category target detection module OD The logarithmic output of represents the number of known foreign body categories, represents a learnable parameter for alleviating class imbalance, Represents an exponential function.

10. The contact network 2C detection and analysis method based on the universal foreign body detection model according to claim 6, characterized in that: The density-based K-means clustering method determines the number of categories of the unknown category foreign objects, including: The initial number of categories of unknown foreign objects is set to 1; Adopting an iterative method, until the intra-class distances of all categories are less than a second threshold, the final number of categories is determined as the number of categories of foreign objects of unknown categories; Among them, the iterative method includes: Perform K-means clustering on the target frame according to the number of categories of the current unknown category foreign objects, and calculate the intra-category distance of each category; If the intra-class distances of all categories are less than the preset second threshold, the number of categories of the current unknown category foreign objects is used as the final number of categories, otherwise the number of categories is increased and K-means clustering is performed again on the target box.

Images