Complex railway multi-mode arc detection method

Abstract

The invention provides a complex railway multi-modal arc detection method, and relates to the technical field of image recognition, and the method comprises the steps: obtaining the multi-modal image data of a complex railway; constructing an arc detection model by using the feature extraction layer, the feature selection layer and the transferable feature learning layer; inputting multi-modal image data into the arc detection model, and training by using a loss function to obtain a trained arc detection model; wherein the trained arc detection model is used for analyzing complex railway image data to obtain an arc detection result, and multi-modal arc detection of the complex railway is completed. According to the invention, the problem of low arc detection precision in a complex railway environment is solved.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a method for detecting arcs in complex railway multi-modes. Background Art

[0002] In recent years, image-based arc detection algorithms have gradually attracted attention. This method is more intuitive than signal-based detection and only requires one camera, making it suitable for use in actual environments. For example, ultraviolet light detection uses devices such as ultraviolet cameras, flame detectors, and photomultiplier tubes (PMTs) to improve detection accuracy through narrow-band filters. Although ultraviolet light detection can effectively shield ambient light sources and background interference, photomultiplier tubes are sensitive to the working environment, have weak output signals and high signal-to-noise ratios, and have limited sampling frequencies, and cannot completely shield interference such as sunlight. Infrared light signal detection determines the occurrence of an arc by monitoring real-time temperature and infrared wavelength signals. It is not sensitive to ambient light, but other high-temperature objects are prone to misjudgment.

[0003] However, there are many challenges in the operation of trains, such as glare, reflected light, and interference factors of complex overhead lines, which make glare and reflected light similar to arc light, increase noise, and easily lead to detection errors. Therefore, the method of using a single sensor visible light image detection has limitations and is difficult to effectively detect arcs in complex environments. To address the above challenges, Huang et al. designed a deep learning model that integrates visible light and infrared images, making full use of the high temperature characteristics of arcs and the advantages of infrared images in high temperature target detection. The model effectively integrates multiple sensor information by dynamically adjusting the credibility weights of different image modalities, significantly reducing the misjudgment rate of a single detection method in specific scenarios. However, this method uses multiple models for infrared images and visible light images respectively during the detection process, and fails to fully integrate the features of the two images to achieve mutual correction. Given the "black box" characteristics of deep learning networks, how to design a network architecture with "interpretability" to better integrate and decompose the features of infrared and visible light images, thereby improving the accuracy of arc detection in complex environments, remains a key research issue. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a complex railway multi-modal arc detection method which solves the problem of low arc detection accuracy in a complex railway environment.

[0005] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is: a complex railway multi-modal arc detection method, comprising: S1: Acquire multimodal image data of a complex railway; wherein the multimodal image data includes a visible light image and an infrared light image; S2: Construct an arc detection model using the feature extraction layer, feature selection layer, and transferable feature learning layer; S3: Input the multimodal image data into the arc detection model, and train it using the loss function to obtain a trained arc detection model; wherein the trained arc detection model is used to analyze complex railway image data, obtain arc detection results, and complete arc detection of complex railway multimodality.

[0006] Furthermore, the arc detection model includes: A feature extraction layer, used for fusing and extracting features from the visible light image and the infrared light image to obtain multi-modal arc features; wherein the multi-modal arc features include low-rank features and sparse features; The feature selection layer is used to extract the associated features of low-quality objects and high-quality objects in the multimodal image data based on the predicted bounding box and quality evaluation index to obtain high-quality sample features; The transferable feature learning layer is used to learn the multimodal arc features using the high-quality sample features, and obtain arc detection results based on similarity scores.

[0007] Furthermore, the expression of the multi-modal arc characteristic is: ; ; in, represents the multi-modal arc features learned in step t, It means that the soft threshold operator can represent the activation function, represents the trainable variables in the convolutional layer, represents the image after the fusion of infrared and natural light, represents the parameter that adjusts the trade-off between low-rank coefficient and sparse coefficient, represents the representation coefficient learned in step t-1, Represents feature information, represents the representation coefficient learned in step t-1, represents the trainable parameters in the convolutional layer, represents the trainable parameters in the convolutional layer, represents the convolution operation, represents the update weight, i.e. the momentum coefficient, represents the transpose of the complete dictionary, Represents input data, Represents the threshold for shrinking the input matrix and is a learnable variable.

[0008] Furthermore, the transferable feature learning layer includes: A convolution module, used for embedding the high-quality sample feature and the multi-modal arc feature into a corresponding feature space to obtain a high-quality feature space vector and an arc feature space vector respectively; The fully connected module obtains a joint probability distribution set as an arc detection result through optimal transportation based on the high-quality feature space vector and the arc feature space vector.

[0009] Furthermore, the expression of the joint probability distribution set is: ; ; ; in, express and The set of all joint probability distributions, The feature space vector representing the high-quality sample feature library, Represents the joint probability distribution matrix, which is used to describe the distribution mapping relationship between high-quality sample features and low-quality sample features. It refers to the dimension of the feature matrix, which is related to the batch size and the size of the high-quality sample library. represents the marginal distribution of the joint probability distribution, with dimensions corresponding to the batch size, represents another marginal distribution of the joint probability distribution, and its dimension corresponds to the size of the high-quality sample feature library. Represents the high-quality sample space. , represents the feature alignment weights calculated by the joint probability matrix, represents the optimization objective function in the optimal transmission problem, It represents the feature similarity matrix, usually calculated based on cosine similarity. It means solving the variable that makes the objective function reach the maximum value. represents the target value in the optimal transport problem, represents the entropy of the joint probability distribution matrix, which is used to regularize the objective function. represents the weight of the constraint item, represents the KL divergence, which measures the difference between the joint distribution and the target distribution. represents the regularization parameter, represents the elements of the joint probability matrix, i.e., the matching probability between specific high-quality and low-quality features, represents the normalized joint probability distribution, which is used to further optimize the alignment characteristics.

[0010] Furthermore, the similarity score includes a high-quality sample feature similarity score and a multi-modal arc feature similarity score, and their specific expressions are: ; ; ; in, Represents the feature similarity score of high-quality samples, i.e., low-quality samples The weight of is used to reflect its importance in the joint probability distribution. Represents the operation of taking the maximum value, , and Represents the elements of each column in each row of the joint probability matrix, corresponding to the The matching weight between low-quality features and each high-quality feature, Represents the multimodal arc feature similarity score, i.e., high-quality samples The weight of is used to reflect its importance in feature matching. Represents a single element in the joint probability matrix, i.e., a low-quality sample and high quality samples The matching probability, Indicates whether the sample is a positive sample with high similarity. Indicates low quality samples The total weight of is usually the same as the row maximum in the joint probability matrix, Indicates the size of the low-quality sample set and the number of low-quality samples in the batch. Indicates high quality sample The total weight of is usually calculated by summing the columns of the joint probability matrix. Represents the size of the high-quality sample set and the number of high-quality samples stored in the feature selection module.

[0011] Furthermore, the loss function is expressed as: ; in, Represents the loss function result, which is used to optimize the similarity between the features of low-quality samples and high-quality samples. It indicates whether sample i is considered a positive sample with high similarity. represents the batch size, that is, the number of low-quality and high-quality sample features in the current training. represents the feature similarity function, represents the high-quality sample feature vector, represents the low-quality sample feature vector, represents the temperature parameter of the contrast loss, which is used to control the smoothness of the similarity distribution. Indicates the range of the number of low-quality samples and high-quality samples.

[0012] The beneficial effects of the present invention are as follows: a complex railway multimodal arc detection method, which fuses visible light and infrared images, designs an arc detection model, and provides learnable representation features for decomposing low-rank features and sparse features in the fused image. (1) Using a transferable imitation learning arc detection model, the representation ability of low-quality examples can be improved under the supervision of high-quality examples; (2) An unsupervised transferable representation learning network is designed based on optimization theory, avoiding dependence on any predefined threshold, and can achieve global and local consistency through unsupervised clustering, thereby automatically discovering the differences in arc features in different scenarios, greatly improving the accuracy of arc detection in complex railways. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein: Figure 1 It is an exemplary flow chart of a complex railway multi-modal arc detection method according to some embodiments of the present specification. DETAILED DESCRIPTION

[0014] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0015] Example Figure 1 FIG. 1 is an exemplary flow chart of a complex railway multi-modal arc detection method according to some embodiments of this specification. Figure 1 As shown, the process includes the following steps. In some embodiments, the process can be executed by a processor.

[0016] S1: Acquire multimodal image data of complex railways.

[0017] The multimodal image data is image data in a complex railway train operation process. For example, the multimodal image data may include visible light images and infrared light images.

[0018] In some embodiments, the processor can use non-contact detection equipment such as visible light cameras, infrared cameras, laser geometric parameter acquisition devices, and ultraviolet arc detection devices to monitor the operation process of complex railway trains and obtain multimodal image data.

[0019] S2: Construct an arc detection model using the feature extraction layer, feature selection layer, and transferable feature learning layer.

[0020] The arc detection model is used to analyze the multimodal image data of complex railways to obtain a neural network model of arc detection results. There can be multiple types of arc detection models. For example, the type of arc detection model can include a convolutional neural network model.

[0021] In some embodiments, the input of the arc detection model may be multi-modal image data, and the output of the arc detection model may be an arc detection result.

[0022] The arc detection result is a detection result reflecting the information related to the occurrence of an arc. For example, the arc detection result may include information such as the occurrence of the arc and the arc parameters.

[0023] In some embodiments, the structure of the arc detection model is as follows: the arc detection model includes a feature extraction layer, a feature selection layer, and a transferable feature learning layer. The outputs of the feature extraction layer and the feature selection layer serve as inputs to the transferable feature learning layer, and the output of the transferable feature learning layer serves as the final output of the arc detection model.

[0024] The feature extraction layer is used to fuse and extract features from the visible light image and the infrared light image to obtain multimodal arc features; the input of the feature extraction layer may include the visible light image and the infrared light image, and the output may include multimodal arc features.

[0025] The multimodal arc feature is a set of features of various modalities related to the arc in the image. For example, the multimodal arc feature may include low-rank features and sparse features.

[0026] In some embodiments, the expression of the multi-modal arc characteristic may be: ; ; in, represents the multi-modal arc features learned in step t, It means that the soft threshold operator can represent the activation function, represents the trainable variables in the convolutional layer, represents the image after the fusion of infrared and natural light, represents the parameter that adjusts the trade-off between low-rank coefficient and sparse coefficient, represents the representation coefficient learned in step t-1, Represents feature information, represents the representation coefficient learned in step t-1, represents the trainable parameters in the convolutional layer, represents the trainable parameters in the convolutional layer, represents the convolution operation, represents the update weight, i.e. the momentum coefficient, represents the transpose of the complete dictionary, Represents input data, Represents the threshold for shrinking the input matrix and is a learnable variable.

[0027] In some embodiments, the processor may fuse the visible light image and the infrared light image, perform feature extraction on the fused image, and obtain multi-modal arc features.

[0028] The feature selection layer is used to extract the associated features of low-quality objects and high-quality objects in the multimodal image data based on the predicted bounding box and the quality evaluation index to obtain high-quality sample features. The input of the feature selection layer may include multimodal image data, and the output may include high-quality sample features.

[0029] High-quality sample features are sample features with high quality evaluation indexes.

[0030] In some embodiments, the processor can extract the associated features of low-quality objects and high-quality objects in the multimodal image data by calculating the quality evaluation indicators of the predicted bounding box and the actual bounding box, and perform layered processing on the samples based on the high-quality threshold and the low-quality threshold to obtain high-quality sample features.

[0031] In some embodiments, the output result of the feature selection layer may also include corresponding medium-quality sample features and low-quality sample features obtained after stratified processing based on the high-quality threshold and the low-quality threshold.

[0032] The transferable feature learning layer is used to learn the multimodal arc features using the high-quality sample features, and obtain arc detection results based on similarity scores. The input of the transferable feature learning layer may include high-quality sample features for multimodal arc features, and the output may include arc detection results.

[0033] In some embodiments, the transferable feature learning layer may include a convolutional module and a fully connected module. For example, the transferable feature learning layer may include 3 consecutive 3×3 convolutional modules and two fully connected modules.

[0034] The convolution module is used to embed the high-quality sample features and the multi-modal arc features into the corresponding feature space to obtain a high-quality feature space vector and an arc feature space vector respectively.

[0035] A high-quality feature space vector is a feature space vector composed of high-quality sample features.

[0036] The arc feature space vector is a feature space vector composed of multi-modal arc features.

[0037] In some embodiments, the processor can embed the high-quality sample features into the corresponding feature space of the multimodal arc features, freeze the parameters of the convolution module input with the high-quality sample features (the dimension of the embedding layer is 256, and the dimension of the hidden layer is set to 512), and obtain the high-quality feature space vector and the arc feature space vector respectively.

[0038] The fully connected module is used to obtain a joint probability distribution set as an arc detection result through optimal transportation based on the high-quality feature space vector and the arc feature space vector.

[0039] The joint probability distribution set is a set of joint probability distributions reflecting all high-quality feature space vectors and arc feature space vectors.

[0040] In some embodiments, the expression of the joint probability distribution set may be: ; ; ; in, express and The set of all joint probability distributions, The feature space vector representing the high-quality sample feature library, Represents the joint probability distribution matrix, which is used to describe the distribution mapping relationship between high-quality sample features and low-quality sample features. It refers to the dimension of the feature matrix, which is related to the batch size and the size of the high-quality sample library. represents the marginal distribution of the joint probability distribution, with dimensions corresponding to the batch size, represents another marginal distribution of the joint probability distribution, and its dimension corresponds to the size of the high-quality sample feature library. Represents the high-quality sample space. , represents the feature alignment weights calculated by the joint probability matrix, represents the optimization objective function in the optimal transmission problem, It represents the feature similarity matrix, usually calculated based on cosine similarity. It means solving the variable that makes the objective function reach the maximum value. represents the target value in the optimal transport problem, represents the entropy of the joint probability distribution matrix, which is used to regularize the objective function. represents the weight of the constraint item, represents the KL divergence, which measures the difference between the joint distribution and the target distribution. represents the regularization parameter, represents the elements of the joint probability matrix, i.e., the matching probability between specific high-quality and low-quality features, represents the normalized joint probability distribution, which is used to further optimize the alignment characteristics.

[0041] The similarity score is a score reflecting the similarity between the high-quality feature space vector and the arc feature space vector. For example, the similarity score may include a high-quality sample feature similarity score and a multimodal arc feature similarity score.

[0042] The high-quality sample feature similarity score is a score indicating how close the high-quality sample feature is to the multimodal arc feature.

[0043] The multimodal arc feature similarity score is a score indicating how close the multimodal arc feature is to the high-quality sample feature.

[0044] In some embodiments, the expressions of the high-quality sample feature similarity score and the multi-modal arc feature similarity score may be: ; ; ; in, Represents the feature similarity score of high-quality samples, i.e., low-quality samples The weight of is used to reflect its importance in the joint probability distribution. Represents the operation of taking the maximum value, , and Represents the elements of each column in each row of the joint probability matrix, corresponding to the The matching weight between low-quality features and each high-quality feature, Represents the multimodal arc feature similarity score, i.e., high-quality samples The weight of is used to reflect its importance in feature matching. Represents a single element in the joint probability matrix, i.e., a low-quality sample and high quality samples The matching probability, Indicates whether the sample is a positive sample with high similarity. Indicates low quality samples The total weight of is usually the same as the row maximum in the joint probability matrix, Indicates the size of the low-quality sample set and the number of low-quality samples in the batch. Indicates high quality sample The total weight of is usually calculated by summing the columns of the joint probability matrix. Represents the size of the high-quality sample set and the number of high-quality samples stored in the feature selection module.

[0045] In some embodiments, the processor may define high similarity samples as positive samples and other samples as negative samples. When the ratio of positive to negative samples exceeds 50%, for negative filling, a virtual feature is obtained by synthesizing a uniformly mixed multimodal arc feature and its farthest high-quality sample feature.

[0046] In some embodiments, the expression of the virtual feature is: ; in, Represents virtual features, i.e., low-quality features after optimization The estimated value of Represents the original low-quality sample The characteristic vector of Indicates the features used to calculate low-quality samples and high-quality sample features The minimum cosine similarity between Indicates the The transposed form of the high-quality sample feature vector.

[0047] In some embodiments, the processor may map the multi-modal arc signature to a high-quality sample signature based on the previous iteration. Update the high-quality sample features to obtain updated high-quality sample features.

[0048] In some embodiments, the expression of the updated high-quality sample feature may be: ; in, Represents the updated high-quality sample features, that is, high-quality sample features The updated result in the t+1th iteration is, Represents the update weight, i.e., the momentum coefficient, which is used to balance the weight between the current feature and the newly calculated feature. Represents high-quality sample features The updated result in the tth iteration is, Represents high-quality sample features in the initial state , Represents the size of the high-quality sample set, which is the total number of high-quality feature samples stored. Indicates the length A vector of all 1s.

[0049] S3: Inputting the multimodal image data into the arc detection model, and performing training using a loss function to obtain a trained arc detection model.

[0050] In some embodiments, the arc detection model can be obtained by training multiple labeled training samples. For example, multiple labeled training samples can be input into the initial arc detection model, and a loss function can be constructed by using the labels and the results of the initial arc detection model. The parameters of the initial arc detection model can be iteratively updated based on the loss function by gradient descent or other methods. When the preset conditions are met, the model training is completed and a trained arc detection model is obtained. The preset conditions can be that the loss function converges, the number of iterations reaches a threshold, etc.

[0051] In some embodiments, the training samples may include multimodal image data, the labels may be corresponding arc detection results, and the labels may be manually annotated.

[0052] In some embodiments, the loss function may be expressed as: ; in, Represents the loss function result, which is used to optimize the similarity between the features of low-quality samples and high-quality samples. It indicates whether sample i is considered a positive sample with high similarity. represents the batch size, that is, the number of low-quality and high-quality sample features in the current training. represents the feature similarity function, represents the high-quality sample feature vector, represents the low-quality sample feature vector, represents the temperature parameter of the contrast loss, which is used to control the smoothness of the similarity distribution. Indicates the range of the number of low-quality samples and high-quality samples.

[0053] In some embodiments, the arc detection model can be trained by unsupervised training samples. and Select and use the fully connected layers to randomly initialize, using features from previous batches to increase the The size of the small batch data processing, by defining and The similarity matrix of is used to optimize the arc detection model using the unsupervised learning loss function to obtain the final arc detection result; Yes The result of concatenating with its neighbor features is that when selecting neighbor features, the average of each At least selected Second-rate.

[0054] The optimization theory results are used to optimize and Similarity parameters.

[0055] In some embodiments, the expression of the optimization theoretical result can be: ; in, Represents the optimization theory result, that is, the joint probability distribution matrix between low-quality sample features, which is used to describe the mapping relationship between low-quality features and their neighborhood features. represents an optimal transmission method that combines entropy regularization and KL divergence constraints to solve the optimal matching problem between features. Represents the similarity matrix between low-quality sample features, represents the number of neighborhood samples, that is, the size of the set of adjacent features used to describe each low-quality sample. Represents the batch size, which refers to the number of low-quality samples in the current training batch. It means solving the variable that makes the objective function reach the maximum value. Represents the joint probability matrix in the objective function and the similarity matrix The trace operation measures the overall similarity of sample feature matching. represents the entropy regularization parameter, which is used to control the joint probability distribution The smoothness of represents the joint probability distribution The entropy is used to regularize the objective function to prevent an overly concentrated distribution during the optimization process. Represents the joint probability distribution matrix, which is used to describe the matching relationship between features. Represents the joint probability matrix elements.

[0056] In some embodiments, the processor may define Ensure OK is a vector, using the first optimization loss pair Optimization is performed so that the arc detection model can extract the global characteristics of the arc.

[0057] In some embodiments, the expression for the optimization loss may be: ; in, Represents the global feature loss, which is used to optimize the global matching of low-quality samples and their neighborhood features. Batch size, that is, the number of samples in the current training, Represents the loss function of a single sample, measuring the estimated probability distribution With features consistency, represents the probability distribution, The feature vector representing the quality sample, The number of neighborhood features, that is, the number of high-quality neighborhood features provided for each low-quality sample, Represents the neighborhood feature vector, which comes from a high-quality feature library. Indicates the degree of smoothness used to control the similarity distribution. Indicates the sample and neighborhood features The matching probability.

[0058] In some embodiments, the processor can utilize an anchor-neighbor exchange strategy to construct a second optimization loss so that the anchor feature and its nearest neighbor features Optimization is performed so that the arc detection model can extract the local features of the arc.

[0059] In some embodiments, the expression of the second optimization loss is: ; in, represents the second optimization loss result, Represents the local matching loss for sample i, measuring the estimated probability distribution and the eigenvector The difference between represents the normalized probability distribution of the extended neighborhood, which is consistent with the sample The neighborhood characteristics are related. represents the local eigenvector of sample i, Represents the normalized probability distribution of sample i, which is used to describe the direct matching relationship between low-quality samples and their neighborhood features.

[0060] In some embodiments, the expression of the unsupervised learning loss function may be: ; in, represents the unsupervised learning loss result, represents the global loss function, represents the local loss function.

[0061] In some embodiments, the processor may use a trained arc detection model to analyze complex railway image data, obtain arc detection results, and complete arc detection of complex railway multi-modality.

[0062] In some embodiments of the present specification, a complex railway multimodal arc detection method is provided, which fuses visible light and infrared images, designs an arc detection model, and provides learnable representation features for decomposing low-rank features and sparse features in the fused image. (1) Using a transferable imitation learning arc detection model, the representation ability of low-quality examples can be improved under the supervision of high-quality examples; (2) An unsupervised transferable representation learning network is designed based on optimization theory, avoiding dependence on any predefined threshold, and can achieve global and local consistency through unsupervised clustering, thereby automatically discovering the differences in arc features in different scenarios, greatly improving the accuracy of arc detection in complex railways.

Claims

1. A complex railway multi-modal arc detection method, characterized in that: include: S1: Acquire multimodal image data of a complex railway; wherein the multimodal image data includes a visible light image and an infrared light image; S2: Construct an arc detection model using the feature extraction layer, feature selection layer, and transferable feature learning layer; S3: Input the multimodal image data into the arc detection model, and train it using the loss function to obtain a trained arc detection model; wherein the trained arc detection model is used to analyze complex railway image data, obtain arc detection results, and complete arc detection of complex railway multimodality.

2. The complex railway multi-modal arc detection method according to claim 1 is characterized in that: The arc detection model includes: A feature extraction layer, used for fusing and extracting features from the visible light image and the infrared light image to obtain multi-modal arc features; wherein the multi-modal arc features include low-rank features and sparse features; The feature selection layer is used to extract the associated features of low-quality objects and high-quality objects in the multimodal image data based on the predicted bounding box and quality evaluation index to obtain high-quality sample features; The transferable feature learning layer is used to learn the multimodal arc features using the high-quality sample features, and obtain arc detection results based on similarity scores.

3. The complex railway multi-modal arc detection method according to claim 2 is characterized in that: The expression of the multi-mode arc characteristic is: ； ； in, represents the multi-modal arc features learned in step t, It means that the soft threshold operator can represent the activation function, represents the trainable variables in the convolutional layer, represents the image after the fusion of infrared and natural light, represents the parameter that adjusts the trade-off between low-rank coefficient and sparse coefficient, represents the representation coefficient learned in step t-1, Represents feature information, represents the representation coefficient learned in step t-1, represents the trainable parameters in the convolutional layer, represents the trainable parameters in the convolutional layer, represents the convolution operation, represents the update weight, i.e. the momentum coefficient, represents the transpose of the complete dictionary, Represents input data, Represents the threshold for shrinking the input matrix and is a learnable variable.

4. The complex railway multi-modal arc detection method according to claim 2, characterized in that: The transferable feature learning layer includes: A convolution module, used for embedding the high-quality sample feature and the multi-modal arc feature into a corresponding feature space to obtain a high-quality feature space vector and an arc feature space vector respectively; The fully connected module obtains a joint probability distribution set as an arc detection result through optimal transportation based on the high-quality feature space vector and the arc feature space vector.

5. The complex railway multi-modal arc detection method according to claim 4, characterized in that: The expression of the joint probability distribution set is: ； ； ； in, express and The set of all joint probability distributions, The feature space vector representing the high-quality sample feature library, Represents the joint probability distribution matrix, which is used to describe the distribution mapping relationship between high-quality sample features and low-quality sample features. It refers to the dimension of the feature matrix, which is related to the batch size and the size of the high-quality sample library. represents the marginal distribution of the joint probability distribution, with dimensions corresponding to the batch size, represents another marginal distribution of the joint probability distribution, and its dimension corresponds to the size of the high-quality sample feature library. Represents the high-quality sample space. , represents the feature alignment weights calculated by the joint probability matrix, represents the optimization objective function in the optimal transmission problem, It is a feature similarity matrix, usually calculated based on cosine similarity. It means solving the variable that makes the objective function reach the maximum value. represents the target value in the optimal transport problem, represents the entropy of the joint probability distribution matrix, which is used to regularize the objective function. represents the weight of the constraint item, represents the KL divergence, which measures the difference between the joint distribution and the target distribution. represents the regularization parameter, represents the elements of the joint probability matrix, i.e., the matching probability between specific high-quality and low-quality features, represents the normalized joint probability distribution, which is used to further optimize the alignment characteristics.

6. The complex railway multi-modal arc detection method according to claim 5, characterized in that: The similarity score includes a high-quality sample feature similarity score and a multi-modal arc feature similarity score, and their specific expressions are: ； ； ； in, Represents the feature similarity score of high-quality samples, i.e., low-quality samples The weight of is used to reflect its importance in the joint probability distribution. Represents the operation of taking the maximum value, , and Represents the elements of each column in each row of the joint probability matrix, corresponding to the The matching weight between low-quality features and each high-quality feature, Represents the multimodal arc feature similarity score, i.e., high-quality samples The weight of is used to reflect its importance in feature matching. Represents a single element in the joint probability matrix, i.e., a low-quality sample and high quality samples The matching probability, Indicates whether the sample is a positive sample with high similarity. Indicates low quality samples The total weight of is usually the same as the row maximum in the joint probability matrix, Indicates the size of the low-quality sample set and the number of low-quality samples in the batch. Indicates high quality sample The total weight of is usually calculated by summing the columns of the joint probability matrix. Represents the size of the high-quality sample set and the number of high-quality samples stored in the feature selection module.

7. The complex railway multi-modal arc detection method according to claim 1, characterized in that: The expression of the loss function is: ； in, Represents the loss function result, which is used to optimize the similarity between the features of low-quality samples and high-quality samples. It indicates whether sample i is considered a positive sample with high similarity. represents the batch size, that is, the number of low-quality and high-quality sample features in the current training. represents the feature similarity function, represents the high-quality sample feature vector, represents the low-quality sample feature vector, represents the temperature parameter of the contrast loss, which is used to control the smoothness of the similarity distribution. Indicates the range of the number of low-quality samples and high-quality samples.

Images