An unsupervised learning and multi-source data track anomaly target detection method

Abstract

This invention provides a method for detecting orbital anomalies using unsupervised learning and multi-source data. The method includes: acquiring a grayscale image I of the original orbital image. RGB and depth map I D A single-mode encoding / decoding network is constructed using an encoder and a decoder to convert grayscale image I... RGB and depth map I D The input is fed into a single-modal codec network to obtain the initial feature map F. RGB and F D ; Construct a Mamba-like feature cross-transformation module to transform the initial feature map F RGB and F D The input is fed into the Mamba-like feature cross-transformation module to obtain the transformed feature map F. T‑D and F T‑RGB ;Transfer grayscale image I RGB Depth Map I D Initial feature map F RGB F D Transformation Feature Map F T‑D and F T‑RGB The data is input into the anomaly map generation module, where a final anomaly map is generated using a hybrid attention approach combining product and superposition. Based on the pixel values ​​of each coordinate point in the final anomaly map, the identification and localization of anomalous targets in the original track image are achieved. This invention leverages the robustness and anti-interference capabilities of multimodal data augmentation algorithms to improve the intelligence level of intelligent track inspection.

Description

Technical Field

[0001] This invention relates to the field of intelligent inspection technology for rail transit, and in particular to a method for detecting track anomalies using unsupervised learning and multi-source data. Background Technology

[0002] Track anomaly target detection is one of the important tasks of track inspection. Currently, most intelligent track anomaly target detection methods are target detection methods. Target detection uses labeled files of anomaly data to supervise the learning of neural networks, thereby achieving accurate detection of targets of known categories.

[0003] However, the target detection method performs poorly in the field of orbital anomaly detection for the following two reasons.

[0004] 1) Anomalous targets are rare. Collecting abundant anomalous data during routine track inspections is challenging. The size and variety of training datasets significantly limit the accuracy of target detection algorithms. Furthermore, data annotation is a costly and labor-intensive task. Therefore, the application of target detection algorithms in track anomaly detection has certain limitations.

[0005] 2) The categories of anomalous targets are infinite. The track is an open environment, and various static objects from the natural environment can potentially intrude onto the track, such as fallen wrenches, loose fasteners, and cardboard boxes. Clearly, the categories of anomalous targets cannot be completely exhaustively listed. Target detection algorithms cannot identify anomalous targets that are not categorized. Therefore, target detection algorithms cannot meet all the requirements for track anomalous target inspection.

[0006] With the continuous breakthroughs in computer vision technology, an unsupervised learning method that does not rely on abnormal samples has rapidly emerged in industrial product defect detection. This method can detect abnormal targets simply by learning the feature distribution of normal samples, effectively solving the two problems mentioned above. Therefore, some scholars have applied the concept of unsupervised learning to the field of track inspection and achieved some success. However, most existing unsupervised track anomaly detection methods rely solely on visible light images. Under natural light, the similarity between the background and foreground can cause anomalies to go unnoticed, leading to missed detections and false positives. Depth information offers a solution to improve detection performance because there is a clear difference in spatial depth distribution between anomalies and the background. However, how to rationally and efficiently utilize visible light images and depth information needs to be considered. Summary of the Invention

[0007] This invention provides an unsupervised learning and multi-source data method for detecting orbital anomalies, effectively achieving anomaly image identification and target pixel localization.

[0008] To achieve the above objectives, the present invention adopts the following technical solution.

[0009] An unsupervised learning and multi-source data-based method for detecting orbital anomalies includes:

[0010] I. Obtain the grayscale image of the original orbital image to be used for anomaly target detection RGB and depth map I D A single-mode encoding / decoding network is constructed using an encoder and a decoder to process the grayscale image I. RGB and the depth map I D The input is fed into the single-modal encoding / decoding network to obtain the initial feature map F. RGB and F D ;

[0011] Construct a Mamba-like feature cross-transformation module to transform the initial feature map F RGB and F D The input is fed into the Mamba-like feature cross-transformation module to obtain the transformed feature map F. T-D and F T-RGB ;

[0012] The grayscale image I RGB Depth Map I D Initial feature map F RGB F D Transformation Feature Map F T-D and F T-RGB The input is fed into the anomaly map generation module with multiple decisions, and the final anomaly map is generated through a hybrid attention method of product and superposition. Based on the pixel value of each coordinate point of the final anomaly map, the anomaly target in the original orbit image is identified and located.

[0013] Preferably, the grayscale image I of the original orbital image to be acquired for abnormal target detection is... RGB and depth map I D A single-mode encoding / decoding network is constructed using an encoder and a decoder to process the grayscale image I. RGB and the depth map I D The input is fed into the single-modal encoding / decoding network to obtain the initial feature map F. RGB and F D ,include:

[0014] A single-modal encoding and decoding network is constructed using an encoder and a decoder. The encoder consists of alternating 3×3 and 5×5 convolutional layers and ReLU activation functions, while the decoder consists of alternating bilinear interpolation upsampling layers, 3×3 convolutional layers, activation function layers, and Dropout layers.

[0015] I. Obtain the grayscale image of the original orbital image to be used for anomaly target detection RGB and depth map I D Grayscale image IRGB and depth map I D When inputting a grayscale image into a single-modal encoder-decoder network, the number of feature map channels in each encoder layer increases with the number of layers. Conversely, when inputting a depth image, the number of feature map channels in each encoder layer increases with the number of layers, while the number of feature map channels in each decoder layer decreases with the number of layers. After the grayscale and depth images are upsampled by their respective encoders, the initial feature map F is obtained. RGB and F D The single-mode codec network also outputs a grayscale image of the reconstructed orbit. R-RGB and depth map I R-D .

[0016] Preferably, the Mamba-like feature cross-transformation module is used to transform the initial feature map F RGB and F D The input is fed into the Mamba-like feature cross-transformation module to obtain the transformed feature map F. T-D and F T-RGB ,include:

[0017] A Mamba-like feature cross-transformation module for cross-modal feature projection is constructed. This Mamba-like feature cross-transformation module includes a convolutional block and a Mamba-like block. The convolutional block consists of three 1×1 convolutional layers. The Mamba-like block consists of a forget gate branch and a multi-path joint convolutional branch. The forget gate branch consists of a normalization layer (Norm), a linear transformation layer (Linear), and a nonlinear SiLU activation function. The multi-path joint convolutional branch consists of a linear layer, a convolutional layer, a nonlinear SiLU activation function, a skip connection, and an addition operation. The number of channels is adjusted to the average number of channels in the two modalities and then converted to the number of channels in the projected modality.

[0018] The initial feature map F D and F RGB The inputs are fed into a Mamba-like feature cross-transformation module with 256 and 128 channels respectively, first yielding feature maps. and feature map The number of channels is 128 and 256 respectively, Conv 3 The convolution operation representing a three-layer convolutional layer has a 1×1 kernel;

[0019]

[0020] Then, by focusing on local feature representation through the forget gate branch, the feature map is obtained. and

[0021]

[0022] The feature map is then obtained through multi-path joint convolution branches. and The multi-path joint convolution branch specifically includes two convolutional layer branches and one skip connection. The sum of the three branches is obtained. and

[0023] Feature map and Element-wise dot product is performed, and attention weights are allocated between feature channels through a linear layer to obtain the feature map. and

[0024]

[0025] Transformed feature map F T-D From feature map and feature map The sum is obtained as the transformed feature map F. T-RGB From feature map and feature map Add them together to get;

[0026]

[0027] Preferably, the grayscale image I is... RGB Depth Map I D Initial feature map F RGB F D Transformation Feature Map F T-D and F T-RGB The input is fed into the multi-decision anomaly map generation module, which generates the final anomaly map through a hybrid attention method of product and superposition. Based on the pixel value of each coordinate point in the final anomaly map, the module identifies and locates anomalous targets in the original orbital image, including:

[0028] Image I RGB and I R-RGB Calculate the Euclidean distance between the pixels of two images, store all calculated Euclidean distance values ​​in an array, and generate a grayscale reconstruction anomaly map (AM) based on all stored Euclidean distance values. R-Texture ;

[0029] Image I D and I R-D Calculate the Euclidean distance between the pixels of two images, store all calculated Euclidean distance values ​​in an array, and generate a depth reconstruction anomaly map (AM) based on all stored Euclidean distance values. R-Spatial ;

[0030] Image F RGB and F T-RGBCalculate the Euclidean distance between the pixels of two images, store all calculated Euclidean distance values ​​in an array, and generate a depth transformation anomaly map (AM) based on all stored Euclidean distance values. T-Spatial ;

[0031] Image F D and F T-D Calculate the Euclidean distance between the pixels of two images, store all calculated Euclidean distance values ​​in an array, and generate a grayscale conversion anomaly image (AM) based on all stored Euclidean distance values. T-Texture ;

[0032] The grayscale anomaly map AM is obtained by multiplying two anomaly maps within the same modality. Texture and depth anomaly map AM Spatial ;

[0033] AM Texture =AM R-Texture *AM T-Texture

[0034] AM Spatial =AM R-Spatial *AM T-Spatial

[0035] Gray-scale anomaly image AM Texture and depth anomaly map AM Spatial Adding them together yields the final anomaly map AM. Final ;

[0036] AM Final =AM Texture +AM Spatial

[0037] Final Anomaly Graph AM Final Each pixel value at coordinate point represents the anomaly score for that point, ranging from 0 to 1. The maximum score is the final anomaly map AM. Final The abnormal score is calculated based on the pixel coordinates of the abnormal image. If the abnormal score of a point is greater than a threshold, the point is identified as an abnormal point. Multiple abnormal points are aggregated to form an abnormal target, thus achieving the localization of the abnormal target.

[0038] Preferably, the method further includes:

[0039] A four-branch similarity learning collaborative constraint module is constructed, with the joint optimization objective being to minimize the dual differences within a single modality and across modalities. The loss function value is calculated, and the parameter values ​​of the single-modal encoding / decoding network, the Mamba-like feature cross-transformation module, and the anomaly graph generation module are adjusted based on the loss function value. This completes the training process of the single-modal encoding / decoding network, the Mamba-like feature cross-transformation module, and the anomaly graph generation module, resulting in the trained single-modal encoding / decoding network, the Mamba-like feature cross-transformation module, and the anomaly graph generation module.

[0040] The four-branch similarity learning collaborative constraint module Loss total The reconstruction loss function Loss R and transformation loss function Loss T Added together;

[0041] Loss total =Loss R +Loss T

[0042] Reconstruction Loss Function R Using the original and reconstructed orbit images as input, calculate the pixel-wise cosine similarity error CosineSimilarity().mean, which is derived from the bimodal independent loss. and Add them together to get;

[0043]

[0044] I RGB For the original grayscale image, I D For the original depth map, I R-RGB To reconstruct the grayscale image, I R-D To reconstruct the depth map;

[0045] Transformation loss function Loss T Using the initial feature map and the transformed feature map as input, calculate the pixel-wise cosine similarity error CosineSimilarity().mean, which is derived from the bimodal independent loss. and Add them together to get;

[0046]

[0047] F RGB For the initial grayscale feature map, F D For the initial depth feature map, F T-D To transform the deep feature map, F T-RGB To convert the grayscale feature map.

[0048] As can be seen from the technical solutions provided by the embodiments of the present invention described above, the present invention solves the two major problems faced in the field of track inspection: the scarcity of abnormal target data and the infinite number of target categories. It effectively alleviates the problem that single visible light images are easily affected by factors such as lighting, leading to a decrease in algorithm performance. The present invention utilizes the robustness and anti-interference capabilities of multimodal data enhancement algorithms to improve the intelligence level of intelligent track inspection.

[0049] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and will become apparent from the description or may be learned by practice of the invention. Attached Figure Description

[0050] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 A flowchart illustrating the processing of an unsupervised learning and multi-source data orbital anomaly target detection method provided in an embodiment of the present invention;

[0052] Figure 2 This is a diagram illustrating the detection effect of an abnormal orbital target provided in an embodiment of the present invention. Detailed Implementation

[0053] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0054] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or couplings. The term “and / or” as used herein includes any and all combinations of one or more of the associated listed items.

[0055] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0056] To facilitate understanding of the embodiments of the present invention, the following will provide further explanation and description with reference to the accompanying drawings and several specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.

[0057] This invention provides an unsupervised learning and multi-source data-based method for detecting orbital anomalies. The method takes matched orbital grayscale and depth maps as input, extracts and reconstructs features from each input using a single-modal lightweight encoder-decoder, then uses a Mamba-like feature cross-transformation module to achieve feature projection between modalities, and a four-branch similarity learning collaborative constraint module to guide network training, ensuring optimization of both single-modal reconstruction and cross-modal transformation. Finally, a multi-decision anomaly map generation strategy is used to generate the final fused anomaly map, thereby achieving anomaly image identification and target pixel localization.

[0058] The processing flow of the unsupervised learning and multi-source data orbital anomaly target detection method provided in this embodiment of the invention is as follows: Figure 1 As shown, the processing steps include the following:

[0059] Step 1: Using the registered and calibrated orbit grayscale image and depth map as input, construct a lightweight single-mode codec network to extract and reconstruct single-mode normal orbit features.

[0060] Step 1 specifically includes the following steps:

[0061] Step 1.1: Obtain the grayscale image I of the original orbital image to be used for anomaly target detection. RGB and depth map I D A single-mode encoding / decoding network is constructed using an encoder and a decoder to convert grayscale image I... RGB and depth map I D The input is fed into a single-mode codec network.

[0062] Step 1.2: The encoder consists of 9 layers, alternating between 3×3 and 5×5 convolutional layers and the ReLU activation function.

[0063] Step 1.3: When the grayscale image is input, as the number of layers increases, the number of feature map channels in each layer of the encoder increases, in the following order: 32, 64, 128, 128, 256.

[0064] Step 1.4: When the depth map is input, as the number of layers increases, the number of feature map channels in each layer of the encoder increases, in the following order: 32, 32, 64, 128, 128.

[0065] Step 1.5: The decoder consists of 20 layers, alternating between bilinear interpolation upsampling layers, 3×3 convolutional layers, activation function layers, and Dropout layers.

[0066] Step 1.6: As the number of layers increases, the number of feature map channels in each layer of the decoder decreases, in the following order: 128, 64, 32, 16, 8, 3.

[0067] Step 1.7: After upsampling the grayscale image and depth image through their respective encoders, the initial feature map F is obtained. RGB and F D The single-modal encoding / decoding network also outputs a grayscale image of the reconstructed orbit. R-RGB and depth map I R-D .

[0068] Step 2: Design a Mamba-like feature cross-transformation module to reveal the feature transformation rules between modalities, complete the cross-modal feature mutual projection, and enhance the consistent representation pattern of normal features; the initial feature map F... RGB and F D The input is fed into the Mamba-like feature cross-transformation module to obtain the transformed feature map F. T-D and F T-RGB .

[0069] Step 2 specifically includes the following steps:

[0070] Step 2.1, the input to this module is the initial feature map F. RGB and F D The number of channels is 256 and 128 respectively, and the output is the transformed feature map F. T-D and F T-RGB The number of channels are 128 and 256 respectively;

[0071] Step 2.2, the Mamba-like feature cross-transformation module consists of two parts: a convolutional block and a Mamba-like block;

[0072] Step 2.3: The convolutional block consists of three 1×1 convolutional layers. The number of channels is adjusted to the average of the number of channels in the two modalities, and then converted to the number of channels in the projected modality. Taking the conversion of grayscale modal features to deep modal features as an example, the feature map is obtained. Taking the conversion of deep modal features to grayscale modal features as an example, the feature map is obtained. Conv 3 The convolution operation representing a three-layer convolutional layer has a 1×1 kernel;

[0073]

[0074] Step 2.4: The Mamba-like block consists of a forget gate branch and a multi-path joint convolution branch;

[0075] Step 2.5: The forget gate branch consists of a normalized layer (Norm), a linear transformation layer (Linear), and a nonlinear SiLU activation function. It focuses on local feature representation to obtain the feature map. and

[0076]

[0077] Step 2.6: The multi-path joint convolutional branch consists of a linear layer, a convolutional layer, a nonlinear SiLU activation function, skip connections, and an addition operation, resulting in a feature map. and The multi-path joint convolution branch specifically includes two convolutional layer branches and one skip connection. The sum of the three branches is obtained. and

[0078] Step 2.7, Feature Map and Element-wise dot product is performed, and attention weights are allocated between feature channels through a linear layer to obtain the feature map. and

[0079]

[0080] Step 2.8, Transform feature map F T-D From feature map and feature map The sum is obtained as the transformed feature map F. T-RGB From feature map and feature map Add them together to get;

[0081]

[0082] Step 3: A multi-decision anomaly map generation module is used to generate the final anomaly map through a hybrid attention method of product and superposition. The maximum score of the anomaly map is used as the anomaly score to complete the anomaly image identification and target pixel localization.

[0083] Step 3.1, calculate image I using the Euclidean distance function. RGB and I R-RGB Image I D and I R-D Feature map F RGB and F T-RGBFeature map F D and F T-D The pixel-level differences between them generate four anomaly maps, namely the grayscale reconstruction anomaly map AM. R-Texture Deep reconstruction anomaly map AM R-Spatial Depth Conversion Anomaly Map AM T-Spatial and grayscale conversion anomaly map AM T-Texture ;

[0084] Image I RGB and I R-RGB Calculate the Euclidean distance between the pixels of two images, store all calculated Euclidean distance values ​​in an array, and generate a grayscale reconstruction anomaly map (AM) based on all stored Euclidean distance values. R-Texture .

[0085] Image I D and I R-D Calculate the Euclidean distance between the pixels of two images, store all calculated Euclidean distance values ​​in an array, and generate a depth reconstruction anomaly map (AM) based on all stored Euclidean distance values. R-Spatial .

[0086] Image F RGB and F T-RGB Calculate the Euclidean distance between the pixels of two images, store all calculated Euclidean distance values ​​in an array, and generate a depth transformation anomaly map (AM) based on all stored Euclidean distance values. T-Spatial .

[0087] Image F D and F T-D Calculate the Euclidean distance between the pixels of two images, store all calculated Euclidean distance values ​​in an array, and generate a grayscale conversion anomaly image (AM) based on all stored Euclidean distance values. T-Texture .

[0088] Step 3.2: Perform a dot product calculation on two anomaly maps within the same modality to obtain the grayscale anomaly map AM. Texture and depth anomaly map AM Spatial That is, an abnormality is confirmed only if both results are judged to be abnormal.

[0089] AM Texture =AM R-Texture *AM T-Texture

[0090] AM Spatial =AM R-Spatial *AM T-Spatial

[0091] Step 3.3: Integrate the anomaly detection results from different modalities by adding the results together, and combine the grayscale anomaly map AM. Textureand depth anomaly map AM Spatial Adding them together yields the final anomaly map AM. Final This enables the complementarity and enhancement of cross-modal anomaly information, with the maximum anomaly map score being used as the anomaly score of the detected sample, thereby achieving accurate localization of the abnormal target.

[0092] AM Final =AM Texture +AM Spatial

[0093] Final Anomaly Graph AM Final Each pixel value at coordinate point represents the anomaly score for that point, ranging from 0 to 1. The maximum score is the final anomaly map AM. Final Abnormal scores.

[0094] In the anomaly image pixel coordinates, if the anomaly score of a point is greater than a threshold, the point is located as an anomaly. Multiple anomaly points are aggregated to form an anomaly target, thus realizing the location of the anomaly target.

[0095] Step 4: Design a four-branch similarity learning collaborative constraint module with the joint optimization objective of minimizing both intramodal and intermodal differences. Calculate the loss function value and adjust the parameter values ​​of the aforementioned single-modal encoding / decoding network, Mamba-like feature cross-transformation module, and anomaly graph generation module based on the loss function value. Complete the training process of the aforementioned single-modal encoding / decoding network, Mamba-like feature cross-transformation module, and anomaly graph generation module to obtain the trained single-modal encoding / decoding network, Mamba-like feature cross-transformation module, and anomaly graph generation module.

[0096] Step 4 specifically includes the following steps:

[0097] Step 4.1, Four-branch similarity learning collaborative constraint module Loss total The reconstruction loss function Loss R and transformation loss function Loss T Added together;

[0098] Loss total =Loss R +Loss T

[0099] Step 4.2, Reconstruct the Loss Function R Using the original and reconstructed orbit images as input, calculate the pixel-wise cosine similarity error CosineSimilarity().mean, which is derived from the bimodal independent loss. and Add them together to get;

[0100]

[0101] Step 4.3, Transform Loss Function T Using the initial feature map and the transformed feature map as input, calculate the pixel-wise cosine similarity error CosineSimilarity().mean, which is derived from the bimodal independent loss. and Add them together to get;

[0102]

[0103] As shown in Table 1, this invention selects I-AUROC as the accuracy index for anomaly image discrimination, and P-AUROC and P-AUPRO as the accuracy indexes for anomaly target pixel localization. The maximum value of the three is 1, and the closer to 1, the higher the accuracy. The track anomaly target detection algorithm proposed in this invention can achieve an anomaly image detection accuracy of 97.00% and an anomaly target pixel localization accuracy of 96.60%, which is significantly better than the existing unsupervised anomaly target detection algorithms. Figure 2 This is a diagram illustrating the detection effect of an abnormal orbital target provided in an embodiment of the present invention.

[0104] In the processing of multimodal orbital data, this invention employs four steps: a single-modal lightweight encoder / decoder, a Mamba-like feature cross-conversion module, a four-branch similarity learning collaborative constraint module, and a multi-decision anomaly graph generation strategy. By utilizing an unsupervised learning method to learn the features of normal orbital samples, this invention avoids the problem of low algorithm accuracy due to the scarcity of anomaly samples and effectively supplements the content of orbital inspection.

[0105] Table 1 Comparison of accuracy evaluation indicators for track anomaly target detection algorithms

[0106]

[0107] In summary, this invention proposes a novel deep learning neural network for the automatic detection of anomalous targets in orbital surface detection. This method overcomes the scarcity and infinite variety of anomalous targets in orbital scenes by utilizing unsupervised learning and cross-modal data, thus improving detection accuracy and speed. This invention proposes a dual fusion processing mode at the decision and feature levels for cross-modal data, effectively improving the reliability of the detection results. The Mamba-like feature cross-transformation module proposed in this invention achieves interaction and correction between grayscale and depth feature representations through nonlinear attention and connection operations, effectively improving the discriminative ability of anomalous images.

[0108] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing the present invention.

[0109] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.

[0110] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for apparatus or system embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The apparatus and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0111] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for unsupervised learning and multi-source data track anomaly target detection, characterized in that, The method comprises the steps of: obtain a gray image I of an original track image to be subjected to abnormal target detection RGB and a depth image I D , construct a single-modal encoding and decoding network by using an encoder and a decoder, input the gray image I RGB and the depth image I D into the single-modal encoding and decoding network, obtain initial feature maps F RGB and F D , and a gray image I R-RGB and a depth image I R-D of a reconstructed track image; Constructing a Mamba-like feature cross conversion module, inputting the initial feature maps F RGB and F D into the Mamba-like feature cross conversion module to obtain converted feature maps F T-D and F T-RGB ; The grayscale image I RGB Depth Map I D Initial feature map F RGB F D The grayscale image of the reconstructed orbit is I R-RGB and depth map I R-D Transformation Feature Map F T-D and F T-RGB The input is fed into the anomaly map generation module with multiple decisions, and the final anomaly map is generated through a hybrid attention method of product and superposition. The anomaly target in the original orbit image is identified and located based on the pixel value of each coordinate point of the final anomaly map. The constructing Mamba feature cross conversion module inputs the initial feature maps F RGB and F D into the Mamba feature cross conversion module to obtain the converted feature maps F T-D and F T-RGB , comprising: A Mamba-like feature cross conversion module is constructed to project features between cross-modalities, the Mamba-like feature cross conversion module comprises a convolution block and a Mamba-like block, the convolution block comprises three 1×1 convolution layers, the Mamba-like block is composed of a forgetting gate branch and a multi-path joint convolution branch, the forgetting gate branch is composed of a normalization layer Norm, a linear transformation layer Linear and a nonlinear SiLU activation function, the multi-path joint convolution branch is composed of a linear layer, a convolution layer, a nonlinear SiLU activation function, a skip connection and an addition operation, and the channel number is adjusted to the average of the channel numbers of the two modalities and then converted into the channel number of the projected modality; The initial feature map F D and F RGB are input into a Mamba class feature cross conversion module, the channel numbers are 256 and 128 respectively, to obtain feature maps F D 1 and F RGB 1 , the channel numbers are 128 and 256 respectively, Conv 3 represents a convolution operation with a three-layer cascaded convolution kernel of 1x1. Through the forgetting gate branch, the local feature expression is focused, and a feature map F is obtained D 2 and F RGB 2 ; The feature map F is obtained through the multi-path joint convolution branch again D 3 and F RGB 3 The multi-path joint convolution branch specifically includes two convolution layer branches and a skip connection, and F is obtained by adding the three branches D 3 and F RGB 3 ; Feature map F D 2 and F D 3 Element-wise point multiplication, and pass through a linear layer, realize the attention weight distribution between feature channels, get feature map F D 4 and F RGB 4 ; transformed feature map F T-D from the feature map F D 1 and the feature map F D 4 is added to obtain a transformed feature map F T-RGB from the feature map F RGB 1 and the feature map F RGB 4 is added to obtain; 。 2. The method of claim 1, wherein, The gray image I of the original track image to be subjected to abnormal target detection RGB And a depth map I D , using an encoder and a decoder to build a single-modal encoding and decoding network, inputting the gray image I RGB And the depth map I D Into the single-modal encoding and decoding network, obtaining an initial feature map F RGB And F D , comprising: A single-modality encoding and decoding network is constructed by using an encoder and a decoder, the encoder is composed of 3×3 and 5×5 convolution layers and activation functions ReLU in an alternating manner, and the decoder is composed of a bilinear difference upsampling layer, a 3×3 convolution layer, an activation function layer and a Dropout layer in an alternating manner; I. Obtain the grayscale image of the original orbital image to be used for anomaly target detection RGB and depth map I D Grayscale image I RGB and depth map I D When inputting a grayscale image into a single-modal encoder-decoder network, the number of feature map channels in each encoder layer increases with the number of layers. Conversely, when inputting a depth image, the number of feature map channels in each encoder layer increases with the number of layers, while the number of feature map channels in each decoder layer decreases with the number of layers. After the grayscale and depth images are upsampled by their respective encoders, the initial feature map F is obtained. RGB and F D The single-mode codec network also outputs a grayscale image of the reconstructed orbit. R-RGB and depth map I R-D .

3. The method of claim 2, wherein, The gray-scale image I RGB , the depth map I D , the initial feature map F RGB , F D , the converted feature map F T-D and F T-RGB are input into a multiple decision abnormality map generation module, and a final abnormality map is generated through a hybrid attention mode of multiplication and superposition, and the identification and positioning of the abnormal target in the original track image are realized according to the pixel value of each coordinate point of the final abnormality map. The image I RGB and I R-RGB The Euclidean distance is calculated point by point between two image pixels, all the calculated Euclidean distance values are stored in an array, and a gray-scale reconstruction anomaly map AM is generated according to all the stored Euclidean distance values R-Texture ; The image I D and I R-D The Euclidean distance is calculated point by point between two image pixels, all the calculated Euclidean distance values are stored in an array, and a depth reconstruction anomaly map AM R-Spatial is generated according to all the stored Euclidean distance values. F RGB and F T-RGB The two image pixel points are calculated point by point to obtain the Euclidean distance, all the calculated Euclidean distance values are stored in an array, and a depth conversion anomaly map AM T-Spatial is generated according to all the stored Euclidean distance values. F D and F T-D The Euclidean distance is calculated for two image pixels point by point, all the calculated Euclidean distance values are stored in an array, and a grayscale conversion anomaly map AM T-Texture is generated according to all the stored Euclidean distance values. Point multiplication calculation between two anomaly maps within the same modality, resulting in a grayscale anomaly map AM Texture and a depth anomaly map AM Spatial ; The grayscale anomaly map AM Texture and the depth anomaly map AM Spatial are added to obtain a final anomaly map AM Final ; the final anomaly map AM Final The pixel value of each coordinate point of the final anomaly map AM represents the anomaly score of the point, which belongs to 0-1, and the maximum score is the anomaly score of the final anomaly map AM Final If the anomaly score of the coordinate point is greater than the threshold value, the point is located as an anomaly point under the anomaly pixel coordinate, and a plurality of anomaly points are aggregated to form an anomaly target, so that the anomaly target is located.

4. The method according to any one of claims 1 to 3, characterized in that, The method further comprises the steps of: A four-branch similarity learning collaborative constraint module is constructed to minimize the double differences within a single modality and between cross-modalities as a joint optimization target, a loss function value is calculated, the parameter values of the single-modality encoding and decoding network, the Mamba-like feature cross conversion module and the anomaly graph generation module are adjusted according to the loss function value, the training process of the single-modality encoding and decoding network, the Mamba-like feature cross conversion module and the anomaly graph generation module is completed, and the trained single-modality encoding and decoding network, Mamba-like feature cross conversion module and anomaly graph generation module are obtained. The four-branch similarity learning collaborative constraint module Loss total By the reconstruction loss function Loss R And the conversion loss function Loss T Add up; Reconstruction loss function Loss R With the original track image and the reconstructed track image as input, the pixel-wise cosine similarity error CosineSimilarity().mean is calculated, and the bimodal independent loss Loss RGB R and Loss D R are added together; I RGB is the original grayscale image, I D is the original depth image, I R-RGB is the reconstructed grayscale image, I R-D is the reconstructed depth image; The conversion loss function Loss T With the initial feature map and the conversion feature map as input, the pixel-by-pixel cosine similarity error CosineSimilarity().mean is calculated, and the bimodal independent loss Loss RGB T And Loss D T are added together. F RGB is an initial gray-scale feature map, F D is an initial depth feature map, F T-D is a converted depth feature map, F T-RGB is a converted gray-scale feature map.

Images