Unsupervised learning and multi-source data track abnormal target detection method

Abstract

The invention provides an unsupervised learning and multi-source data track abnormal target detection method. The method comprises the following steps: acquiring a grey-scale map IRGB and a depth map ID of an original track image, constructing a single-mode coding and decoding network by adopting an encoder and a decoder, and inputting the grey-scale map IRGB and the depth map ID into the single-mode coding and decoding network to obtain initial feature maps FRGB and FD; constructing a similar Mama feature cross conversion module, and inputting the initial feature maps FRGB and FD into the similar Mama feature cross conversion module to obtain converted feature maps FT-D and FT-RGB; and inputting the grey-scale image IRGB, the depth image ID, the initial feature images FRGB and FD and the conversion feature images FT-D and FT-RGB into an abnormal image generation module, generating a final abnormal image through a mixed attention mode of product and superposition, and recognizing and positioning an abnormal target in the original track image according to a pixel value of each coordinate point of the final abnormal image. According to the invention, the robustness and anti-interference performance of the algorithm are enhanced by using the multi-modal data, and the intelligent level of track intelligent inspection is improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent inspection technology for rail transit, and in particular to a method for detecting track anomalies using unsupervised learning and multi-source data. Background Technology

[0002] Track anomaly target detection is one of the important tasks of track inspection. Currently, most intelligent track anomaly target detection methods are target detection methods. Target detection uses labeled files of anomaly data to supervise the learning of neural networks, thereby achieving accurate detection of targets of known categories.

[0003] However, the target detection method performs poorly in the field of orbital anomaly detection for the following two reasons.

[0004] 1) Anomalous targets are rare. Collecting abundant anomalous data during routine track inspections is challenging. The size and variety of training datasets significantly limit the accuracy of target detection algorithms. Furthermore, data annotation is a costly and labor-intensive task. Therefore, the application of target detection algorithms in track anomaly detection has certain limitations.

[0005] 2) The categories of anomalous targets are infinite. The track is an open environment, and various static objects from the natural environment can potentially intrude onto the track, such as fallen wrenches, loose fasteners, and cardboard boxes. Clearly, the categories of anomalous targets cannot be completely exhaustively listed. Target detection algorithms cannot identify anomalous targets that are not categorized. Therefore, target detection algorithms cannot meet all the requirements for track anomalous target inspection.

[0006] With the continuous breakthroughs in computer vision technology, an unsupervised learning method that does not rely on abnormal samples has rapidly emerged in industrial product defect detection. This method can detect abnormal targets simply by learning the feature distribution of normal samples, effectively solving the two problems mentioned above. Therefore, some scholars have applied the concept of unsupervised learning to the field of track inspection and achieved some success. However, most existing unsupervised track anomaly detection methods rely solely on visible light images. Under natural light, the similarity between the background and foreground can cause anomalies to go unnoticed, leading to missed detections and false positives. Depth information offers a solution to improve detection performance because there is a clear difference in spatial depth distribution between anomalies and the background. However, how to rationally and efficiently utilize visible light images and depth information needs to be considered. Summary of the Invention

[0007] This invention provides an unsupervised learning and multi-source data method for detecting orbital anomalies, effectively achieving anomaly image identification and target pixel localization.

[0008] To achieve the above objectives, the present invention adopts the following technical solution.

[0009] An unsupervised learning and multi-source data-based method for detecting orbital anomalies includes:

[0010] I. Obtain the grayscale image of the original orbital image to be used for anomaly target detection RGB and depth map I D A single-mode encoding / decoding network is constructed using an encoder and a decoder to process the grayscale image I. RGB and the depth map I D The input is fed into the single-modal encoding / decoding network to obtain the initial feature map F. RGB and F D ;

[0011] Construct a Mamba-like feature cross-transformation module to transform the initial feature map F RGB and F D The input is fed into the Mamba-like feature cross-transformation module to obtain the transformed feature map F. T-D and F T-RGB ;

[0012] The grayscale image I RGB Depth Map I D Initial feature map F RGB F D Transformation Feature Map F T-D and F T-RGB The input is fed into the anomaly map generation module with multiple decisions, and the final anomaly map is generated through a hybrid attention method of product and superposition. Based on the pixel value of each coordinate point of the final anomaly map, the anomaly target in the original orbit image is identified and located.

[0013] Preferably, the grayscale image I of the original orbital image to be acquired for abnormal target detection is... RGB and depth map I D A single-mode encoding / decoding network is constructed using an encoder and a decoder to process the grayscale image I. RGB and the depth map I D The input is fed into the single-modal encoding / decoding network to obtain the initial feature map F. RGB and F D ,include:

[0014] A single-modal encoding and decoding network is constructed using an encoder and a decoder. The encoder consists of alternating 3×3 and 5×5 convolutional layers and ReLU activation functions, while the decoder consists of alternating bilinear interpolation upsampling layers, 3×3 convolutional layers, activation function layers, and Dropout layers.

[0015] I. Obtain the grayscale image of the original orbital image to be used for anomaly target detection RGB and depth map I D Grayscale image IRGB and depth map I D When inputting a grayscale image into a single-modal encoder-decoder network, the number of feature map channels in each encoder layer increases with the number of layers. Conversely, when inputting a depth image, the number of feature map channels in each encoder layer increases with the number of layers, while the number of feature map channels in each decoder layer decreases with the number of layers. After the grayscale and depth images are upsampled by their respective encoders, the initial feature map F is obtained. RGB and F D The single-mode codec network also outputs a grayscale image of the reconstructed orbit. R-RGB and depth map I R-D .

[0016] Preferably, the Mamba-like feature cross-transformation module is used to transform the initial feature map F RGB and F D The input is fed into the Mamba-like feature cross-transformation module to obtain the transformed feature map F. T-D and F T-RGB ,include:

[0017] A Mamba-like feature cross-transformation module for cross-modal feature projection is constructed. This Mamba-like feature cross-transformation module includes a convolutional block and a Mamba-like block. The convolutional block consists of three 1×1 convolutional layers. The Mamba-like block consists of a forget gate branch and a multi-path joint convolutional branch. The forget gate branch consists of a normalization layer (Norm), a linear transformation layer (Linear), and a nonlinear SiLU activation function. The multi-path joint convolutional branch consists of a linear layer, a convolutional layer, a nonlinear SiLU activation function, a skip connection, and an addition operation. The number of channels is adjusted to the average number of channels in the two modalities and then converted to the number of channels in the projected modality.

[0018] The initial feature map F D and F RGB The inputs are fed into a Mamba-like feature cross-transformation module with 256 and 128 channels respectively, first yielding feature maps. and feature map The number of channels is 128 and 256 respectively, Conv 3 The convolution operation representing a three-layer convolutional layer has a 1×1 kernel;

[0019]

[0020] Then, by focusing on local feature representation through the forget gate branch, the feature map is obtained. and

[0021]

[0022] The feature map is then obtained through multi-path joint convolution branches. and The multi-path joint convolution branch specifically includes two convolutional layer branches and one skip connection. The sum of the three branches is obtained. and

[0023] Feature map and Element-wise dot product is performed, and attention weights are allocated between feature channels through a linear layer to obtain the feature map. and

[0024]

[0025] Transformed feature map F T-D From feature map and feature map The sum is obtained by transforming the feature map F. T-RGB From feature map and feature map Add them together to get;

[0026]

[0027] Preferably, the grayscale image I is... RGB Depth Map I D Initial feature map F RGB F D Transformation Feature Map F T-D and F T-RGB The input is fed into the multi-decision anomaly map generation module, which generates the final anomaly map through a hybrid attention method of product and superposition. Based on the pixel value of each coordinate point in the final anomaly map, the module identifies and locates anomalous targets in the original orbital image, including:

[0028] Image I RGB and I R-RGB Calculate the Euclidean distance between the pixels of two images, store all calculated Euclidean distance values ​​in an array, and generate a grayscale reconstruction anomaly map (AM) based on all stored Euclidean distance values. R-Texture ;

[0029] Image I D and I R-D Calculate the Euclidean distance between the pixels of two images, store all calculated Euclidean distance values ​​in an array, and generate a depth reconstruction anomaly map (AM) based on all stored Euclidean distance values. R-Spatial ;

[0030] Image F RGB and F T-RGBCalculate the Euclidean distance between the pixels of two images, store all calculated Euclidean distance values ​​in an array, and generate a depth transformation anomaly map (AM) based on all stored Euclidean distance values. T-Spatial ;

[0031] Image F D and F T-D Calculate the Euclidean distance between the pixels of two images, store all calculated Euclidean distance values ​​in an array, and generate a grayscale conversion anomaly image (AM) based on all stored Euclidean distance values. T-Texture ;

[0032] The grayscale anomaly map AM is obtained by multiplying two anomaly maps within the same modality. Texture and depth anomaly map AM Spatial ;

[0033] AM Texture =AM R-Texture *AM T-Texture

[0034] AM Spatial =AM R-Spatial *AM T-Spatial

[0035] Gray-scale anomaly image AM Texture and depth anomaly map AM Spatial Adding them together yields the final anomaly map AM. Final ;

[0036] AM Final =AM Texture +AM Spatial

[0037] Final Anomaly Graph AM Final Each pixel value at coordinate point represents the anomaly score for that point, ranging from 0 to 1. The maximum score is the final anomaly map AM. Final The abnormal score is calculated based on the pixel coordinates of the abnormal image. If the abnormal score of a point is greater than a threshold, the point is identified as an abnormal point. Multiple abnormal points are aggregated to form an abnormal target, thus achieving the localization of the abnormal target.

[0038] Preferably, the method further includes:

[0039] A four-branch similarity learning collaborative constraint module is constructed, with the joint optimization objective being to minimize the dual differences within a single modality and across modalities. The loss function value is calculated, and the parameter values ​​of the single-modal encoding / decoding network, the Mamba-like feature cross-transformation module, and the anomaly graph generation module are adjusted based on the loss function value. This completes the training process of the single-modal encoding / decoding network, the Mamba-like feature cross-transformation module, and the anomaly graph generation module, resulting in the trained single-modal encoding / decoding network, the Mamba-like feature cross-transformation module, and the anomaly graph generation module.

[0040] The four-branch similarity learning collaborative constraint module Loss total The reconstruction loss function Loss R and transformation loss function Loss T Added together;

[0041] Loss total =Loss R +Loss T

[0042] Reconstruction Loss Function R Using the original and reconstructed orbit images as input, calculate the pixel-wise cosine similarity error CosineSimilarity().mean, which is derived from the bimodal independent loss. and Add them together to get;

[0043]

[0044] I RGB For the original grayscale image, I D For the original depth map, I R-RGB To reconstruct the grayscale image, I R-D To reconstruct the depth map;

[0045] Transformation loss function Loss T Using the initial feature map and the transformed feature map as input, calculate the pixel-wise cosine similarity error CosineSimilarity().mean, which is derived from the bimodal independent loss. and Add them together to get;

[0046]

[0047] F RGB For the initial grayscale feature map, F D For the initial depth feature map, F T-D To transform the deep feature map, F T-RGB To convert the grayscale feature map.

[0048] As can be seen from the technical solutions provided by the embodiments of the present invention described above, the present invention solves the two major problems faced in the field of track inspection: the scarcity of abnormal target data and the infinite number of target categories. It effectively alleviates the problem that single visible light images are easily affected by factors such as lighting, leading to a decrease in algorithm performance. The present invention utilizes the robustness and anti-interference capabilities of multimodal data enhancement algorithms to improve the intelligence level of intelligent track inspection.

[0049] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and will become apparent from the description or may be learned by practice of the invention. Attached Figure Description

[0050] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 A flowchart illustrating the processing of an unsupervised learning and multi-source data orbital anomaly target detection method provided in an embodiment of the present invention;

[0052] Figure 2 This is a diagram illustrating the detection effect of an abnormal orbital target provided in an embodiment of the present invention. Detailed Implementation

[0053] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0054] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or couplings. The term “and / or” as used herein includes any and all combinations of one or more of the associated listed items.

[0055] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0056] To facilitate understanding of the embodiments of the present invention, the following will provide further explanation and description with reference to the accompanying drawings and several specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.

[0057] This invention provides an unsupervised learning and multi-source data-based method for detecting orbital anomalies. The method takes matched orbital grayscale and depth maps as input, extracts and reconstructs features from each input using a single-modal lightweight encoder-decoder, then uses a Mamba-like feature cross-transformation module to achieve feature projection between modalities, and a four-branch similarity learning collaborative constraint module to guide network training, ensuring optimization of both single-modal reconstruction and cross-modal transformation. Finally, a multi-decision anomaly map generation strategy is used to generate the final fused anomaly map, thereby achieving anomaly image identification and target pixel localization.

[0058] The processing flow of the unsupervised learning and multi-source data orbital anomaly target detection method provided in this embodiment of the invention is as follows: Figure 1 As shown, the processing steps include the following:

[0059] Step 1: Using the registered and calibrated orbit grayscale image and depth map as input, construct a lightweight single-mode codec network to extract and reconstruct single-mode normal orbit features.

[0060] Step 1 specifically includes the following steps:

[0061] Step 1.1: Obtain the grayscale image I of the original orbital image to be used for anomaly target detection. RGB and depth map I D A single-mode encoding / decoding network is constructed using an encoder and a decoder to convert grayscale image I... RGB and depth map I D The input is fed into a single-mode codec network.

[0062] Step 1.2: The encoder consists of 9 layers, alternating between 3×3 and 5×5 convolutional layers and the ReLU activation function.

[0063] Step 1.3: When the grayscale image is input, as the number of layers increases, the number of feature map channels in each layer of the encoder increases, in the following order: 32, 64, 128, 128, 256.

[0064] Step 1.4: When the depth map is input, as the number of layers increases, the number of feature map channels in each layer of the encoder increases, in the following order: 32, 32, 64, 128, 128.

[0065] Step 1.5: The decoder consists of 20 layers, alternating between bilinear interpolation upsampling layers, 3×3 convolutional layers, activation function layers, and Dropout layers.

[0066] Step 1.6: As the number of layers increases, the number of feature map channels in each layer of the decoder decreases, in the following order: 128, 64, 32, 16, 8, 3.

[0067] Step 1.7: After upsampling the grayscale image and depth image through their respective encoders, the initial feature map F is obtained. RGB and F D The single-modal encoding / decoding network also outputs a grayscale image of the reconstructed orbit. R-RGB and depth map I R-D .

[0068] Step 2: Design a Mamba-like feature cross-transformation module to reveal the feature transformation rules between modalities, complete the cross-modal feature mutual projection, and enhance the consistent representation pattern of normal features; the initial feature map F... RGB and F D The input is fed into the Mamba-like feature cross-transformation module to obtain the transformed feature map F. T-D and F T-RGB .

[0069] Step 2 specifically includes the following steps:

[0070] Step 2.1, the input to this module is the initial feature map F. RGB and F D The number of channels is 256 and 128 respectively, and the output is the transformed feature map F. T-D and F T-RGB The number of channels are 128 and 256 respectively;

[0071] Step 2.2, the Mamba-like feature cross-transformation module consists of two parts: a convolutional block and a Mamba-like block;

[0072] Step 2.3: The convolutional block consists of three 1×1 convolutional layers. The number of channels is adjusted to the average of the number of channels in the two modalities, and then converted to the number of channels in the projected modality. Taking the conversion of grayscale modal features to deep modal features as an example, the feature map is obtained. Taking the conversion of deep modal features to grayscale modal features as an example, the feature map is obtained. Conv 3 The convolution operation representing a three-layer convolutional layer has a 1×1 kernel;

[0073]

[0074] Step 2.4: The Mamba-like block consists of a forget gate branch and a multi-path joint convolution branch;

[0075] Step 2.5: The forget gate branch consists of a normalized layer (Norm), a linear transformation layer (Linear), and a nonlinear SiLU activation function. It focuses on local feature representation to obtain the feature map. and

[0076]

[0077] Step 2.6: The multi-path joint convolutional branch consists of a linear layer, a convolutional layer, a nonlinear SiLU activation function, skip connections, and an addition operation, resulting in a feature map. and The multi-path joint convolution branch specifically includes two convolutional layer branches and one skip connection. The sum of the three branches is obtained. and

[0078] Step 2.7, Feature Map and Element-wise dot product is performed, and attention weights are allocated between feature channels through a linear layer to obtain the feature map. and

[0079]

[0080] Step 2.8, Transform feature map F T-D From feature map and feature map The sum is obtained by transforming the feature map F. T-RGB From feature map and feature map Add them together to get;

[0081]

[0082] Step 3: A multi-decision anomaly map generation module is used to generate the final anomaly map through a hybrid attention method of product and superposition. The maximum score of the anomaly map is used as the anomaly score to complete the anomaly image identification and target pixel localization.

[0083] Step 3.1, calculate image I using the Euclidean distance function. RGB and I R-RGB Image I D and I R-D Feature map F RGB and F T-RGBFeature map F D and F T-D The pixel-level differences between them generate four anomaly maps, namely the grayscale reconstruction anomaly map AM. R-Texture Deep reconstruction anomaly map AM R-Spatial Depth Conversion Anomaly Map AM T-Spatial and grayscale conversion anomaly map AM T-Texture ;

[0084] Image I RGB and I R-RGB Calculate the Euclidean distance between the pixels of two images, store all calculated Euclidean distance values ​​in an array, and generate a grayscale reconstruction anomaly map (AM) based on all stored Euclidean distance values. R-Texture .

[0085] Image I D and I R-D Calculate the Euclidean distance between the pixels of two images, store all calculated Euclidean distance values ​​in an array, and generate a depth reconstruction anomaly map (AM) based on all stored Euclidean distance values. R-Spatial .

[0086] Image F RGB and F T-RGB Calculate the Euclidean distance between the pixels of two images, store all calculated Euclidean distance values ​​in an array, and generate a depth transformation anomaly map (AM) based on all stored Euclidean distance values. T-Spatial .

[0087] Image F D and F T-D Calculate the Euclidean distance between the pixels of two images, store all calculated Euclidean distance values ​​in an array, and generate a grayscale conversion anomaly image (AM) based on all stored Euclidean distance values. T-Texture .

[0088] Step 3.2: Perform a dot product calculation on two anomaly maps within the same modality to obtain the grayscale anomaly map AM. Texture and depth anomaly map AM Spatial That is, an abnormality is confirmed only if both results are judged to be abnormal.

[0089] AM Texture =AM R-Texture *AM T-Texture

[0090] AM Spatial =AM R-Spatial *AM T-Spatial

[0091] Step 3.3: Integrate the anomaly detection results from different modalities by adding the results together, and combine the grayscale anomaly map AM. Textureand depth anomaly map AM Spatial Adding them together yields the final anomaly map AM. Final This enables the complementarity and enhancement of cross-modal anomaly information, with the maximum anomaly map score being used as the anomaly score of the detected sample, thereby achieving accurate localization of the abnormal target.

[0092] AM Final =AM Texture +AM Spatial

[0093] Final Anomaly Graph AM Final Each pixel value at coordinate point represents the anomaly score for that point, ranging from 0 to 1. The maximum score is the final anomaly map AM. Final Abnormal scores.

[0094] In the anomaly image pixel coordinates, if the anomaly score of a point is greater than a threshold, the point is located as an anomaly. Multiple anomaly points are aggregated to form an anomaly target, thus realizing the location of the anomaly target.

[0095] Step 4: Design a four-branch similarity learning collaborative constraint module with the joint optimization objective of minimizing both intramodal and intermodal differences. Calculate the loss function value and adjust the parameter values ​​of the aforementioned single-modal encoding / decoding network, Mamba-like feature cross-transformation module, and anomaly graph generation module based on the loss function value. Complete the training process of the aforementioned single-modal encoding / decoding network, Mamba-like feature cross-transformation module, and anomaly graph generation module to obtain the trained single-modal encoding / decoding network, Mamba-like feature cross-transformation module, and anomaly graph generation module.

[0096] Step 4 specifically includes the following steps:

[0097] Step 4.1, Four-branch similarity learning collaborative constraint module Loss total The reconstruction loss function Loss R and transformation loss function Loss T Added together;

[0098] Loss total =Loss R +Loss T

[0099] Step 4.2, Reconstruct the Loss Function R Using the original and reconstructed orbit images as input, calculate the pixel-wise cosine similarity error CosineSimilarity().mean, which is derived from the bimodal independent loss. and Add them together to get;

[0100]

[0101] Step 4.3, Transform Loss Function T Using the initial feature map and the transformed feature map as input, calculate the pixel-wise cosine similarity error CosineSimilarity().mean, which is derived from the bimodal independent loss. and Add them together to get;

[0102]

[0103] As shown in Table 1, this invention selects I-AUROC as the accuracy index for anomaly image discrimination, and P-AUROC and P-AUPRO as the accuracy indexes for anomaly target pixel localization. The maximum value of the three is 1, and the closer to 1, the higher the accuracy. The track anomaly target detection algorithm proposed in this invention can achieve an anomaly image detection accuracy of 97.00% and an anomaly target pixel localization accuracy of 96.60%, which is significantly better than the existing unsupervised anomaly target detection algorithms. Figure 2 This is a diagram illustrating the detection effect of an abnormal orbital target provided in an embodiment of the present invention.

[0104] In the processing of multimodal orbital data, this invention employs four steps: a single-modal lightweight encoder / decoder, a Mamba-like feature cross-conversion module, a four-branch similarity learning collaborative constraint module, and a multi-decision anomaly graph generation strategy. By utilizing an unsupervised learning method to learn the features of normal orbital samples, this invention avoids the problem of low algorithm accuracy due to the scarcity of anomaly samples and effectively supplements the content of orbital inspection.

[0105] Table 1 Comparison of accuracy evaluation indicators for track anomaly target detection algorithms

[0106]

[0107] In summary, this invention proposes a novel deep learning neural network for the automatic detection of anomalous targets in orbital surface detection. This method overcomes the scarcity and infinite variety of anomalous targets in orbital scenes by utilizing unsupervised learning and cross-modal data, thus improving detection accuracy and speed. This invention proposes a dual fusion processing mode at the decision and feature levels for cross-modal data, effectively improving the reliability of the detection results. The Mamba-like feature cross-transformation module proposed in this invention achieves interaction and correction between grayscale and depth feature representations through nonlinear attention and connection operations, effectively improving the discriminative ability of anomalous images.

[0108] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing the present invention.

[0109] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.

[0110] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for apparatus or system embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The apparatus and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0111] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for detecting orbital anomalies using unsupervised learning and multi-source data, characterized in that, include: I. Obtain the grayscale image of the original orbital image to be used for anomaly target detection RGB and depth map I D A single-mode encoding / decoding network is constructed using an encoder and a decoder to process the grayscale image I. RGB and the depth map I D The input is fed into the single-modal encoding / decoding network to obtain the initial feature map F. RGB and F D ; Construct a Mamba-like feature cross-transformation module to transform the initial feature map F RGB and F D The input is fed into the Mamba-like feature cross-transformation module to obtain the transformed feature map F. T-D and F T-RGB ; The grayscale image I RGB Depth Map I D Initial feature map F RGB F D Transformation Feature Map F T-D and F T-RGB The input is fed into the anomaly map generation module with multiple decisions, and the final anomaly map is generated through a hybrid attention method of product and superposition. Based on the pixel value of each coordinate point of the final anomaly map, the anomaly target in the original orbit image is identified and located.

2. The method according to claim 1, characterized in that, The grayscale image I of the original orbital image to be used for abnormal target detection is obtained as described above. RGB and depth map I D A single-mode encoding / decoding network is constructed using an encoder and a decoder to process the grayscale image I. RGB and the depth map I D The input is fed into the single-modal encoding / decoding network to obtain the initial feature map F. RGB and F D ,include: A single-modal encoding and decoding network is constructed using an encoder and a decoder. The encoder consists of alternating 3×3 and 5×5 convolutional layers and ReLU activation functions, while the decoder consists of alternating bilinear interpolation upsampling layers, 3×3 convolutional layers, activation function layers, and Dropout layers. I. Obtain the grayscale image of the original orbital image to be used for anomaly target detection RGB and depth map I D Grayscale image I RGB and depth map I D When inputting a grayscale image into a single-modal encoder-decoder network, the number of feature map channels in each encoder layer increases with the number of layers. Conversely, when inputting a depth image, the number of feature map channels in each encoder layer increases with the number of layers, while the number of feature map channels in each decoder layer decreases with the number of layers. After the grayscale and depth images are upsampled by their respective encoders, the initial feature map F is obtained. RGB and F D The single-mode codec network also outputs a grayscale image of the reconstructed orbit. R-RGB and depth map I R-D .

3. The method according to claim 2, characterized in that, The aforementioned Mamba feature cross-transformation module converts the initial feature map F RGB and F D The input is fed into the Mamba-like feature cross-transformation module to obtain the transformed feature map F. T-D and F T-RGB ,include: A Mamba-like feature cross-transformation module for cross-modal feature projection is constructed. This Mamba-like feature cross-transformation module includes a convolutional block and a Mamba-like block. The convolutional block consists of three 1×1 convolutional layers. The Mamba-like block consists of a forget gate branch and a multi-path joint convolutional branch. The forget gate branch consists of a normalization layer (Norm), a linear transformation layer (Linear), and a nonlinear SiLU activation function. The multi-path joint convolutional branch consists of a linear layer, a convolutional layer, a nonlinear SiLU activation function, a skip connection, and an addition operation. The number of channels is adjusted to the average number of channels in the two modalities and then converted to the number of channels in the projected modality. The initial feature map F D and F RGB The inputs are fed into a Mamba-like feature cross-transformation module with 256 and 128 channels respectively, first yielding feature map F. D 1 and feature map F RGB 1 The number of channels is 128 and 256 respectively, Conv 3 The convolution operation representing a three-layer convolutional layer has a 1×1 kernel; Then, by focusing on local feature representation through the forgetting gate branch, feature map F is obtained. D 2 and F RGB 2 ; The feature map F is then obtained through multi-path joint convolution branches. D 3 and F RGB 3 The multi-path joint convolution branch specifically includes two convolutional layer branches and one skip connection. The sum of the three branches yields F. D 3 and F RGB 3 ; Feature map F D 2 and F D 3 Element-wise dot product is performed, and attention weights are allocated between feature channels through a linear layer to obtain the feature map F. D 4 and F RGB 4 ; Transformed feature map F T-D From feature map F D 1 and feature map F D 4 The sum is obtained by transforming the feature map F. T-RGB From feature map F RGB 1 and feature map F RGB 4 Add them together to get; 4. The method according to claim 2, characterized in that, The grayscale image I is described RGB Depth Map I D Initial feature map F RGB F D Transformation Feature Map F T-D and F T-RGB The input is fed into the multi-decision anomaly map generation module, which generates the final anomaly map through a hybrid attention method of product and superposition. Based on the pixel value of each coordinate point in the final anomaly map, the module identifies and locates anomalous targets in the original orbital image, including: Image I RGB and I R-RGB Calculate the Euclidean distance between the pixels of two images, store all calculated Euclidean distance values ​​in an array, and generate a grayscale reconstruction anomaly map (AM) based on all stored Euclidean distance values. R-Texture ; Image I D and I R-D Calculate the Euclidean distance between the pixels of two images, store all calculated Euclidean distance values ​​in an array, and generate a depth reconstruction anomaly map (AM) based on all stored Euclidean distance values. R-Spatial ; Image F RGB and F T-RGB Calculate the Euclidean distance between the pixels of two images, store all calculated Euclidean distance values ​​in an array, and generate a depth transformation anomaly map (AM) based on all stored Euclidean distance values. T-Spatial ; Image F D and F T-D Calculate the Euclidean distance between the pixels of two images, store all calculated Euclidean distance values ​​in an array, and generate a grayscale conversion anomaly image (AM) based on all stored Euclidean distance values. T-Texture ; The grayscale anomaly map AM is obtained by multiplying two anomaly maps within the same modality. Texture and depth anomaly map AM Spatial ; AM Texture =AM R-Texture *AM T-Texture AM Spatial =AM R-Spatial *AM T-Spatial Gray-scale anomaly image AM Texture and depth anomaly map AM Spatial Adding them together yields the final anomaly map AM. Final ; AM Final =AM Texture +AM Spatial Final Anomaly Graph AM Final Each pixel value at coordinate point represents the anomaly score for that point, ranging from 0 to 1. The maximum score is the final anomaly map AM. Final The abnormal score is calculated based on the pixel coordinates of the abnormal image. If the abnormal score of a point is greater than a threshold, the point is identified as an abnormal point. Multiple abnormal points are aggregated to form an abnormal target, thus achieving the localization of the abnormal target.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: A four-branch similarity learning collaborative constraint module is constructed, with the joint optimization objective being to minimize the dual differences within a single modality and across modalities. The loss function value is calculated, and the parameter values ​​of the single-modal encoding / decoding network, the Mamba-like feature cross-transformation module, and the anomaly graph generation module are adjusted based on the loss function value. This completes the training process of the single-modal encoding / decoding network, the Mamba-like feature cross-transformation module, and the anomaly graph generation module, resulting in the trained single-modal encoding / decoding network, the Mamba-like feature cross-transformation module, and the anomaly graph generation module. The four-branch similarity learning collaborative constraint module Loss total The reconstruction loss function Loss R and transformation loss function Loss T Added together; Loss total =Loss R +Loss T Reconstruction Loss Function R Using the original and reconstructed orbit images as input, calculate the pixel-wise cosine similarity error CosineSimilarity().mean, which is derived from the bimodal independent loss. RGB R and Loss D R Add them together to get; I RGB For the original grayscale image, I D For the original depth map, I R-RGB To reconstruct the grayscale image, I R-D To reconstruct the depth map; Transformation loss function Loss T Using the initial feature map and the transformed feature map as input, calculate the pixel-wise cosine similarity error CosineSimilarity().mean, and use the bimodal independent loss Loss. RGB T and Loss D T Add them together to get; F RGB For the initial grayscale feature map, F D For the initial depth feature map, F T-D To transform the deep feature map, F T-RGB To convert the grayscale feature map.

Images