A railway multi-temporal change detection method and system based on cross-domain generated images

Abstract

The application discloses a railway multi-time change detection method and system based on cross-domain generated images, comprising the following steps: acquiring a UAV single-time abnormal image of a highway scene; obtaining a railway scene front-time image set based on an image generation model; obtaining a railway scene post-time image set based on an image editing model; simulating diversity by style transfer and constructing a change detection data set; constructing and training a change detection model based on state space perception; inputting real railway double-time images into the model for prediction to obtain change information binary images. The application generates a railway scene double-time image set by using easily acquired highway abnormal images, effectively solves the problem of railway scene abnormal data scarcity, and improves the change detection precision of railway surrounding environment risk hidden dangers through a structure perception and state space updating mechanism, and is widely applicable to railway surrounding environment inspection, infrastructure safety detection, traffic scene data enhancement and intelligent analysis and the like.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a method and system for detecting multi-temporal changes in railways based on cross-domain generated images. Background Technology

[0002] Due to various factors such as changes in meteorological conditions, evolution of hydrological conditions, and geological tectonic activities, safety hazards such as landslides, collapses, rockfalls, and foreign object encroachment in the railway surrounding environment are on the rise and are prone to causing secondary disasters. Traditional manual inspection methods are difficult to meet the actual needs of railway safety operation and maintenance.

[0003] With the development of the low-altitude economy, railway inspection technology based on industrial drones has been gradually applied. By detecting changes in drone images of the same railway section acquired at different times, it is possible to automatically identify potential hazards in the railway's surrounding environment to a certain extent. However, in practical applications, the scale of multi-temporal drone image data for a single railway section is enormous, manual comparison and analysis are inefficient, and are greatly affected by subjective experience, making it difficult to reliably and accurately identify subtle or structural changes, resulting in some potential safety risks not being detected in a timely manner.

[0004] Currently, multi-temporal UAV imagery change detection methods mainly fall into two categories: traditional change detection methods and deep learning-based change detection methods. Traditional methods typically rely on manually designed low- or mid-level features for comparative analysis. However, these methods suffer from insufficient feature robustness when facing the complex scene structure and variable environmental background of railways, which are typical linear engineering projects. They are easily affected by changes in lighting, seasons, and background interference, generally exhibiting problems such as inaccurate location of change areas, high false detection rates, and weak generalization ability, making it difficult to meet the needs of fine-grained change detection in the railway surrounding environment. Deep learning-based change detection methods automatically learn high-level semantic features from large-scale labeled samples, offering certain advantages in change detection accuracy. However, these methods are highly dependent on high-quality labeled data. Especially in railway scenarios, acquiring effective change samples that truly reflect changes in disaster hazards or risks is costly and time-consuming. In remote sections, complex terrain, or extreme weather conditions, it is difficult to accumulate a sufficiently large dataset in a short time, severely limiting the practical application of deep learning-based change detection methods in railway surrounding environment monitoring.

[0005] Because acquiring drone imagery samples of highway scenes is relatively easy, and there are many scenes containing anomalies, such as lightweight floating objects intruding into the road surface, slope collapses, fallen trees, and falling rocks, a large number of samples can be provided for multi-temporal change detection of railways. However, due to the significant differences between highway and railway scenes in terms of infrastructure morphology, spatial structure, and surrounding environment layout, it is difficult to directly transfer highway anomaly imagery data to railway scene change detection tasks using existing technologies.

[0006] Although existing image generation, image editing, and cross-domain transfer technologies can achieve visual transformation between different scenes to some extent, there are still common problems in the process of transforming abnormal images of highway scenes into abnormal images of railway scenes. These problems include inconsistencies between the generated results and the actual structure of the railway scene, inaccurate representation of abnormal areas, lack of effective filtering mechanisms for generated data, and lack of targeted data enhancement for railway change detection tasks. As a result, the generated data is difficult to meet the application requirements of railway surrounding environment change detection in terms of accuracy, authenticity, and diversity.

[0007] In summary, current technologies lack a method that can fully utilize readily available highway scene anomaly data, construct a high-quality railway scene dual-temporal change detection dataset through reasonable cross-domain migration and image generation mechanisms, and further support high-precision detection of changes in the railway surrounding environment. Summary of the Invention

[0008] This invention aims to at least solve one of the technical problems existing in the prior art. Therefore, one objective of this invention is to propose a method and system for detecting multi-temporal changes in railways based on cross-domain generated images. This method constructs a cross-domain migration image generation framework based on abnormal images collected by UAVs in highway scenes, and further proposes a method for detecting changes in risks and hazards in railway scenes.

[0009] To achieve the above objectives, this invention provides a method for detecting multi-temporal changes in railways based on cross-domain generated images, comprising the following steps: Step 1: Acquire single-temporal-phase anomalous images of the highway scene from a drone as the original image set; Step 2: Mask the highway area of ​​the images in the original image set and generate corresponding text descriptions. Input the mask, text descriptions and original images into the trained image inpainting model to generate abnormal images of railway scenes. Then, select the previous time-phase abnormal images based on the image evaluation system. Step 3: Obtain the text description of anomaly removal generated in Step 2, input the text description and the previous anomaly image into the trained image editing model to generate anomaly-free images of the railway scene, and filter the anomaly-free image set based on the image evaluation system; Step 4: Input the set of images without anomalies obtained in Step 3 into the style transfer model for random augmentation processing to construct a dual-temporal change detection dataset for railway scenes; Step 5: Construct a change detection network model and train it on the dual-temporal change detection dataset constructed in Step 4 to obtain the weights of the change detection network model; Step 6: Input the two sets of UAV images of the railway scene into the change detection network model, perform weight prediction, and output the binary map of the change detection area.

[0010] Preferably, in step 2, the highway area in the image is first labeled to generate a corresponding region mask, and the text description of the image is obtained; then, the region mask, text description, and original image are input into an existing image inpainting model to generate an abnormal railway scene image; finally, an image generation evaluation system is constructed to evaluate and filter the generated images, and the image with the best evaluation result is selected as the previous-temporal abnormal image for railway scene change detection; step 2 includes the following sub-steps: Step 2-1: Obtain any image from the original image set. Image annotation tools are used to annotate the highway areas, resulting in a mask set to be converted into railway areas. ; Step 2-2: Use an existing open-source multimodal large language model to generate images. Text description ; Steps 2-3: Text Description Perform semantic modification, replacing descriptions related to highway scenarios with descriptions of railway scenarios, and obtain the modified text description. ; Steps 2-4: Extract the original image Mask to be converted { and the revised text description The images are input into an image restoration model to generate multiple images of abnormal railway scenes; Steps 2-5: Construct an evaluation system for generated railway scene images, including two indicators: image sharpness and completeness of generated information; among them, peak signal-to-noise ratio is used. PSNR As an indicator of image sharpness, its calculation formula is: ; in The maximum pixel value is set to 255. MSE The mean square error between the original image and the generated image; when PSNR When the value is greater than 30dB, the image sharpness is deemed to meet the requirements. Then, the Intersection over Union (IOU) ratio of the abnormal regions in the generated image and the original image is calculated as an indicator of the completeness of the generated information. When the IOU is greater than 0.6, the abnormal information is deemed to be completely expressed. Based on this image evaluation system, images that conform to the structural characteristics of railway scenes and clearly express anomaly information are selected as previous-phase railway anomaly scene images. .

[0011] Preferably, in step 3, anomaly removal processing is performed on the previous phase railway anomaly scene images to generate corresponding railway anomaly-free scene images, and these images are then selected based on an image evaluation system. Step 3 includes the following sub-steps: Step 3-1: Mark the potential hazard areas in the previous phase railway anomaly scene images and generate anomaly area masks; Step 3-2: Based on the railway scene text description generated in Step 2-2, modify the keywords related to anomalies to obtain a text description without anomaly information; Step 3-3: Input the text description of the removed abnormal regions in the image and the best generated image selected in Step 2-5 into the image editing model to generate railway abnormality scene images; and select the best post-temporal railway abnormality scene image set based on the image evaluation system in Step 2.

[0012] Preferably, in step 4, the railway anomaly-free scene image set obtained in step 3 is input into the style transfer model, and the images are subjected to random enhancement processing based on seasonal changes, illumination changes, and meteorological conditions such as rain and fog to generate style transfer images, which serve as railway anomaly-free scene images in the later time phase. These images are then combined with the railway anomaly scene images obtained in step 2 in the earlier time phase to construct a dual-time phase change detection dataset for railway scenes. Step 4 includes the following sub-steps: Step 4-1: Use a style transfer model to improve the generalization ability of the change detection model to environmental changes. Set model parameters related to seasonal, meteorological and lighting changes, and perform random style transfer processing on railway scene images without anomalies in the later time phase to generate multiple style variant images. Step 4-2: Label the change regions in the corresponding before and after phase images obtained in Step 2 and Step 4-1. The labeling categories include change regions and background regions. The change regions are highlighted to identify abnormal hazards in the railway's surrounding environment, forming a railway change detection training dataset containing anomaly labeling information.

[0013] Preferably, step 5, based on the dual-temporal image dataset constructed in step 4-2, inputs the dual-temporal images and their corresponding change region locations and category labels into the change detection neural network for training, thereby obtaining the set of weight parameters for the change detection network model. W The change detection network model includes: a dual-temporal feature encoding network, a change-aware state-space modeling network, and a change-aware fusion decoding network.

[0014] Preferably, step 5 includes the following sub-steps: Step 5-1: Transfer the dual-temporal images and Feature encoding networks with completely shared input parameters; Step 5-1-1: For the first s ( s =1,2,3,4) Feature extraction stage, let the first... l The input features of each structure-aware state-space module are denoted as follows: ,in, This represents the output characteristics of the s-stage downsampling unit; Step 5-1-2: Downsample the features The input is fed into multiple structure-aware state space blocks for step-by-step feature transformation to obtain the first... s Stage-based structural perception features Its residual update form is expressed as: ; in, This represents a branch for state-space modeling, used to model long-range spatial dependencies while maintaining linear or near-linear complexity. This represents the structure-aware branch, used to display local geometric structure information in modeled remote sensing images; LN The representation layer normalization operation fuses the outputs of the two branches with the original features element-wise to form the structure-aware features for the current stage. ; The structure-aware branch input feature Z is structurally enhanced through depthwise separable convolution and gating mechanisms, and its expression is: ; in, Indicates nonlinear activation. This represents the sigmoid function. This is represented as element-wise multiplication. This represents a 3×3 depthwise separable convolution. This represents a 1x1 convolution; the output is continuously injected into the encoded feature stream as structural guidance information, enabling the encoder to maintain sensitivity to the continuity of ground boundaries, morphology, and structure during the layer-by-layer abstraction process; Step 5-2: Construct a change-aware state-space modeling module, which integrates the previously acquired front-phase and back-phase features. Input into CPSSM to obtain the s-th stage temporal interaction features at the corresponding scale. ; Step 5-2-1: Constructing dual-temporal fusion features at the s-stage scale , is represented as: ; in, It features dual-time-phase differential channels; The characteristics of the dual-temporal correlation channels can be calculated based on the cosine similarity metric. Step 5-2-2: Merge features Input a change-aware state-space model and perform recursive modeling in the form of state updates; introduce changing state variables at each spatial location. It is then updated based on the current input features, and the calculation formula is as follows: ; in As the initial state variable, set it to the zero tensor; Update the convolution operator for the state; This is a lightweight local modeling operator consisting of a 3×3 Conv-GELU convolution activation operator followed by a 1×1 Conv-GELU convolution activation operator; Step 5-3: Combine the temporal interaction features at four scales The input is fed into the structural consistency decoding network to generate change decoding features; Step 5-3-1: Employ a multi-scale fusion decoding method to... Perform upsampling and fusion; initialize weights using the lowest resolution image. Build initialization features Then, based on the recursive idea, multi-scale step-by-step fusion is performed, and the calculation formula is: ; ; in, Initialize the weights of the convolution kernel for high-resolution features. Indicates the first i Stage-series interaction characteristics, For bilinear interpolation upsampling function, Upsampling to They are stitched together at the same resolution and along the channel dimension. This is a non-linear processing function; Step 5-3-2: Perform binary classification mapping using a 1×1 Conv. The weights of the binary classification convolution kernel can output a probability map of change, represented as follows: ; Step 5-4: During model training, construct the total loss function that combines the joint change detection loss and the structural consistency regularization term. After training, obtain the set of network model weight parameters. W ; Step 5-4-1: Use pixel-wise binary cross-entropy as the change detection loss. , is represented as: ; in Indicates pixel index, For changing truth labels, To predict the probability of change; Step 5-4-2: To demonstrate the key contribution of "structure awareness" and suppress edge spikes, a structural consistency loss is introduced, using the spatial gradient operator. Constrain the predicted boundary to be consistent with the actual boundary: ; in and Let these represent the set of predicted boundary pixel probabilities and the set of true boundary values, respectively, and let the spatial gradient operator be... The Sobel operator can be used for computation; Step 5-4-3: The final total loss function is defined as: ; in, The weighting coefficient for change detection loss, For structure-aware weighting coefficients.

[0015] A system used in a railway multi-temporal change detection method based on cross-domain generated images includes: The image acquisition module is used to perform step 1, acquiring single-temporal anomaly images of the highway scene from the UAV as the original image set; The image processing module is used to perform steps 2 and 3, including masking, text description generation, image restoration and editing, and screening based on the image evaluation system. The data augmentation module is used to perform step 4, which involves style transfer enhancement on images without anomalies and constructing a dual-temporal change detection dataset. The model training module is used to perform step 5, building and training the change detection network model; The change detection module is used to perform step 6, which uses the trained model to perform change detection on the two drone images and outputs a binary image.

[0016] Preferably, the image processing module includes an evaluation unit configured to be based on peak signal-to-noise ratio. PSNR Image filtering is performed by comparing Intersection over Union (IOU). when PSNR When the value is greater than 30dB, the image sharpness is considered to meet the requirements. When IOU is greater than 0.6, the abnormal information is considered to be complete.

[0017] Preferably, the structure and working principle of the change detection network model configured in the model training module are as defined in steps 3, 4 and 5.

[0018] Preferably, the loss function used by the model training module during training is as defined in step 5.

[0019] The advantages of this invention compared to the prior art are: 1. This invention uses easily obtainable and abundant highway anomaly images as the data source, and applies image restoration, image editing, and style transfer models to construct a cross-domain image generation framework to achieve efficient generation of dual-temporal datasets for detecting changes in the surrounding environment of railway scenes, effectively solving the problem of difficulty in obtaining and insufficient quantity of existing railway scene disaster hazard samples.

[0020] 2. This invention constructs an objective image generation evaluation system and combines it with targeted image enhancement strategies to screen generated images, ensuring the rationality and accuracy of generated samples in terms of scene structure and anomaly expression. At the same time, it covers different seasons, lighting and weather conditions, providing high-quality and diverse training samples for change detection models.

[0021] 3. This invention constructs a novel change detection network model, which introduces a structure awareness and state space update mechanism into the network structure, making it sensitive to structural changes and suitable for detecting changes in the railway surrounding environment; and it is trained based on the generated railway scene sample set, effectively improving the accuracy of changing risk and hidden dangers in the railway surrounding environment. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of the railway scene cross-domain migration image generation framework of the present invention; Figure 2 This is an example of the process for generating railway scene images through cross-domain migration of highway scenes according to the present invention; Figure 3 This is a schematic diagram of the railway scene image editing model structure of the present invention; Figure 4 This is an example of a dual-temporal railway scene change detection sample from the present invention; Figure 5 This is a schematic diagram of the dual-temporal change detection network model structure of the present invention; Figure 6 This is an example of the railway scene change detection results of the present invention. Detailed Implementation

[0024] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0025] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "joining" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0026] The invention will be further described in detail with specific example diagrams. The present invention provides a method for detecting multi-temporal changes in railways based on cross-domain generated images, comprising the following steps: Step 1: Acquire a single-temporal anomaly image of the highway scene from a drone, and perform an image scaling operation to obtain a standard image of size 1024×1024, which will serve as the original image set. I .

[0027] Step 2: Input the original image set into the constructed railway scene cross-domain transfer generation framework (e.g., Figure 1 (As shown). First, the highway area in the image is labeled to generate a corresponding region mask, and the text description of the image is obtained. Then, the region mask, text description, and original image are input into an existing image inpainting model to generate an abnormal railway scene image. Finally, an image generation evaluation system is constructed to evaluate and filter the generated images, and the image with the best evaluation result is selected as the previous-temporal abnormal image for railway scene change detection.

[0028] Step 2-1: Using any image from the original image set For example, an image annotation tool is used to annotate the highway area, resulting in a mask set to be converted into a railway area. ; Step 2-2: Use existing open-source multimodal large language models (e.g., but not limited to the Qwen series models) to generate images. Text description ; Steps 2-3: Describe the text Perform semantic modification, replacing descriptions related to the highway scene with descriptions of the railway scene ("highway" replaced with "railway"), and obtain the modified text description. ; Steps 2-4: Extract the original image Mask to be converted { and the revised text description The input is fed into an image inpainting model (e.g., but not limited to the FLUX-ControlNet-Inpainting model) to generate multiple images of railway anomaly scenes. Specifically, according to... Figure 2 The structure and generation process are as follows: text descriptions are input into the generation network, and region masks are input into the control network, providing positional and structural constraints for the generation process; Steps 2-5: Construct an evaluation system for generated railway scene images, including two indicators: image sharpness and completeness of generated information. Peak signal-to-noise ratio (PSNR) is used as the evaluation metric. PSNR As an indicator of image sharpness, its calculation formula is: ; in This represents the maximum pixel value, typically set to 255. MSE This represents the mean square error between the original image and the generated image. PSNR When the overlap is greater than 30dB, the image sharpness is considered to meet the requirements. Then, the overlap between the anomalous regions before and after generation, i.e., the intersection-overall ratio (IOU), is calculated as an indicator of the completeness of the generated information. The IOU threshold is set to 0.6; values ​​higher than this threshold indicate complete expression of anomalous information. Based on the above evaluation system, images that conform to the structural characteristics of railway scenes and clearly express anomalous information are selected as the previous-phase railway anomalous scene images. .

[0029] Step 3: Perform anomaly removal processing on the previous time-phase railway anomaly scene images to generate corresponding railway anomaly-free scene images, and then filter them based on the evaluation system.

[0030] Step 3-1: Mark the potential hazard areas in the previous phase railway anomaly scene images and generate anomaly area masks; Step 3-2: Based on the railway scene text description generated in Step 2-2, modify the keywords related to anomalies to obtain a text description without anomaly information; Step 3-3: As Figure 3 As shown, the text description of the removed abnormal regions in the image and the best generated image selected in steps 2-5 are input into the image editing model (e.g., but not limited to the ICEdit model) to generate railway abnormality scene images; and the best post-temporal railway abnormality scene image set is obtained by filtering based on the image evaluation system in step 2.

[0031] Step 4: Input the set of railway scene images without anomalies obtained in Step 3 into the style transfer model, perform random enhancement processing on the images to account for seasonal changes, illumination changes and meteorological conditions such as rain and fog, and generate style transfer images as railway scene images without anomalies in the later time phase. Combined with the railway scene images with anomalies obtained in Step 2 in the earlier time phase, construct a railway scene two-phase change detection dataset.

[0032] Step 4-1: To improve the generalization ability of the change detection model to environmental changes, a style transfer model (such as but not limited to CycleGAN) is adopted. The model parameters related to seasonal, meteorological and lighting changes are set, and random style transfer processing is performed on the railway scene images without anomalies in the later time phase to generate multiple style variant images. Step 4-2: Annotate the corresponding time-phase images obtained in Step 2 and Step 4-1 to identify areas of change. The annotation categories include changed areas and background areas. The changed areas are highlighted to identify potential anomalies in the railway's surrounding environment, forming a railway change detection training dataset containing anomaly annotation information. Some example images are shown below. Figure 4 As shown.

[0033] Step 5: Construct a change detection neural network model, such as Figure 5 As shown, the model comprises three parts: a dual-temporal feature encoding network, a change-aware state-space modeling network, and a change-aware fusion decoding network. Based on the dual-temporal image dataset constructed in step 4-2, the dual-temporal images and their corresponding change region locations and category labels are input into the change detection neural network for training, thereby obtaining the set of weight parameters for the change detection network model. W .

[0034] Step 5-1: Transfer the dual-temporal images and A feature encoding network with completely shared input parameters ensures that features from different time phases reside in the same feature space, avoiding the introduction of additional temporal bias. In a preferred embodiment, the encoder consists of four cascaded feature extraction stages, each stage including a downsampling unit and several structure-aware state space modules (SASSMs). Step 5-1-1: For the first s ( s =1,2,3,4) Feature extraction stage, let the first... l The input features of each structure-aware state-space module are denoted as follows: ,in, This represents the output characteristics of the s-stage downsampling unit; Step 5-1-2: Downsample the features The input is fed into multiple structure-aware state space blocks for step-by-step feature transformation to obtain the first... s Stage-based structural perception features Its residual update form is expressed as: ; in, This represents a branch for state-space modeling, used to model long-range spatial dependencies while maintaining linear or near-linear complexity. This represents the structure-aware branch, used to display local geometric structure information in modeled remote sensing images. LN The layer normalization operation is performed. The outputs of the two branches are fused element-wise with the original features to form the structure-aware features for the current stage. More specifically, the structure-aware branch input feature Z is structurally enhanced through depthwise separable convolution and gating mechanisms, and its expression is: ; in, Indicates nonlinear activation. This represents the sigmoid function. This is represented as element-wise multiplication. This represents a 3×3 depthwise separable convolution. This represents a 1x1 convolution; the output is continuously injected into the encoded feature stream as structural guidance information, enabling the encoder to maintain sensitivity to the continuity of ground boundaries, morphology, and structure during the layer-by-layer abstraction process; Step 5-2: Construct the Change Perception State Space Modeling Module (CPSM) to integrate the previously acquired pre- and post-temporal features. Input into CPSSM to obtain the s-th stage temporal interaction features at the corresponding scale. ; Step 5-2-1: Constructing dual-temporal fusion features at the s-stage scale , represented as: ; in, It features dual-time-phase differential channels; The characteristics of the dual-temporal correlation channels can be calculated based on the cosine similarity metric. Step 5-2-2: Input the fused features into the change-aware state-space model and perform recursive modeling in the form of state updates. Specifically, introduce a changing state variable at each spatial location. It is then updated based on the current input features, and the calculation formula is as follows: ; in As the initial state variable, it is usually set as the zero tensor. To update the convolution operator for the state, This is a lightweight local modeling operator consisting of a 3×3 Conv-GELU convolution activation operator followed by a 1×1 Conv-GELU convolution activation operator; Step 5-3: Combine the temporal interaction features at four scales The input is fed into the structural consistency decoding network to generate change decoding features; Step 5-3-1: Employ a multi-scale fusion decoding method to... Upsampling and fusion are performed. Specifically, the weights are initialized using the lowest resolution image. Build initialization features Then, based on the recursive idea, multi-scale step-by-step fusion is performed, and the calculation formula is: ; ; in, Initialize the weights of the convolution kernel for high-resolution features. Indicates the first i Stage-series interaction characteristics, For bilinear interpolation upsampling function, Upsampling to They are stitched together at the same resolution and along the channel dimension. This is a non-linear processing function; Step 5-3-2: Perform binary classification mapping using a 1×1 Conv. The weights of the binary classification convolution kernel can output a probability map of change, represented as follows: ; Step 5-4: During model training, construct the total loss function that combines the joint change detection loss and the structural consistency regularization term. After training, obtain the set of network model weight parameters. W ; Step 5-4-1: Use pixel-wise binary cross-entropy as the change detection loss. , represented as: ; in Indicates pixel index, For changing truth labels, To predict the probability of change.

[0035] Step 5-4-2: To demonstrate the key contribution of "structure awareness" and suppress edge spikes, a structural consistency loss is introduced, using the spatial gradient operator. Constrain the predicted boundary to be consistent with the actual boundary: ; in and Let these represent the set of predicted boundary pixel probabilities and the set of true boundary values, respectively, and let the spatial gradient operator be... The Sobel operator can be used for computation; Step 5-4-3: The final total loss function is defined as: ; in, The weighting coefficient for change detection loss, For structure-aware weighting coefficients.

[0036] Step 6: Input the two real-collected UAV images into the trained change detection network, and utilize the model weights. W Perform inference and prediction, and output a binary map of the railway surrounding environment change detection area. A specific example is shown below. Figure 6 As shown.

[0037] To more clearly illustrate the specific embodiments of the present invention, an example is provided: The working principle of this invention is based on the technical path of "cross-domain data generation → high-quality dataset construction → dedicated model training". Its core lies in solving the bottleneck problem of scarce samples for railway scene change detection and improving detection accuracy through a proprietary network structure. Its systematic working principle is as follows: 1. Cross-Domain Migration and Data Generation: This invention cleverly leverages the ease of obtaining abnormal image data from highway scenes. Through an "image inpainting" model, under the joint control of text description (semantic guidance) and region masking (structural constraints), abnormal images from highway scenes are "translated" into abnormal images from railway scenes, achieving cross-domain migration from the "source domain" (highway) to the "target domain" (railway). Subsequently, through an "image editing" model, while preserving the background structure of the railway scene, anomalies in the previous temporal image are precisely removed based on the modified text description, generating a corresponding anomaly-free subsequent temporal image. This closed-loop "generation-removal" process ensures a high degree of consistency and rationality in the constructed dual-temporal image pair in terms of scene structure and change logic.

[0038] 2. Data Quality Assurance: To ensure the effectiveness of the generated images, this invention introduces a quantitative image evaluation system. Peak Signal-to-Noise Ratio (PSNR), as an objective indicator, guarantees the overall clarity and image quality of the generated images, avoiding the introduction of excessive noise. Intersection over Union (IOU) is specifically used to evaluate the information fidelity of anomalous regions during cross-domain generation, ensuring that key anomaly features are accurately transmitted and expressed. By setting clear thresholds (PSNR > 30dB, IOU > 0.6) for screening, the quality of the generated data is controlled from the source. In addition, style transfer is used for data augmentation, simulating different seasons, weather, and lighting conditions, greatly enhancing the diversity of the dataset, thereby improving the generalization ability and robustness of the subsequently trained change detection model.

[0039] 3. Core innovation of the change detection network: To solve the problem of detecting complex structures and subtle changes in railway linear engineering scenarios, this invention designs a dedicated change detection network model.

[0040] Structure-aware encoding: The structure-aware state-space module (SASSM) introduced into the feature encoding network uses a structure-aware branch (SCM) that employs depthwise separable convolution and gating mechanisms to explicitly extract and enhance local geometric structures and edge information in images. This allows the network to maintain high sensitivity to the boundaries and morphology of key features such as railway tracks and slopes when extracting deep features, laying the foundation for accurate identification of structural changes (such as landslide edges).

[0041] Change-aware modeling: Change-aware state-space modeling network (CPSSM) constructs a fusion feature G s (Including original features, difference features, and related features), comprehensively capturing the difference information between two temporal images. Employing a state-space model for recursive updates, it can effectively model long-distance dependencies in image sequences, thereby better understanding the context of scene changes and suppressing false alarms caused by irrelevant factors such as illumination changes.

[0042] Structural consistency constraint: A structural consistency loss is introduced into the loss function, using the spatial gradient operator to directly constrain the boundaries of the predicted change regions to remain consistent with the true boundaries. This regularization term effectively suppresses the "spurt" phenomenon at the edges of the predicted results, prompting the network to generate change maps with clear and smooth boundaries, further improving the spatial accuracy of the detection results.

[0043] In summary, this invention, through its interconnected technical design, first breaks through the data bottleneck and creates high-quality labeled data; then, considering the special characteristics of the railway scenario, it designs a dedicated detection model that perceives structures, is adept at differences, and has clear boundaries, ultimately achieving high-precision and robust change detection of potential hazards in the railway's surrounding environment.

[0044] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.

[0045] The following are all formula characters extracted based on the content of this invention and their Chinese definitions:

Claims

1. A method for detecting multi-temporal changes in railways based on cross-domain generated images, characterized in that, Includes the following steps: Step 1: Acquire single-temporal-phase anomalous images of the highway scene from a drone as the original image set; Step 2: Perform highway area masking on the images in the original image set and generate corresponding text descriptions. Input the mask, text descriptions and original images into the trained image inpainting model to generate abnormal railway scene images. Then, filter the abnormal images from the previous time phase based on the image evaluation system. Step 3: Obtain the text description for removing anomalies generated in Step 2, input the text description and the previous anomaly image into the trained image editing model to generate anomaly-free images of the railway scene, and filter out anomaly-free image set based on the image evaluation system; Step 4: Input the set of images without anomalies obtained in Step 3 into the style transfer model for random augmentation processing to construct a dual-temporal change detection dataset for railway scenes; Step 5: Construct a change detection network model and train it on the dual-temporal change detection dataset constructed in Step 4 to obtain the weights of the change detection network model; Step 6: Input the two UAV images of the railway scene collected in two phases into the change detection network model, perform weight prediction, and output a binary map of the change detection area.

2. The method for detecting multi-temporal changes in railways based on cross-domain generated images according to claim 1, characterized in that: In step 2, firstly, the highway area in the image is labeled, a corresponding region mask is generated, and the text description of the image is obtained; then, the region mask, text description, and original image are input into an existing image inpainting model to generate an abnormal railway scene image; finally, an image generation evaluation system is constructed to evaluate and filter the generated images, and the image with the best evaluation result is selected as the previous-temporal abnormal image for railway scene change detection; step 2 includes the following sub-steps: Step 2-1: Obtain any image from the original image set. The highway areas were labeled using an image annotation tool to obtain a mask set to be converted into railway areas. ; Step 2-2: Use an existing open-source multimodal large language model to generate images. Text description ; Steps 2-3: Describe the text Perform semantic modification, replacing descriptions related to highway scenarios with descriptions of railway scenarios, and obtain the modified text description. ; Steps 2-4: Extract the original image Mask to be converted { and the revised text description The images are input into an image restoration model to generate multiple images of abnormal railway scenes; Steps 2-5: Construct an evaluation system for generated railway scene images, including two indicators: image sharpness and completeness of generated information; among them, peak signal-to-noise ratio (PSNR) is... PSNR As an indicator of image sharpness, its calculation formula is: ； in The maximum pixel value is set to 255. MSE The mean square error between the original image and the generated image; when PSNR When the value is greater than 30dB, the image sharpness is deemed to meet the requirements. Then, the Intersection over Union (IOU) ratio of the abnormal regions in the generated image and the original image is calculated as an indicator of the completeness of the generated information. When the IOU is greater than 0.6, the abnormal information is deemed to be completely expressed. Based on this image evaluation system, images that conform to the structural characteristics of railway scenes and clearly express anomaly information are selected as previous-phase railway anomaly scene images. .

3. The method for detecting multi-temporal changes in railways based on cross-domain generated images according to claim 1, characterized in that: In step 3, anomaly removal processing is performed on the previous time-phase railway anomaly scene images to generate corresponding railway anomaly-free scene images, which are then filtered based on the image evaluation system. Step 3 includes the following sub-steps: Step 3-1: Mark the potential hazard areas in the previous phase railway anomaly scene images and generate anomaly area masks; Step 3-2: Based on the railway scene text description generated in Step 2-2, modify the keywords related to anomalies to obtain a text description without anomaly information; Step 3-3: Input the text description of the removed abnormal regions in the image and the best generated image selected in Step 2-5 into the image editing model to generate railway abnormality scene images; and select the best post-temporal railway abnormality scene image set based on the image evaluation system in Step 2.

4. The method for detecting multi-temporal changes in railways based on cross-domain generated images according to claim 1, characterized in that: In step 4, the railway anomaly-free scene image set obtained in step 3 is input into the style transfer model. The images undergo random enhancement processing to account for seasonal changes, illumination changes, and meteorological conditions such as rain and fog, generating style transfer images as the later-phase railway anomaly-free scene images. Combined with the earlier-phase railway anomaly scene images obtained in step 2, a dual-phase change detection dataset for railway scenes is constructed. Step 4 includes the following sub-steps: Step 4-1: Use a style transfer model to improve the generalization ability of the change detection model to environmental changes. Set model parameters related to seasonal, meteorological and lighting changes, and perform random style transfer processing on the later-phase railway scene images without anomalies to generate multiple style variant images. Step 4-2: Label the change regions in the corresponding before and after phase images obtained in Step 2 and Step 4-1. The labeling categories include change regions and background regions. The change regions are highlighted to identify abnormal hazards in the railway's surrounding environment, forming a railway change detection training dataset containing anomaly labeling information.

5. The method for detecting multi-temporal changes in railways based on cross-domain generated images according to claim 1, characterized in that: Step 5, based on the dual-temporal image dataset constructed in Step 4-2, inputs the dual-temporal images and their corresponding change region locations and category labels into the change detection neural network for training, thereby obtaining the set of weight parameters for the change detection network model. W ; The change detection network model includes: a dual-temporal feature encoding network, a change-aware state-space modeling network, and a change-aware fusion decoding network.

6. The method for detecting multi-temporal changes in railways based on cross-domain generated images according to claim 5, characterized in that: Step 5 includes the following sub-steps: Step 5-1: Transfer the dual-temporal images and Feature encoding networks with completely shared input parameters; Step 5-1-1: For the first s ( s =1,2,3,4) Feature extraction stage, let the first... l The input features of each structure-aware state-space module are denoted as follows: ,in, This represents the output characteristics of the s-stage downsampling unit; Step 5-1-2: Downsample the features The input is fed into multiple structure-aware state space blocks for step-by-step feature transformation to obtain the first... s Stage-based structural perception features Its residual update form is expressed as: ； in, This represents a branch for state-space modeling, used to model long-range spatial dependencies while maintaining linear or near-linear complexity. This represents the structure-aware branch, used to display local geometric structure information in modeled remote sensing images; LN The representation layer normalization operation fuses the outputs of the two branches with the original features element-wise to form the structure-aware features for the current stage. ; The structure-aware branch input feature Z is structurally enhanced through depthwise separable convolution and gating mechanisms, and its expression is: ； in, Indicates nonlinear activation. This represents the sigmoid function. This is represented as element-wise multiplication. This represents a 3×3 depthwise separable convolution. This represents a 1x1 convolution; the output is continuously injected into the encoded feature stream as structural guidance information, enabling the encoder to maintain sensitivity to the continuity of ground boundaries, morphology, and structure during the layer-by-layer abstraction process; Step 5-2: Construct a change-aware state-space modeling module, which integrates the previously acquired front-phase and back-phase features. Input into CPSSM to obtain the s-th stage temporal interaction features at the corresponding scale. ; Step 5-2-1: Constructing dual-temporal fusion features at the s-stage scale , is represented as: ； in, It features dual-time-phase differential channels; The characteristics of the dual-temporal correlation channels can be calculated based on the cosine similarity metric. Step 5-2-2: Merge the features Input a change-aware state-space model and perform recursive modeling in the form of state updates; introduce changing state variables at each spatial location. It is then updated based on the current input features, and the calculation formula is as follows: ； in As the initial state variable, set it to the zero tensor; Update the convolution operator for the state; This is a lightweight local modeling operator consisting of a 3×3 Conv-GELU convolution activation operator followed by a 1×1 Conv-GELU convolution activation operator; Step 5-3: Combine the temporal interaction features at four scales The input is fed into the structural consistency decoding network to generate change decoding features; Step 5-3-1: Employ a multi-scale fusion decoding method to... Perform upsampling and fusion; initialize weights using the lowest resolution image. Build initialization features Then, based on the recursive idea, multi-scale step-by-step fusion is performed, and the calculation formula is: ； ； in, Initialize the weights of the convolution kernel for high-resolution features. Indicates the first i Stage-series interaction characteristics, For bilinear interpolation upsampling function, Upsampling to They are stitched together at the same resolution and along the channel dimension. This is a non-linear processing function; Step 5-3-2: Perform binary classification mapping using a 1×1 Conv. The weights of the binary classification convolution kernel can output a probability map of change, represented as follows: ； Step 5-4: During model training, construct the total loss function that combines the joint change detection loss and the structural consistency regularization term. After training, obtain the set of network model weight parameters. W ; Step 5-4-1: Use pixel-wise binary cross-entropy as the change detection loss. , is represented as: ； in Indicates pixel index, For changing truth labels, To predict the probability of change; Step 5-4-2: To demonstrate the key contribution of "structure awareness" and suppress edge spikes, a structural consistency loss is introduced, using the spatial gradient operator. Constrain the predicted boundary to be consistent with the actual boundary: ； in and Let these represent the set of predicted boundary pixel probabilities and the set of true boundary values, respectively, and let the spatial gradient operator be... The Sobel operator can be used for computation; Step 5-4-3: The final total loss function is defined as: ； in, For change detection loss weighting coefficients, For structure-aware weighting coefficients.

7. A system used in the railway multi-temporal change detection method based on cross-domain generated images as described in claim 1, characterized in that, include: The image acquisition module is used to perform step 1, acquiring single-temporal anomaly images of the highway scene from the UAV as the original image set; The image processing module is used to perform steps 2 and 3, including masking, text description generation, image restoration and editing, and screening based on the image evaluation system. The data augmentation module is used to perform step 4, which involves style transfer enhancement on images without anomalies and constructing a dual-temporal change detection dataset. The model training module is used to perform step 5, building and training the change detection network model; The change detection module is used to perform step 6, which uses the trained model to perform change detection on the two drone images and outputs a binary image.

8. The system used in the railway multi-temporal change detection method based on cross-domain image generation according to claim 7, characterized in that: The image processing module includes an evaluation unit configured to be based on peak signal-to-noise ratio. PSNR Image filtering is performed by comparing Intersection over Union (IOU). when PSNR When the value is greater than 30dB, the image sharpness is considered to meet the requirements. When IOU is greater than 0.6, the abnormal information is considered to be complete.

9. The system used in the railway multi-temporal change detection method based on cross-domain image generation according to claim 7, characterized in that: The change detection network model configured in the model training module has the structure and working principle as defined in steps 3, 4 and 5.

10. The system used in the railway multi-temporal change detection method based on cross-domain image generation according to claim 7, characterized in that: The loss function used by the model training module during training is as defined in step 5.

Images