A training method, detection method, and system for a domain-generalized detection model for foreign object intrusion detection in railways.

Abstract

This invention discloses a training method, detection method, and system for a domain-generalized detection model for foreign object intrusion detection in railways. Belonging to the fields of railway traffic safety monitoring and computer vision technology, the invention includes: semantic decoupling data synthesis to generate a synthetic image with pixel-level annotations; model training, initializing the target detection model backbone as a vision-language model encoder, constructing frozen text prototypes and learnable background prototypes, and calculating the sum of detection loss and text prototype alignment loss as the total loss; and the detection method, inputting the monitoring image into the trained model and directly outputting the foreign object category, location, and confidence level. This invention, employing the above method and system, overcomes the limitation of scarce real samples, suppresses overfitting to environmental artifacts, improves the average detection accuracy in unseen target domains, and achieves robust domain-generalized detection across railway lines and weather conditions.

Description

Technical Field

[0001] This invention relates to the fields of railway traffic safety monitoring and computer vision technology, and in particular to a training method, detection method and system for a domain generalization detection model for detecting foreign object intrusions into railway lines. Background Technology

[0002] Foreign object intrusion into railway tracks (such as falling rocks, pedestrians, construction tools, etc.) is one of the main sources of risk threatening railway traffic safety. Deep learning-based visual object detection technology is widely integrated into railway video surveillance systems due to its flexible deployment and low cost. A typical solution uses fixed high-definition cameras to continuously capture images of the track area, uses object detection methods to identify foreign objects, and triggers a warning mechanism.

[0003] However, in practical applications, the following problems exist: there is a significant domain shift between the training dataset and the real deployment scenario. Specifically, the training data is usually collected from only a few existing lines under specific seasons, normal weather conditions, and fixed camera configurations. This results in extremely limited coverage of the dataset in terms of track structure (e.g., ballast / ballastless), background vegetation, lighting conditions, imaging perspective, and the semantic distribution of foreign objects. Furthermore, since foreign object events themselves occur very infrequently, the training set is mostly manually constructed and captured, making it difficult to cover the diverse intrusion scenarios that may occur in actual operation. Therefore, although the model performs well on the training lines, its performance drops sharply under new lines or extreme weather conditions, leading to an increase in false positives or false negatives.

[0004] To address the aforementioned issues, some existing technologies have explored various approaches. For example, one method involves fine-tuning a text-to-image generation model and utilizing a large language model to generate refined cues, synthesizing training images under different environmental conditions, and then performing domain-adaptive training together with the original images. However, these methods typically require task-specific fine-tuning of the image generation model and rely on real source domain images. Another approach uses the target category name as a text cue to generate a complete synthetic image, then extracts a foreground mask using a segmentation model, and finally pastes the foreground onto the real background image at random positions and scales. This type of method does not model the geometric, scale, or occlusion constraints between the target and the background, resulting in images with insufficient physical plausibility.

[0005] In summary, existing methods still face the following shortcomings when applied to railway foreign object intrusion detection: First, the combination of foreign object categories and environmental conditions in the training data is insufficient, and existing synthesis methods lack modeling of track geometric constraints and occlusion relationships, resulting in limited physical rationality of the generated images; Second, the model tends to rely on environmentally specific features such as track texture, seasonal vegetation, and weather lighting during training, and its detection performance significantly decreases when deployed to new lines or in extreme weather conditions. Summary of the Invention

[0006] The purpose of this invention is to provide a training method, detection method, and system for a domain-generalized detection model for foreign object intrusion detection in railways, aiming to solve the problem of decreased detection performance caused by the scarcity of real foreign object samples and environmental domain offset. By generating physically reasonable labeled synthetic data through semantic decoupling data synthesis, and combining it with vision-language alignment regularization training, the model can robustly detect foreign objects in environments without railway presence, relying solely on source domain data.

[0007] To achieve the above objectives, this invention provides a training method for a domain-generalized detection model for railway foreign object intrusion detection, comprising: S1. Semantic decoupling data synthesis: Using a large visual language model, structured instructions guide it to generate foreground cues that describe only the visual attributes of foreign objects and background cues that describe only the context of the railway environment. Based on foreground cues, a foreground image with foreign objects as the main subject is generated through a text-to-image generation model, and the foreground binary mask is extracted. Based on background cues, a background image of a railway scene is generated using a text-to-image generation model; Within the orbital boundary region of the background image, based on the collaborative constraints between the scale prior derived from the standard track gauge and the background depth prior, the foreground image and the background image are geometrically fused and the occlusion relationship is simulated to generate a synthetic image and the corresponding pixel-level mask. S2, Model Training: The backbone network of the object detection model is initialized as the visual encoder of a pre-trained vision-language model; Construct frozen text prototypes and learnable background prototypes corresponding to the foreign object categories; The target detection model is trained using existing limited railway scene foreign object intrusion annotation images and synthetic images in the source domain. The sum of the detection loss and the text prototype alignment loss is calculated as the total loss, and the model parameters are updated accordingly. Among them, the text prototype alignment loss is used to semantically align the detected query features with the text prototype.

[0008] Preferred methods for geometric fusion based on standard gauge scale priors and background depth priors include: The background image is semantically segmented for railway scenes, and the identified track and ballast class pixel regions are merged and defined as the track boundary region. Within the orbital boundary area, at least one depth interval is determined based on the background depth map, and at least one depth interval is randomly selected from it. In each selected depth interval, at least one pixel position is randomly sampled as the center point of the bottom edge of the foreground object. For each sampled bottom edge center point position Based on the orbit semantic segmentation results, the ordinate of the bottom edge center point is... The pixel distance between the left and right rails is measured and used as the local track gauge pixel distance. ; According to standard railway gauge Distance from local track gauge pixels Scale the foreground object to the target pixel width using the following formula. : ; in, The actual width of the foreground object; Dense depth estimation of the background image is performed using a depth estimation algorithm to obtain a background depth map. ; Sampling position of the center point of the bottom edge of the foreground object Background depth Target depth as a foreground object ; Foreground binary mask Mapping to the background image coordinate system yields the pixel region covered by the foreground mask. ; For pixel region Each pixel position in If satisfied Then determine the position of the foreground. Obscured by the background, and the foreground is masked. In position The number of cases handled was 0. Generate a synthetic image that conforms to the actual occlusion relationship based on the cropped foreground mask. and the corresponding pixel-level mask .

[0009] Preferably, text prototype alignment loss The calculation methods include: The text prototypes include frozen text prototypes corresponding to the foreign object category and learnable background prototypes; L2 normalization is performed on both the frozen text prototypes and the learnable background prototypes to obtain a normalized text prototype set. ,in, This represents the total number of foreign object categories. Corresponding to the Foreign objects, Corresponding background class; For each detection query feature The unit vector is obtained by two layers of MLP projection and L2 normalization. ; Calculate temperature-scaled cosine similarity: , ; in, For temperature coefficient, For the first Normalized vectors of prototypes, This indicates that the query index is being checked. To detect the total number of queries, ; The prediction results of the detection model are matched with the ground truth labels using Hungarian matching. Based on the matching results, the predicted queries that match successfully are used as the positive sample set. And from the unmatched predicted queries, those predicted queries with a maximum intersection-union ratio (MOU) lower than a threshold and a confidence level lower than a threshold are selected as a reliable background sample set. ; Define target label: ; in, For the first The true category index for each query, The index corresponding to the background prototype; Calculate the text prototype alignment loss in InfoNCE form: ; in, Indicates the first The similarity between a query and its target label. Indicates the first The query and the first Similarity between prototypes of each category Traverse 1 to , Corresponding background class.

[0010] Preferably, the large language model is a visual language model with instruction-following capabilities, the text-to-image generation model is a high-fidelity text-to-image model, and a foreground binary mask is used. Extracted using a salient target segmentation network.

[0011] Preferably, the object detection model is based on the DETR series detection framework, the pre-trained visual-language model is CLIP, and the visual encoder is ResNet-50 or ViT.

[0012] Preferably, freeze the text prototype. Category via CLIP text encoder The natural language description is encoded to obtain it; The text encoder keeps its parameters constant during training; Learnable background prototype These are trainable vectors, which, together with the text prototypes, constitute the prototype set. .

[0013] This invention also provides a domain-generalized detection method for detecting foreign object intrusion in railways, comprising: D1. Obtain the railway surveillance image to be detected; D2. Input the monitoring image into the domain generalization detection model. The domain generalization detection model is trained using the training method described above for a domain generalization detection model for railway foreign object intrusion detection. D3, the domain generalization detection model directly outputs the category, bounding box coordinates, and confidence score of foreign objects in the image; in the inference stage, the detection model does not require target domain samples, domain labels, or online fine-tuning.

[0014] This invention also provides a domain-generalized detection system for detecting foreign object intrusions in railways, comprising: The image acquisition module is used to acquire railway monitoring images; The detection module is used to load the domain generalization detection model trained by the above-mentioned training method for detecting foreign objects intruding into railways, and to execute the above-mentioned domain generalization detection method for detecting foreign objects intruding into railways on the monitoring image, and output the category, bounding box coordinates and confidence of the foreign object target. The alarm module is used to trigger a warning signal based on the output of the detection module.

[0015] Therefore, the present invention employs the above-mentioned training method, detection method, and system for a domain generalization detection model for railway foreign object intrusion detection, which has the following beneficial effects: Overcoming the limitation of scarce real-world samples: A foreground-background semantic decoupling cue generation mechanism utilizes a large language model to generate foreground and background cuees separately. This drives an untuned text-to-image generation model to synthesize foreground and railway background images. Furthermore, by incorporating track semantic segmentation, scale priors derived from standard track gauge, and depth priors, collaborative geometric constraints are applied to generate pixel-level annotated synthetic data that conforms to track gauge scale and occlusion relationships. The entire synthesis process requires no task-specific training or fine-tuning of the generation model and does not rely on real-world target domain samples. It can generate highly diverse synthetic training data at low cost and on a large scale, effectively solving the problems of scarce real-world samples and incomplete semantic coverage related to foreign object intrusion on railway tracks.

[0016] Significantly improves cross-domain generalization ability: Through regularized training with visual-language alignment, frozen text prototypes and learnable background prototypes are constructed. Text prototype alignment loss is introduced into the detection model training, projecting the detection query features onto the joint semantic space of the pre-trained visual-language model. This explicitly guides the model to learn domain-invariant semantic features, suppressing overfitting to environment-specific cues such as track texture, seasonal changes, weather conditions, or camera perspective. Ablation experiments show that, when trained using only source domain data, the average detection accuracy in unseen target domains is improved from 56.2% to 74.4%, significantly reducing both false negative and false positive rates.

[0017] Achieving out-of-the-box detection without target domain dependency: The detection model of this invention only requires end-to-end training with source domain labeled images and synthetic images, without any target domain samples, domain labels, or online fine-tuning. During the inference phase, it can directly process monitoring images from unseen railway lines, different seasons, or extreme weather, outputting reliable foreign object categories, locations, and confidence levels, meeting the practical deployment needs of railway safety systems for low cost and high adaptability.

[0018] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the training method, detection method, and overall structure of a domain generalization detection model for foreign object intrusion detection in railways, as described in this invention. Figure 2 This is a schematic diagram of the overall process of decoupled synthetic data generation in an embodiment of the training method of a domain generalization detection model for foreign object intrusion detection in railways according to the present invention. Figure 3 This is a synthetic image and corresponding labeled example diagram of an embodiment of the training method of a domain generalization detection model for foreign object intrusion detection in railways according to the present invention; Figure 4 This is a schematic diagram of visual-language alignment regularization training, which is an embodiment of the training method of a domain generalization detection model for railway foreign object intrusion detection according to the present invention. Figure 5 This is an example diagram of the detection results in the target domain of an embodiment of the domain generalization detection method for foreign object intrusion detection in railways according to the present invention. Detailed Implementation

[0020] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0021] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0022] In this application, the source domain refers to the environment or distribution of the labeled data used during model training, such as surveillance images collected from one or more specific railway lines, or data under specific season, weather, and lighting conditions; the target domain refers to the new environment or unknown distribution that the model faces during actual deployment or testing, such as entirely new railway lines, different seasons, weather, or camera perspectives; domain generalization refers to the ability of the model to maintain robust performance in unseen out-of-distribution scenarios (such as different railway lines, seasons, weather, or lighting conditions) by training only on labeled data in the source domain and deploying without relying on any target domain samples, domain labels, or online adjustments.

[0023] In this application, the "cooperative constraint between the scale prior derived from the standard track gauge and the background depth prior" refers to: using the standard track gauge as a scale reference, mapping the track gauge pixel distance segmented in the image to the target scaling size of the foreground object; simultaneously, using the foreground and background position information provided by the background depth map, determining the placement depth of the foreground object and simulating realistic occlusion relationships. The two work together to solve the problem of physically reasonable placement of foreign objects in top-view images where absolute scale information is lacking.

[0024] Example 1 like Figure 1 As shown, this embodiment provides a training method, detection method, and system for a domain-generalized detection model for railway foreign object intrusion detection. The core of this method lies in constructing a detection model that can generalize to unseen railway environments by relying solely on source domain data through semantically decoupled data synthesis and vision-language aligned regularized training.

[0025] I. Semantic Decoupling of Data Synthesis This embodiment utilizes a foreground-background semantic decoupling mechanism for cue generation and synthesis. Leveraging a large visual language model, structured instructions guide the generation of foreground cuees describing only visual attributes such as the object's color, material, shape, and geometry, and background cuees describing only the railway environment context, including track layout, lighting, weather, and camera viewpoint. Foreground images are synthesized based on the foreground cuees, and background images of the railway scene are synthesized based on the background cuees. These are then fused to generate physically plausible synthetic training data with pixel-level annotations. For example... Figure 2 As shown, the specific process for generating decoupled synthetic data includes the following steps.

[0026] (1) Structured semantic prompt generation First, real railway scene images from multiple perspectives and under various environmental conditions are collected as background priors, and a list of railway foreign object categories is compiled. Two types of prompt generation tasks are then performed using a large visual language model (VLM, such as Qwen): Background information generation: For each real railway image, guide VLM to extract structured context information, including track alignment (straight / curved), camera height (elevated / eye-level), viewpoint direction, surrounding environment (field / city / tunnel), light intensity, and weather conditions.

[0027] Foreground cues generation: For each object category, the VLM is constrained by instructions to describe only its intrinsic visual attributes (color, material, shape, surface texture, geometry), and the introduction of environment, function or subjective semantics is prohibited.

[0028] The generated prompts are stored in the background prompt library and the foreground prompt library, respectively, and used as control signals for subsequent image synthesis.

[0029] (2) Foreground and background distinguishing image synthesis Input the two types of prompts into the image generation model as high-fidelity text: The background prompt inputs high-fidelity text into the image generation model to generate a high-fidelity railway scene background image.

[0030] Foreground cues are input into a high-fidelity text image generation model, which synthesizes a target-centered foreground image against a uniform and simple background.

[0031] Subsequently, a salient object segmentation network is used to process the foreground image, extracting a foreground binary mask with sharp boundaries and semantic purity to form a high-quality foreground asset library.

[0032] (3) Geometric constraint fusion and occlusion simulation The foreground and background are physically composited in a reasonable manner. The specific steps are as follows: The background image is semantically segmented for railway scenes, and the identified track and ballast class pixel regions are merged and defined as the track boundary region.

[0033] Within the orbital boundary area, at least one depth interval is determined based on the background depth map, and at least one depth interval is randomly selected from it. Within each selected depth interval, at least one pixel position is randomly sampled as the center point of the bottom edge of the foreground object to ensure that the sampling points have good coverage and randomness in the depth dimension.

[0034] Introducing a scale prior derived from standard gauge (this embodiment uses standard railway gauge) (Based on the standard gauge). This embodiment uses the scale information provided by the standard gauge and the spatial location information provided by the depth map for collaborative constraints: the scale prior is used to determine the size of the foreign object, and the depth prior is used to determine where the foreign object is placed and its occlusion relationship. Together, they solve the problem of the lack of absolute scale information in the top-view image. Let the local gauge pixel distance between the two rails at the sampling point be . The actual width of the object is Then scale the foreground object to the target pixel width using the following formula. : ; Dense depth estimation of the background image is performed using a depth estimation algorithm to obtain a background depth map. A higher depth value indicates a greater distance from the camera.

[0035] Sampling position of the center point of the bottom edge of the foreground object Background depth The target depth of the foreground object .

[0036] Foreground binary mask Mapping to the background image coordinate system yields the pixel region it covers. .

[0037] For pixel region Each pixel position in If satisfied Then determine the position of the foreground. Obscured by the background, and the foreground is masked. In position The number of cases handled was 0.

[0038] Finally, a synthetic image that conforms to the actual occlusion relationship is generated based on the cropped foreground binary mask. and its corresponding pixel-level mask . Figure 3 The synthesized image and its corresponding pixel-level binary mask annotation are shown as an example.

[0039] II. Regularization Training for Visual-Language Alignment Based on obtaining high-quality synthetic data, to further improve the model's generalization ability in unknown railway environments, such as... Figure 4 As shown, this embodiment constructs frozen text prototypes and learnable background prototypes corresponding to foreign object categories through semantic alignment training of frozen text prototypes and learnable background prototypes. During the training of the detection model, a text prototype alignment loss is introduced to semantically align the detection query features with the prototypes, thereby suppressing the model's dependence on domain-related cues such as track texture, seasonal changes, weather conditions, or camera perspective. The specific process is as follows: (1) Visual-Language Space Alignment Initialization The backbone network of the object detection model (DINO-DETR in this embodiment) is initialized as a pre-trained visual encoder of the visual-language model (ResNet-50 visual encoder of CLIP in this embodiment), so that its initial feature distribution is aligned with the semantic space defined by the large-scale language model.

[0040] (2) Text Prototype Construction For each foreign object category Using CLIP text encoder to generate fixed embeddings And introduce learnable background prototypes. This constitutes a complete prototype set. .

[0041] (3) Text prototype alignment loss For each detection query feature The unit vector is obtained by two layers of MLP projection and L2 normalization. Index for each foreign object category And the background class, respectively calculate the normalized text prototype. and Calculate temperature-scaled cosine similarity: , ; in, To detect the total number of queries, For temperature coefficient, For the first Normalized vectors of prototypes, This indicates that the query index is being checked. ; The prediction results of the detection model are matched with the ground truth labels using Hungarian matching. Based on the matching results, the predicted queries that match successfully are used as the positive sample set. And from the unmatched predicted queries, those predicted queries with a maximum intersection-union ratio (MOU) lower than a threshold and a confidence level lower than a threshold are selected as a reliable background sample set. ; Define target label: ; in, For the first The true category index for each query, The index corresponding to the background prototype; Calculate the text prototype alignment loss in InfoNCE form: ; in, Indicates the first The similarity between a query and its target label. Indicates the first The query and the first Similarity between prototypes of each category This means iterating through all categories (foreground + background), i.e. .

[0042] The total loss function is the sum of the standard detection loss and the text prototype alignment loss: ; in, This is the original target detection loss function (e.g., the loss of DINO-DETR).

[0043] This embodiment uses existing finite railway scene foreign object intrusion annotation images in the source domain and the above-mentioned synthetic images to train the target detection model.

[0044] III. Generalized Detection Model for Foreign Object Intrusion in Railway Scenarios After training, a domain-generalized detection model is obtained. During the inference phase, the model is deployed to the railway monitoring system, with input from monitoring images of railway lines that were not trained, as well as images from different seasons or extreme weather conditions. The model directly outputs the object category, bounding box coordinates, and confidence score, without any target domain fine-tuning or online adaptation.

[0045] IV. Experimental Verification To verify the effectiveness of this invention, ablation experiments were conducted on a real railway scene dataset. The experiments included two categories of foreign objects: pedestrians and falling rocks. The source domain detection accuracy and the average target domain detection accuracy are shown in Table 1.

[0046] Table 1 Ablation Experiment

[0047] As shown in Table 1, after adding synthetic data (Model 1), the average detection accuracy of the model in the target domain increased from 56.2% to 68.1%; further adding CLIP initialization (Model 2) improved the accuracy to 72.5%; and adding text prototype alignment loss (Model 3) further improved the accuracy to 74.4%. This indicates that the semantic decoupling data synthesis and visual-language alignment regularization training of this invention can effectively improve the model's generalization ability in unknown scenarios.

[0048] Figure 5 This demonstrates a comparison of the detection results of the present invention and a baseline model in the target domain scene. From Figure 5 As can be seen from this, the present invention can more accurately locate and classify abnormal targets, effectively reduce the false alarm rate, and significantly suppress false alarms.

[0049] Therefore, this invention employs the aforementioned training method, detection method, and system for a domain-generalized detection model for railway foreign object intrusion detection. Through semantically decoupled data synthesis, it generates pixel-level labeled synthetic data that conforms to track gauge scale and occlusion relationships, effectively alleviating the problem of scarce real samples. Through visual-language alignment regularization training, it suppresses overfitting of the model to domain-related cues such as track texture, season, and weather, enhancing the learning of domain-invariant semantic features. Experiments show that in unseen target domains, this invention improves the average detection accuracy from 56.2% to 74.4%, significantly reducing false negatives and false positives, and achieving stable detection across lines, seasons, and weather conditions without the need for target domain data, meeting the actual deployment requirements of railway safety systems.

[0050] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A training method for a domain-generalized detection model for foreign object intrusion detection in railways, characterized in that, include: S1. Semantic decoupling data synthesis: Using a large visual language model, structured instructions guide it to generate foreground cues that describe only the visual attributes of foreign objects and background cues that describe only the context of the railway environment. Based on foreground cues, a foreground image with foreign objects as the main subject is generated through a text-to-image generation model, and the foreground binary mask is extracted. Based on background cues, a background image of a railway scene is generated using a text-to-image generation model; Within the orbital boundary region of the background image, based on the collaborative constraints between the scale prior derived from the standard track gauge and the background depth prior, the foreground image and the background image are geometrically fused and the occlusion relationship is simulated to generate a synthetic image and the corresponding pixel-level mask. S2, Model Training: The backbone network of the object detection model is initialized as the visual encoder of a pre-trained vision-language model; Construct frozen text prototypes and learnable background prototypes corresponding to the foreign object categories; The target detection model is trained using existing limited railway scene foreign object intrusion annotation images and synthetic images in the source domain. The sum of the detection loss and the text prototype alignment loss is calculated as the total loss, and the model parameters are updated accordingly. Among them, the text prototype alignment loss is used to semantically align the detected query features with the text prototype.

2. The training method for a domain generalization detection model for foreign object intrusion detection in railways according to claim 1, characterized in that, Specific methods for geometric fusion based on standard gauge scale priors and background depth priors include: The background image is semantically segmented for railway scenes, and the identified track and ballast class pixel regions are merged and defined as the track boundary region. Within the orbital boundary area, at least one depth interval is determined based on the background depth map, and at least one depth interval is randomly selected from it. In each selected depth interval, at least one pixel position is randomly sampled as the center point of the bottom edge of the foreground object. For each sampled bottom edge center point position Based on the orbit semantic segmentation results, the ordinate of the bottom edge center point is... The pixel distance between the left and right rails is measured and used as the local track gauge pixel distance. ; According to standard railway gauge Distance from local track gauge pixels Scale the foreground object to the target pixel width using the following formula. : ； in, The actual width of the foreground object; Dense depth estimation of the background image is performed using a depth estimation algorithm to obtain a background depth map. ; Sampling position of the center point of the bottom edge of the foreground object Background depth Target depth as a foreground object ; Foreground binary mask Mapping to the background image coordinate system yields the pixel region covered by the foreground mask. ; For pixel region Each pixel position in If satisfied Then determine the position of the foreground. Obscured by the background, and the foreground is masked. In position The number of cases handled was 0. Generate a synthetic image that conforms to the actual occlusion relationship based on the cropped foreground mask. and the corresponding pixel-level mask .

3. The training method for a domain generalization detection model for foreign object intrusion detection in railways according to claim 1, characterized in that, Text prototype alignment loss The calculation methods include: The text prototypes include frozen text prototypes corresponding to the foreign object category and learnable background prototypes; L2 normalization is performed on both the frozen text prototypes and the learnable background prototypes to obtain a normalized text prototype set. ,in, This represents the total number of foreign object categories. Corresponding to the Foreign object Corresponding background class; For each detection query feature The unit vector is obtained by two layers of MLP projection and L2 normalization. ; Calculate temperature-scaled cosine similarity: ， ； in, For temperature coefficient, This indicates that the query index is being checked. , To detect the total number of queries, For the first Normalized vectors of prototypes; The prediction results of the detection model are matched with the ground truth labels using Hungarian matching. Based on the matching results, the predicted queries that match successfully are used as the positive sample set. And from the unmatched predicted queries, those predicted queries with a maximum intersection-union ratio (MOU) lower than a threshold and a confidence level lower than a threshold are selected as a reliable background sample set. ; Define target label: ； in, For the first The true category index for each query, The index corresponding to the background prototype; Calculate the text prototype alignment loss in InfoNCE form: ； in, Indicates the first The similarity between a query and its target label. Indicates the first The query and the first Similarity between prototypes of each category.

4. The training method for a domain generalization detection model for foreign object intrusion detection in railways according to claim 1, characterized in that, The large language model is a visual language model with instruction-following capabilities; the text-to-image generation model is a high-fidelity text-to-image model; and a foreground binary mask is used. Extracted using a salient target segmentation network.

5. The training method for a domain generalization detection model for foreign object intrusion detection in railways according to claim 1, characterized in that, The object detection model is based on the DETR series detection framework, with the pre-trained visual-language model being CLIP and the visual encoder being ResNet-50 or ViT.

6. The training method for a domain generalization detection model for foreign object intrusion detection in railways according to claim 3, characterized in that, Frozen text prototype Category via CLIP text encoder The natural language description is encoded to obtain it; The text encoder keeps its parameters constant during training; Learnable background prototype These are trainable vectors, which, together with the text prototypes, constitute the prototype set. .

7. A domain-generalized detection method for detecting foreign object intrusion in railways, characterized in that, include: D1. Obtain the railway surveillance image to be detected; D2. Input the monitoring image into the domain generalization detection model. The domain generalization detection model is trained using the training method of the domain generalization detection model for railway foreign object intrusion detection as described in any one of claims 1 to 6. D3, the domain generalization detection model directly outputs the category, bounding box coordinates, and confidence score of foreign objects in the image; in the inference stage, the detection model does not require target domain samples, domain labels, or online fine-tuning.

8. A domain-generalized detection system for detecting foreign object intrusion in railways, characterized in that, include: The image acquisition module is used to acquire railway monitoring images; The detection module is used to load the domain generalization detection model trained by the training method of the domain generalization detection model for railway foreign object intrusion detection as described in any one of claims 1 to 6, and to execute the domain generalization detection method for railway foreign object intrusion detection as described in claim 7 on the monitoring image, and output the category, bounding box coordinates and confidence level of the foreign object target. The alarm module is used to trigger a warning signal based on the output of the detection module.

Images