Multi-modal image target extraction method for environment along railway

Abstract

The invention discloses a railway line environment multi-modal image target extraction method, which comprises the following steps of S1, acquiring a visible light image along a railway through an unmanned aerial vehicle, and performing three-dimensional field reconstruction based on NeRF to obtain a three-dimensional scene; s2, defining a target visibility evaluation function, and finding a visual angle enabling the target visibility evaluation function to be maximum; s2, using NeRF to render the image under the visual angle in the three-dimensional scene obtained in the step S1, and re-projecting a rendering result as an optimal visual angle image under the visual angle; s3, preprocessing data of different modes from different platforms, then converting the data and the optimal view angle image obtained in the S2 into a unified coordinate system, and obtaining a multi-mode coarse registration relation through affine transformation; and S4, target area selection and target extraction are carried out based on the improved Ground DINO network. The method effectively improves the reliability and robustness of a target detection result in a complex railway line scene.

Description

Technical Field

[0001] This invention relates to the field of intelligent interpretation technology of multimodal remote sensing images, specifically to a method for extracting targets from multimodal images of railway environment. Background Technology

[0002] With the continuous expansion of my country's railway network and the sustained growth of its operational mileage, the safety monitoring of railway infrastructure and the identification of potential environmental hazards along the lines have become core tasks for ensuring railway transportation safety. Traditional manual inspection methods are limited by low efficiency and limited coverage, making it difficult to meet the all-weather, high-precision, and real-time inspection needs of modern railways. The rapid development of remote sensing technology has provided a new technical path for intelligent railway inspection. The comprehensive application of multimodal remote sensing data provides multi-dimensional and multi-scale observation capabilities for railway line environmental monitoring. However, the special characteristics of railway scenarios pose unique technical challenges to multimodal image target extraction technology, urgently requiring targeted solutions to problems such as poor cross-modal data adaptability, inconsistent feature representation, and dynamic environmental interference.

[0003] Single-sensor technologies such as optical imaging and infrared thermal imaging were once widely used in railway facility inspection. Optical imaging can identify defects visible to the naked eye; infrared thermal imaging can detect abnormal equipment temperatures and hidden faults. However, both are susceptible to interference in complex environments, resulting in limited detection accuracy and robustness.

[0004] With the rise of various sensing technologies such as hyperspectral imaging, LiDAR, and millimeter-wave radar, multimodal image fusion has become an important development direction for improving environmental monitoring capabilities along railway lines. However, the significant differences between multi-source heterogeneous data present technical challenges for efficient fusion and target extraction.

[0005] The environment along railway lines is complex, with train aerodynamic disturbances and seasonal changes in buildings and vegetation interfering with detection. Key railway facilities are diverse in type and form, and traditional detection methods based on target templates or local features lack versatility and scalability. Real-time monitoring data is massive, and existing fusion algorithms are computationally expensive and complex, making them difficult to deploy in the field.

[0006] Currently, methods to improve the extraction of targets along railway lines can be mainly divided into three categories: traditional image processing methods rely on edge detection, template matching and other technologies, which are not robust enough in complex environments; single-modal deep learning methods are based on optical images, and some networks perform well in specific tasks, but generally suffer from insufficient cross-modal adaptability and limited robustness due to interference from complex environments; multimodal data fusion methods improve the recognition effect by fusing multi-source data, but the quality of data from different modalities is inconsistent, and the registration error between different modalities may lead to inaccurate target extraction results, and robustness and adaptability still need to be improved.

[0007] In summary, existing methods for extracting targets from UAV images along railway lines struggle to balance real-time performance and accuracy when dealing with challenges such as limited viewpoint selection in complex environments, inconsistent cross-modal feature representation, and dynamic interference. Summary of the Invention

[0008] To address the problems existing in the prior art, this invention proposes a method for target extraction from multimodal images of railway line environments that improves target visibility, matching accuracy, and robustness.

[0009] Therefore, the present invention adopts the following technical solution:

[0010] A method for target extraction from multimodal images of railway line environment includes the following steps:

[0011] S1 uses a drone to acquire visible light images of the railway line, which are then used for 3D field reconstruction via NeRF to obtain a 3D scene. ;

[0012] S2, define the target visibility evaluation function, and find the viewpoint that maximizes the target visibility evaluation function; use NeRF to evaluate the 3D scene obtained in S1. Render the image from that perspective, and then reproject the rendered result to the optimal viewpoint image from that perspective. ;

[0013] S3 preprocesses data from different modalities on different platforms, and then compares the preprocessed modal data with the optimal viewpoint image obtained in S2. Transform to a unified coordinate system through affine transformation. Initial registration is achieved, resulting in multimodal coarse registration relationships;

[0014] S4, target region selection and target extraction based on the improved GroundingDINO network, including:

[0015] S41, Construct an improved GroundingDINO network;

[0016] S42, train the improved GroundingDINO network constructed in S41;

[0017] S43, based on the trained improved GroundingDINO network, achieves target region selection and target extraction.

[0018] In step S2 above:

[0019] The target visibility evaluation function The formula is:

[0020] ,

[0021] In the formula, These represent a viewpoint direction in a spherical coordinate system, namely the pitch angle and the azimuth angle; This represents all targets to be detected; Indicates the first A goal in perspective Signal-to-noise ratio at the following levels; Indicates the first The importance weight of each objective;

[0022] The three-dimensional scene obtained in S1 In this process, grid sampling is used to traverse multiple viewpoints to find the formula for the viewpoint that maximizes the evaluation function:

[0023] ,

[0024] in, To maximize the evaluation function from the perspective of perspective.

[0025] In step S3 above: the data of different modalities include optical images, depth maps and infrared images; the preprocessing includes denoising, geometric correction and preliminary RANSAC registration.

[0026] In step S41 above:

[0027] The improved GroundingDINO network takes images of different modalities as input. , and natural language text The output is in the form of natural language text. Describe the corresponding target region location and confidence level; the improved GroundingDINO network includes a dual-branch visual encoder, a feature alignment and adaptive fusion module, a language encoder, a language-guided query module, a language-visual cross-modal interaction structure, a fine-grained localization module, and a detection head, wherein:

[0028] The dual-branch visual encoder includes a branch A visual encoder and a branch B visual encoder, used for image extraction. , Features, in which branch A visual encoder for images The feature map is obtained through processing. Branch B visual encoder for images The feature map is obtained through processing. ;

[0029] The feature alignment and adaptive fusion module is used to adjust the input feature map. and To process, first analyze the feature map. and Scale and channel matching are performed to make them fusionable; then, a multi-head attention structure is introduced to fuse them to obtain a fused feature map. Multi-head attention structures can adaptively adjust feature maps. and exist The proportion of contribution in;

[0030] The language encoder is used to process the input natural language text. The text feature vector is obtained through processing. ;

[0031] The language-guided query module is used to process the input text feature vector. and fusion feature map Language-guided query selection will filter out image features that are highly relevant to the text as language query vectors;

[0032] The language-visual cross-modal interaction structure is used to process the input text feature vector. fusion feature map The target response map is obtained by processing the language query vector.

[0033] The refined localization module is used to process the input target response map. First, the target response map is adaptively thresholded to obtain a preliminary candidate region set. Then, for each preliminary candidate region, the corresponding initial target confidence is calculated based on its activation intensity in the target response map. The preliminary candidate region set is then screened a second time to improve the effectiveness of the candidate regions.

[0034] The detection head is used to further perform bounding box regression on the screened candidate regions to restore the precise location and scale of the target, enhance or correct the initial target confidence to obtain the corrected confidence, thereby improving the reliability of the target extraction results, and finally outputting the target region location and confidence.

[0035] In step S42 above:

[0036] Based on the multimodal coarse registration relationship established in S3, reconstruction loss and consistency loss are introduced to form the first combined loss function; cross-entropy loss, bounding box regression loss, mask loss and language matching loss are introduced to form the second combined loss function.

[0037] Using the combined loss function one and combined loss function two, a large number of unlabeled visible light images and their corresponding optimal viewpoint images are used as inputs to the dual-branch visual encoder. Specifically, the input of the branch A visual encoder is the visible light image, and the input of the branch A visual encoder is the corresponding optimal viewpoint image. Annotated data with text descriptions is used as input to the language encoder. Based on the detection head trained with high-quality manually annotated samples, the improved GroundingDINO network is optimized and trained. The cross-modal alignment ability, semantic understanding ability, and target localization ability are simultaneously optimized through weight summation to obtain the trained improved GroundingDINO network.

[0038] In step S43 above:

[0039] Based on the improved GroundingDINO network trained by S42, the input of the visual encoder in branch A is replaced with a raw visible light image acquired by a drone in S1. The input to the branch B visual encoder is replaced with the original visible light image. The corresponding optimal viewpoint image obtained from S2 The input natural language text for the language encoder Replace with the desired text instructions to obtain the target region and target extraction results.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] 1. Enhanced Free-View 3D Reconstruction and Observation Capabilities: The NeRF-based implicit 3D reconstruction technology supports free-viewpoint selection, overcoming the limitations of fixed viewpoints in traditional multi-view images. It can maximize the visibility of targets and texture details, significantly improving target recognition accuracy.

[0042] 2. Efficient fusion and feature unification of multimodal data: Through deep cross-modal coupling enhancement network, adaptive alignment and fusion of multi-source data such as optical images, infrared images and depth maps are realized to form a unified and stable feature representation and improve the efficiency of collaborative utilization of multi-source data.

[0043] 3. Model robustness and convergence performance optimization: By combining multi-head attention mechanism, the global optimization capability and robustness of the model are enhanced, the influence of local optimum traps is reduced, and high accuracy and high stability of target detection and extraction are ensured.

[0044] 4. Intelligent Interaction and Dynamic Target Query: A language-guided dynamic query mechanism is introduced, supporting natural language command input to achieve dynamic identification and localization of multiple target categories. This enhances the intelligent interaction level and human-computer collaboration flexibility of the method, and expands its application scenarios.

[0045] 5. Significantly improved overall performance and engineering application value: Overall, it improves the accuracy, efficiency and real-time response capability of multi-platform and multi-modal target extraction along railway lines, providing reliable technical support for intelligent railway inspection and safety early warning, and has strong engineering application prospects and promotion potential. Attached Figure Description

[0046] Figure 1 This is a flowchart of the target extraction method of the present invention;

[0047] Figure 2 The diagram shows a multimodal remote sensing image, (a) showing the optical image and depth map, and (b) showing the optical image and infrared image.

[0048] Figure 3 This is a schematic diagram of the improved GroundingDINO network structure in this invention;

[0049] Figure 4 This is a target extraction map of a multimodal image along a railway line in an embodiment of the present invention. Detailed Implementation

[0050] The technical solution of the invention will be clearly and completely described below with reference to the accompanying drawings and embodiments. Obviously, the following embodiments are only some embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] Example

[0052] The method for target extraction from multimodal images of the railway line environment in this embodiment, such as... Figure 1 As shown, it includes the following steps:

[0053] S1, Multi-view UAV image acquisition and NeRF 3D field reconstruction, as detailed below:

[0054] S11, multi-view drone image acquisition.

[0055] Use drones to collect visible light images of the railway line from different perspectives (different altitudes, angles, and flight paths). Each image needs to record its camera's intrinsic parameters (focal length, principal point) and extrinsic parameters (i.e., the camera's attitude and position in the world coordinate system). These intrinsic and extrinsic parameters are collectively referred to as... .

[0056] In the present invention, all images are image data.

[0057] S12, NeRF 3D field reconstruction.

[0058] Neural radiation field (NeRF) is an implicit 3D modeling method that essentially uses neural networks (typically multilayer perceptrons). This method uses a NerfStudio network structure to fit the color and density distribution at any location and direction in the scene. In this embodiment, the NerfStudio network structure is used for 3D field reconstruction. The trained model is then used to obtain an implicit 3D representation of the 3D scene based on the input multi-view UAV imagery. It is a differentiable, continuous 3D scene. This scene can be used to render realistic images from any angle, providing spatially consistent visual input for subsequent object detection, matching, and cross-modal fusion. The specific operation is as follows:

[0059] First, the image obtained from S1 and its corresponding intrinsic and extrinsic parameters are processed to obtain voxel coordinates. With viewpoint direction As input to the NerfStudio network, utilizing Fit density With radiance :

[0060] (1)

[0061] in, High-frequency mapping function; density function Used to measure the density of matter at a point, and to control whether that point "blocks" light; radiance function Indicates from the perspective direction The color emitted by that point during observation.

[0062] Use volume rendering to composite each ray From near to far ( arrive The color value of the light source is integrated over all sampled points on the light source, accumulating the color and transparency to generate the RGB value of each pixel in the 3D scene.

[0063] (2)

[0064] in, For the predicted pixel color, Let be the transparency of the light from the starting point to position 𝑡 (i.e., the probability that the light from the starting point to position 𝑡 is not blocked).

[0065] After generating the RGB values ​​of each pixel in a 3D scene, an implicit 3D representation of the 3D scene is obtained. .

[0066] S2, Optimal observation angle selection and reprojection.

[0067] Since targets may be obscured or unclear from certain viewpoints, a function is needed to measure "how easily a target is visible from the current viewpoint." Based on the spatial distribution of the targets to be inspected, a target visibility evaluation function is defined. :

[0068] (4)

[0069] In the formula, Representing a viewpoint direction in a spherical coordinate system, respectively, the pitch angle and the azimuth angle. This represents all targets to be detected. Indicates the first A goal in perspective The signal-to-noise ratio (SNR) at a given point in time can be understood as how easily a signal can be identified from that perspective. Indicates the first The importance weight of each objective may vary, with some key components potentially having a higher weight than others.

[0070] Implicit 3D Scene Constructed by NeRF In this process, grid sampling is used to traverse multiple viewpoints to find the viewpoint that maximizes the evaluation function, i.e.:

[0071] (5)

[0072] After determining the optimal viewpoint, NeRF is used to render the image of the 3D scene obtained from S1 at that viewpoint, and the NeRF rendering result is reprojected as the optimal viewpoint image. This image is used for subsequent feature extraction. Theoretically, this image contains the clearest and most easily identifiable target.

[0073] S3, multimodal data access and coarse registration.

[0074] Integrating data from different platforms and different modalities into a single geometric space. For example... Figure 2 As shown, in this embodiment, the data of different modalities includes optical images, depth maps, and infrared images, wherein: optical images It is a high-resolution visible light image used to reveal details of the earth's surface texture; depth map Information from LiDAR or stereo vision provides 3D geometric structure information; infrared imagery. Capturing thermal information can reveal structures or damage invisible to the naked eye. This is also relevant to optical imaging. Infrared imaging Depth map Preprocessing is performed, including denoising, geometric correction, and preliminary RANSAC registration. The purpose of denoising is to eliminate image noise and improve the stability of subsequent registration and feature extraction; geometric correction can correct imaging distortion and ensure spatial consistency among images; preliminary RANSAC registration uses the random sampling consensus algorithm.

[0075] The preprocessed modal data are compared with the optimal viewpoint image obtained from S2. Transform to a unified coordinate system through affine transformation. Achieve coarse registration and obtain multimodal coarse registration relationships.

[0076] S4. Target region selection and target extraction are performed based on the improved GroundingDINO network. The specific operations are as follows:

[0077] S41, construct an improved GroundingDINO network.

[0078] like Figure 3 As shown, the GroundingDINO network is improved from a single-branch visual encoder to a dual-branch visual encoder, and feature alignment and adaptive fusion modules are introduced to process the output of the dual-branch visual encoder. The input data for the improved GroundingDINO network consists of images of different modalities. , and natural language text The output is in the form of natural language text. Describe the corresponding target region location and confidence level. The improved GroundingDINO network includes a dual-branch visual encoder, a feature alignment and adaptive fusion module, a language encoder, a language-guided query module, a language-visual cross-modal interaction structure, a fine-grained localization module, and a detection head, wherein:

[0079] The dual-branch visual encoder includes a branch A visual encoder and a branch B visual encoder, used for image extraction. , Features, in which branch A visual encoder for images The feature map is obtained through processing. Branch B visual encoder for images The feature map is obtained through processing. The dual-branch design can handle multimodal images, overcoming the original network's dependence on a single RGB image. The visual encoder is a Transformer-based encoder.

[0080] The feature alignment and adaptive fusion module is used to adjust the input feature map. and To process, first analyze the feature map. and Scale and channel matching are performed to make them fusionable; then, a multi-head attention structure is introduced to fuse them to obtain a fused feature map. Multi-head attention structures can adaptively adjust feature maps. and exist The contribution ratio of the network enables the network to autonomously select the most advantageous information for current recognition in scenarios such as low light, occlusion, and complex structures, thereby significantly improving the stability and robustness of the network in different scenarios.

[0081] The language encoder is used to process the input natural language text. The text feature vector is obtained through processing. The natural language text For task description, it supports multiple text combinations and can further improve target positioning accuracy through compound commands.

[0082] The language-guided query module is used to process the input text feature vector. and fusion feature map Language-guided query selection filters out image features highly relevant to the text as the language query vector. Specifically, it calculates the text feature vector. Included text features and fused feature maps The similarity of the included image features is used to obtain an association score through matrix multiplication. The features with the highest scores are then selected to initialize the language query vector.

[0083] The language-visual cross-modal interaction structure is used to process the input text feature vector. fusion feature map The target response map is obtained by processing the language query vector.

[0084] The refined localization module is used to process the input target response map. First, the target response map is adaptively thresholded to obtain a preliminary candidate region set. Then, for each preliminary candidate region, the corresponding initial target confidence is calculated based on its activation intensity in the target response map. The preliminary candidate region set is then screened a second time to improve the effectiveness of the candidate regions.

[0085] The detection head is used to further perform bounding box regression on the screened candidate regions to restore the precise location and scale of the target, enhance or correct the initial target confidence to obtain the corrected confidence, thereby improving the reliability of the target extraction results, and finally outputting the target region location and confidence.

[0086] S42 trains the improved GroundingDINO network constructed in S41.

[0087] Based on the multimodal coarse registration relationship established in S3, reconstruction loss and consistency loss are introduced to form the first combined loss function; cross-entropy loss, bounding box regression loss, mask loss and language matching loss are introduced to form the second combined loss function.

[0088] Using the combined loss function one and combined loss function two, a large number of unlabeled visible light images and their corresponding optimal viewpoint images are used as inputs to the dual-branch visual encoder. Specifically, during training, the input of the visual encoder for branch A is the visible light image, and the input of the visual encoder for branch B is the corresponding optimal viewpoint image. Annotated data with text descriptions (e.g., garbage dumps, plastic bags, dust nets, etc.) are used as inputs to the language encoder. Based on the detection head trained with high-quality manually annotated samples, the improved GroundingDINO network is optimized and trained. The cross-modal alignment ability, semantic understanding ability, and target localization ability are simultaneously optimized through weight summation to obtain the trained improved GroundingDINO network.

[0089] S43, based on the trained improved GroundingDINO network, achieves target region selection and target extraction.

[0090] Based on the improved GroundingDINO network trained by S42, the input of the visual encoder in branch A is replaced with a raw visible light image acquired by a drone in S1. The input to the branch B visual encoder is replaced with the original visible light image. The corresponding optimal viewpoint image obtained from S2 The input natural language text for the language encoder Replace with the desired text instructions to obtain the target region and target extraction results.

[0091] like Figure 4 As shown, the target extraction results in typical scenarios along the railway line are visualized and analyzed. The target region bounding box output by the improved GroundingDINO network is overlaid on the detection image and its corresponding depth map and infrared image as the background to show the target extraction results.

[0092] The visualization results show that the improved GroundingDINO network can stably locate key targets along the railway line, and the boundary of the target area has a high degree of consistency with the actual target location. By introducing multimodal coarse registration relationship, the final target extraction result maintains spatial consistency under multiple modalities, the target area localization is more stable, and the reliability and robustness of target detection results in complex railway line scenarios are effectively improved.

Claims

1. A method for extracting targets from multimodal images of the railway environment, characterized in that, Includes the following steps: S1 uses a drone to acquire visible light images of the railway line, which are then used for 3D field reconstruction via NeRF to obtain a 3D scene. ; S2, define the target visibility evaluation function, and find the viewpoint that maximizes the target visibility evaluation function; use NeRF to evaluate the 3D scene obtained in S1. Render the image from that perspective, and then reproject the rendered result to the optimal viewpoint image from that perspective. ; S3 preprocesses data from different modalities on different platforms, and then compares the preprocessed modal data with the optimal viewpoint image obtained in S2. Transform to a unified coordinate system through affine transformation. Initial registration is achieved, resulting in multimodal coarse registration relationships; S4, target region selection and target extraction based on the improved GroundingDINO network, including: S41, Construct an improved GroundingDINO network; S42, train the improved GroundingDINO network constructed in S41; S43, based on the trained improved GroundingDINO network, achieves target region selection and target extraction.

2. The method for extracting targets from multimodal images of the railway environment according to claim 1, characterized in that, In S2: The target visibility evaluation function The formula is: ， In the formula, These represent a viewpoint direction in a spherical coordinate system, namely the pitch angle and the azimuth angle; This represents all targets to be detected; Indicates the first A goal in perspective Signal-to-noise ratio at the following levels; Indicates the first The importance weight of each objective; The three-dimensional scene obtained in S1 In this process, grid sampling is used to traverse multiple viewpoints to find the formula for the viewpoint that maximizes the evaluation function: ， in, To maximize the evaluation function from the perspective of perspective.

3. The method for extracting targets from multimodal images of railway environment according to claim 2, characterized in that, In S3: the data of different modalities include optical images, depth maps and infrared images; the preprocessing includes denoising, geometric correction and preliminary RANSAC registration.

4. The method for extracting targets from multimodal images of railway environment according to claim 1, characterized in that, In S41: The improved GroundingDINO network takes images of different modalities as input. , and natural language text The output is in the form of natural language text. Describe the corresponding target region location and confidence level; the improved GroundingDINO network includes a dual-branch visual encoder, a feature alignment and adaptive fusion module, a language encoder, a language-guided query module, a language-visual cross-modal interaction structure, a fine-grained localization module, and a detection head, wherein: The dual-branch visual encoder includes a branch A visual encoder and a branch B visual encoder, used for image extraction. , Features, in which branch A visual encoder for images The feature map is obtained through processing. Branch B visual encoder for images The feature map is obtained through processing. ; The feature alignment and adaptive fusion module is used to perform feature mapping on the input feature map. and To process, first analyze the feature map. and Scale and channel matching are performed to make them fusionable; then, a multi-head attention structure is introduced to fuse them to obtain a fused feature map. Multi-head attention structures can adaptively adjust feature maps. and exist The proportion of contribution in; The language encoder is used to process the input natural language text. The text feature vector is obtained through processing. ; The language-guided query module is used to process the input text feature vector. and fusion feature map Language-guided query selection will filter out image features that are highly relevant to the text as language query vectors; The language-visual cross-modal interaction structure is used to process the input text feature vector. fusion feature map The target response map is obtained by processing the language query vector. The refined localization module is used to process the input target response map. First, the target response map is adaptively thresholded to obtain a preliminary candidate region set. Then, for each preliminary candidate region, the corresponding initial target confidence is calculated based on its activation intensity in the target response map. The preliminary candidate region set is then filtered a second time to improve the effectiveness of the candidate regions. The detection head is used to further perform bounding box regression on the screened candidate regions to restore the precise location and scale of the target, enhance or correct the initial target confidence to obtain the corrected confidence, thereby improving the reliability of the target extraction results, and finally outputting the target region location and confidence.

5. The method for extracting targets from multimodal images of the railway environment according to claim 4, characterized in that, In S42: Based on the multimodal coarse registration relationship established in S3, reconstruction loss and consistency loss are introduced to form the first combined loss function; cross-entropy loss, bounding box regression loss, mask loss and language matching loss are introduced to form the second combined loss function. Using the combined loss function one and combined loss function two, a large number of unlabeled visible light images and their corresponding optimal viewpoint images are used as inputs to the dual-branch visual encoder. Specifically, the input of the branch A visual encoder is the visible light image, and the input of the branch A visual encoder is the corresponding optimal viewpoint image. Annotated data with text descriptions is used as input to the language encoder. Based on the detection head trained with high-quality manually annotated samples, the improved GroundingDINO network is optimized and trained. The cross-modal alignment ability, semantic understanding ability, and target localization ability are simultaneously optimized through weight summation to obtain the trained improved GroundingDINO network.

6. The method for extracting targets from multimodal images of railway environment according to claim 5, characterized in that, In S43: Based on the improved GroundingDINO network trained by S42, the input of the visual encoder in branch A is replaced with a raw visible light image acquired by a drone in S1. The input to the branch B visual encoder is replaced with the original visible light image. The corresponding optimal viewpoint image obtained from S2 The input natural language text for the language encoder Replace with the desired text instructions to obtain the target region and target extraction results.

Images