Open vocabulary 3d inference gaussian sputtering method based on implicit text queries

Abstract

The application discloses an open vocabulary 3D reasoning Gaussian sputtering method based on implicit text query. The application comprises the following steps: 1, selecting a representative view set for reference; 2, using a visual language model to analyze implicit query text, and generating a target detection bounding box on the representative view set, and using a segmentation model to segment the target detection bounding box to generate a refined 2D target mask; 3, refining the 2D target mask to obtain a refined mask; 4, embedding the refined mask into a 3D Gaussian to realize a 3D target mask aligned with the implicit query text; through the optimized refined mask, the optimization process of single-dimensional mask coding is guided, and finally a 3D target mask accurately matched with the implicit query text is realized. Experiments on multiple models and data sets show that the open vocabulary 3D reasoning Gaussian sputtering method based on implicit text query can effectively improve the speed and performance of implicit segmentation in the 3D Gaussian.

Description

Technical Field

[0001] This invention proposes an open-vocabulary 3D inference Gaussian sputtering method based on implicit text query for segmentation tasks of implicit targets in 3D space. Background Technology

[0002] Open-vocabulary 3D query technology has recently garnered significant attention, with existing methods such as Language Embedded Neural Radiation Field (LERF) and 3D Language Gaussian Sputtering (LangSplat) demonstrating their ability to generate 3D object masks based on explicit instructions. However, real-world human interactions often rely on implicit instructions (e.g., "what can be used to hold cookies") rather than explicit terms like "plate." Current 3D systems built upon these methods typically require explicit object labeling to perform tasks, exhibiting limited performance in understanding user intent through implicit instructions. This limitation hinders the development of cognitive architectures for 3D scene understanding, despite the potential these systems have shown in spatial intelligence and robotics.

[0003] This invention proposes an innovative task of open-vocabulary 3D reasoning, which requires generating accurate 3D target masks from implicit query texts involving complex reasoning. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by providing an open-vocabulary 3D inference Gaussian sputtering method based on implicit text query.

[0005] This invention modifies existing datasets (3D-OVS and LERF-Mask) by manually relabeling explicit instructions as implicit queries while preserving the original segmentation results.

[0006] The Gaussian sputtering method for open-vocabulary 3D inference based on implicit text query includes the following steps:

[0007] Step 1: Select a representative set of reference views ;

[0008] Step 2: Obtain the 2D target mask;

[0009] Using a visual language model to parse implicit query text and in representative view collections Generate target detection bounding boxes .

[0010] In obtaining the target detection bounding box Then, the segmentation model is used to detect the bounding boxes of the targets. Segmentation is performed to generate a refined 2D target mask. .

[0011] Step 3: Refine the 2D target mask to obtain a refined mask;

[0012] For the same implicit query text While the inference and segmentation results of multimodal large language models and segmentation models in different representative views are mostly correct and point to the same target, an optimization mechanism is still needed to ensure the accuracy of 3D segmentation. This invention proposes a 2D target mask optimization method, which uses a maximum clique mining algorithm to select the most likely correct 2D target mask in representative views. It should be particularly noted that erroneous segmentation results in representative views will directly affect the accuracy of the final 3D segmentation.

[0013] Step 4: Embed the refined mask into a 3D Gaussian mask to achieve an accurate 3D target mask aligned with the implicit query text. This invention guides the optimization process of single-dimensional mask encoding through the optimized refined mask, ultimately achieving a 3D target mask that precisely matches the implicit query text.

[0014] Furthermore, step (1) is as follows:

[0015] Multimodal large language models can locate the corresponding target in each view based on implicit query text. However, using all existing views and relying on a multimodal large language model for implicit information reasoning would result in a significant computational burden. Given the perspective similarity between different views within the same scene, this invention proposes selecting only a small set of representative views. This can meet the inference needs of multimodal large language models.

[0016] First, a hierarchical clustering algorithm based on the camera position in the world coordinate system is used to obtain representative views from the camera's perspective:

[0017]

[0018] in This represents a hierarchical clustering algorithm. This represents the number of clusters, and its value is the same as the number of representative views. This represents the set of camera positions in the existing view. This represents a view clustering group. Views within the same cluster have similar shooting perspectives.

[0019] Secondly, after clustering, the mean value of camera positions within each cluster is calculated. The camera view closest to the mean position is selected as the representative view of that cluster, thus forming a representative view set. .

[0020] Furthermore, step (2) is as follows:

[0021] This invention employs the Qwen2.5-VL-7B-Instruct visual language model to parse implicit query text. and in representative view collections Generate target detection bounding boxes Implicit query text is about to be executed. and representative view collection As input to the Tongyi Qianwen 2.5 model, this model maps natural language descriptions to visual space coordinates through deep semantic understanding, achieving coarse target localization under open vocabulary conditions. Represents a representative set of views The number of representative views in the data.

[0022] For the same implicit query text This invention uses the Tongyi Qianwen 2.5 visual language model to parse implicit query text. and in representative view collections Generate target detection bounding boxes In obtaining the target detection bounding box Subsequently, this invention utilizes the Segment Anything Model (SAM) to generate a refined 2D target mask. Specifically, the target detection bounding box As a prompt for input segmentation model, a representative set of views is also included. Input the segmentation model; the SAM segmentation model, through its powerful zero-shot segmentation capability, outputs a refined 2D target mask that strictly corresponds to the target region in a representative view. This process fully integrates cueing engineering and adaptive segmentation techniques to ensure high segmentation accuracy even in complex scenes. The specific formula is as follows:

[0023]

[0024]

[0025] in, Representing a visual language model, Represents the segmentation model;

[0026] Furthermore, step (3) is as follows:

[0027] In three-dimensional space, the scale of the same target is invariant. Based on this characteristic, this invention uses the target scale as the core criterion for determining whether different view segmentation results belong to the same target. Specific implementation includes:

[0028] 3-1. Map the 2D target mask to three-dimensional spatial coordinates using the camera pose matrix to obtain a set of three-dimensional coordinate points;

[0029] 3-2. Calculate the standard deviation of the three-dimensional coordinate point set and use this standard deviation as the target scale. The target scale is solved as follows:

[0030]

[0031] in, , and Representing three-dimensional coordinate points The coordinates.

[0032] 3-3. The most likely correct 2D target mask in the representative view is selected using the maximum clique mining algorithm, as follows:

[0033] 3-3-1. Constructing a similarity graph: Define an undirected, unweighted graph. Vertex set The set of edges representing 2D target masks. This represents the set of similarities between 2D target masks;

[0034] 3-3-2. Determine the similarity criterion: When the target scale ratio of any two 2D target masks is greater than 1, the similarity criterion is determined. At that time, establish the connection edges of the 2D target mask. ,otherwise ;

[0035] 3-3-3. Filtering using the maximum clique mining algorithm: The correct mask recognition problem is abstracted into the maximum clique problem in graph theory, and a backtracking algorithm is used to solve the problem in the undirected, unweighted graph. The largest group The maximum clique obtained by solving is the refined mask. .

[0036] Furthermore, step (4) is as follows:

[0037] To achieve targeted segmentation of the query target (while excluding other irrelevant objects in the scene), this invention innovatively employs single-dimensional mask encoding for 3D Gaussian allocation, specifically including:

[0038] 4-1. Targeted segmentation: Only perform 3D Gaussian encoding on the 3D target corresponding to the implicit query text;

[0039] 4-2 Differentiable Rendering: Directly generating target segmentation masks using differentiable rasterization technology. ;

[0040] 4-3. Construct a composite loss function that integrates binary cross-entropy and the Dice coefficient. In refining the mask Under the supervision of [the relevant authority], the single-dimensional mask encoding is optimized, and the final output three-dimensional target mask achieves semantic-level alignment with the implicit query text.

[0041] 4-4. The 3D Gaussian field with one-dimensional mask encoding saved after training can display the target segmentation results according to the specified viewpoint.

[0042] Furthermore, the present invention also provides an open-vocabulary 3D inference Gaussian sputtering system based on implicit text query, comprising:

[0043] Dataset module: Selects a representative set of reference views;

[0044] 2D target mask acquisition module: It uses a visual language model to parse implicit query text and generates target detection bounding boxes on a representative view set. It then uses a segmentation model to segment the target detection bounding boxes and generate a refined 2D target mask.

[0045] The refined mask acquisition module refines the 2D target mask by mapping it to 3D spatial coordinates using the camera pose matrix, resulting in a set of 3D coordinate points. It then calculates the standard deviation of each point in the 3D coordinate set and uses this standard deviation as the target scale. The most likely correct 2D target mask in the representative view is selected by the maximum clique mining algorithm to obtain the refined mask;

[0046] 3D Target Mask Acquisition Module: Performs 3D Gaussian encoding only on the 3D targets corresponding to the implicit query text, and directly generates the target segmentation mask using differentiable rasterization technology. Construct a composite loss function that integrates binary cross-entropy and the Dice coefficient. In refining the mask Under the supervision of [the system], the single-dimensional mask encoding is optimized, and the final output three-dimensional target mask achieves semantic-level alignment with the implicit query text.

[0047] Furthermore, the present invention also provides an open-vocabulary 3D inference Gaussian sputtering device based on implicit text query, wherein the method described in the present invention is provided within the device.

[0048] The beneficial effects of this invention are as follows:

[0049] (1) Open vocabulary 3D reasoning is defined as the core capability of 3D intelligent system for the first time; (2) Multimodal large language model reasoning, multi-view target mask refinement and efficient single-dimensional mask encoding are integrated; (3) A new benchmark for 3D query based on implicit instructions is established, and experiments show that the method has excellent reasoning ability.

[0050] To enhance the model's ability to understand implicit human commands, this invention proposes a Gaussian sputtering method for open-vocabulary 3D reasoning based on implicit text queries, addressing the novel task of open-vocabulary 3D reasoning. This method outputs an accurate 3D target mask based on a given implicit query text. Evaluation on the open-vocabulary 3D reasoning dataset annotated by this invention demonstrates superior performance in handling this type of task, achieving state-of-the-art results in the field. The proposed method can advance the frontier development of open-vocabulary 3D querying and is expected to find applications in a wide range of fields such as spatial intelligence and robotics. Attached Figure Description

[0051] Figure 1 This is a visualization of the invention.

[0052] Figure 2 This is a complete flowchart of the present invention;

[0053] Figure 3 The results of testing this invention and other existing methods on the 3D-OVS dataset are shown in the figure.

[0054] Figure 4 The graph shows the results of testing this invention and other existing methods on the LERF-Mask dataset; Detailed Implementation

[0055] The method of the present invention and its detailed parameters are described in further detail below.

[0056] like Figure 1 As shown, the method of this invention addresses this challenge through three core components: First, it uses a multimodal large language model to parse the implicit query, clusters representative views based on camera location, and then uses a segmentation model (SAM) to obtain a 2D target mask; second, it constructs a similarity map and uses a maximum clique mining algorithm to determine the most likely correct 2D target mask as a refined mask, thus solving the problem of inconsistent masks across views; finally, it designs an efficient one-dimensional mask encoding scheme, embedding the refined mask into a 3D Gaussian to achieve an accurate 3D target mask aligned with the implicit query text.

[0057] This invention evaluates open-vocabulary 3D inference performance on two benchmark datasets:

[0058] (1) 3D-OVS dataset: contains target objects with a long tail distribution. Each scene provides multiple sets of explicit text annotations for objects. It is designed specifically for 3D open vocabulary semantic segmentation tasks.

[0059] (2) LERF-Mask dataset: As an extended version of LERF-Localization, it provides higher-precision query object mask annotation, suitable for evaluation of 3D open vocabulary segmentation tasks. To build a 3D inference benchmark, this invention converts the original explicit text into an implicit representation.

[0060] The specific configuration is as follows: Number of clusters The implicit text was parsed and representative view targets were located using the Qwen2.5-VL-7B-Instruct visual language model. Batch inference was used to accelerate the processing. 3D Gaussian sputtering (3DGS) was used for initial training of 10,000 steps (default parameters). After freezing the 3D Gaussian parameters, an optimizer that only optimizes the one-dimensional mask encoding was used for training. The Adam optimizer was used for training (learning rate 0.5), and convergence was achieved in 50 iterations. All experiments were completed on the NVIDIA RTX-4090 GPU platform.

[0061] like Figure 2 As shown, the workflow of this invention first clusters the cameras of existing views to select representative views. Then, using a generalized query and segmentation model, these representative views are located and segmented based on the input implicit query text. The similarity between 2D segmentation masks in the representative views is calculated to construct a similarity map, which is used to infer more refined masks. Finally, under the supervision of the refined masks, the 3D Gaussian one-dimensional mask encoding is optimized to achieve accurate 3D segmentation consistent with the implicit query text.

[0062] The Gaussian sputtering method for open-vocabulary 3D inference based on implicit text query is described in the following steps:

[0063] Step (1) Select a representative set of reference views:

[0064] Multimodal large language models can locate the corresponding target in each view based on implicit query text. However, using all existing views and relying on a multimodal large language model for implicit information reasoning would result in a significant computational burden. Given the perspective similarity between different views within the same scene, this invention proposes selecting only a small set of representative views. This can meet the inference requirements of multimodal large language models. The specific implementation uses a hierarchical clustering algorithm based on camera position in the world coordinate system.

[0065]

[0066] Where HC(·) represents the hierarchical clustering algorithm, This represents the number of clusters, and its value is the same as the number of representative views. This represents the set of camera positions in the existing view. Representative view clusters are formed. Views within the same cluster have similar shooting angles. After clustering, the mean camera position within each cluster is calculated, and the camera angle closest to the mean position is selected as the representative view for that cluster, ultimately forming a set of representative views. .

[0067] Step (2) Obtain the 2D target mask:

[0068] This invention uses the Tongyi Qianwen 2.5 visual language model to parse implicit query text. and in representative view collections Generate target detection bounding boxes Implicit query text is about to be executed. and representative view collection As input to the Tongyi Qianwen 2.5 visual language model, this model maps natural language descriptions to visual space coordinates through deep semantic understanding, achieving coarse target localization under open vocabulary conditions.

[0069] In obtaining the target detection bounding box Subsequently, this invention utilizes the Segment Anything Model (SAM) to generate a refined 2D target mask. Specifically, the target detection bounding box As a prompt for input segmentation model, a representative set of views is also included. Input the segmentation model; the SAM segmentation model, through its powerful zero-shot segmentation capability, outputs a refined 2D target mask that strictly corresponds to the target region in a representative view. This process fully integrates cueing engineering and adaptive segmentation techniques to ensure high segmentation accuracy even in complex scenes. The specific formula is as follows:

[0070]

[0071]

[0072] in, Representing a visual language model, Represents the segmentation model;

[0073] Step (3) Refine the 2D target mask to obtain the refined mask:

[0074] In three-dimensional space, the scale of the same target is invariant. Based on this characteristic, this invention uses the target scale as the core criterion for determining whether different view segmentation results belong to the same target. Specific implementation includes:

[0075] 3-1. Map the 2D target mask to three-dimensional spatial coordinates using the camera pose matrix to obtain a set of three-dimensional coordinate points;

[0076] 3-2. Calculate the standard deviation of the three-dimensional coordinate point set and use this standard deviation as the target scale. The target scale is solved as follows:

[0077]

[0078] in, , and Representing three-dimensional coordinate points The coordinates.

[0079] 3-3. The most likely correct 2D target mask in the representative view is selected using the maximum clique mining algorithm, as follows:

[0080] 3-3-1. Constructing a similarity graph: Define an undirected, unweighted graph. Vertex set A set of 2D target masks, edge set This represents the set of similarities between 2D target masks.

[0081] 3-3-2. Determine the similarity criterion: When the target scale of any two 2D target masks is greater than or equal to the target scale of the target mask, the similarity criterion is determined. At that time, establish the connection edges of the 2D target mask. ,otherwise .

[0082] 3-3-3. Filtering using the maximum clique mining algorithm: The correct mask recognition problem is abstracted into the maximum clique problem in graph theory, and a backtracking algorithm is used to solve the problem in the undirected, unweighted graph. The largest group The maximum clique obtained by solving is the refined mask. , Expressed as:

[0083]

[0084] Furthermore, step (4) is as follows:

[0085] To achieve targeted segmentation of the query target (while excluding other irrelevant objects in the scene), this invention innovatively employs single-dimensional mask encoding for 3D Gaussian allocation, specifically including:

[0086] 4-1. Targeted segmentation: Perform 3D Gaussian encoding only on the 3D target corresponding to the implicit query text.

[0087] 4-2 Differentiable Rendering: Directly generating target segmentation masks using differentiable rasterization technology. The specific expression is as follows:

[0088]

[0089] in, The feature encoding represents the i-th Gaussian feature. Indicates the number of 3D Gaussian distributions. This represents the opacity property of the i-th Gaussian. This represents the opacity property of the j-th Gaussian.

[0090] 4-3. To optimize one-dimensional mask coding, a composite loss function is constructed that integrates binary cross-entropy (BCE) and Dice coefficient (DICE). :

[0091]

[0092] Refining the mask Under the supervision of [the system], unidimensional mask encoding can converge efficiently, and the final output 3D target mask achieves semantic-level alignment with the implicit query text.

[0093] 4-4. The 3D Gaussian field with one-dimensional mask encoding saved after training can display the target segmentation results according to the specified viewpoint.

[0094] Table 1 below shows the quantitative comparison results of the open-vocabulary 3D inference task on the 3D-OVS and LERF-Mask datasets. Segmentation quality is evaluated using the mIoU and mBIoU metrics. The results demonstrate that the Gaussian sputtering method for open-vocabulary 3D inference based on implicit text queries proposed in this paper effectively improves segmentation performance: it outperforms all state-of-the-art 3D segmentation methods in both mIoU and mBIoU metrics. The evaluation metrics used are mean intersection-over-union (mIoU) and mean boundary intersection-over-union (mBIoU). Figure 3 and 4As shown, on the 3D-OVS dataset, the method of this invention achieves an overall mIoU of 94.3% and an mBIoU of 87.9%, which is significantly better than the comparison methods: large model inference segmentation (mIoU of 83.3%, mBIoU of 77.6%), language embedding neural radiation field (mIoU of 37.2%, mBIoU of 17.4%), Gaussian grouping (mIoU of 69.8%, mBIoU of 66.7%), and 3D language Gaussian splashing (mIoU of 65.0%, mBIoU of 57.7%). On the LERF-Mask dataset, the method of this invention achieves an overall mIoU of 80.6% and an mBIoU of 76.5%, significantly outperforming the comparison methods: large model inference segmentation (mIoU 38.7%, mBIoU 35.5%), language embedding neural radiation field (mIoU 20.7%, mBIoU 15.1%), Gaussian grouping (mIoU 34.3%, mBIoU 30.3%), and 3D language Gaussian splashing (mIoU 23.8%, mBIoU 20.5%).

[0095] Table 1. Quantitative comparison results of open-vocabulary 3D reasoning on the 3D-OVS and LERF-Mask datasets.

[0096]

[0097] This result demonstrates that Language Embedded Neural Radiation Field (LERF) can directly render high-dimensional CLIP features, which can be used to correctly understand some implicit text. However, the segmentation boundaries generated by this method are not clear enough. 3D Language Gaussian Sputtering (LangSplat) uses an autoencoder to compress CLIP features. This compression process leads to some information loss in the rendered CLIP features, thereby reducing its ability to understand implicit text. Therefore, this decline in text understanding directly leads to inaccurate segmentation results. For the Gaussian Grouping method, the error originates from the open-vocabulary detector GroundingDINO, which provides inaccurate localization results for the input implicit text, ultimately leading to incorrect 3D open-vocabulary inference results. The open-vocabulary 3D inference Gaussian sputtering method based on implicit text queries proposed in this invention demonstrates the ability to accurately understand implicit query text and generate precise segmentation results in simple real-world scenarios, achieving better segmentation results.

[0098] Figure 3This paper presents the results of 3D open-vocabulary inference on the simple real-world dataset 3D-OVS. Language Embedding Neural Radiation Field (LERF) can directly render high-dimensional CLIP features and can be used to correctly understand some implicit text, but the segmentation boundaries it produces are not clear enough. 3D Language Gaussian Sputtering (LangSplat) uses an autoencoder to compress CLIP features, which leads to information loss in the rendered CLIP features, thus weakening its ability to understand implicit text. This degradation in text understanding directly results in inaccurate segmentation results. For Gaussian Grouping, the error stems from the inaccurate localization of the input implicit text by the open-vocabulary detector GroundingDINO, ultimately leading to incorrect 3D open-vocabulary inference results. The proposed open-vocabulary 3D inference Gaussian sputtering method (ReasonSplat) based on implicit text queries demonstrates the ability to accurately understand implicit query text and generate precise 3D segmentation results.

[0099] Figure 4 This paper presents the 3D open-vocabulary inference results on the LERF-Mask dataset. Compared to 3D-OVS, the scenes in the LERF-Mask dataset are more complex and better reflect real-world scenarios. The proposed Gaussian sputtering method for open-vocabulary 3D inference based on implicit text queries demonstrates powerful implicit inference capabilities, enabling it to provide accurate 3D segmentation masks even in complex real-world scenes. In contrast, the inference capabilities of contrasting methods drop sharply in complex scenes, leading to a significant degradation in the performance of the final 3D segmentation results.

Claims

1. An open-vocabulary 3D inference Gaussian sputtering method based on implicit text query, characterized in that, Includes the following steps: Step 1: Select a representative set of reference views ; Step 2: Use a visual language model to parse the implicit query text. and in representative view collections Generate target detection bounding boxes Use a segmentation model to detect bounding boxes for targets. Segmentation is performed to generate a refined 2D target mask. ; Step 3: Refine the 2D target mask to obtain a refined mask; 3-1. Map the 2D target mask to three-dimensional spatial coordinates using the camera pose matrix to obtain a set of three-dimensional coordinate points; 3-2. Calculate the standard deviation of each point in the three-dimensional coordinate point set, and use this standard deviation as the target scale. ; 3-3. Use the maximum clique mining algorithm to select the most likely correct 2D target mask from the representative views; 3-3-1. Constructing a similarity graph: Define an undirected, unweighted graph. Vertex set The set of edges representing 2D target masks. This represents the set of similarities between 2D target masks; 3-3-2. Determine the similarity criterion: When the target scale ratio of any two 2D target masks is greater than 1, the similarity criterion is determined. At that time, establish the connection edges of the 2D target mask. ,otherwise ; 3-3-3. Filtering using the maximum clique mining algorithm: The correct mask recognition problem is abstracted into the maximum clique problem in graph theory, and a backtracking algorithm is used to solve the problem in the undirected, unweighted graph. The largest group The maximum clique obtained by solving is the refined mask. ; Step 4: Embed the refined mask into a 3D Gaussian to achieve a 3D target mask aligned with the implicit query text; guide the optimization process of the single-dimensional mask encoding through the optimized refined mask, and finally achieve a 3D target mask that precisely matches the implicit query text.

2. The open-vocabulary 3D inference Gaussian sputtering method based on implicit text query according to claim 1, characterized in that, The specific implementation of step (1) is as follows: First, a hierarchical clustering algorithm based on the camera position in the world coordinate system is used to obtain representative views from the camera's perspective; Secondly, the mean of camera positions within each cluster is calculated, and the camera view closest to the mean position is selected as the representative view of that cluster, ultimately forming a representative view set. .

3. The open-vocabulary 3D inference Gaussian sputtering method based on implicit text query according to claim 1 or 2, characterized in that, The specific implementation of step (2) is as follows: Using a visual language model to parse implicit query text and in representative view collections Generate target detection bounding boxes Implicit query text and representative view collection As input to the model, the model maps natural language descriptions to visual space coordinates through deep semantic understanding, achieving coarse target localization under open vocabulary conditions. In obtaining the target detection bounding box Then, a refined 2D target mask is generated using the segmentation model. The bounding box for object detection As a prompt for input segmentation model, a representative set of views is also included. Input the segmentation model; the segmentation model, through its powerful zero-sample segmentation capability, outputs a refined 2D target mask that strictly corresponds to the target region in a representative view.

4. The open-vocabulary 3D inference Gaussian sputtering method based on implicit text query according to claim 3, characterized in that, Step (4) specifically includes: 4-1. Targeted segmentation: Only perform 3D Gaussian encoding on the 3D target corresponding to the implicit query text; 4-2 Differentiable Rendering: Directly generating target segmentation masks using differentiable rasterization technology. ; 4-3. Construct a composite loss function that integrates binary cross-entropy and the Dice coefficient. In refining the mask Under the supervision of [the relevant authority], the single-dimensional mask encoding is optimized, and the final output three-dimensional target mask achieves semantic-level alignment with the implicit query text. 4-4. The 3D Gaussian field with one-dimensional mask encoding saved after training can display the target segmentation results according to the specified viewpoint.

5. An open-vocabulary 3D inference Gaussian sputtering system based on implicit text query, characterized in that: include: Dataset module: Selects a representative set of reference views; 2D target mask acquisition module: It uses a visual language model to parse implicit query text and generates target detection bounding boxes on a representative view set. It then uses a segmentation model to segment the target detection bounding boxes and generate a refined 2D target mask. The refined mask acquisition module refines the 2D target mask by mapping it to 3D spatial coordinates using the camera pose matrix, resulting in a set of 3D coordinate points. It then calculates the standard deviation of each point in the 3D coordinate set and uses this standard deviation as the target scale. The most likely correct 2D target mask in the representative view is selected by the maximum clique mining algorithm to obtain the refined mask; 3D Target Mask Acquisition Module: Performs 3D Gaussian encoding only on the 3D targets corresponding to the implicit query text, and directly generates the target segmentation mask using differentiable rasterization technology. ; Construct a composite loss function that integrates binary cross-entropy and Dice coefficients. In refining the mask Under the supervision of [the system], the single-dimensional mask encoding is optimized, and the final output three-dimensional target mask achieves semantic-level alignment with the implicit query text.

6. An open-vocabulary 3D inference Gaussian sputtering device based on implicit text query, characterized in that, The device is equipped with the method described in claim 1.

Images