Interactive point cloud instance segmentation method based on three-dimensional Gaussian scattering field

Abstract

The invention provides an interactive point cloud instance segmentation method based on a three-dimensional Gaussian scattering field, and the method comprises the steps: receiving click prompt data of a user, carrying out the calculation of interaction points and weights based on the click prompt data, and obtaining Gaussian features containing interaction prompt information; screening and converting the Gaussian features in the interaction point neighborhood to obtain point cloud batch data; performing foreground and background prediction on the point cloud batch data by using a point cloud segmentation network to obtain a three-dimensional instance label; performing projection mapping on the three-dimensional instance label through a Gaussian grating to obtain a two-dimensional segmentation mask; and performing morphological post-processing on the two-dimensional segmentation mask to obtain a segmentation result. According to the method, the cross-view consistency and the boundary precision of the point cloud segmentation result of the three-dimensional Gaussian scattering field instance are effectively improved, and the segmentation time efficiency is accelerated.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular, to an interactive point cloud instance segmentation method based on a three-dimensional Gaussian scattering field. Background Art

[0002] With the rapid development of neural rendering and three-dimensional reconstruction technologies, the three-dimensional Gaussian scattering field, as an efficient, explicit, and editable three-dimensional scene representation method, has been widely applied in intelligent scene modeling fields such as augmented reality, virtual reality, robot navigation, and three-dimensional digital twins. The three-dimensional Gaussian scattering field explicitly models a large number of Gaussian basis elements in three-dimensional space and combines camera projection to achieve differentiable rasterization rendering, which can realize real-time novel view synthesis while maintaining the image detail quality, and has advantages such as high rendering efficiency, clear structure expression, and controllable representation granularity.

[0003] Although the three-dimensional Gaussian scattering field performs excellently in three-dimensional scene modeling, its research in the direction of three-dimensional semantic segmentation and instance annotation is still in its infancy, especially in aspects such as supporting user interaction, maintaining cross-view consistency, and accurate segmentation boundaries, there are still obvious deficiencies. Currently, the mainstream three-dimensional semantic assignment methods are mostly based on two-dimensional image models, such as generating segmentation masks using the Segment Anything Model (SAM), and mapping two-dimensional labels to three-dimensional space through methods such as feature alignment, label distillation, or projection matching. Although these methods can indirectly achieve three-dimensional label propagation, they have problems such as accuracy depending on two-dimensional segmentation effects, insufficient three-dimensional structure consistency, and blurred boundaries. At the same time, they require multiple stages of processing and external model support, resulting in high system complexity and low deployment and interaction response efficiency.

[0004] In addition, some segmentation methods for three-dimensional Gaussian scattering field scenarios, such as Feature3DGS, Omni-Seg3D, SAGA, etc., attempt to improve the semantic quality in the Gaussian representation through mechanisms such as multi-view Figure 1 consistency optimization, context fusion, and contrast learning. However, such methods usually require additional training, modification, or feature distillation of the original three-dimensional Gaussian scattering field model, do not have a plug-and-play lightweight interaction ability, and are difficult to adapt to the diverse large-scale scene annotation and editing requirements. Summary of the Invention

[0005] In view of this, the present invention provides an interactive point cloud instance segmentation method based on a three-dimensional Gaussian scattering field to solve the above problems.

[0006] The present invention provides an interactive point cloud instance segmentation method based on a three-dimensional Gaussian scattering field, including: receiving click prompt data of a user, calculating interaction points and weights based on the click prompt data to obtain Gaussian features containing interaction prompt information; screening and converting the Gaussian features within the neighborhood of the interaction points to obtain point cloud batch data; using a point cloud segmentation network to perform foreground and background prediction on the point cloud batch data to obtain three-dimensional instance labels; performing projection mapping on the three-dimensional instance labels through a Gaussian raster to obtain a two-dimensional segmentation mask; and performing morphological post-processing on the two-dimensional segmentation mask to obtain a segmentation result.

[0007] In another implementation manner of the present invention, the receiving click prompt data of the user, calculating interaction points and weights based on the click prompt data to obtain Gaussian features containing interaction prompt information includes: receiving click prompt data of the user in an arbitrary perspective image; obtaining two-dimensional coordinates corresponding to the click prompt data, and back-projecting the two-dimensional coordinates into a three-dimensional space ray according to camera parameters; using a Gaussian transparency accumulation model to calculate a depth position at which the three-dimensional space ray first reaches a threshold when penetrating a three-dimensional Gaussian scattering field point cloud scene, which is defined as an interaction point; calculating the Euclidean distance between the interaction point and the center of each Gaussian basis element; using a Gaussian function to calculate the spatial attenuation weight of each Gaussian basis element, and fusing the spatial attenuation weight with the original attributes to obtain Gaussian features containing interaction prompt information.

[0008] In another implementation manner of the present invention, the screening and converting the Gaussian features within the neighborhood of the interaction points to obtain point cloud batch data includes: constructing a three-dimensional neighborhood range centered on the interaction point based on a preset radius and a preset height; screening Gaussian elements located within the three-dimensional space range from the Gaussian features to obtain Gaussian elements within the region of interest; converting the Gaussian elements within the region of interest into a point cloud set represented in a three-dimensional point cloud format; and when the number of points in the point cloud exceeds a set threshold, dividing the point cloud set to obtain point cloud batch data.

[0009] In another implementation manner of the present invention, the using a point cloud segmentation network to perform foreground and background prediction on the point cloud batch data to obtain three-dimensional instance labels includes: inputting the point cloud batch data into the point cloud segmentation network; performing foreground and background classification on each point cloud in the point cloud batch data and obtaining corresponding foreground probabilities and background probabilities; and determining the three-dimensional instance labels of each point according to the comparison result between the foreground probability and the background probability.

[0010] In another implementation of the present invention, the projection mapping of the three-dimensional instance label by the Gaussian grating to obtain a two-dimensional segmentation mask includes: projecting the three-dimensional instance label onto the image plane according to the camera parameters of the target perspective; performing pixel assignment on the foreground Gaussian elements within the perspective range that meet the preset conditions based on the Gaussian rasterization rendering rule; and generating a two-dimensional segmentation mask consistent with the current perspective.

[0011] In another implementation of the present invention, the morphological post-processing of the two-dimensional segmentation mask to obtain a segmentation result includes: performing a closing operation on the two-dimensional segmentation mask to remove boundary cracks and small-area holes; performing edge-aware filling and maximum connected region screening on the unsegmented regions inside the two-dimensional segmentation mask after the closing operation, and retaining the main target region to obtain the segmentation result.

[0012] On the other hand, the present invention provides an interactive point cloud instance segmentation system based on a three-dimensional Gaussian scattering field, including: a calculation module: receiving click prompt data from a user, calculating interaction points and weights based on the click prompt data to obtain Gaussian features containing interaction prompt information; a screening module: screening and converting the Gaussian features within the neighborhood of the interaction points to obtain point cloud batch data; a prediction module: using a point cloud segmentation network to perform foreground and background prediction on the point cloud batch data to obtain three-dimensional instance labels; a mapping module: performing projection mapping on the three-dimensional instance labels by a Gaussian grating to obtain a two-dimensional segmentation mask; a post-processing module: performing morphological post-processing on the two-dimensional segmentation mask to obtain a segmentation result.

[0013] On the other hand, the present invention provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the computer program, the steps of an interactive point cloud instance segmentation method based on a three-dimensional Gaussian scattering field as described in any one of the above are implemented.

[0014] On the other hand, the present invention provides a computer storage medium, characterized in that a computer program is stored on the computer storage medium, and when the computer program is executed by a processor, the steps in an interactive point cloud instance segmentation method based on a three-dimensional Gaussian scattering field as described in any one of the above are implemented.

[0015] The interactive point cloud instance segmentation method based on a three-dimensional Gaussian scattering field of the present invention does not need to rely on a two-dimensional image model and can directly act on a three-dimensional Gaussian scattering field; by fusing spatial weight reconstruction modeling and point cloud-level semantic prediction, the cross-perspective consistency and boundary accuracy of the three-dimensional Gaussian scattering field instance point cloud segmentation result are effectively improved, and the segmentation time efficiency is accelerated. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required in the description of the embodiments or the prior art. By reading the detailed description of the following embodiments, the advantages and benefits in the solutions become clear to those skilled in the art. The accompanying drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present invention. In the accompanying drawings:

[0017] Figure 1 Schematic diagram of the process of an interactive point cloud instance segmentation method based on a three-dimensional Gaussian scattering field according to an embodiment of the present invention.

[0018] Figure 2 Schematic diagram of point cloud instance segmentation in a Gaussian scene according to an embodiment of the present invention and a schematic diagram of efficiency comparison with the existing method.

[0019] Figure 3 Schematic diagram of comparison of multi-view instance segmentation results of different methods in an urban street scene according to an embodiment of the present invention.

[0020] Figure 4 Schematic diagram of the point cloud instance segmentation results of multiple targets in three-dimensional space according to an embodiment of the present invention and a schematic diagram of its detail magnification.

[0021] Figure 5 Schematic diagram of the effect of the post-processing module according to an embodiment of the present invention. Detailed implementation manners

[0022] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the following will clearly and detailedly describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art shall fall within the scope of protection of the embodiments of the present invention.

[0023] Figure 1 Schematic diagram of a process of an interactive point cloud instance segmentation method based on a three-dimensional Gaussian scattering field provided by an embodiment of the present invention. As Figure 1 shown, this embodiment mainly includes:

[0024] S101. Receive the click prompt data of the user, calculate the interaction points and weights based on the click prompt data, and obtain the Gaussian features containing interaction prompt information.

[0025] S102. Screen and transform the Gaussian features within the neighborhood of the interaction points to obtain point cloud batch data.

[0026] S103. Use the point cloud segmentation network to predict the foreground and background of the batched point cloud data to obtain 3D instance labels.

[0027] S104. Perform projection mapping on the 3D instance labels through a Gaussian grating to obtain a 2D segmentation mask.

[0028] S105. Perform morphological post-processing on the 2D segmentation mask to obtain a segmentation result.

[0029] The interactive point cloud instance segmentation method based on a 3D Gaussian scattering field of the present invention does not need to rely on a 2D image model and can directly act on the 3D Gaussian scattering field; by fusing the spatial weight reconstruction model and point cloud-level semantic prediction, it effectively improves the cross-view consistency and boundary accuracy of the 3D Gaussian scattering field instance point cloud segmentation result and speeds up the segmentation time efficiency.

[0030] In another implementation manner of the present invention, the receiving of the click prompt data of the user and the calculation of the interaction points and weights based on the click prompt data to obtain Gaussian features including interaction prompt information include: receiving the click prompt data of the user in an arbitrary perspective image; obtaining the 2D coordinates corresponding to the click prompt data, and back-projecting the 2D coordinates into a 3D space ray according to the camera parameters; using the Gaussian transparency accumulation model to calculate the depth position of the first threshold reached when the 3D space ray penetrates the 3D Gaussian scattering field point cloud scene, which is defined as the interaction point; calculating the Euclidean distance between the interaction point and the center of each Gaussian basis element; using the Gaussian function to calculate the spatial attenuation weight of each Gaussian basis element, and fusing the spatial attenuation weight with the original attributes to obtain Gaussian features including interaction prompt information.

[0031] Exemplarily, the click coordinates (u, v) of the user received on the screen are converted into a 3D ray through inverse projection transformation, parameterized as:

[0032] r(t) = o + t·d, t ≥ 0

[0033] where o is the camera origin and d is the ray direction (unit vector).

[0034] For each Gaussian basis element G i , the mean μ i covariance Σ i , the opacity α i , calculate the interval where the ray intersects it satisfying:

[0035] (r(t) - μ i ) T Σ i -1 (r(t) - μ i ) ≤ τ 2

[0036] Among them, τ is the threshold (such as 3σ).

[0037] Collect all the intersecting Gaussians and sort them in ascending order to obtain the sequence G ={G sq |i = 1, …, N}. i |i = 1, …, N}.

[0038] Initialize the cumulative transmittance T = 1.0 and process the sorted Gaussians in sequence:

[0039] T ← T·(1 - α k )

[0040] Terminate when T < 0.1, record the current Gaussian's as the depth t stop , and define the position of the interaction point as p click :

[0041] p click = r(t stop ) = o + t stop d

[0042] For each Gaussian basis element G i , its spatial center μ i , calculate the Euclidean distance d i :

[0043] d i = ∥μ i - p click ∥

[0044] Construct a spatial attenuation weight map according to the distance, and define the weight w i of each Gaussian using a Gaussian function:

[0045]

[0046] Among them, σ represents the control parameter of spatial attenuation.

[0047] Take w i as an additional channel and splice it with the original Gaussian basis element g i to obtain a new Gaussian basis element:

[0048]

[0049] In another implementation manner of the present invention, screening and transforming the Gaussian features in the neighborhood of the interaction point to obtain point cloud batch data includes: centering on the interaction point, constructing a three-dimensional neighborhood range based on a preset radius and a preset height; screening Gaussian elements located within the three-dimensional space range from the Gaussian features to obtain Gaussian elements within the region of interest; converting the Gaussian elements within the region of interest into a point cloud set represented in a three-dimensional point cloud format; when the number of points in the point cloud exceeds a set threshold, dividing the point cloud set to obtain point cloud batch data.

[0050] Exemplarily, a cylindrical neighborhood centered on the interaction point with a clipping radius of r and a height of h set according to the approximate size to be segmented as needed is constructed:

[0051]

[0052] where (x i , y i , z i ) is the center of the Gaussian basis element.

[0053] The Gaussian set G that meets the above conditions c is converted into a point cloud set P c , and each point contains position and fusion attribute information.

[0054] Let the maximum number of points for single-batch processing threshold be N max = 8192. If the number of points in the point cloud set P c satisfies |P c | > 8192, then P c is divided into sub-batches:

[0055]

[0056] In another implementation manner of the present invention, using a point cloud segmentation network to perform foreground and background prediction on the point cloud batch data to obtain three-dimensional instance labels includes: inputting the point cloud batch data into the point cloud segmentation network; classifying each point cloud in the point cloud batch data into foreground and background, and obtaining corresponding foreground probabilities and background probabilities; determining the three-dimensional instance labels of each point according to the comparison result of the foreground probability and the background probability.

[0057] Exemplarily, each sub-batch of point cloud is input into the point cloud structure segmentation network, using the PointTransformerV3 (PTV3) architecture to extract local and global features. Obtain the foreground probability and the background probability of each point, and generate three-dimensional labels by comparing the probability magnitudes:

[0058]

[0059] In another implementation of the present invention, the three-dimensional instance label is projected and mapped by the Gaussian grating to obtain a two-dimensional segmentation mask, including: projecting the three-dimensional instance label onto the image plane according to the camera parameters of the target view; performing pixel assignment on the foreground Gaussian elements within the view range that meet the preset conditions based on the Gaussian rasterization rendering rule; generating a two-dimensional segmentation mask consistent with the current view.

[0060] Exemplarily, according to the camera projection parameters, the Gaussian basis elements labeled as foreground are mapped from three-dimensional space to the pixels of the image plane. For each pixel point (u, v), the Gaussian projection point set within its neighborhood is analyzed If there is a Gaussian that meets the following conditions, the pixel point is assigned the label c corresponding to this Gaussian k :

[0061]

[0062] (x k , y k ) represents the central value coordinates of the Gaussian basis element after being mapped to the image plane, and τ is an empirical parameter that can be set to 4.

[0063] For all target view images, the above steps are sequentially executed to generate a set of two-dimensional instance segmentation masks with cross-view consistency.

[0064] In another implementation of the present invention, the morphological post-processing of the two-dimensional segmentation mask to obtain the segmentation result includes: performing a closing operation on the two-dimensional segmentation mask to remove boundary cracks and small-area holes; performing edge-aware filling and maximum connected region screening on the unsegmented regions inside the two-dimensional segmentation mask after the closing operation, and retaining the main target region to obtain the segmentation result.

[0065] Exemplarily, as Figure 5 shown, apply a morphological closing operation to the initially generated two-dimensional instance segmentation mask to fill the edge cracks and eliminate small-scale hole regions. Perform hole filling processing in combination with edge information to enhance the internal connectivity of the target region. Perform connected component analysis on the processed mask image, and screen the maximum connected region as the final effective instance region to output a structurally complete two-dimensional segmentation result.

[0066] As Figure 2 , Figure 3 , Figure 4 shown, it is a comparison diagram of instance segmentation of the method of the present invention and the method in the prior art in different scenarios. It can be seen that compared with the existing multi-stage label distillation scheme, the structure of the present invention is lightweight, the interaction is efficient, it has good adaptability and real-time performance, and is applicable to various scenarios such as 3D modeling, augmented reality, and spatial intelligent perception.

[0067] The present invention does not need to rely on a two-dimensional segmentation model and a multi-stage distillation process, and can directly act on three-dimensional Gaussian basis elements. Without changing the structure of the original three-dimensional scene model, it directly performs instance-level semantic annotation on the Gaussian basis elements, fully utilizes the explicit geometric features of the Gaussian basis elements, and realizes a three-dimensional instance segmentation method with efficient interaction, consistent structure, and clear labels, so as to enhance the practical value of the Gaussian scene in three-dimensional understanding, interactive modeling, and intelligent perception.

[0068] On the other hand, the present invention provides an interactive point cloud instance segmentation system based on a three-dimensional Gaussian scattering field, including:

[0069] Computing module: Receiving click prompt data of a user, calculating interaction points and weights based on the click prompt data, and obtaining Gaussian features containing interaction prompt information.

[0070] Filtering module: Filtering and transforming the Gaussian features within the neighborhood of the interaction points to obtain point cloud batch data.

[0071] Prediction module: Using a point cloud segmentation network to perform foreground and background prediction on the point cloud batch data to obtain three-dimensional instance labels.

[0072] Mapping module: Performing projection mapping on the three-dimensional instance labels through a Gaussian grating to obtain a two-dimensional segmentation mask.

[0073] Post-processing module: Performing morphological post-processing on the two-dimensional segmentation mask to obtain a segmentation result.

[0074] The interactive point cloud instance segmentation system based on a three-dimensional Gaussian scattering field of the present invention does not need to rely on a two-dimensional image model and can directly act on a three-dimensional Gaussian scattering field; by fusing spatial weight reconstruction modeling and point cloud-level semantic prediction, it effectively improves the cross-view consistency and boundary accuracy of the three-dimensional Gaussian scattering field instance point cloud segmentation result, and speeds up the segmentation time efficiency.

[0075] On the other hand, an electronic device includes: a processor, a memory, and a communication bus and a communication interface.

[0076] Wherein:

[0077] The processor, the memory, and the communication interface complete communication with each other through the communication bus.

[0078] The communication interface is used for communicating with other electronic devices or servers.

[0079] A processor for executing a program, specifically capable of executing the steps of any one of the above-described interactive point cloud instance segmentation methods based on a three-dimensional Gaussian scattering field in the foregoing embodiments.

[0080] Specifically, the program may include program code, and the program code includes computer operation instructions.

[0081] The processor may be a central processing unit (CPU), or a specific integrated circuit (ASIC, Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application. One or more processors included in the intelligent device may be of the same type of processor, such as one or more CPUs; or may be of different types of processors, such as one or more CPUs and one or more ASICs.

[0082] A memory for storing the program. The memory may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory.

[0083] The program is specifically configured to cause the processor to execute to implement the steps of any one of the interactive point cloud instance segmentation methods based on a three-dimensional Gaussian scattering field described in the embodiments. For the specific implementation of each step in the program, reference may be made to the corresponding descriptions in the steps and units of any one of the above-described interactive point cloud instance segmentation methods based on a three-dimensional Gaussian scattering field, and details are not described herein again. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described devices and modules may refer to the corresponding process descriptions in the foregoing method embodiments.

[0084] An exemplary embodiment of the present application further provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the methods of the embodiments of the present application.

[0085] The method according to the embodiments of the present invention can be implemented in hardware, firmware, or can be implemented as software or computer code that can be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or can be implemented as computer code originally stored in a remote recording medium or a non-transitory machine-readable medium and downloaded through a network and to be stored in a local recording medium, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component (such as RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the method shown herein, the execution of the code converts the general-purpose computer into a dedicated computer for executing the method shown herein.

[0086] So far, specific embodiments of the present invention have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require the particular order or sequential order shown to achieve the desired result.

[0087] It should be noted that all directional indications (such as up, down, left, right, back...) in the embodiments of the present invention are only used to explain the relative positional relationship between components in a certain specific order (as shown in the drawings), and if this specific order changes, then the directional indications will change accordingly.

[0088] In the description of the present invention, the terms "first" and "second" are only used for convenience in describing different components or names, and cannot be understood as indicating or implying an order relationship, relative importance, or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of such features.

[0089] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs. The terms used in the description of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0090] It should be noted that although the specific embodiments of the present invention have been described in detail with reference to the accompanying drawings, it should not be construed as a limitation on the protection scope of the present invention. Within the scope described in the claims, various modifications and deformations that can be made by those skilled in the art without creative efforts still fall within the protection scope of the present invention.

[0091] The examples of the embodiments of the present invention are intended to briefly illustrate the technical features of the embodiments of the present invention, so that those skilled in the art can intuitively understand the technical features of the embodiments of the present invention, and do not serve as an improper limitation on the embodiments of the present invention.

[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An interactive point cloud instance segmentation method based on a three-dimensional Gaussian scattering field, characterized in that It includes: Receiving the click prompt data of the user, calculating the interaction points and weights based on the click prompt data, and obtaining Gaussian features containing interaction prompt information; Screening and transforming the Gaussian features within the neighborhood of the interaction points to obtain point cloud batch data; Using a point cloud segmentation network to perform foreground and background prediction on the point cloud batch data to obtain three-dimensional instance labels; Performing projection mapping on the three-dimensional instance labels through a Gaussian grating to obtain a two-dimensional segmentation mask; Performing morphological post-processing on the two-dimensional segmentation mask to obtain a segmentation result.

2. The method according to claim 1, characterized in that The receiving the click prompt data of the user, calculating the interaction points and weights based on the click prompt data, and obtaining Gaussian features containing interaction prompt information includes: Receiving the click prompt data of the user in an arbitrary perspective image; Obtaining the two-dimensional coordinates corresponding to the click prompt data, and back-projecting the two-dimensional coordinates into a three-dimensional space ray according to the camera parameters; Using a Gaussian transparency accumulation model to calculate the depth position where the three-dimensional space ray first reaches the threshold when penetrating a three-dimensional Gaussian scattering field point cloud scene, which is defined as the interaction point; Calculating the Euclidean distance between the interaction point and the centers of each Gaussian basis element; Using a Gaussian function to calculate the spatial attenuation weights of each Gaussian basis element, and fusing the spatial attenuation weights with the original attributes to obtain Gaussian features containing interaction prompt information.

3. The method according to claim 1, characterized in that The screening and transforming the Gaussian features within the neighborhood of the interaction points to obtain point cloud batch data includes: Constructing a three-dimensional neighborhood range centered on the interaction point based on a preset radius and a preset height; Screening the Gaussian elements located within the three-dimensional space range from the Gaussian features to obtain the Gaussian elements within the region of interest; Converting the Gaussian elements within the region of interest into a point cloud set represented in a three-dimensional point cloud format; When the number of point clouds exceeds a set threshold, dividing the point cloud set to obtain point cloud batch data.

4. The method according to claim 1, wherein The using a point cloud segmentation network to perform foreground and background prediction on the point cloud batch data to obtain three-dimensional instance labels includes: Inputting the point cloud batch data into a point cloud segmentation network; Performing foreground and background classification on each point cloud in the point cloud batch data, and obtaining the corresponding foreground probability and background probability; Determining the three-dimensional instance labels of each point according to the comparison result of the foreground probability and the background probability.

5. The method according to claim 4, wherein The performing projection mapping on the three-dimensional instance labels through a Gaussian grating to obtain a two-dimensional segmentation mask includes: Projecting the three-dimensional instance labels onto the image plane according to the camera parameters of the target perspective; Performing pixel assignment on the foreground Gaussian elements that meet the preset conditions within the perspective range based on the Gaussian rasterization rendering rule; Generating a two-dimensional segmentation mask consistent with the current perspective.

6. The method according to claim 5, wherein The performing morphological post-processing on the two-dimensional segmentation mask to obtain a segmentation result includes: Performing a closing operation on the two-dimensional segmentation mask to remove boundary cracks and small area holes; Performing edge-aware filling and maximum connected region screening on the unsegmented regions inside the two-dimensional segmentation mask after the closing operation, and retaining the main target region to obtain a segmentation result.

7. An interactive point cloud instance segmentation system based on a three-dimensional Gaussian scattering field, characterized in that, It includes: Calculation module: Receives the click prompt data of the user, calculates the interaction points and weights based on the click prompt data, and obtains Gaussian features containing interaction prompt information; Filtering module: Filters and transforms the Gaussian features within the neighborhood of the interaction points to obtain point cloud batch data; Prediction module: Uses a point cloud segmentation network to predict the foreground and background of the point cloud batch data to obtain three-dimensional instance labels; Mapping module: Performs projection mapping on the three-dimensional instance labels through a Gaussian raster to obtain a two-dimensional segmentation mask; Post-processing module: Performs morphological post-processing on the two-dimensional segmentation mask to obtain a segmentation result.

8. An electronic device, characterized in that, Including: A memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the steps of an interactive point cloud instance segmentation method according to any one of claims 1 to 6.

9. A computer storage medium, characterized in that, A computer program is stored on the computer storage medium. When the computer program is executed by a processor, it implements the steps in an interactive point cloud instance segmentation method according to any one of claims 1 to 6.

Images