Instance segmentation methods, apparatus, devices, storage media, and program products

By determining the foreground point in the 3D Gaussian rendering map and using a sparse connection strategy to process the segmentation results, the problem of complex viewpoint selection and inconsistent segmentation in the prior art is solved, and efficient and accurate instance segmentation is achieved.

CN123118985APending Publication Date: 2026-11-17SHANDONG SENTER ELECTRONICS
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202510619371.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-05-14
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

The instance segmentation method based on 3D Gaussian splashing technology has complex model training and viewpoint selection steps, resulting in low segmentation efficiency and inconsistent results, making it difficult to adapt to large-scale scene segmentation.

Method used

By identifying multiple foreground points as cue points in the target rendering map and combining them with a sparse connection strategy to process the 3D segmentation results, the viewpoint selection is simplified and the segmentation accuracy and consistency are improved.

Benefits of technology

It improves the efficiency and accuracy of instance segmentation, is suitable for real-time applications in large-scale scenarios, reduces the differences in segmentation results caused by different perspectives, and ensures the integrity and accuracy of segmentation results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123118985A_ABST
    Figure CN123118985A_ABST
Patent Text Reader

Abstract

This application provides an instance segmentation method, apparatus, device, storage medium, and program product, relating to the field of 3D image processing technology. The method includes: performing 3D Gaussian splashing on 3D Gaussian data of a target scene and determining a target rendering image; determining multiple foreground points of target instances in the target rendering image as cue points of the target instances; segmenting the rendering image based on the cue points of the target instances to obtain a target segmentation mask, and obtaining a first 3D segmentation result of the target instances based on the target segmentation mask; and processing each 3D point in the first 3D segmentation result using a sparse connection strategy to obtain a second 3D segmentation result of the target instances. This application provides a technical solution for instance segmentation in large-scale scenarios such as substations.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of three-dimensional image processing technology, and in particular to an instance segmentation method, apparatus, device, storage medium, and program product. Background Technology

[0002] 3D Gaussian Splatting (3DGS) is a lightweight and high-precision scene representation method that uses tens of thousands of 3D Gaussian primitives. Each primitive is modeled using parameters such as position, covariance, opacity, and spherical harmonics. 3D Gaussian Splatting provides a new technical standard for 3D reconstruction and new perspective generation.

[0003] Among them, the 3D Gaussian instance segmentation method based on 3D Gaussian splashing technology identifies the scene object to which each 3D Gaussian primitive belongs by adding instance information. In this way, all 3D Gaussian primitives belonging to the same object can be segmented to form a whole object with classification attributes.

[0004] Currently, 3D Gaussian instance segmentation methods based on 3D Gaussian splashing technology involve a large number of complex steps such as model training, viewpoint selection, and object encoding, which affect the segmentation efficiency of instance segmentation. Furthermore, different viewpoints may lead to significant differences in segmentation results, increasing the complexity and uncertainty of the processing and making them unsuitable for large-scale scene segmentation. Summary of the Invention

[0005] This application provides an instance segmentation method, apparatus, device, storage medium, and program product, which provides a technical solution for instance segmentation applicable to large-scale scenarios such as substations.

[0006] Firstly, this application provides an instance segmentation method, the method comprising:

[0007] Perform 3D Gaussian splashing on the 3D Gaussian data of the target scene and determine the target rendering map;

[0008] Multiple foreground points of the target instance in the target rendering image are identified as cue points of the target instance;

[0009] Based on the cue points of the target instance, the rendered image is segmented to obtain a target segmentation mask, and based on the target segmentation mask, a first 3D segmentation result of the target instance is obtained;

[0010] A sparse connection strategy is used to process each 3D point in the first 3D segmentation result to obtain the second 3D segmentation result of the target instance.

[0011] In one optional implementation, the step of performing 3D Gaussian splashing on the 3D Gaussian data of the target scene and determining the target rendering image includes:

[0012] In the interactive interface, 3D Gaussian splashing is performed on the 3D Gaussian data of the target scene to obtain multiple rendering images from multiple perspectives.

[0013] Based on the target instance, a target rendering image is determined from the plurality of rendering images.

[0014] In one optional implementation, segmenting the rendered image based on the cue points of the target instance to obtain a target segmentation mask includes:

[0015] Based on the cue points of the target instance, the rendered image is segmented and inferred using a fine-tuned segmentation model;

[0016] Based on the results of the segmentation inference, the target segmentation mask is determined.

[0017] In one optional implementation, determining the target segmentation mask based on the result of the segmentation inference includes:

[0018] Based on the results of the segmentation inference, including multiple segmentation masks, the optimal segmentation mask is determined and used as the target segmentation mask.

[0019] In one optional implementation, the fine-tuned segmentation model is a model obtained by fine-tuning the segmentation base model using the target dataset.

[0020] The target dataset includes an image dataset used to characterize the target scene, and the image data in the target dataset is image data that has undergone data standardization preprocessing.

[0021] In one alternative implementation, the cue points include key points and / or feature points of the target instance, the key points and / or feature points being used to describe the shape and / or boundaries of the target instance.

[0022] In one optional implementation, and based on a target segmentation mask, a first 3D segmentation result of the target instance is obtained, the method further includes:

[0023] Obtain the contour coordinates of the target instance in the target segmentation mask;

[0024] The contour coordinates of the target instance are transmitted to the interactive interface, and in the interactive interface, a specific point in the contour coordinates of the target instance is identified and marked as a foreground seed point.

[0025] Based on the foreground seed points, the target segmentation mask is projected into 3D space to obtain the first 3D segmentation result of the target instance.

[0026] In one alternative implementation, before transmitting the outline of the target instance to the interactive interface, the method further includes:

[0027] Based on the contour coordinates of the target instance, a cutting pattern of the target instance is determined in the rendered image;

[0028] The category of the target instance is obtained by reasoning from the segmentation map of the target instance using a fine-tuned target detection model.

[0029] In one optional implementation, transmitting the contour coordinates of the target instance to the interactive interface, and identifying and labeling specific points in the contour coordinates of the target instance as foreground seed points in the interactive interface includes:

[0030] The outline coordinates of the target instance and the category of the target instance are transmitted to the interactive interface;

[0031] In the interactive interface, based on the category of the target instance, specific points in the contour coordinates of the target instance are identified and labeled as foreground seed points.

[0032] In one optional implementation, the fine-tuned target detection model is a model obtained by fine-tuning the target detection base model using a target dataset;

[0033] The target dataset includes an image dataset used to characterize the target scene, and the image data in the target dataset is image data that has undergone data standardization preprocessing.

[0034] In one optional implementation, the step of employing a sparse connection strategy to process each 3D point in the first 3D segmentation result to obtain a second 3D segmentation result for the target instance includes:

[0035] Each 3D point in the first 3D segmentation result is taken as a node in the graph. Multiple connection edges are constructed for each node using a sparse connection strategy, and the comprehensive weight of the connection edges is calculated.

[0036] Based on the comprehensive weights of the connecting edges, a graph segmentation algorithm is used to perform graph segmentation, resulting in a second 3D segmentation result for the target instance.

[0037] In an optional implementation, the step of treating each 3D point in the first 3D segmentation result as a node in a graph, constructing multiple connection edges for each node using a sparse connection strategy, and calculating the comprehensive weight of the connection edges includes:

[0038] Each 3D point in the first 3D segmentation result is taken as a node in the graph, and the attribute information of the node includes spatial coordinates and normal vector;

[0039] A sparse connection strategy is used to construct multiple connection edges for each node, and the comprehensive weight of each connection edge is calculated. The comprehensive weight includes the product of the spatial term weight and the normal vector term weight.

[0040] In one optional implementation, based on the comprehensive weights of the connecting edges, a graph segmentation algorithm is used to perform graph segmentation to obtain a second 3D segmentation result for the target instance, including:

[0041] Based on the combined weight of the connecting edges, and under the constraint of minimizing the total weight of the cut edges, a graph cutting algorithm is used to perform graph segmentation, resulting in the second 3D segmentation result of the target instance.

[0042] Secondly, this application provides an instance segmentation method for a substation scenario, the method comprising:

[0043] Perform 3D Gaussian splashing on the 3D Gaussian data of multiple devices in a substation scenario, and obtain a rendered image from the target viewpoint;

[0044] Multiple foreground points of the target instance in the target rendering image are identified as cue points of the target instance;

[0045] Based on the cue points of the target instance, the rendered image is segmented to obtain a target segmentation mask;

[0046] The target segmentation mask is projected into 3D space to obtain the first 3D segmentation result of the target instance;

[0047] A sparse connection strategy is used to process each 3D point in the first 3D segmentation result to obtain the second 3D segmentation result of the target instance.

[0048] In one optional implementation, segmenting the rendered image based on the cue points of the target instance to obtain a target segmentation mask includes:

[0049] Based on the cue points of the target instance, the rendered image is segmented using a fine-tuned segmentation model; the fine-tuned segmentation model is a model obtained by fine-tuning the segmentation base model using the target dataset; wherein, the target dataset includes an image dataset of the substation scene, and the image data in the target dataset is image data that has undergone data standardization preprocessing.

[0050] Based on the results of the segmentation inference, the target segmentation mask is determined.

[0051] Thirdly, this application provides an instance segmentation apparatus, the apparatus comprising: a first acquisition module, a first determination module, and a first segmentation module;

[0052] The first acquisition module is used to perform 3D Gaussian splashing on the 3D Gaussian data of multiple objects in the target scene and obtain a rendered image from the target viewpoint;

[0053] The first determining module is used to determine multiple foreground points of the target instance in the target rendering image as cue points of the target instance;

[0054] The first segmentation module is used for:

[0055] Based on the cue points of the target instance, the rendered image is segmented to obtain a target segmentation mask;

[0056] The target segmentation mask is projected into 3D space to obtain the first 3D segmentation result of the target instance;

[0057] A sparse connection strategy is used to process each 3D point in the first 3D segmentation result to obtain the second 3D segmentation result of the target instance.

[0058] Fourthly, this application provides an instance segmentation device for a substation scenario, the device comprising: a second acquisition module, a second determination module, and a second segmentation module;

[0059] The second acquisition module is used to perform 3D Gaussian splashing on the 3D Gaussian data of multiple devices in the substation scene and obtain a rendering image from the target viewpoint;

[0060] The second determining module is used to determine multiple foreground points of the target instance in the target rendering image as cue points of the target instance;

[0061] The second segmentation module is used for:

[0062] Based on the cue points of the target instance, the rendered image is segmented to obtain a target segmentation mask;

[0063] The target segmentation mask is projected into 3D space to obtain the first 3D segmentation result of the target instance;

[0064] A sparse connection strategy is used to process each 3D point in the first 3D segmentation result to obtain the second 3D segmentation result of the target instance.

[0065] Fifthly, this application provides an electronic device, comprising: at least one processor and a memory;

[0066] The memory stores computer-executed instructions;

[0067] The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the method as described in either the first or second aspect.

[0068] In a sixth aspect, this application provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the method described in either the first or second aspect.

[0069] In a seventh aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method described in either the first or second aspect.

[0070] This application provides an instance segmentation method, apparatus, device, storage medium, and program product. The instance segmentation method includes: performing 3D Gaussian splashing on 3D Gaussian data of multiple objects in a target scene and obtaining a target rendering image; determining multiple foreground points of target instances in the target rendering image as cue points of the target instances; and segmenting the rendering image based on the cue points of the target instances to obtain a target segmentation mask. This target segmentation mask is used to locate the target instance region, and since it is determined based on the cue points of the target instances, it can achieve accurate location of the instance region. Then, based on the target segmentation mask, a first 3D segmentation result of the target instance is obtained; and a sparse connection strategy is used to process each 3D point corresponding to the target instance in the first 3D segmentation result to obtain a second 3D segmentation result of the target instance. Based on this, the embodiments of this application regard each 3D point corresponding to the target instance in the first 3D segmentation result as a node in the graph. By adopting a sparse connection strategy, the 3D points of the target instance in the first 3D segmentation result are used as foreground seed points. The graph is divided into two subsets: one containing foreground seed points and the other excluding foreground points, thereby achieving the purpose of segmenting the 3D Gaussian model.

[0071] Based on the above description, this application embodiment directly performs 3D Gaussian splashing on multiple objects in the target scene and obtains the target rendering image, reducing complex viewpoint selection and object encoding steps. This not only improves the efficiency of instance segmentation but also simplifies the viewpoint selection step, making the method more suitable for real-time applications and large-scale scene segmentation tasks. It also reduces the differences in segmentation results caused by different viewpoints, improving segmentation accuracy and consistency. Furthermore, by determining multiple foreground points of the target instance as cue points in the target rendering image, the target instance can be identified and segmented more accurately. Moreover, this application employs a sparse connection strategy to process each 3D point of the target instance in the first 3D segmentation result. This not only effectively handles complex structures in large-scale scenes but also ensures the integrity and accuracy of the segmentation results while maintaining computational efficiency. Attached Figure Description

[0072] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0073] Figure 1 A flowchart illustrating an instance segmentation method provided in this application embodiment. Figure 1 ;

[0074] Figure 2 A flowchart illustrating an instance segmentation method provided in this application embodiment. Figure 2 ;

[0075] Figure 3 A flowchart illustrating an instance segmentation method for a substation scenario provided in an embodiment of this application;

[0076] Figure 4 A schematic diagram illustrating an instance segmentation method for a substation scenario provided in an embodiment of this application;

[0077] Figure 5 This is a schematic diagram of the structure of an instance segmentation device provided in an embodiment of this application;

[0078] Figure 6 A schematic diagram of the structure of an instance segmentation device for a substation scenario provided in an embodiment of this application;

[0079] Figure 7 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application.

[0080] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0081] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0082] In this document, it should be understood that the terminology used is for convenience of understanding only and does not imply any limitation on its meaning. Furthermore, any number of elements in the accompanying drawings is for illustrative purposes only and not for limitation, and any naming is for distinction only and has no limiting meaning.

[0083] The data involved in this application may be data authorized by the user or fully authorized by all parties, and the collection, dissemination, and use of the data shall comply with the requirements of relevant national laws and regulations.

[0084] 3D Gaussian Splatting (3DGS) is a lightweight and high-precision scene representation method that uses tens of thousands of 3D Gaussian primitives. Each primitive is modeled using parameters such as position, covariance, opacity, and spherical harmonics. 3D Gaussian Splatting provides a new technical standard for 3D reconstruction and new perspective generation.

[0085] Among them, the 3D Gaussian instance segmentation method based on 3D Gaussian splashing technology identifies the scene object to which each 3D Gaussian primitive belongs by adding instance information. In this way, all 3D Gaussian primitives belonging to the same object can be segmented to form a whole object with classification attributes.

[0086] Currently, 3D Gaussian instance segmentation methods based on 3D Gaussian splashing technology involve a large number of complex steps such as model training, viewpoint selection, and object encoding, which affect the segmentation efficiency of instance segmentation. Furthermore, different viewpoints may lead to significant differences in segmentation results, increasing the complexity and uncertainty of the processing and making them unsuitable for large-scale scene segmentation.

[0087] Based on this, the technical concept of this application is as follows: A target rendering image is determined from multiple rendering images generated by 3D Gaussian splashing to reduce complex viewpoint selection and object encoding steps. Multiple foreground points of the target instance are determined as cue points in the target rendering image to accurately identify and segment the target instance. A sparse connection strategy is employed to process each 3D point of the target instance in the first 3D segmentation result, enabling the embodiments of this application to effectively handle complex structures in large-scale scenes.

[0088] The technical solutions shown in this application will now be described in detail through specific embodiments. It should be noted that the following embodiments may exist independently or in combination with each other; for identical or similar content, the description will not be repeated in different embodiments.

[0089] Figure 1 A flowchart illustrating the steps of an instance segmentation method provided for an exemplary embodiment of this application. (Refer to...) Figure 1 The method includes the following steps:

[0090] S101 performs 3D Gaussian splashing on the 3D Gaussian data of the target scene and obtains the target rendering image.

[0091] Optionally, the target scenario can include urban streets, rural roads, or highways within an autonomous driving environment. In this target scenario, 3D Gaussian splashing technology can be used to segment vehicles, pedestrians, traffic signs, and other obstacles on the road to facilitate path planning and safe driving.

[0092] Optionally, the target scenario can also include building clusters, green spaces, parks, and road networks in architectural design and urban planning. In this target scenario, 3D Gaussian splashing technology can help planners perform building layout analysis, green space design, and traffic flow management.

[0093] Optionally, the target scenario can also include equipment layout in substations, transmission lines, and related infrastructure. In this target scenario, 3D Gaussian splashing technology can be used to achieve precise positioning and status monitoring of equipment, improving maintenance efficiency and safety.

[0094] It should be understood that the above target scenarios are merely examples and are not intended to limit the specific implementation of this application. Any other scenario in which 3D Gaussian data can be obtained can be used as a scenario in this application.

[0095] In this embodiment, the 3D Gaussian data originates from LiDAR, stereo cameras, depth sensors, or other 3D scanning devices. The 3D Gaussian data is represented by a set of Gaussian distribution parameters, each parameter set corresponding to an object or a portion of an object in the scene. Specifically, each Gaussian distribution may consist of a mean vector (representing the center position of the object), a covariance matrix (representing the shape and orientation of the object), and weights (representing the importance or probability of the object).

[0096] In data processing, 3D Gaussian data is used to represent the spatial distribution and geometric features of objects in a scene. By parametrically representing the Gaussian distribution, complex 3D data can be effectively simplified while preserving key spatial information. In instance segmentation, 3D Gaussian data is used to identify and distinguish different objects in a scene. By analyzing the parameters of the Gaussian distribution, object localization, classification, and segmentation can be achieved. Using 3D Gaussian data can significantly reduce data volume, improve computational efficiency, and enhance robustness to noise, making it suitable for processing complex objects and details in large-scale scenes.

[0097] It should be understood that in three-dimensional space, each object or part of an object can be represented by a Gaussian distribution. A Gaussian distribution describes the object's center position, shape, and orientation through a mean vector and a covariance matrix. 3D Gaussian splashing is the process of applying a Gaussian distribution to the entire scene. By superimposing multiple Gaussian distributions, a comprehensive probability density function can be formed, representing the spatial distribution of all objects in the scene. By projecting the result of 3D Gaussian splashing onto a two-dimensional plane, a rendering image can be generated. The rendering image preserves the spatial information and feature distribution of the objects, facilitating subsequent image processing and analysis. In the rendering image, different objects are represented by color, brightness, or other visual features.

[0098] Optionally, multiple perspectives (e.g., top view, side view, oblique view) can be selected to represent the target scene. The 3D Gaussian splashing result is projected onto a 2D plane from each perspective to generate a corresponding rendering image, resulting in multiple rendering images. The target rendering image is then determined from these multiple rendering images. The methods for determining the target rendering image can include the following:

[0099] Choose a high-quality rendering image with clear details, or, depending on the specific application requirements, choose a rendering image that best showcases key features or object relationships, or choose a rendering image that contains the most key information to ensure that important objects and features are not obscured or omitted.

[0100] In the embodiments of this application, the target rendering image can also be selected manually or by using image analysis algorithms and machine learning models. This application does not impose any special limitations on this.

[0101] For example, in an autonomous driving scenario, a target rendering image needs to be selected from multiple rendering images for real-time decision-making. The target rendering image can be selected based on the following criteria: the edges of vehicles and pedestrians in the image are clear, the viewpoint that can show the road ahead and potential obstacles is selected, and all key areas (such as intersections and traffic signals) are visible.

[0102] Based on the above description, the 3D Gaussian splashing technique transforms complex point cloud data into a Gaussian distributed parameterized representation, significantly reducing the amount of data, lowering computational complexity, and improving processing efficiency. It should be understood that the Gaussian distribution effectively captures the geometric features and spatial relationships of objects and exhibits strong robustness to noise.

[0103] Furthermore, 3D Gaussian splashing allows for more accurate representation of object boundaries and shapes, improving segmentation precision. Moreover, 3D Gaussian splashing technology supports generating renderings from multiple perspectives, providing comprehensive scene analysis capabilities and enabling more accurate identification and understanding of object relationships in complex scenes. Finally, Gaussian splashing technology can effectively handle complex structures in large-scale scenes, ensuring the integrity and accuracy of segmentation results, making it suitable for scenarios requiring the processing of large-scale data, such as autonomous driving, industrial inspection, and substations.

[0104] S102, determine multiple foreground points of the target instance in the target rendering image as cue points of the target instance.

[0105] Optionally, the target rendering image can be displayed in the interactive interface, and multiple data points in the area where the target instance is located can be selected as multiple foreground points, which can then be used as prompt points for the target instance.

[0106] Optionally, the target rendered image is loaded and displayed in the interactive interface. This interface supports basic operations such as zooming and panning, allowing users to view different parts of the image. In one instance, the interface may provide multiple selection tools (such as rectangular selection, free selection, and polygon selection) to allow users to flexibly select the target instance.

[0107] Next, the user marks target instances on the rendered image using selection tools; for example, the user can select the boundaries of an instance by clicking or dragging. Multiple data points are extracted from the user-selected target instance region. These data points can be based on the image's pixel coordinates or other features. By analyzing the density, color, and other features of the selected region, multiple foreground points are identified.

[0108] Optionally, the data (such as coordinates and feature values) of these prompt points can be output or stored for subsequent processing and analysis.

[0109] Based on this, the embodiments of this application allow users to directly participate in the target instance selection process through an interactive interface, improving the accuracy and flexibility of identification. Furthermore, this application allows users to manually select the target instance area, effectively avoiding background interference and improving the accuracy of foreground element identification.

[0110] It should be understood that the embodiments of this application can also analyze the target rendering image, such as edge detection, to identify the outline of the target instance, and then apply a feature extraction algorithm to identify key points in the rendering image as prompt points.

[0111] S103, based on the cue points of the target instance, the rendered image is segmented to obtain a target segmentation mask, and based on the target segmentation mask, a first 3D segmentation result of the target instance is obtained.

[0112] It should be understood that cue points for target instances are used to provide key locations for target instance segmentation. A segmentation mask, used to identify pixels of different regions or objects in an image, is a binary or multi-valued image of the same size as the original image, where the value of each pixel represents the pixel's category or label in the original image. Typically, a value of 0 in the segmentation mask represents the background, while non-zero values ​​represent the foreground or a specific object.

[0113] Segmentation masks can include: Binary masks: each pixel has only two possible values ​​(e.g., 0 and 1), used to distinguish foreground from background. Multi-class masks: each pixel can have multiple values, each value corresponding to a specific category or object; this mask is used for multi-class segmentation tasks. It should be understood that segmentation masks can accurately identify the boundaries of target objects, helping to improve the accuracy of object recognition and segmentation.

[0114] Optionally, a suitable image segmentation algorithm, such as GrabCut, or a deep learning-based segmentation network (such as Mask R-CNN), can be selected. User-marked cue points can be used as input to the segmentation algorithm to guide it to segment the target instance more accurately. After that, the segmentation algorithm outputs a segmentation mask of the target instance, in which the region of the target instance is marked.

[0115] Next, the target segmentation mask is projected into 3D space to obtain the first 3D segmentation result. For example, the target segmentation mask can be combined with the original 3D data (such as point clouds) and camera parameters (such as intrinsic and extrinsic parameters) to obtain the first 3D segmentation result.

[0116] S104, using a sparse connection strategy, each 3D point in the first 3D segmentation result is processed to obtain the second 3D segmentation result of the target instance.

[0117] Sparse connectivity is a common approach used when processing large-scale data (such as 3D point clouds). It aims to reduce computational complexity by decreasing the number of connections between data points, while maintaining or improving the accuracy and effectiveness of the processing results.

[0118] In this embodiment, the 3D point cloud data of the target instance is obtained from the first 3D segmentation result. The 3D point cloud data of the target instance represents the initially segmented target instance.

[0119] In this embodiment, a sparse connectivity strategy is used to define a neighborhood for the 3D point cloud data of the target instance, typically based on Euclidean distance or other similarity metrics. Then, this embodiment applies the sparse connectivity strategy to construct a sparse graph structure in the 3D point cloud data of the target instance. Each node in the sparse graph structure represents a 3D point in the 3D point cloud data of the target instance, and the edges of the sparse graph structure represent connections between 3D points. Optionally, a graph traversal algorithm (such as DFS or BFS) can be used to identify connected components in the sparse graph structure. Connected components represent possible sub-instances or refined segmentation regions. Then, depending on application requirements, the identified connected components are merged or retained to form the final second 3D segmentation result.

[0120] As described above, the sparse connection strategy significantly reduces computation and improves processing efficiency by limiting the number and range of connections. Furthermore, through sparse connections and connected component analysis, target instances can be identified and segmented more precisely, noise can be removed, and segmentation accuracy can be improved.

[0121] Based on the above description, this application embodiment directly performs 3D Gaussian splashing on multiple objects in the target scene and obtains the target rendering image, reducing complex viewpoint selection and object encoding steps. This not only improves the efficiency of instance segmentation but also simplifies the viewpoint selection step, making the method more suitable for real-time applications and large-scale scene segmentation tasks. It also reduces the differences in segmentation results caused by different viewpoints, improving segmentation accuracy and consistency. Furthermore, by determining multiple foreground points of the target instance as cue points in the target rendering image, the target instance can be identified and segmented more accurately. Moreover, this application employs a sparse connection strategy to process each 3D point of the target instance in the first 3D segmentation result, effectively handling complex structures in large-scale scenes and ensuring the integrity and accuracy of the segmentation results while maintaining computational efficiency.

[0122] Below, in Figure 1 Based on the illustrated embodiment, combined with Figure 2 The above instance segmentation method will be explained in detail.

[0123] Figure 2 A flowchart illustrating an instance segmentation method provided as an example of an embodiment of this application. Figure 2 Please see. Figure 2 The method may include:

[0124] S201, in the interactive interface, performs 3D Gaussian splashing on the 3D Gaussian data of the target scene to obtain multiple rendering images from multiple perspectives.

[0125] It should be understood that in the interactive interface, users can select or adjust different viewpoints to obtain rendered images from multiple perspectives. Optionally, the interactive interface can display key information, such as current viewpoint parameters and Gaussian distribution properties, to help users understand and analyze the data.

[0126] These multi-view renderings are used to provide users with 3D Gaussian data for analyzing target scenes from different perspectives.

[0127] S202, Based on the target instance, determine the target rendering image from the plurality of rendering images, that is, the target instance is included in the rendering image from the target's perspective.

[0128] Optionally, the target rendering image can be selected from the rendering image corresponding to the best viewpoint of the target instance. For example, a viewpoint optimization algorithm can be used to select the best viewpoint of the target instance.

[0129] Based on this, a better segmentation foundation can be provided for the subsequent segmentation of target instances.

[0130] S203, determine multiple foreground points of the target instance in the target rendering image as cue points of the target instance.

[0131] This step can be referenced. Figure 1 The relevant description of step S102 in the embodiments will not be repeated here.

[0132] Optionally, the prompt points include key points and / or feature points of the target instance, which are used to describe the shape and / or boundaries of the target instance.

[0133] The extraction of prompt points can be achieved by manual selection by the user or automatic acquisition by an algorithm, and this application embodiment does not impose any special limitations on this.

[0134] In some examples, detection algorithms (such as SIFT, SURF, ORB) are used to detect key points in the target rendering map. These points are usually located at salient feature locations of the target instance, such as headlights, wheels, etc.

[0135] In this embodiment, the shape of the target instance or the boundary of the target instance can be constructed based on these cue points, improving the accuracy and reliability of subsequent target instance segmentation. Furthermore, the cue points are applicable to complex scenes and multi-view data, and can handle the challenges posed by occlusion and changes in viewpoint. Therefore, in complex scenes, they can more effectively identify and distinguish similar objects.

[0136] S204, based on the cue points of the target instance, perform segmentation inference on the rendered image using a fine-tuned segmentation model.

[0137] In this step, a pre-trained segmentation base model is first selected. It should be understood that this model can be considered to have been trained on a large-scale dataset.

[0138] Optionally, the fine-tuned segmentation model is a model obtained by fine-tuning the segmentation base model using the target dataset; wherein, the target dataset includes an image dataset for characterizing the target scene, and the image data in the target dataset is image data that has undergone data standardization preprocessing.

[0139] The process of fine-tuning the segmentation basis model using the target dataset may include:

[0140] First, collect an image dataset related to the target scene. These images should include images of the target instance from multiple perspectives and under various conditions (such as different lighting and weather). Then, annotate the image data to generate a segmentation label (mask) for each image for supervised learning.

[0141] Optionally, embodiments of this application may further perform image data standardization preprocessing, including: adjusting the image to a uniform size to meet model input requirements; performing color standardization on the image (e.g., mean subtraction, standard deviation normalization) to reduce the impact of illumination variations; and applying data augmentation techniques (e.g., rotation, flipping, cropping, noise addition) to increase the diversity of the dataset and improve the model's generalization ability.

[0142] Optionally, in this embodiment of the application, a segmentation base model (such as Mask R-CNN or DeepLab) pre-trained on a large-scale dataset is selected as the object detection base model, and the base model architecture (such as the number of categories in the output layer) is adjusted according to the needs of the target task.

[0143] Subsequently, the segmentation base model is fine-tuned using the target dataset to better adapt the fine-tuned segmentation model to specific scenarios, resulting in more accurate and reliable segmentation results in specific application scenarios.

[0144] Optionally, the fine-tuning method in this application embodiment can be adjusted according to different target tasks and datasets to be applicable to various application scenarios and target instances.

[0145] Based on the above description, this application can effectively utilize the target dataset to fine-tune the segmentation base model, thereby improving the segmentation ability of the fine-tuned segmentation model in specific scenarios.

[0146] S205, Based on the results of the segmentation inference, determine the target segmentation mask.

[0147] The result of segmentation inference can be a probability map or a preliminary segmentation mask, representing the probability that each pixel belongs to the target instance. A suitable threshold is selected, and the probability map is converted into a binary mask. Pixels with probabilities greater than the threshold are marked as part of the target instance, and the rest are marked as background.

[0148] Afterwards, morphological operations (such as dilation, erosion, opening, and closing operations) can be applied to remove noise and small isolated regions, smooth target boundaries, and identify and retain the largest connected components to ensure the integrity of target instances and remove possible false detection regions.

[0149] The embodiments of this application can also use the aforementioned prompt points (key points and feature points) to constrain and adjust the segmentation mask to ensure that the segmentation mask accurately covers the key areas of the target instance.

[0150] Optionally, determining the target segmentation mask based on the result of the segmentation inference includes:

[0151] Based on the results of the segmentation inference, including multiple segmentation masks, the optimal segmentation mask is determined and used as the target segmentation mask.

[0152] The optimal segmentation mask needs to accurately cover the target instance and minimize missegmentation and omissions.

[0153] In this embodiment, the output of the fine-tuned segmentation model includes multiple segmentation masks, each of which corresponds to a possible target instance.

[0154] Optionally, the overlap between each segmentation mask and the target instance can be calculated to evaluate its accuracy, or the area and shape features of each segmentation mask can be analyzed to evaluate its accuracy. Then, the optimal segmentation mask can be determined based on the accuracy of multiple segmentation masks.

[0155] In an alternative example, if multiple segmentation masks are all accurate, they can be fused together to form the optimal segmentation mask.

[0156] Based on the above description, this embodiment uses the optimal segmentation mask as the target segmentation mask, which can improve the reliability of the segmentation results and make this embodiment adaptable to different scenarios and target instances.

[0157] S206, Based on the target segmentation mask, obtain the first 3D segmentation result of the target instance.

[0158] This target segmentation mask defines the pixel region of the target instance in the target rendering map.

[0159] In this embodiment of the application, 3D data corresponding to the target segmentation mask is obtained. The 3D data can be point cloud data obtained by LiDAR, stereo vision or structured light scanning. Then, the target segmentation mask is mapped into 3D space to obtain the first 3D segmentation result of the target instance.

[0160] Optionally, obtaining the first 3D segmentation result of the target instance based on the target segmentation mask includes the following steps:

[0161] Step 1: Obtain the contour coordinates of the target instance in the target segmentation mask.

[0162] In this embodiment, edge detection algorithms (such as Canny edge detection) or contour detection algorithms can be used to extract contour coordinates in the segmentation mask, wherein the contour coordinates consist of a series of point coordinates that define the boundary of the target instance.

[0163] In some examples, a smoothing algorithm can also be used to smooth the contour coordinates in the segmentation mask.

[0164] Next, the extracted contour coordinates are formatted into the desired output format (such as a list or array). Optionally, the contour coordinates can be output to the interactive interface or saved as a file for further analysis or processing.

[0165] Based on the above description, the contour coordinates of target instances can be effectively extracted from the segmentation mask, providing a reliable foundation for subsequent analysis and applications.

[0166] Step 2: Transmit the contour coordinates of the target instance to the interactive interface, and in the interactive interface, identify and mark specific points in the contour coordinates of the target instance as foreground seed points.

[0167] Optionally, the contour coordinates can be converted to a format suitable for transmission (such as JSON, XML, or other data formats) for parsing in the interactive interface. Then, based on the contour coordinates of the target instance, the contour of the target instance is drawn in the interactive interface using a graphics library (such as HTML5Canvas or OpenGL), ensuring that the display of the contour is consistent with the scale and position of the target rendered image.

[0168] In one instance, algorithms (such as centroid calculation and maximum curvature point detection) can be used to automatically select specific points on the contour as seed points, and the calibrated foreground seed points can be displayed in the interface with different colors or markers (such as circles and crosses).

[0169] In another instance, a specific point in the outline coordinates of the target instance can be selected as the foreground seed point by clicking or dragging in the interface.

[0170] Based on the above description, the embodiments of this application allow users to intuitively view and select specific points on the contour through an interactive interface. Furthermore, by calibrating foreground seed points, more accurate initial conditions can be provided for subsequent instance segmentation and analysis.

[0171] Optionally, before transmitting the outline of the target instance to the interactive interface, the method further includes:

[0172] First, based on the contour coordinates of the target instance, a cutout of the target instance is determined in the rendered image.

[0173] In this embodiment, the cropping image is the region of the target instance clipped from the rendered image using the contour coordinates of the target instance. Specifically, the region of the target instance can be determined by calculating the minimum bounding box of the contour. Based on the determined cropping region, the cropping image of the target instance is extracted from the rendered image. It is ensured that the cropped image retains sufficient background information to facilitate subsequent target segmentation.

[0174] Then, the fine-tuned target detection model is used to infer the segmentation map of the target instance to obtain the category of the target instance.

[0175] In this embodiment, the cutout image of the target instance is input into the target detection model. It is important to note that the input cutout image must meet the input requirements of the fine-tuned target detection model (e.g., size, color channels). Through inference, the category label and confidence score of the target instance are obtained. Optionally, the category with the highest confidence score is selected as the final category of the target instance.

[0176] In one optional implementation, the fine-tuned target detection model is a model obtained by fine-tuning the target detection base model using a target dataset;

[0177] The target dataset includes an image dataset used to characterize the target scene, and the image data in the target dataset is image data that has undergone data standardization preprocessing.

[0178] In this embodiment, firstly, an image dataset related to the target scene is collected as the target dataset. This image dataset contains image data of the target instance from multiple perspectives and under various conditions (such as different lighting and weather). Then, the image data is labeled to generate target detection labels (such as bounding boxes and category labels) for each image, which are then used for supervised learning.

[0179] Optionally, embodiments of this application may further include data standardization preprocessing of the target dataset: adjusting images to a uniform size to meet model input requirements; performing color standardization on images (e.g., mean subtraction, standard deviation normalization) to reduce the impact of illumination variations; and applying data augmentation techniques (e.g., rotation, flipping, cropping, noise addition) to increase the diversity of the dataset and improve the model's generalization ability.

[0180] Optionally, a pre-trained object detection base model (such as Faster R-CNN, YOLO, or SSD) on a large-scale dataset may be selected as the object detection base model in this application embodiment.

[0181] Subsequently, the segmentation base model is fine-tuned using the target dataset to better adapt the fine-tuned segmentation model to specific scenarios, resulting in more accurate and reliable segmentation results in specific application scenarios.

[0182] Optionally, transmitting the contour coordinates of the target instance to the interactive interface, and identifying and labeling specific points in the contour coordinates of the target instance as foreground seed points in the interactive interface, includes:

[0183] First, the outline coordinates of the target instance and the category of the target instance are transmitted to the interactive interface.

[0184] Optionally, before transmission, the outline coordinates and category information can be formatted into a suitable transmission format (such as JSON or XML) to facilitate parsing in the interactive interface. During transmission, the outline of the target instance can be drawn using graphics libraries (such as HTML5 Canvas or OpenGL) in the interactive interface, and the category information of the target instance can be displayed, which can be shown in the form of labels or icons.

[0185] Then, in the interactive interface, based on the category of the target instance, a specific point in the contour coordinates of the target instance is identified and labeled as a foreground seed point.

[0186] Optionally, based on the category of the target instance, a specific strategy can be applied to identify specific points on the contour. For example, for a vehicle, the front or rear of the vehicle might be selected as the foreground seed point; for a pedestrian, the head or feet might be selected as the foreground seed point.

[0187] In this embodiment, these specific points can be automatically selected and calibrated using algorithms (such as centroid calculation and maximum curvature point detection), or they can be manually selected and calibrated in an interactive manner.

[0188] In some examples, embodiments of this application may also provide a confirmation button or other interactive methods to allow the user to confirm the selected foreground seed point.

[0189] Based on the above description, the embodiments of this application can provide accurate initial conditions for subsequent instance segmentation by calibrating foreground seed points.

[0190] The third step is to project the target segmentation mask into 3D space based on the foreground seed points to obtain the first 3D segmentation result of the target instance.

[0191] In this embodiment, a foreground seed point is used as a reference to project a target segmentation mask onto 3D space to obtain a first 3D segmentation result. For example, the target segmentation mask is combined with the original 3D data (such as point cloud) and camera parameters (such as intrinsic and extrinsic parameters) to obtain the first 3D segmentation result.

[0192] It should be understood that the embodiments of this application can accurately reconstruct the 3D shape and position of the target instance by combining a 2D target segmentation mask and foreground seed points.

[0193] S207, take each 3D point in the first 3D segmentation result as a node in the graph, construct multiple connection edges for each node using a sparse connection strategy, and calculate the comprehensive weight of the connection edges.

[0194] It should be understood that in the fields of computer vision and graphics processing, a "graph" is a mathematical structure used to represent relationships between objects. A graph consists of nodes (also called vertices) and edges, where nodes represent objects and edges represent relationships or connections between objects. In 3D segmentation results, each 3D point is considered a node in the graph. Nodes typically contain attribute information associated with that point, such as spatial coordinates (x, y, z) and other features (such as normal vectors, color, etc.). Edges represent the connections between nodes. In a 3D point cloud graph, edges typically connect adjacent nodes. Edges can be directed or undirected, depending on the application scenario. Each edge can have a weight representing the strength or similarity of the relationship between nodes. Weights can be calculated based on factors such as geometric distance, attribute similarity, etc.

[0195] Therefore, the spatial and attribute relationships between 3D points can be intuitively represented through the graph structure.

[0196] In this embodiment, each point in the first 3D segmentation result is considered as a node in a graph. Each node contains its 3D coordinate information (x, y, z). Then, the neighboring nodes of each node can be constructed based on the k-nearest neighbors (k-NN) and the radius neighborhood, and then connecting edges can be constructed between each node and its neighboring nodes.

[0197] Optionally, the step of treating each 3D point in the first 3D segmentation result as a node in a graph, constructing multiple connection edges for each node using a sparse connection strategy, and calculating the comprehensive weight of the connection edges includes:

[0198] First, each 3D point in the first 3D segmentation result is taken as a node in the graph, wherein the attribute information of the node includes spatial coordinates and normal vector.

[0199] In this context, the spatial coordinates of each 3D point are typically represented as (x, y, z), corresponding to the point's position in 3D space. Spatial coordinates are a fundamental component of graph structures, used to calculate geometric relationships (such as distances) between nodes and to construct connecting edges.

[0200] A normal vector is a unit vector, represented as (nx, ny, nz), used to describe the orientation of a surface containing a 3D point. Normal vectors can be calculated using the local geometry of a point cloud. Common methods include: neighborhood plane fitting: fitting a plane within the local neighborhood of a point, with the normal vector being the normal to that plane; or surface normal estimation: estimating the direction of the normal on a surface. Normal vectors are used to capture surface features, helping to identify surface shape and orientation.

[0201] Then, a sparse connection strategy is used to construct multiple connection edges for each node, and the comprehensive weight of each connection edge is calculated. The comprehensive weight includes the product of the spatial term weight and the normal vector term weight.

[0202] In this embodiment, the algorithm of radius neighborhood and k-nearest neighbor is used to construct the connection edges between nodes and neighboring points to avoid computational explosion of fully connected nodes. During the implementation process, through multiple experiments and debugging, the neighborhood radius was set to 4 and the number of neighboring points was set to 30 to achieve the best segmentation result.

[0203] Optionally, the comprehensive weight of the edge can be calculated by incorporating Gaussian properties. The comprehensive weight is the product of the spatial term weight and the normal vector term weight.

[0204] The weights of the spatial terms are calculated as follows:

[0205]

[0206] In the formula, p i p j Let p be a node and its neighboring points in the graph. i -p j ‖ represents the Euclidean distance between a node and its neighboring nodes.

[0207] The farther the node is from its neighbors, the smaller its weight, and the decay rate increases from σ. s control.

[0208] The weights of the normal vector terms are calculated as follows:

[0209]

[0210] In the formula, n i nj Let the normal vectors of the nodes and their neighbors in the graph be...

[0211] The normal vector term measures the similarity of the angle between normal vectors; the smaller the angle, the higher the weight, and the decay rate is determined by σ. n control.

[0212] Overall weighting:

[0213] w(i,j)=w s (i,j)·w n (i,j)

[0214] In the embodiments of this application, the comprehensive weight takes into account both spatial and attribute similarity, so that points that are similar in both space and attributes are more likely to be retained in the same segmentation instance during segmentation.

[0215] S208, based on the comprehensive weight of the connecting edges, graph segmentation is performed using a graph cutting algorithm to obtain the second 3D segmentation result of the target instance.

[0216] It should be understood that in the fields of computer vision and graphics processing, graph cutting algorithms can be used to segment graph structures. By utilizing the combined weights of connected edges, graph cutting algorithms can effectively segment a graph into different parts, thereby achieving more refined 3D segmentation of the target instance.

[0217] Graph cutting algorithms include: Normalized Cut: Segmentation is performed by minimizing the ratio of the cutting cost to the internal connectivity of the subgraph. Graph Cuts: Segmentation is performed using the maximum flow minimum cut theory, suitable for binary segmentation. Spectral Clustering: Segmentation is performed using the Laplacian matrix of the graph, suitable for multi-class segmentation. Optionally, embodiments of this application may select a suitable graph cutting algorithm based on specific application requirements and graph characteristics.

[0218] Next, depending on the selected algorithm, necessary parameters (such as the number of clusters and the number of iterations) can be set, and the graph segmentation algorithm can be run to segment the graph. The algorithm outputs a segmentation label for each node, indicating the subgraph to which it belongs. The segmentation labels are mapped back to the 3D point cloud to generate a second 3D segmentation result for the target instance.

[0219] In the embodiments of this application, the graph segmentation algorithm can achieve fine segmentation of the target instance, identify different structures and parts, and improve the accuracy of segmentation and the ability to preserve details.

[0220] In one instance, based on the combined weights of the connecting edges, a graph segmentation algorithm is used to perform graph segmentation, resulting in a second 3D segmentation result for the target instance, including:

[0221] Based on the combined weight of the connecting edges, and under the constraint of minimizing the total weight of the cut edges, a graph cutting algorithm is used to perform graph segmentation, resulting in the second 3D segmentation result of the target instance.

[0222] In this embodiment, under the constraint of minimizing the sum of the comprehensive weights of the cut edges, the starting direction of path exploration is optimized by using the mask points generated by instance segmentation, and finally the graph is divided into two subsets: one containing foreground seed points and the other excluding foreground points, thus realizing the segmentation of the 3D Gaussian model.

[0223] Secondly, referring to Figure 3 This application also provides an instance segmentation method for a substation scenario.

[0224] It should be understood that in substation scenarios, equipment is densely distributed in space, the geometric structure is complex, and there are dynamic operation scenarios. Traditional segmentation methods based on two-dimensional images or point clouds often result in blurred instance boundaries due to severe occlusion, uneven lighting, and dynamic interference, making it difficult to meet the centimeter-level accuracy requirements of scenarios such as equipment status monitoring and automatic inspection by robotic arms.

[0225] The instance segmentation method for substation scenes provided in this application utilizes 3D Gaussian splash rendering, a fine-tuned segmentation model, a fine-tuned classification model, and a graph cut algorithm. By leveraging the multi-dimensional joint representation of the geometric features and material properties of equipment surfaces in substations, it effectively solves problems such as blurred edges of irregularly shaped equipment and adhesion of multiple targets, achieving interactive, high-precision, and real-time 3D instance segmentation.

[0226] Reference Figure 3 and Figure 4 The method includes:

[0227] S301 performs 3D Gaussian splashing on the 3D Gaussian data of the substation scene and determines the target rendering map.

[0228] Optionally, the interactive interface can perform splash rendering on the 3D Gaussian data of various devices in the substation scene, select any suitable viewpoint, and determine the rendering image under that viewpoint as the target rendering image.

[0229] In the substation scenario, 3D Gaussian data of various equipment can be obtained through laser scanning, photogrammetry or other 3D modeling techniques, including point cloud data or 3D model data.

[0230] Multiple perspectives (e.g., top view, side view, oblique view) can be selected to represent the substation scene. The 3D Gaussian splashing result is projected onto a 2D plane from each perspective to generate a corresponding rendering image, resulting in multiple rendering images. The target rendering image is then determined from these multiple rendering images. The methods for determining the target rendering image can include the following:

[0231] Choose a high-quality rendering image with clear details, or, based on the application requirements of the substation scenario, choose a rendering image corresponding to the perspective that best displays key features or object relationships, or choose a rendering image containing the most key information to ensure that target instances and features are not obscured or omitted.

[0232] In the embodiments of this application, the target rendering image can also be selected manually or by using image analysis algorithms and machine learning models. This application does not impose any special limitations on this.

[0233] S302, determine multiple foreground points of the target instance in the target rendering image as cue points of the target instance.

[0234] Optionally, select any number of foreground points on the target instance within the field of view of the target rendering map as cue points.

[0235] The number can be determined according to specific needs, and this application embodiment does not impose any special limitations on this.

[0236] For a description of this step, please refer to the description of step S102, which will not be repeated here.

[0237] S303, based on the cue points of the target instance, the rendered image is segmented to obtain a target segmentation mask, and based on the target segmentation mask, a first 3D segmentation result of the target instance is obtained.

[0238] In this embodiment, based on the cue points of the target instance, the fine-tuned segmentation model is invoked to perform segmentation inference on the rendered image, the highest quality segmentation mask in the inference results is obtained as the target segmentation mask, and an array is output to characterize the coordinate position of the target segmentation mask.

[0239] The segmentation mask is processed to obtain the coordinate points on the smooth boundary contour of the mask, and the coordinate points are stored in a list a.

[0240] Using coordinate points on the smooth boundary contour, the region is cropped in the rendered image to obtain the cutout map of the target instance. Then, a fine-tuned target detection model is invoked to infer the cutout map of the target instance and determine its category. It is worth noting that the description of the fine-tuned target detection model can be found in the previous embodiment and will not be repeated here.

[0241] Return the list of coordinate points 'a' and the instance category to the interactive interface. The points in the list of coordinate points 'a' are determined as foreground seed points. Project the segmentation mask onto the 3D space to obtain the first 3D segmentation result.

[0242] S304, using a sparse connection strategy, each 3D point in the first 3D segmentation result is processed to obtain the second 3D segmentation result of the target instance.

[0243] Sparse connectivity is a common approach used when processing large-scale data (such as 3D point clouds). It aims to reduce computational complexity by decreasing the number of connections between data points, while maintaining or improving the accuracy and effectiveness of the processing results.

[0244] In this embodiment, the 3D point cloud data of the target instance is obtained from the first 3D segmentation result. The 3D point cloud data of the target instance represents the initially segmented target instance.

[0245] Then, each 3D point in the first 3D segmentation result is regarded as a node in the graph. A sparse connection strategy is used to construct connecting edges, and the comprehensive weight of the edges is calculated. Under the constraint of minimizing the total weight of the cut edges, the graph is divided into two subsets: one containing the foreground seed point and the other excluding the foreground seed point. Finally, the instance segmentation of the 3D Gaussian model is achieved, and the second 3D segmentation result is obtained.

[0246] This application transforms the point cloud segmentation problem into a graph minimum cut problem. It utilizes the cueing effect of the segmentation mask generated under two-dimensional instance segmentation to guide the minimum cut algorithm to prioritize preserving the complete topological structure of instances, as detailed below:

[0247] The segmentation mask of the entity in the substation scenario is projected onto 3D space to obtain the first 3D segmentation result in the absence of depth information.

[0248] Then, each point in the first 3D segmentation result is regarded as a node in the graph. The node attributes include spatial coordinates, normal vectors, etc. The edge between the starting point and the point is constructed using the radius neighborhood and k-nearest neighbor algorithms to avoid computational explosion of fully connected components. During the implementation process, through multiple experiments and debugging, the neighborhood radius was set to 4 and the number of nearest neighbors was set to 30 to achieve the best segmentation results.

[0249] The comprehensive weight of the edge is calculated by incorporating Gaussian properties. The comprehensive weight is the product of the spatial term weight and the normal vector term weight. The spatial term weight is calculated as follows:

[0250]

[0251] In the formula, p i p j Let p be two points in the diagram. i -p j ‖ represents the Euclidean distance between two points.

[0252] The greater the distance between two points, the smaller the weight, and the decay rate increases from σ. s control.

[0253] The weights of the normal vector terms are calculated as follows:

[0254]

[0255] In the formula, n i n j Let be the normal vectors of two points in the graph. The normal vector term measures the similarity of the angle between the normal vectors; the smaller the angle, the higher the weight. The decay rate is determined by σ. n control.

[0256] Overall weighting:

[0257] w(i,j)=w s (i,j)·w n (i,j)

[0258] Based on the above, the comprehensive weight of this application takes into account both spatial and attribute similarity, so that points that are similar in both space and attributes are more likely to be retained in the same segment during segmentation.

[0259] Under the constraint of minimizing the sum of the comprehensive weights of the cut edges, the starting direction of path exploration is optimized by using the mask points generated by instance segmentation. Finally, the graph is divided into two subsets: one containing foreground seed points and the other excluding foreground points, thus achieving the segmentation of the target instance.

[0260] Optionally, the step of segmenting the rendered image based on the cue points of the target instance to obtain a target segmentation mask includes:

[0261] Based on the cue points of the target instance, a fine-tuned segmentation model is used to perform segmentation inference on the rendered image. The fine-tuned segmentation model is a model obtained by fine-tuning the segmentation base model using the target dataset. The target dataset includes an image dataset of the substation scene, and the image data in the target dataset is image data that has undergone data standardization preprocessing. A description of the fine-tuned segmentation model can be found in the previous embodiment and will not be repeated here.

[0262] In this embodiment, the feature values ​​of the image annotation dataset obtained in the substation scenario are preprocessed by data standardization. The standardized dataset is then divided into a training set and a test set in an 8:2 ratio for fine-tuning and testing of the segmentation model.

[0263] Segmentation Model Original SAM Original SAM2 SAM2 after fine-tuning Acc 0.64 0.70 0.82

[0264] On the same dataset, the above segmentation models were tested, and the evaluation metrics (Acc) are shown in the table above. The SAM2 fine-tuned model with the highest Acc evaluation metric was selected as the final segmentation model. SAM (Segment Anything Model) is an image segmentation model.

[0265] Based on the results of the segmentation inference, the target segmentation mask is determined as follows:

[0266] Based on the results of the segmentation inference, including multiple segmentation masks, the optimal segmentation mask is determined and used as the target segmentation mask.

[0267] In one optional implementation, the fine-tuned segmentation model is a model obtained by fine-tuning the segmentation base model using the target dataset.

[0268] The target dataset includes an image dataset used to characterize the substation scenario, and the image data in the target dataset is image data that has undergone data standardization preprocessing.

[0269] In one alternative implementation, the cue points include key points and / or feature points of the target instance, the key points and / or feature points being used to describe the shape and / or boundaries of the target instance.

[0270] In one optional implementation, and based on a target segmentation mask, a first 3D segmentation result of the target instance is obtained, the method further includes:

[0271] Obtain the contour coordinates of the target instance in the target segmentation mask;

[0272] The contour coordinates of the target instance are transmitted to the interactive interface, and in the interactive interface, a specific point in the contour coordinates of the target instance is identified and marked as a foreground seed point.

[0273] Based on the foreground seed points, the target segmentation mask is projected into 3D space to obtain the first 3D segmentation result of the target instance.

[0274] In one alternative implementation, before transmitting the outline of the target instance to the interactive interface, the method further includes:

[0275] Based on the contour coordinates of the target instance, a cutting pattern of the target instance is determined in the rendered image;

[0276] The category of the target instance is obtained by reasoning from the segmentation map of the target instance using a fine-tuned target detection model.

[0277] In one optional implementation, transmitting the contour coordinates of the target instance to the interactive interface, and identifying and labeling specific points in the contour coordinates of the target instance as foreground seed points in the interactive interface includes:

[0278] The outline coordinates of the target instance and the category of the target instance are transmitted to the interactive interface;

[0279] In the interactive interface, based on the category of the target instance, specific points in the contour coordinates of the target instance are identified and labeled as foreground seed points.

[0280] In one optional implementation, the fine-tuned target detection model is a model obtained by fine-tuning the target detection base model using a target dataset;

[0281] The target dataset includes an image dataset used to characterize the substation scenario, and the image data in the target dataset is image data that has undergone data standardization preprocessing.

[0282] In one optional implementation, the step of employing a sparse connection strategy to process each 3D point in the first 3D segmentation result to obtain a second 3D segmentation result for the target instance includes:

[0283] Each 3D point in the first 3D segmentation result is taken as a node in the graph. Multiple connection edges are constructed for each node using a sparse connection strategy, and the comprehensive weight of the connection edges is calculated.

[0284] Based on the comprehensive weights of the connecting edges, a graph segmentation algorithm is used to perform graph segmentation, resulting in a second 3D segmentation result for the target instance.

[0285] In an optional implementation, the step of treating each 3D point in the first 3D segmentation result as a node in a graph, constructing multiple connection edges for each node using a sparse connection strategy, and calculating the comprehensive weight of the connection edges includes:

[0286] Each 3D point in the first 3D segmentation result is taken as a node in the graph, and the attribute information of the node includes spatial coordinates and normal vector;

[0287] A sparse connection strategy is used to construct multiple connection edges for each node, and the comprehensive weight of each connection edge is calculated. The comprehensive weight includes the product of the spatial term weight and the normal vector term weight.

[0288] In one optional implementation, based on the comprehensive weights of the connecting edges, a graph segmentation algorithm is used to perform graph segmentation to obtain a second 3D segmentation result for the target instance, including:

[0289] Based on the combined weight of the connecting edges, and under the constraint of minimizing the total weight of the cut edges, a graph cutting algorithm is used to perform graph segmentation, resulting in the second 3D segmentation result of the target instance.

[0290] Based on the above description, the embodiments of this application combine the minimum cut algorithm with instance segmentation and object detection, and utilize the collaborative work of multiple models to achieve efficient segmentation and classification of instances in 3D scenes through dynamic interactive methods. The segmentation and classification accuracy in complex substation scenes reaches 82%, and the 3D Gaussian model instance segmentation time is less than 200ms.

[0291] This method improves interactivity and guidance, and balances accuracy and efficiency through algorithm fusion and human-computer interaction, achieving 3D scene Gaussian model instance segmentation in large-scale scenarios.

[0292] Reference Figure 5 This application provides an instance segmentation device 500, which includes: a first acquisition module 501, a first determination module 502, and a first segmentation module 503;

[0293] The first acquisition module 501 is used to perform 3D Gaussian splashing on the 3D Gaussian data of multiple objects in the target scene and acquire the rendered image from the target viewpoint;

[0294] The first determining module 502 is used to determine multiple foreground points of the target instance in the target rendering image as cue points of the target instance;

[0295] The first segmentation module 503 is used for:

[0296] Based on the cue points of the target instance, the rendered image is segmented to obtain a target segmentation mask, and the target segmentation mask is projected into 3D space to obtain the first 3D segmentation result of the target instance;

[0297] A sparse connection strategy is used to process each 3D point in the first 3D segmentation result to obtain the second 3D segmentation result of the target instance.

[0298] In one optional implementation, the first acquisition module 501 is specifically used to: perform 3D Gaussian splashing on the 3D Gaussian data of the target scene in the interactive interface to obtain multiple rendering images from multiple perspectives.

[0299] Based on the target instance, determine the rendering image from the target's perspective from multiple rendering images.

[0300] In one optional implementation, the first segmentation module 503 is specifically used for:

[0301] Based on the cue points of the target instance, the rendered image is segmented and inferred using a fine-tuned segmentation model;

[0302] Based on the results of the segmentation inference, the target segmentation mask is determined.

[0303] In one optional implementation, the first segmentation module 503 is specifically used for:

[0304] Based on the results of the segmentation inference, including multiple segmentation masks, the optimal segmentation mask is determined and used as the target segmentation mask.

[0305] In one optional implementation, the fine-tuned segmentation model is a model obtained by fine-tuning the segmentation base model using the target dataset.

[0306] The target dataset includes an image dataset used to characterize the target scene, and the image data in the target dataset is image data that has undergone data standardization preprocessing.

[0307] In one alternative implementation, the cue points include key points and / or feature points of the target instance, the key points and / or feature points being used to describe the shape and / or boundaries of the target instance.

[0308] In one optional implementation, the first segmentation module 503 is specifically used for:

[0309] Obtain the contour coordinates of the target instance in the target segmentation mask;

[0310] The contour coordinates of the target instance are transmitted to the interactive interface, and in the interactive interface, a specific point in the contour coordinates of the target instance is identified and marked as a foreground seed point.

[0311] Based on the foreground seed points, the target segmentation mask is projected into 3D space to obtain the first 3D segmentation result of the target instance.

[0312] In an optional implementation, the first segmentation module 503 is further configured to:

[0313] Based on the contour coordinates of the target instance, a cutting pattern of the target instance is determined in the rendered image;

[0314] The category of the target instance is obtained by reasoning from the segmentation map of the target instance using a fine-tuned target detection model.

[0315] In one optional implementation, the first segmentation module 503 is specifically used for:

[0316] The outline coordinates of the target instance and the category of the target instance are transmitted to the interactive interface;

[0317] In the interactive interface, based on the category of the target instance, specific points in the contour coordinates of the target instance are identified and labeled as foreground seed points.

[0318] In one optional implementation, the fine-tuned target detection model is a model obtained by fine-tuning the target detection base model using a target dataset;

[0319] The target dataset includes an image dataset used to characterize the target scene, and the image data in the target dataset is image data that has undergone data standardization preprocessing.

[0320] In one optional implementation, the first segmentation module 503 is specifically used for:

[0321] Each 3D point in the first 3D segmentation result is taken as a node in the graph. Multiple connection edges are constructed for each node using a sparse connection strategy, and the comprehensive weight of the connection edges is calculated.

[0322] Based on the comprehensive weights of the connecting edges, a graph segmentation algorithm is used to perform graph segmentation, resulting in a second 3D segmentation result for the target instance.

[0323] In one optional implementation, the first segmentation module 503 is specifically used for:

[0324] Each 3D point in the first 3D segmentation result is taken as a node in the graph, and the attribute information of the node includes spatial coordinates and normal vector;

[0325] A sparse connection strategy is used to construct multiple connection edges for each node, and the comprehensive weight of each connection edge is calculated. The comprehensive weight includes the product of the spatial term weight and the normal vector term weight.

[0326] In one optional implementation, the first segmentation module 503 is specifically used for:

[0327] Based on the combined weight of the connecting edges, and under the constraint of minimizing the total weight of the cut edges, a graph cutting algorithm is used to perform graph segmentation, resulting in the second 3D segmentation result of the target instance.

[0328] Reference Figure 6 This application provides an instance segmentation device 600 for a substation scenario. The device 600 includes: a second acquisition module 601, a second determination module 602, and a second segmentation module 603.

[0329] The second acquisition module 601 is used to perform 3D Gaussian splashing on the 3D Gaussian data of multiple objects in the substation scene and obtain a rendering image from the target viewpoint;

[0330] The second determining module 602 is used to determine multiple foreground points of the target instance in the target rendering image as cue points of the target instance;

[0331] The second segmentation module 603 is used for:

[0332] Based on the cue points of the target instance, the rendered image is segmented to obtain a target segmentation mask, and the target segmentation mask is projected into 3D space to obtain the first 3D segmentation result of the target instance;

[0333] A sparse connection strategy is used to process each 3D point in the first 3D segmentation result to obtain the second 3D segmentation result of the target instance.

[0334] In one optional implementation, the second acquisition module 601 is specifically used to: perform 3D Gaussian splashing on the 3D Gaussian data of the substation scene in the interactive interface to obtain multiple rendering images from multiple perspectives.

[0335] Based on the target instance, determine the rendering image from the target's perspective from multiple rendering images.

[0336] In one optional implementation, the second segmentation module 603 is specifically used for:

[0337] Based on the cue points of the target instance, the rendered image is segmented and inferred using a fine-tuned segmentation model;

[0338] Based on the results of the segmentation inference, the target segmentation mask is determined.

[0339] In one optional implementation, the second segmentation module 603 is specifically used for:

[0340] Based on the results of the segmentation inference, including multiple segmentation masks, the optimal segmentation mask is determined and used as the target segmentation mask.

[0341] In one optional implementation, the fine-tuned segmentation model is a model obtained by fine-tuning the segmentation base model using the target dataset.

[0342] The target dataset includes an image dataset used to characterize the substation scenario, and the image data in the target dataset is image data that has undergone data standardization preprocessing.

[0343] In one alternative implementation, the cue points include key points and / or feature points of the target instance, the key points and / or feature points being used to describe the shape and / or boundaries of the target instance.

[0344] In one optional implementation, the second segmentation module 603 is specifically used for:

[0345] Obtain the contour coordinates of the target instance in the target segmentation mask;

[0346] The contour coordinates of the target instance are transmitted to the interactive interface, and in the interactive interface, a specific point in the contour coordinates of the target instance is identified and marked as a foreground seed point.

[0347] Based on the foreground seed points, the target segmentation mask is projected into 3D space to obtain the first 3D segmentation result of the target instance.

[0348] In an optional implementation, the second segmentation module 603 is further configured to:

[0349] Based on the contour coordinates of the target instance, a cutting pattern of the target instance is determined in the rendered image;

[0350] The category of the target instance is obtained by reasoning from the segmentation map of the target instance using a fine-tuned target detection model.

[0351] In one optional implementation, the second segmentation module 603 is specifically used for:

[0352] The outline coordinates of the target instance and the category of the target instance are transmitted to the interactive interface;

[0353] In the interactive interface, based on the category of the target instance, specific points in the contour coordinates of the target instance are identified and labeled as foreground seed points.

[0354] In one optional implementation, the fine-tuned target detection model is a model obtained by fine-tuning the target detection base model using a target dataset;

[0355] The target dataset includes an image dataset used to characterize the substation scenario, and the image data in the target dataset is image data that has undergone data standardization preprocessing.

[0356] In one optional implementation, the second segmentation module 603 is specifically used for:

[0357] Each 3D point in the first 3D segmentation result is taken as a node in the graph. Multiple connection edges are constructed for each node using a sparse connection strategy, and the comprehensive weight of the connection edges is calculated.

[0358] Based on the comprehensive weights of the connecting edges, a graph segmentation algorithm is used to perform graph segmentation, resulting in a second 3D segmentation result for the target instance.

[0359] In one optional implementation, the second segmentation module 603 is specifically used for:

[0360] Each 3D point in the first 3D segmentation result is taken as a node in the graph, and the attribute information of the node includes spatial coordinates and normal vector;

[0361] A sparse connection strategy is used to construct multiple connection edges for each node, and the comprehensive weight of each connection edge is calculated. The comprehensive weight includes the product of the spatial term weight and the normal vector term weight.

[0362] In one optional implementation, the second segmentation module 603 is specifically used for:

[0363] Based on the combined weight of the connecting edges, and under the constraint of minimizing the total weight of the cut edges, a graph cutting algorithm is used to perform graph segmentation, resulting in the second 3D segmentation result of the target instance.

[0364] Figure 7 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Figure 7 As shown, the controller provided in this embodiment includes at least one processor 801 and a memory 802. The processor 801 and the memory 802 are connected via a bus 803.

[0365] In a specific implementation, at least one processor 801 executes computer execution instructions stored in memory 802, causing at least one processor 801 to execute the method in the above method embodiment.

[0366] The specific implementation process of processor 801 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0367] In the above Figure 7 In the illustrated embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0368] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage.

[0369] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0370] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the method described in the above-described method embodiments.

[0371] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the above method embodiments.

[0372] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0373] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0374] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0375] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0376] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0377] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An instance segmentation method, characterized in that, The method includes: Perform 3D Gaussian splashing on the 3D Gaussian data of the target scene and determine the target rendering map; Multiple foreground points of the target instance in the target rendering image are identified as cue points of the target instance; Based on the cue points of the target instance, the rendered image is segmented to obtain a target segmentation mask, and based on the target segmentation mask, a first 3D segmentation result of the target instance is obtained; A sparse connection strategy is used to process each 3D point in the first 3D segmentation result to obtain the second 3D segmentation result of the target instance.

2. The instance segmentation method according to claim 1, characterized in that, The process of performing 3D Gaussian splashing on the 3D Gaussian data of the target scene and determining the target rendering image includes: In the interactive interface, 3D Gaussian splashing is performed on the 3D Gaussian data of the target scene to obtain multiple rendering images from multiple perspectives. Based on the target instance, a target rendering image is determined from the plurality of rendering images.

3. The method according to claim 1, characterized in that, The segmentation of the rendered image based on the cue points of the target instance to obtain the target segmentation mask includes: Based on the cue points of the target instance, the rendered image is segmented and inferred using a fine-tuned segmentation model; Based on the results of the segmentation inference, the target segmentation mask is determined.

4. The method according to claim 3, characterized in that, Based on the results of the segmentation inference, the target segmentation mask is determined as follows: Based on the results of the segmentation inference, including multiple segmentation masks, the optimal segmentation mask is determined and used as the target segmentation mask.

5. The method according to claim 3, characterized in that, The fine-tuned segmentation model is the model obtained by fine-tuning the segmentation base model using the target dataset; The target dataset includes an image dataset used to characterize the target scene, and the image data in the target dataset is image data that has undergone data standardization preprocessing.

6. The method according to claim 3, characterized in that, The prompt points include key points and / or feature points of the target instance, which are used to describe the shape and / or boundaries of the target instance.

7. The method according to any one of claims 1-6, characterized in that, Based on the target segmentation mask, a first 3D segmentation result of the target instance is obtained, and the method further includes: Obtain the contour coordinates of the target instance in the target segmentation mask; The contour coordinates of the target instance are transmitted to the interactive interface, and in the interactive interface, a specific point in the contour coordinates of the target instance is identified and marked as a foreground seed point. Based on the foreground seed points, the target segmentation mask is projected into 3D space to obtain the first 3D segmentation result of the target instance.

8. The method according to claim 7, characterized in that, Before transmitting the outline of the target instance to the interactive interface, the method further includes: Based on the contour coordinates of the target instance, a cutting pattern of the target instance is determined in the rendered image; The category of the target instance is obtained by reasoning from the segmentation map of the target instance using a fine-tuned target detection model.

9. The method according to claim 8, characterized in that, Transmitting the contour coordinates of the target instance to the interactive interface, and identifying and labeling specific points in the contour coordinates of the target instance as foreground seed points in the interactive interface, includes: The outline coordinates of the target instance and the category of the target instance are transmitted to the interactive interface; In the interactive interface, based on the category of the target instance, specific points in the contour coordinates of the target instance are identified and labeled as foreground seed points.

10. The method according to claim 8, characterized in that, The fine-tuned target detection model is the model obtained by fine-tuning the target detection base model using the target dataset; The target dataset includes an image dataset used to characterize the target scene, and the image data in the target dataset is image data that has undergone data standardization preprocessing.

11. The method according to any one of claims 1-6, characterized in that, The step of using a sparse connection strategy to process each 3D point in the first 3D segmentation result to obtain the second 3D segmentation result of the target instance includes: Each 3D point in the first 3D segmentation result is taken as a node in the graph. Multiple connection edges are constructed for each node using a sparse connection strategy, and the comprehensive weight of the connection edges is calculated. Based on the comprehensive weights of the connecting edges, a graph segmentation algorithm is used to perform graph segmentation, resulting in a second 3D segmentation result for the target instance.

12. The method according to claim 11, characterized in that, The step of treating each 3D point in the first 3D segmentation result as a node in a graph, constructing multiple connection edges for each node using a sparse connection strategy, and calculating the comprehensive weight of the connection edges includes: Each 3D point in the first 3D segmentation result is taken as a node in the graph, and the attribute information of the node includes spatial coordinates and normal vector; A sparse connection strategy is used to construct multiple connection edges for each node, and the comprehensive weight of each connection edge is calculated. The comprehensive weight includes the product of the spatial term weight and the normal vector term weight.

13. The method according to claim 11, characterized in that, Based on the comprehensive weights of the connected edges, graph segmentation is performed using a graph cutting algorithm to obtain the second 3D segmentation result of the target instance, including: Based on the combined weight of the connecting edges, and under the constraint of minimizing the total weight of the cut edges, a graph cutting algorithm is used to perform graph segmentation, resulting in the second 3D segmentation result of the target instance.

14. A method for instance segmentation in a substation scenario, characterized in that, The method includes: Perform 3D Gaussian splashing on the 3D Gaussian data of the substation scene and determine the target rendering image; Multiple foreground points of the target instance in the target rendering image are identified as cue points of the target instance; Based on the cue points of the target instance, the rendered image is segmented to obtain a target segmentation mask, and based on the target segmentation mask, a first 3D segmentation result of the target instance is obtained; A sparse connection strategy is used to process each 3D point in the first 3D segmentation result to obtain the second 3D segmentation result of the target instance.

15. The method according to claim 14, characterized in that, The segmentation of the rendered image based on the cue points of the target instance to obtain the target segmentation mask includes: Based on the cue points of the target instance, the rendered image is segmented and inferred using a fine-tuned segmentation model; the fine-tuned segmentation model is a model obtained by fine-tuning the segmentation base model using the target dataset; wherein, the target dataset includes an image dataset of the substation scene, and the image data in the target dataset is image data that has undergone data standardization preprocessing; Based on the results of the segmentation inference, the target segmentation mask is determined.

16. An instance segmentation device, characterized in that, The device includes: a first acquisition module, a first determination module, and a first segmentation module; The first acquisition module is used to perform 3D Gaussian splashing on the 3D Gaussian data of multiple objects in the target scene and obtain a rendered image from the target viewpoint; The first determining module is used to determine multiple foreground points of the target instance in the target rendering image as cue points of the target instance; The first segmentation module is used for: Based on the cue points of the target instance, the rendered image is segmented to obtain a target segmentation mask, and the target segmentation mask is projected into 3D space to obtain the first 3D segmentation result of the target instance; A sparse connection strategy is used to process each 3D point in the first 3D segmentation result to obtain the second 3D segmentation result of the target instance.

17. An instance segmentation device for a substation scenario, characterized in that, The device includes: a second acquisition module, a second determination module, and a second segmentation module; The second acquisition module is used to perform 3D Gaussian splashing on the 3D Gaussian data of multiple devices in the substation scene and obtain a rendering image from the target viewpoint; The second determining module is used to determine multiple foreground points of the target instance in the target rendering image as cue points of the target instance; The second segmentation module is used for: Based on the cue points of the target instance, the rendered image is segmented to obtain a target segmentation mask; The target segmentation mask is projected into 3D space to obtain the first 3D segmentation result of the target instance; A sparse connection strategy is used to process each 3D point in the first 3D segmentation result to obtain the second 3D segmentation result of the target instance.

18. An electronic device, characterized in that, include: At least one processor and memory; The memory stores computer-executed instructions; The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the method as described in any one of claims 1-15.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the method as described in any one of claims 1-15.

20. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method as described in any one of claims 1-15.