Three-dimensional interactive object segmentation method and system based on positive sample learning

Abstract

This disclosure relates to the fields of computer vision and 3D interactive segmentation technology. A 3D interactive object segmentation method and system based on positive sample learning is proposed. First, positive sample learning is performed on unlabeled image pixels based on a positive dataset to determine the unlabeled image classification labels and obtain semantic segmentation label information for the scene image. Based on the camera viewpoint corresponding to each image, the coordinates of the camera's line of sight and shooting angle parameters are obtained to perform scene representation and obtain a semantic probability distribution of the objects in the image. The obtained semantic probability distribution is combined with the semantic segmentation label information to obtain a 3D segmented image of the interactive object. This method can generate a 3D segmented image of labeled objects from a scene image while providing images with fewer viewpoints.

Description

Technical Field

[0001] The present disclosure relates to the technical fields related to computer vision and three-dimensional interactive segmentation, and more specifically, to a three-dimensional interactive target segmentation method and system based on positive sample learning. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] Interactive segmentation is a key technology in computer vision. It aims to accurately and efficiently distinguish target objects from background and other content in images or 3D data through human-computer interaction, thereby achieving high-precision recognition and detailed segmentation of target objects. This technology has broad applications in fields such as medical image analysis, remote sensing image processing, virtual reality, and game development.

[0004] Existing interactive segmentation methods can be roughly divided into two categories: traditional interactive segmentation methods and deep learning-based interactive segmentation methods. Traditional interactive segmentation methods struggle to accurately identify and segment complex scenes, such as those with high-dimensional features, complex textures, and overlapping structures. Unlike traditional methods, deep learning-based interactive segmentation methods leverage the powerful feature extraction and pattern recognition capabilities of deep neural networks to automatically capture complex image features and improve segmentation accuracy.

[0005] The inventors discovered that existing methods can effectively perform interactive image segmentation. However, in the field of 3D interactive segmentation, although 3D data contains rich spatial information, achieving accurate interactive segmentation directly in 3D space faces many challenges. Acquiring 3D datasets and fully annotating them is often extremely time-consuming. Summary of the Invention

[0006] In order to solve the above problems, the present disclosure proposes a three-dimensional interactive target segmentation method and system based on positive sample learning, which can generate a three-dimensional segmented image of annotated objects from a scene image under the condition of providing images with fewer perspectives.

[0007] In order to achieve the above objectives, the present disclosure adopts the following technical solutions:

[0008] One or more embodiments provide a three-dimensional interactive object segmentation method based on positive sample learning, comprising the following steps:

[0009] Select some images from the 3D scene image to be segmented for target annotation, build a positive dataset, and use the unlabeled images as the unlabeled dataset;

[0010] Interactively segment the positive dataset and the unlabeled dataset, perform positive sample learning on the unlabeled image pixels based on the positive dataset, determine the classification labels of the unlabeled images, and obtain the semantic segmentation label information of the scene image;

[0011] According to the camera parameters corresponding to each image, the scene is expressed to obtain the semantic probability distribution of the target in the image;

[0012] The obtained semantic probability distribution is combined with the semantic segmentation label information, and according to the obtained perspective to be identified, a three-dimensional segmentation image of the interactive object under the corresponding perspective is obtained.

[0013] A 3D interactive object segmentation system based on positive sample learning, including:

[0014] Positive sample segmentation module: configured to obtain part of the 3D scene image to be segmented for target labeling, construct a positive class dataset, and use the unlabeled images as the unlabeled dataset;

[0015] Positive Sample Learning Module: This module is configured to interactively segment the positive dataset and the unlabeled dataset, perform positive sample learning on the unlabeled image pixels based on the positive dataset, determine the classification labels of the unlabeled images, and obtain semantic segmentation label information for the scene images.

[0016] Scene representation module: This module is configured to represent the scene based on the camera parameters corresponding to each image and obtain the semantic probability distribution of the objects in the image;

[0017] Fusion segmentation module: configured to combine the obtained semantic probability distribution with the semantic segmentation label information, and obtain a three-dimensional segmentation image of the interactive object under the corresponding perspective according to the obtained perspective to be identified.

[0018] An electronic device includes a memory and a processor, and computer instructions stored in the memory and executed on the processor. When the computer instructions are executed by the processor, the steps in the above-mentioned three-dimensional interactive target segmentation method based on positive sample learning are completed.

[0019] A computer-readable storage medium is used to store computer instructions, which, when executed by a processor, complete the steps in the above-mentioned three-dimensional interactive object segmentation method based on positive sample learning.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] The 3D interactive segmentation method disclosed in this paper can predict the corresponding scene based on the camera's pose within a set camera area. It can also segment target objects selected from scenes that cannot be directly captured by the camera, achieving semantic segmentation output for objects in the entire area without blind spots. The proposed interactive segmentation can separate annotated objects from the scene image and, combined with scene representation, realize 3D interactive segmentation.

[0022] The advantages of the present disclosure and additional advantages will be described in detail in the following specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings, which constitute a part of the present disclosure, are used to provide a further understanding of the present disclosure. The exemplary embodiments of the present disclosure and their descriptions are used to explain the present disclosure but do not constitute a limitation of the present disclosure.

[0024] Figure 1 is a flowchart of a three-dimensional interactive object segmentation method based on positive sample learning according to Example 1 of the present disclosure;

[0025] Figure 2 3D interactive object segmentation process diagram of the 3D interactive object segmentation system based on positive sample learning in Example 1 of the present disclosure;

[0026] Figure 3 This is a schematic diagram of manual graffiti-style annotation in the model training of Example 1 of the present disclosure. DETAILED DESCRIPTION

[0027] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

[0028] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present disclosure belongs.

[0029] It should be noted that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof. It should be noted that, in the absence of conflict, the various embodiments in the present disclosure and the features in the embodiments can be combined with each other. The embodiments will be described in detail below with reference to the accompanying drawings.

[0030] Example 1

[0031] In the technical solutions disclosed in one or more embodiments, Figures 1 to 3 As shown in FIG, the three-dimensional interactive object segmentation method based on positive sample learning includes the following steps:

[0032] Step 1: Select some images in the 3D scene image to be segmented for target annotation, build a positive class dataset, and use the unlabeled images as the unlabeled dataset;

[0033] Step 2: Interactively segment the positive dataset and the unlabeled dataset, perform positive sample learning on the unlabeled image pixels based on the positive dataset, determine the classification label of the unlabeled image, and obtain the semantic segmentation label information of the scene image;

[0034] Step 3: According to the camera viewpoint corresponding to each image, the coordinates of the camera's line of sight and the shooting angle parameters are obtained to express the scene and obtain the semantic probability distribution of the target in the image;

[0035] Step 4: Combine the obtained semantic probability distribution with the semantic segmentation label information, and obtain a three-dimensional segmentation image of the interactive object under the corresponding perspective according to the obtained perspective to be identified.

[0036] The 3D interactive segmentation method in this embodiment can predict the corresponding scene within a camera's set area based on the camera's position. It can also segment selected target objects in scenes that are not directly captured by the camera, achieving semantic segmentation output for objects across the entire area without blind spots. The proposed interactive segmentation can separate annotated objects from the scene image, and combined with scene representation, it can realize 3D interactive segmentation.

[0037] Specifically, step 1 includes the following steps:

[0038] Step 11: Acquire a three-dimensional scene image, including a multi-view image and an interactive image, and perform feature extraction on each image to obtain a feature image;

[0039] Specifically, the size of the feature image is consistent with the size of the scene image, and the pixel value of the feature image is the feature value of the corresponding scene image pixel; there are interactive targets in the interactive image, and the target can be annotated. The multi-view image is the unannotated image scene image to be identified;

[0040] Step 12: Select the interactive image for target annotation, build a positive dataset based on the annotated image and the corresponding feature image, and use the unannotated images in the 3D scene image and the corresponding feature image as the unannotated dataset;

[0041] Select images from the acquired 3D scene graph and label the target objects of interest to obtain manually labeled images. Combine them with the feature images of the images to form a positive dataset, and the feature maps of the remaining unlabeled images are used as an unlabeled dataset.

[0042] In step 2, an interactive segmentation module is constructed, and the positive class dataset and the unlabeled dataset are input into the interactive segmentation module to obtain the probability that the unlabeled image pixels belong to the positive class;

[0043] Specifically, we build an interactive segmentation module based on PU Learning and train it to obtain the probability map of pixels in multi-view images belonging to interactive objects.

[0044] Among them, PU Learning is called positive sample unlabeled learning;

[0045] Specifically, the interactive segmentation module can be configured to perform interactive segmentation using a multi-layer perceptron network (MLP network for short) to obtain the probability that the unlabeled image pixels belong to the positive class.

[0046] Furthermore, a suitable threshold is set. If the probability of the unlabeled image pixels belonging to the positive class is greater than the threshold, they are considered to be in the positive class. Otherwise, the remaining unlabeled image pixels are regarded as unlabeled classes to obtain semantic segmentation label information.

[0047] In step 3, the method for determining the semantic probability distribution of the target in the image includes the following steps:

[0048] Step 31: Obtain camera parameters of the camera in the scene area to be identified;

[0049] Step 32: For each camera viewpoint, generate a ray from the camera center to the captured image plane, and uniformly sample a series of three-dimensional coordinates (x, y, z) on the ray;

[0050] Step 33: The three-dimensional coordinates of each sampling point and the corresponding sight direction parameters are used to form a feature vector, which is input into the trained scene representation network for scene representation to obtain the semantic probability distribution of the target in the image;

[0051] The sight direction parameter is the two-dimensional viewing angle information, which can be obtained by converting the three-dimensional coordinates and camera parameters, including θ, Together they form a five-dimensional vector

[0052] Among them, θ, Parameters used to represent the viewing direction or camera pose, θ usually represents the azimuth, which is the rotation angle around the vertical axis (usually the z-axis), which describes the offset of the viewing direction in the horizontal plane; It is generally expressed as the elevation angle, which is the angle measured downward from the positive z-axis and describes the degree to which the viewing direction is tilted relative to the vertical.

[0053] Specifically, the scene representation network uses the Neural Radiance Field (NeRF) as the basic model, obtains the semantic probability distribution of the target in the image based on the scene representation function φ(x), and expresses the scene based on the Neural Radiance Field to obtain the semantic probability distribution of the target in the image;

[0054] The two inputs to the NeRF are the position coordinates (x, y, z) in 3D space and the viewing direction (usually represented by a unit vector). Adding a semantic segmentation head to the network allows it to output a probability distribution, or semantics. The NeRF output includes the radiant intensity (color), volume density, and semantics at that location.

[0055] In step 4, the obtained semantic probability distribution is combined with the above-mentioned semantic segmentation label information, and the semantic probability distribution is integrated with the semantic segmentation label information generated by the interactive segmentation module to synthesize the image under the perspective and generate the corresponding semantic segmentation map, that is, to obtain a three-dimensional segmentation image of the interactive object.

[0056] During training, NeRF attempts to minimize the reconstruction loss and iteratively adjusts the network weights so that the image obtained through volume rendering, based on the learned radiation field and density information, is as close as possible to the actual photograph. This includes correctly predicting the color under known perspectives, while also ensuring coherence and consistency when viewed from new perspectives. Once NeRF training is complete, given a new camera pose (i.e., the new perspective the user wants to render), NeRF is able to generate images from the new perspective as follows:

[0057] Step 41. Traverse each pixel of the new perspective and calculate the path of the ray corresponding to the pixel in 3D space;

[0058] Step 42: Use the MLP network to estimate the color and density of multiple sampling points along the ray.

[0059] Step 43: Apply volume rendering technology to calculate the color of the final pixel based on the color and transparency of all sampling points along the ray, and obtain the image under the perspective to be recognized.

[0060] Here, transparency is the volume density.

[0061] The network composed of interactive segmentation module and scene representation network is as follows Figure 2 As shown in the figure, [R, t] is the camera external parameter, K is the camera internal parameter, u is the image plane coordinate, that is, (x, y); v is the viewing direction, and the above θ, This embodiment also provides a process for training the above-mentioned interactive segmentation module and scene expression network, which is described below.

[0062] In some embodiments, a method for training an interactive segmentation module comprises the following steps:

[0063] Step S1: Obtain a data set, perform feature extraction on the images in the data set, and obtain a feature atlas;

[0064] Specifically, for the dataset X, a feature extraction module is used to extract features from the dataset to obtain a feature map with the same size as the original image. The pixel values ​​of the feature maps are the feature values ​​of the pixels corresponding to the original image, forming a feature map set T.

[0065] Step S2: Manually label the selected images, use the labeled images and feature maps as the positive dataset, and use the unlabeled feature images as the unlabeled dataset;

[0066] Select images from the dataset X and manually mark the objects you want to segment with a graffiti-style annotation (e.g. Figure 3 shown), Figure 3 The middle figure is an example of manual annotation. Assuming that the manual annotation is completely correct, the annotated pixel gets the label value 1. The feature x of the annotated pixel, its label value s = 1, and the annotated label m = 1 constitute the positive class dataset X. P , unlabeled pixel features, unlabeled labels m = 0 constitute the unlabeled dataset X U .

[0067] Step S3: The positive dataset and the unlabeled dataset are input into the interactive segmentation module for training to obtain a probability image of the unlabeled pixels belonging to the labeled target object;

[0068] Specifically, the interactive segmentation module has two outputs, P(y=1|x;u1) and h(x;u2). Among them, P(y=1|x;u1) represents the probability that the sample belongs to the positive class, and h(x;u2) represents the probability that the positive sample in the unlabeled sample is correctly labeled;

[0069] Furthermore, the probability h(x;u2) that the positive samples in the unlabeled samples are correctly labeled is identified by the trained neural network.

[0070] Since there are only positive class labels and no negative class labels in the data set, there is a labeling bias problem, which makes it easy for overfitting to occur during training. In this embodiment, a neural network outputs h(x;u2) in this module, and uses it to construct a loss function to achieve better results.

[0071] Then, a suitable threshold t is set according to the actual data set. When P(y=1|x;u1) is greater than t, the pixel is considered to belong to the positive class and its pixel value is set to 1. Otherwise, it is set to 0, and finally a two-dimensional segmentation map of the labeled object is obtained.

[0072] Step S4: construct a positive class loss function based on the probability image obtained by recognition and the actual probability. p And the loss function Loss of the unlabeled class u The total loss function is used to calculate the loss function value, and the weight parameters of the interactive segmentation module are updated. The iterative cycle is repeated until the iteration termination condition is met to obtain the trained interactive segmentation module.

[0073] Optional, interactive segmentation module loss function, as follows:

[0074]

[0075] Loss total =Loss p +λ·Loss u (3)

[0076] Among them, Loss p Represents the positive class loss function, as shown in formula (2), where |Pixels P | represents the number of positive data sets, y is the label value, which is always 1 in this case, and P(y=1|x;u1) represents the posterior probability of positive data; |Pixels U | represents the number of unlabeled data sets, F(·) represents a multiplication operation; h(x;u2) represents the probability that a positive sample in an unlabeled sample is correctly labeled, and P(y=1|x;u1) is the same as above.

[0077] Loss u represents the loss function of the unlabeled class, Q(x) is a learnable parameter, which is used to adjust the proportion of the corresponding loss value during training; λ is the weight of the loss function, which is needed to adjust the ratio of labeled and unlabeled samples due to the imbalance.

[0078] Furthermore, the process of training the scene expression network is also included, including the following:

[0079] Step S5: Estimate the camera parameters (including position, orientation, etc.) of each image in the dataset X. For each ray starting from the camera center, sample multiple 3D points along the ray path.

[0080] Step S6: Input the camera parameters of each image, the multiple 3D points obtained by sampling, and the probability map generated by the interactive segmentation module into the scene representation function to output the semantic probability distribution;

[0081] Among them, the scene expression function is obtained by fitting the NeRF network.

[0082] Step S7: Integrate the semantic probability distribution and the semantic segmentation label information generated by the interactive segmentation module to synthesize the image under the viewing angle and generate a corresponding semantic segmentation map;

[0083] Step S8: Calculate the segmentation loss based on the obtained semantic segmentation map S , update the parameters of the scene representation network, execute step S5, iterate the training until the semantic segmentation result under the new perspective that has not been seen before can be generated, and obtain the trained scene representation network;

[0084] The parameters of the scene expression network include the weights and biases in the neural network and other learnable parameters; the loss function Loss S Cross entropy loss can be used;

[0085] In this embodiment, the semantic segmentation label information generated by the interactive segmentation module is used to S , learning the semantic boundaries of labeled objects. Finally, through the integration process, the images under different perspectives are synthesized and the corresponding semantic segmentation maps are generated. By repeatedly iterating the training process and updating the network parameters, semantic segmentation results can be generated for new perspectives that have not been seen before.

[0086] In order to illustrate the effect of the method in this embodiment, a comparative experiment was conducted on the NOVS dataset with other methods. The comparison results are shown in Table 1.

[0087]

[0088] In the table, Graph-cut (3D) indicates the method of applying graph cut theory (GraphCut) for semantic segmentation;

[0089] NVOS, short for Neural Volumetric Object Selection, is an existing method for interactive image segmentation using deep learning.

[0090] ISRF, the full name of which is Interactive Segmentation of Radiance Fields, is also an existing method for interactive image segmentation using deep learning.

[0091] Ours, represents the segmentation method of this embodiment;

[0092] The experimental data results were quantified using the intersection over union (IoU) and accuracy (Acc). For both metrics, the larger the value, the better the effect. Table 1 shows a comparison of these metrics. The quantitative results of the method of the present invention are superior to those of the comparison algorithms, indicating that the segmentation method of this embodiment has higher segmentation accuracy.

[0093] Example 2

[0094] Based on Example 1, this embodiment provides a three-dimensional interactive object segmentation system based on positive sample learning, including:

[0095] Positive sample segmentation module: configured to obtain part of the 3D scene image to be segmented for target labeling, construct a positive class dataset, and use the unlabeled images as the unlabeled dataset;

[0096] Positive Sample Learning Module: This module is configured to interactively segment the positive dataset and the unlabeled dataset, perform positive sample learning on the unlabeled image pixels based on the positive dataset, determine the classification labels of the unlabeled images, and obtain semantic segmentation label information for the scene images.

[0097] Scene representation module: This module is configured to represent the scene based on the camera parameters corresponding to each image and obtain the semantic probability distribution of the objects in the image;

[0098] Fusion segmentation module: configured to combine the obtained semantic probability distribution with the semantic segmentation label information to obtain a three-dimensional segmentation image of the interactive object.

[0099] It should be noted here that the various modules in this embodiment correspond one-to-one to the various steps in Example 1, and the specific implementation processes are the same, which will not be repeated here.

[0100] Example 3

[0101] This embodiment provides an electronic device, including a memory and a processor, and computer instructions stored in the memory and running on the processor. When the computer instructions are run by the processor, the steps in the three-dimensional interactive target segmentation method based on positive sample learning in Example 1 are completed.

[0102] Example 4

[0103] This embodiment provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the steps in the three-dimensional interactive object segmentation method based on positive sample learning in Example 1 are completed.

[0104] The electronic devices proposed in the present disclosure can be mobile terminals and non-mobile terminals. Non-mobile terminals include desktop computers, and mobile terminals include smart phones (such as Android phones, IOS phones, etc.), smart glasses, smart watches, smart bracelets, tablets, laptops, personal digital assistants, and other mobile Internet devices that can communicate wirelessly.

[0105] It should be understood that in the present disclosure, the processor may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), off-the-shelf field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0106] The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0107] During implementation, each step of the above method can be performed by hardware integrated logic circuits in a processor or by software instructions. The steps of the method disclosed in this disclosure can be directly implemented as execution by a hardware processor, or by a combination of hardware and software modules in the processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. The storage medium is located in a memory, and the processor reads the information in the memory and, in conjunction with its hardware, performs the steps of the above method. To avoid repetition, a detailed description is not given here. Those skilled in the art will appreciate that the units and algorithm steps of the various examples described in the embodiments disclosed herein can be implemented using electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this disclosure.

[0108] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0109] In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a division of logical functions. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0110] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0111] The foregoing description is merely a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Those skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present disclosure shall be included within the scope of protection of the present disclosure.

[0112] Although the above describes the specific implementation methods of the present disclosure in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present disclosure. Those skilled in the art should understand that on the basis of the technical solution of the present disclosure, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present disclosure.

Claims

1. A three-dimensional interactive object segmentation method based on positive sample learning, characterized by: The steps include: Select some images from the 3D scene image to be segmented for target annotation, build a positive dataset, and use the unlabeled images as the unlabeled dataset; Interactively segment the positive dataset and the unlabeled dataset, perform positive sample learning on the unlabeled image pixels based on the positive dataset, determine the classification labels of the unlabeled images, and obtain the semantic segmentation label information of the scene image; Construct an interactive segmentation module based on PU Learning for interactive segmentation. The interactive segmentation module adopts a multi-layer perceptron network. According to the camera parameters corresponding to each image, the scene is expressed to obtain the semantic probability distribution of the target in the image; Estimate the camera parameters of each image in the acquired dataset, and for each ray starting from the camera center, sample multiple 3D points along the ray’s path; The camera parameters of each image, the sampled 3D points, and the probability map generated by the interactive segmentation module are input into the scene representation function to output the semantic probability distribution; The obtained semantic probability distribution is combined with the semantic segmentation label information, and a 3D segmentation image of the interactive object under the corresponding perspective is obtained according to the obtained perspective to be identified; The interactive segmentation module training method comprises the following steps: Obtain a data set, perform feature extraction on the images in the data set, and obtain a feature atlas; Manually label the selected images, use the labeled images and feature maps as the positive dataset, and use the unlabeled feature images as the unlabeled dataset; The positive dataset and the unlabeled dataset are used as input to the interactive segmentation module for training, and a probability image of the unlabeled pixels belonging to the labeled target object is obtained; According to the probability image obtained by recognition and the actual probability, a total loss function including the loss function of the positive class and the loss function of the unlabeled class is constructed to calculate the loss function value, and the weight parameters of the interactive segmentation module are updated. The iterative loop is repeated until the iteration termination condition is met to obtain the trained interactive segmentation module. The loss function of the interactive segmentation module is as follows: （1） （2） （3） in, Represents the positive class loss function, as shown in formula (2), where Indicates the number of positive data sets, y is the label value, in this case, its value is always 1, Represents the posterior probability of positive data; represents the number of unlabeled datasets, Represents multiplication operation; Indicates the probability that the positive sample in the unlabeled sample is correctly labeled, Same as above; represents the loss function for the unlabeled class, It is a learnable parameter, which is used to adjust the proportion of the corresponding loss value during training; It is the weight of the loss function, which is needed to adjust the ratio of labeled and unlabeled samples due to the imbalance.

2. The three-dimensional interactive object segmentation method based on positive sample learning according to claim 1, characterized in that: Acquire a three-dimensional scene image, including a multi-view image and an interactive image, and perform feature extraction on each image to obtain a feature image; the interactive image includes an interactive target; The interactive images are selected for target annotation, and a positive dataset is constructed based on the annotated images and the corresponding feature images. The unannotated images in the 3D scene images and the corresponding feature images are used as the unannotated dataset.

3. The three-dimensional interactive object segmentation method based on positive sample learning according to claim 1, characterized in that: A threshold is set, and the unlabeled image pixels are classified as positive if the probability of belonging to the positive class is greater than the threshold. Otherwise, they are classified as unlabeled class, and the semantic segmentation label information is obtained.

4. The three-dimensional interactive object segmentation method based on positive sample learning according to claim 1, characterized in that The method for determining the semantic probability distribution of an object in an image comprises the following steps: Get the camera parameters of the camera in the scene area to be recognized; For each camera viewpoint, generate a ray from the camera center to the captured image plane and uniformly sample a series of 3D coordinates on the ray; The three-dimensional coordinates of each sampling point and the corresponding line of sight direction parameters constitute a feature vector, which is input into the trained scene representation network for scene representation to obtain the semantic probability distribution of the target in the image.

5. The three-dimensional interactive object segmentation method based on positive sample learning according to claim 4, characterized in that: The scene representation network uses the neural radiance field as the basic model and obtains the semantic probability distribution of the target in the image based on the scene representation function; The process of training the scene representation network also includes the following: The semantic probability distribution is integrated with the semantic segmentation label information generated by the interactive segmentation module to synthesize the image under the viewing angle and generate the corresponding semantic segmentation map. The segmentation loss is calculated based on the obtained semantic segmentation map, and the parameters in the scene expression function of the scene expression network are updated. The training is iterated until the semantic segmentation result under the new perspective that has not been seen before can be generated, and the trained scene expression network is obtained.

6. The three-dimensional interactive object segmentation method based on positive sample learning according to claim 1, characterized in that: The semantic probability distribution and the semantic segmentation label information generated by the interactive segmentation module are integrated to synthesize the image under the perspective, including the following steps: Traverse each pixel of the new perspective and calculate the path of the ray corresponding to the pixel in 3D space; Use the MLP network to estimate the color and density of multiple sampling points along the ray; Volume rendering technology is applied to calculate the color of the final pixel according to the color and transparency of all sampling points along the ray, and the image under the synthetic viewing angle to be recognized is obtained.

7. A system using the three-dimensional interactive object segmentation method based on positive sample learning according to any one of claims 1 to 6, characterized in that: include: Positive sample segmentation module: configured to obtain part of the 3D scene image to be segmented for target labeling, construct a positive class dataset, and use the unlabeled images as the unlabeled dataset; Positive Sample Learning Module: This module is configured to interactively segment the positive dataset and the unlabeled dataset, perform positive sample learning on the unlabeled image pixels based on the positive dataset, determine the classification labels of the unlabeled images, and obtain semantic segmentation label information for the scene images. Scene representation module: This module is configured to represent the scene based on the camera parameters corresponding to each image and obtain the semantic probability distribution of the objects in the image; Fusion segmentation module: configured to combine the obtained semantic probability distribution with the semantic segmentation label information, and obtain a three-dimensional segmentation image of the interactive object under the corresponding perspective according to the obtained perspective to be identified.

8. An electronic device, characterized in that: The invention comprises a memory and a processor, and computer instructions stored in the memory and executed on the processor. When the computer instructions are executed by the processor, the steps of the three-dimensional interactive target segmentation method based on positive sample learning according to any one of claims 1 to 6 are completed.

9. A computer-readable storage medium, characterized in that Used to store computer instructions, which, when executed by a processor, complete the steps of the three-dimensional interactive target segmentation method based on positive sample learning according to any one of claims 1 to 6.

Images