Unmanned driving image instance segmentation method and system based on prototype contrast learning

Abstract

The application discloses an unmanned image instance segmentation method and system based on prototype contrast learning, and relates to the technical field of image instance segmentation. The method comprises the following steps: inputting a plurality of acquired unmanned images into a student model in an image instance segmentation model, extracting image instance features, and acquiring prototype knowledge specific to the image instance features in a query manner based on prototype knowledge migrated to the student model, combining the image instance features, and outputting an image instance segmentation result; wherein the image instance segmentation model comprises a teacher model and a student model, the teacher model uses a feature extraction network and a prototype knowledge acquisition network to respectively acquire sample image features and prototype knowledge similar to the features, the teacher model is trained based on contrast learning, and the prototype knowledge learned by the trained teacher model is migrated to the student model. The application guarantees the accuracy of instance segmentation and simplifies the size of the instance segmentation model through prototype contrast learning and knowledge distillation.

Description

Technical Field

[0001] This invention relates to the field of image instance segmentation technology, and in particular to an image instance segmentation method and system for autonomous driving based on prototype contrastive learning. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute advanced technology.

[0003] Currently, the realization of autonomous driving is limited by hardware. Most autonomous vehicles need to upload real-time signals and road images acquired by sensors to cloud computing servers for data analysis and image processing before returning the results to the vehicle to perform corresponding driving operations. However, in situations with poor network signals, the transmission of data to and from the cloud platform can cause severe network latency, and in extreme cases, the inability to send signals can lead to serious safety hazards. Therefore, how to deploy machine learning models on lightweight edge devices such as vehicle sensors for image processing to achieve safe and stable autonomous driving is a pressing problem that needs to be solved.

[0004] Image instance segmentation is a fundamental computer vision problem that aims to provide different pixel-level labels for different instances belonging to the same object class, where the number of instances in each class in an image is unknown. Therefore, instance segmentation can be defined as a technique that simultaneously solves object detection and semantic segmentation problems. Instance segmentation is significantly more difficult than semantic segmentation because it also requires distinguishing between nearby and occluded object instances. Traditional image instance segmentation methods rely on expert knowledge to design local descriptive symbols to identify specific instances. However, with the development of deep learning, robust feature representations with different levels of abstraction can be automatically learned from images. Subsequently, the development of image instance segmentation has gradually shifted towards developing better-performing network frameworks and more optimized training procedures.

[0005] Existing image instance segmentation models are complex and difficult to deploy directly on lightweight devices. Moreover, segmentation models deployed on lightweight devices not only require high model accuracy but also higher segmentation efficiency. Segmentation efficiency refers to the computational cost of the segmentation algorithm, which requires efficient real-time computation, such as acceptable memory / storage requirements and low processor load. Existing image instance segmentation schemes are difficult to meet these requirements. Summary of the Invention

[0006] To address the shortcomings of the existing technologies, this invention provides a method and system for autonomous driving image instance segmentation based on prototype contrastive learning. This solves the problem of achieving high-precision, high-efficiency, lightweight image instance segmentation during autonomous driving. Through prototype contrastive learning, fine-grained prototype knowledge specific to each instance is learned. Furthermore, through knowledge distillation, the prototype knowledge learned by the teacher model is transferred to a lightweight student model. Image instance segmentation is then achieved through this lightweight student model, ensuring segmentation accuracy while reducing the size of the instance segmentation model and minimizing computational costs.

[0007] Firstly, this disclosure provides a method for autonomous driving image instance segmentation based on prototype contrastive learning, including:

[0008] Multiple autonomous driving images are acquired and input into the student model in the image instance segmentation model. Image instance features are extracted, and prototype knowledge specific to the image instance features is obtained based on the prototype knowledge transferred to the student model and a query method. Then, based on the obtained prototype knowledge and image instance features, the image instance segmentation result is output.

[0009] The image instance segmentation model includes a teacher model and a student model. The teacher model includes a feature extraction network and a prototype knowledge acquisition network. The feature extraction network extracts features from the input sample image, and the prototype knowledge acquisition network learns prototype knowledge similar to the features of the sample image. The teacher model is trained based on contrastive learning, and the prototype knowledge learned by the trained teacher model is transferred to the student model.

[0010] Secondly, this disclosure provides an autonomous driving image instance segmentation system based on prototype contrastive learning, including:

[0011] The image acquisition module is used to acquire multiple images of the autonomous vehicle.

[0012] The image instance segmentation module is used to input the acquired image into the student model in the image instance segmentation model, extract image instance features, and obtain prototype knowledge specific to image instance features based on the prototype knowledge transferred to the student model in a query-based manner. Then, based on the obtained prototype knowledge and image instance features, the module outputs the image instance segmentation result.

[0013] The image instance segmentation model includes a teacher model and a student model. The teacher model includes a feature extraction network and a prototype knowledge acquisition network. The feature extraction network extracts features from the input sample image, and the prototype knowledge acquisition network learns prototype knowledge similar to the features of the sample image. The teacher model is trained based on contrastive learning, and the prototype knowledge learned by the trained teacher model is transferred to the student model.

[0014] Thirdly, this disclosure also provides an electronic device, including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the steps of the method described in the first aspect.

[0015] Fourthly, this disclosure also provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the steps of the method described in the first aspect.

[0016] The above one or more technical solutions have the following beneficial effects:

[0017] 1. This invention provides an image instance segmentation method and system for autonomous driving based on prototype contrastive learning, which solves the problem of not being able to achieve high-precision, high-efficiency, lightweight image instance segmentation in autonomous driving. Through prototype contrastive learning, it learns instance-specific fine-grained prototype knowledge, making prototypes of the same class similar and prototypes of different classes more distant. Furthermore, through knowledge distillation, a teacher-student model is constructed to achieve instance-specific prototype knowledge transfer and application to lightweight student models in different scenarios, thereby achieving image instance segmentation through the student model.

[0018] 2. This invention helps the model learn task-specific prototypes better by adding frequency prior knowledge, and through a query mechanism based on cosine similarity and key matching, the learned prototype knowledge can be transferred to different instance segmentation scenarios, making it more widely applicable. Attached Figure Description

[0019] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0020] Figure 1 This is an overall flowchart of the autonomous driving image instance segmentation method based on prototype contrastive learning as described in Embodiment 1 of the present invention;

[0021] Figure 2 This is a schematic diagram of the image instance segmentation model training process in Embodiment 1 of the present invention. Detailed Implementation

[0022] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0023] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0024] Example 1

[0025] As described in the background section, to deploy instance segmentation models on lightweight devices, in addition to high model accuracy, higher efficiency in image instance segmentation is also required. Therefore, this embodiment provides an autonomous driving image instance segmentation method based on prototype contrastive learning. A set of prototypes is learned for all types of instances in the image. Training is performed using a constructed contrastive loss to make instances of the same type more similar to the prototypes, thereby learning a set of instance-specific prototype knowledge. The learned low-level features are then used for instance discrimination. Based on knowledge distillation, the prototype knowledge learned in this way is transferred from a high-performing teacher model to a lightweight student model, resulting in a lightweight student model with performance comparable to the teacher model. This lightweight student model is used for image instance segmentation, ensuring segmentation accuracy while reducing the scale of the instance segmentation model and minimizing computational cost.

[0026] The autonomous driving image instance segmentation method proposed in this embodiment, such as... Figure 1 As shown, it includes:

[0027] Multiple autonomous driving images are acquired and input into the student model in the image instance segmentation model. Image instance features are extracted, and prototype knowledge specific to the image instance features is obtained based on the prototype knowledge transferred to the student model and a query method. Then, based on the obtained prototype knowledge and image instance features, the image instance segmentation result is output.

[0028] The image instance segmentation model includes a teacher model and a student model. The teacher model includes a feature extraction network and a prototype knowledge acquisition network. The feature extraction network extracts features from the input sample image, and the prototype knowledge acquisition network learns prototype knowledge similar to the features of the sample image. The teacher model is trained based on contrastive learning, and the prototype knowledge learned by the trained teacher model is transferred to the student model.

[0029] In the above method, prototype knowledge specific to image instances is learned through a teacher model. Based on knowledge distillation, the prototype knowledge learned from a well-performing and trained teacher model is transferred to a lightweight student model. This is then distilled to obtain a lightweight student model with performance comparable to the teacher model. Image instance segmentation is achieved through this lightweight student model. Therefore, in the method described in this embodiment, the construction and training of the teacher model and student model (i.e., the image instance segmentation model) are fundamental to achieving high-precision image instance segmentation. The image instance segmentation model will be introduced below.

[0030] In this embodiment, firstly, multiple autonomous driving images are acquired, and different instances in the autonomous driving images are labeled. The labeled autonomous driving images are used as a training sample set to train an image instance segmentation model. This image instance segmentation model includes a teacher model and a student model, wherein the teacher model includes a feature extraction network and a prototype knowledge acquisition network. For example... Figure 2 As shown, the training process of the above image instance segmentation model includes the following steps:

[0031] Step 1: Input multiple sample images simultaneously into a pre-trained feature extraction network for feature extraction. In this embodiment, the feature extraction network is a ResNet101 backbone network, which is used to extract features from the input sample images.

[0032] Step 2: Utilize prototype knowledge to acquire prototype knowledge that is similar to the features of the sample images learned by the network. First, a set of randomly initialized prototype features is preset. To predict the attributes of an image instance, where p n Let n represent the nth prototype feature. For each prototype feature, locate the relevant image instance, and then calculate the image instance-specific visual features, i.e., update the prototype feature as follows:

[0033] P 1 =f(XS),S=h(X T P 0 (1)

[0034] Taking the i-th sample image as an example, X i Let X represent the features extracted from the i-th sample image, and let S represent the feature set of the sample images. This represents the features of the i-th sample image and the prototype features p in the initial prototype feature set. n The similarity is calculated by using prototype features specific to related image instances in X, aggregated by S; h() is a column-regularized softmax function; f() represents the activation function, consisting of two cascaded fully connected layers with ReLU activation functions, used to refine the prototype features. Compared to the original sample image features, P1 Explicitly capture specific visual features of the sample image for each instance, such as Features of image instances in the aggregated sample image feature set X that are related to the first prototype feature p1 in the initial prototype feature set.

[0035] In fact, during the process of prototype knowledge learning, the attributes of an image instance refer to the features or attributes used to describe and distinguish different categories. In this embodiment, prototype features are used to characterize the attributes of an image instance, which is an abstract representation of the observation and understanding of things. By associating samples with their attributes, the prototype learning algorithm can learn the prototype or representative sample of each category and classify them according to the similarity between the sample and the prototype. Taking the instance of classifying image samples into two types, dogs and cats, by the prototype learning algorithm as an example, the attributes are the features of the image samples, such as fur color, tail shape, ear size, etc. The above attributes are associated with the image samples to form an attribute-value correspondence. According to the attribute settings: (1) The attributes of dogs are set as: fur color (attribute) - brown (value), tail shape (attribute) - curled (value), ear size (attribute) - large (value); (2) The attributes of cats are set as: fur color (attribute) - gray (value), tail shape (attribute) - long and thin (value), ear size (attribute) - medium (value). Finally, image instance classification based on prototype features specific to the image instance can be achieved.

[0036] In this embodiment, an initial prototype feature set is first set up using the above method. By calculating visual features specific to the image instance, the initial prototype features are updated to obtain prototype features that are closer to the features of the sample image, so as to facilitate subsequent image classification.

[0037] Furthermore, considering instance variance, the initial prototype feature P 0 It cannot accurately locate relevant instances for all sample images. In equation (1) above, with P 0 In comparison, P 1 The visual patterns corresponding to different instances are more specific to a given image feature X, therefore P is further... 1 Considered as based on P 0 The prototype features corresponding to the instance updated for the target image. That is, by continuously iterating through the following equation (2), using P k+1 Replace P k Gradually adjust the prototype features specific to the image instance, that is:

[0038] P k+1 =f(Xh(X) T P k (2)

[0039] Through the above method, better instance prototype features are obtained based on features extracted from the pre-trained backbone network. In addition, the prototype features obtained after K iterations are prototype knowledge, which is used for subsequent calculations.

[0040] Step 3: Train the feature extraction network and prototype knowledge acquisition network based on contrastive learning, and jointly optimize the backbone network and prototype knowledge.

[0041] First, the input image x is processed by a feature extraction network to obtain the input image features q. Then, the input image features q are processed by a prototype knowledge acquisition network to obtain the prototype knowledge corresponding to the image features q, i.e., the prototype features P. k To further optimize the feature extraction network and prototype knowledge acquisition network, a contrastive loss function between image features and prototype features is constructed, which compares image feature q with its similar prototype features. (Treat it as a positive sample) Close the distance and identify prototype features that are not similar to it. To increase the distance, the specific methods are as follows:

[0042]

[0043] Where τ is the temperature hyperparameter.

[0044] Secondly, considering the often-related relationships between visual attributes, to enhance the prediction of instance attribute combinations, the prototype knowledge acquisition network constrains attribute prototypes through competition of unrelated instance prototype features and sharing of related instance prototype features. To represent the semantic relationships between instance prototype features, a classification network is added after the prototype knowledge acquisition network for classification, i.e.:

[0045]

[0046] In the above formula, cat represents tensor concatenation along the channel dimension, and f() represents the activation function, which is the same as the activation function in formula (1) above.

[0047] Image category classification is achieved through the above classification. Compared to instances, categories are a more abstract concept, representing a class of similar instances. That is, instances in an image are specific individuals that belong to a certain category, and multiple instances can belong to the same category. This embodiment, through category classification, can learn the common semantic relationships of instances belonging to the same category, assisting in prototype optimization.

[0048] By using the cross-entropy loss described above, the correlation of prototype features is removed, forcing feature competition between prototype features from instances and feature sharing between prototype features of the same instance. Thus, after optimization, the training of the teacher model is complete.

[0049] Step 4: Transfer the prototype knowledge learned by the trained teacher model to the student model.

[0050] Based on the construction and training of the above model, multiple autonomous driving images are acquired and input into the student model in the image instance segmentation model. The feature extraction network in the student model extracts image instance features, and based on the prototype knowledge transferred to the student model, prototype knowledge specific to the image instance features is obtained in a query-based manner. Then, based on the obtained prototype knowledge and image instance features, the image instance segmentation result is output.

[0051] Specifically, based on the instance input characteristics of the image, a subset of prototype features P related to the instance is dynamically found. M ={P1,P2,…,P M}, 1≤M≤N. Feature encoding based on a pre-trained teacher model consists of two parts. First, a pre-trained feature extraction network is used to encode multiple input images, obtaining the image feature set x. e Simultaneously, using the prototype feature subset P M P in equation (1) 0 The enhanced prototype feature set is obtained through the prototype knowledge acquisition network, and then compared with x. e By splicing, we get x j Compared to ordinary fine-tuning, x obtained based on prototype knowledge... j It has a higher feature learning ability, and then based on this feature x j Image instance segmentation is achieved by obtaining the corresponding prediction output through a lightweight student model. During this process, the weights of the teacher model are completely frozen and do not participate in optimization updates.

[0052] In this embodiment, based on the teacher model trained through steps 1-3 above, a student model is trained separately using the same dataset as the teacher model. The network parameters in the student model are independent of those in the teacher model. The student model has shallower network layers and fewer model parameters than the teacher model, enabling deployment on lightweight devices. During the separate training of the student model, it extracts features from the sample images. Based on the prototype knowledge transferred to the student model (obtained through teacher model training), it directly determines prototype features specific to the sample image instance features through dynamic lookup, and performs image instance segmentation based on these prototype features.

[0053] The above query method refers to treating the image to be segmented as a query. To dynamically select appropriate prototype knowledge for different inputs (i.e., to achieve instance-level querying), each piece of prototype knowledge is associated as a value with a learnable key, stored in shared memory space as key-value pairs: {(k1,P1),(k2,P2),…,(k...} M ,PM Ideally, such a query should be based on the input instance itself. Therefore, a simple query encoding function q(x), such as FFN, is introduced into the student model to encode the input image to be segmented into a key. The corresponding prototype knowledge is directly retrieved through the key value, and the corresponding feature x is obtained based on this prototype knowledge. j The corresponding predicted output is obtained through a lightweight student network.

[0054] Specifically, based on acquiring the corresponding prototype knowledge, the acquired prototype knowledge (i.e., prototype features) is combined with image features x. e By concatenating the features, we obtain feature x. j After splicing, the segmentation head network generates a pixel-level segmentation mask for each pixel position. The segmentation head network consists of two cascaded fully connected layers with ReLU activation functions, and uses the same classification head as formula (4) to predict a category label for each pixel, indicating which target category or background the pixel belongs to. The classification head is a fully connected layer, and finally outputs the image instance segmentation result.

[0055] This embodiment minimizes the error between the student model's training and the true label by transferring prototype knowledge, thereby minimizing the difference between the student model's prediction and the teacher model's soft objective. Minimizing the error with the true label is achieved through cross-entropy loss. The student model is trained and optimized using cross-entropy loss. The optimized student model has instance segmentation performance similar to the teacher model, but with fewer model parameters, making it effectively applicable to edge devices for image instance segmentation.

[0056] As another implementation, frequency prior knowledge can be added to help the model better learn task-specific prototypes. Specifically, a prototype frequency table H is maintained only during training of the t-th task. t ={h1,h2,…,h M Each entry represents a prototype feature P selected before task t. i The normalized frequency. Therefore, for input instance x, the set K of the top-N matching keys. x for:

[0057]

[0058] The query mechanism based on cosine similarity γ(·) for query and key matching is performed in an instance-based manner, making the entire framework task-independent. This means that the above method does not require explicit task boundaries during training and does not require task identifiers during testing, and can be transferred to different instance segmentation scenarios.

[0059] The superiority of the segmentation method described in this embodiment is further illustrated by the following simulation experiments. As shown in Table 1, the experiments were conducted using the publicly available MS-COCO (Microsoft Common Objects in Context) dataset, and the average accuracy (AP) was measured using multiple IOU thresholds. IOU is the ratio of the intersection to the union of instance predictions and manually annotated targets. AP, AP50, and AP75 represent the evaluation using all, greater than 0.5, and greater than 0.75 instance predictions, respectively.

[0060] Table 1 compares the accuracy of instance segmentation results of the method described in this embodiment with other algorithms.

[0061]

[0062] Compared with existing technologies, the method described in this embodiment greatly reduces overfitting and improves the performance of instance segmentation.

[0063] Example 2

[0064] This embodiment provides an autonomous driving image instance segmentation system based on prototype contrastive learning, including:

[0065] The image acquisition module is used to acquire multiple images of the autonomous vehicle.

[0066] The image instance segmentation module is used to input the acquired image into the student model in the image instance segmentation model, extract image instance features, and obtain prototype knowledge specific to image instance features based on the prototype knowledge transferred to the student model in a query-based manner. Then, based on the obtained prototype knowledge and image instance features, the module outputs the image instance segmentation result.

[0067] The image instance segmentation model includes a teacher model and a student model. The teacher model includes a feature extraction network and a prototype knowledge acquisition network. The feature extraction network extracts features from the input sample image, and the prototype knowledge acquisition network learns prototype knowledge similar to the features of the sample image. The teacher model is trained based on contrastive learning, and the prototype knowledge learned by the trained teacher model is transferred to the student model.

[0068] Example 3

[0069] This embodiment provides an electronic device, including a memory and a processor, as well as computer instructions stored in the memory and running on the processor. When the processor executes the computer instructions, it completes the steps in the prototype-based contrastive learning-based autonomous driving image instance segmentation method described above.

[0070] Example 4

[0071] This embodiment also provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the steps in the prototype-based contrastive learning-based autonomous driving image instance segmentation method described above.

[0072] The steps and methods described in Embodiments 2 to 4 above correspond to those in Embodiment 1. For detailed implementation methods, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor, and enabling the processor to perform any method of the present invention. Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0073] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0074] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for segmenting autonomous driving image instances based on prototype contrastive learning, characterized in that, include: Multiple autonomous driving images are acquired and input into the student model in the image instance segmentation model. Image instance features are extracted, and prototype knowledge specific to the image instance features is obtained based on the prototype knowledge transferred to the student model and a query method. Then, based on the obtained prototype knowledge and image instance features, the image instance segmentation result is output. The image instance segmentation model includes a teacher model and a student model. The teacher model includes a feature extraction network and a prototype knowledge acquisition network. The feature extraction network extracts features from the input sample image, and the prototype knowledge acquisition network learns prototype knowledge similar to the features of the sample image. The teacher model is trained based on contrastive learning, and the prototype knowledge learned by the trained teacher model is transferred to the student model. The method of acquiring prototype knowledge that is similar to the features of sample images learned by the network includes: A set of randomly initialized prototype features is preset. Based on the features of the sample image, the relevant instances of each prototype feature are located, and visual features specific to the sample image instances are calculated and used as new prototype features. Using the new prototype features as the initial prototype features, the process is iterated continuously to eventually obtain prototype features specific to the sample image instance.

2. The autonomous driving image instance segmentation method based on prototype contrastive learning as described in claim 1, characterized in that, The training of the teacher model based on contrastive learning includes: Construct a contrastive loss function between sample image features and prototype features, and then convert the sample image features... q Similar prototype features Close the distance, and identify the dissimilar prototype features. To increase the distance, the loss function is: in, It is a temperature hyperparameter.

3. The autonomous driving image instance segmentation method based on prototype contrastive learning as described in claim 1, characterized in that, The query-based method for obtaining prototype knowledge specific to image instance features includes: The image is treated as a query, and each transferred prototype knowledge is associated as a value with a learnable key, stored in a shared memory space in the form of key-value pairs. The image to be segmented input into the student model is encoded into a key, and the corresponding prototype knowledge can be retrieved directly through the key value.

4. The autonomous driving image instance segmentation method based on prototype contrastive learning as described in claim 3, characterized in that it further... include: Based on the acquired prototype knowledge and the features of the input image, the prototype knowledge and image features are concatenated to obtain the concatenated features. Based on the splicing features, a lightweight student network is used to obtain the corresponding prediction output, thereby achieving image instance segmentation.

5. An image instance segmentation system for autonomous driving based on prototype contrastive learning, characterized in that, include: The image acquisition module is used to acquire multiple images of the autonomous vehicle. The image instance segmentation module is used to input the acquired image into the student model in the image instance segmentation model, extract image instance features, and obtain prototype knowledge specific to image instance features based on the prototype knowledge transferred to the student model in a query-based manner. Then, based on the obtained prototype knowledge and image instance features, the module outputs the image instance segmentation result. The image instance segmentation model includes a teacher model and a student model. The teacher model includes a feature extraction network and a prototype knowledge acquisition network. The feature extraction network extracts features from the input sample image, and the prototype knowledge acquisition network learns prototype knowledge similar to the features of the sample image. The teacher model is trained based on contrastive learning, and the prototype knowledge learned by the trained teacher model is transferred to the student model. The method of acquiring prototype knowledge that is similar to the features of sample images learned by the network includes: A set of randomly initialized prototype features is preset. Based on the features of the sample image, the relevant instances of each prototype feature are located, and visual features specific to the sample image instances are calculated and used as new prototype features. Using the new prototype features as the initial prototype features, the process is iterated continuously to eventually obtain prototype features specific to the sample image instance.

6. The autonomous driving image instance segmentation system based on prototype contrastive learning as described in claim 5, characterized in that, The query-based method for obtaining prototype knowledge specific to image instance features includes: The image is treated as a query, and each transferred prototype knowledge is associated as a value with a learnable key, stored in a shared memory space in the form of key-value pairs. The image to be segmented input into the student model is encoded into a key, and the corresponding prototype knowledge can be retrieved directly through the key value.

7. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, complete the steps of an autonomous driving image instance segmentation method based on prototype contrastive learning as described in any one of claims 1-4.

8. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, complete the steps of an autonomous driving image instance segmentation method based on prototype contrastive learning as described in any one of claims 1-4.

Images