Segmentation detection model training method, segmentation detection method, device and equipment

Abstract

Embodiments of the invention provide an image segmentation model training method and apparatus, and an electronic device. The method comprises the steps of obtaining a labeled sample set and an unlabeled image set; performing distillation training on the student model by using the teacher model and the sample images in the labeled sample set to obtain a student model after distillation training; the student model after distillation training is used for screening the unlabeled image set, a difficult sample image is obtained, and the difficult sample image represents an initial image whose prediction confidence is lower than a preset confidence threshold when the student model after distillation training is used for prediction; according to the method, the difficult sample image is acquired, the annotation information corresponding to the difficult sample image is acquired, the student model after distillation training is trained by using the difficult sample image and the corresponding annotation information, the trained segmentation detection model is acquired, the annotation cost can be effectively saved through the training method, and the image segmentation model obtained through training is more accurate.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a training method, segmentation detection method, apparatus, and device for a segmentation detection model. Background Technology

[0002] Image segmentation is a core technology in computer vision, aiming to divide an image into multiple regions or objects with consistent meaning. It has wide applications in medical imaging, autonomous driving, and industrial inspection. With the development of deep learning, methods based on convolutional neural networks (CNNs), such as U-Net, Mask R-CNN, and YOLOv8-seg, have become mainstream in image segmentation and have achieved significant progress in multiple tasks. However, these deep learning methods typically require large amounts of labeled data for training, especially in image segmentation tasks, where pixel-level accurate annotation is not only labor-intensive but also costly. Summary of the Invention

[0003] In view of this, embodiments of this application propose a training method, segmentation detection method, apparatus, and device for a segmentation detection model, which can effectively save annotation costs while improving the accuracy of the trained image segmentation model.

[0004] In a first aspect, embodiments of this application provide a training method for a segmentation detection model. The method includes: acquiring a labeled sample set and an unlabeled image set, wherein the labeled sample set includes multiple sample images and segmentation detection result labels corresponding to each sample image; the unlabeled image set includes multiple initial images; performing distillation training on a student model using a teacher model and the sample images in the labeled sample set to obtain a distilled-trained student model; using the distilled-trained student model to filter the unlabeled image set to obtain difficult sample images, wherein the difficult sample images represent initial images in the unlabeled image set whose prediction confidence is lower than that of other initial images in the unlabeled image set excluding the difficult sample images when using the distilled-trained student model for prediction; acquiring the annotation information corresponding to the difficult sample images; and training the distilled-trained student model using the difficult sample images and the corresponding annotation information to obtain a trained segmentation detection model.

[0005] Secondly, embodiments of this application provide a segmentation detection method, the method comprising: acquiring an image to be detected; performing segmentation detection on the image to be detected using a segmentation detection model to obtain a segmentation detection result of the image to be detected, wherein the segmentation detection model is obtained by training a student model trained by distillation using difficult sample images, the student model trained by distillation is obtained by training the student model based on a teacher model and a labeled sample set, the difficult sample images are obtained by filtering an unlabeled image set using the student model trained by distillation, and the difficult sample images represent images whose prediction confidence is lower than other images in the unlabeled image set except for the difficult sample images when making predictions using the student model trained by distillation.

[0006] Thirdly, embodiments of this application provide a training apparatus for a segmentation detection model. The apparatus includes: a data acquisition module for acquiring a labeled sample set and an unlabeled image set, wherein the labeled sample set includes multiple sample images and segmentation detection result labels corresponding to each sample image, and the unlabeled image set includes multiple initial images; a first training module for distilling a student model using a teacher model and the sample images in the labeled sample set to obtain a distilled-trained student model; an image filtering module for filtering the unlabeled image set using the distilled-trained student model to obtain difficult sample images, wherein the difficult sample images are those whose prediction confidence is lower than that of other initial images in the unlabeled image set excluding the difficult sample images when using the distilled-trained student model for prediction; a labeling information acquisition module for acquiring labeling information for the difficult sample images; and a sample update module for training the distilled-trained student model using the difficult sample images and the corresponding labeling information to obtain a trained segmentation detection model.

[0007] In one possible implementation, the first segmentation detection result includes the probability that the segmented object in the initial image belongs to multiple classification categories. The image filtering module includes a first segmentation detection submodule, an uncertainty value acquisition submodule, a confidence value acquisition submodule, and an image filtering submodule. The first segmentation detection submodule is used to perform segmentation detection on multiple initial images in the unlabeled image set using the student model trained by distillation to obtain a first segmentation detection result for each initial image. The uncertainty value acquisition submodule is used to determine the prediction uncertainty value of the initial image based on the first segmentation detection result. The confidence value acquisition submodule is used to obtain the prediction confidence value of the initial image based on the prediction uncertainty value of the initial image, wherein the prediction uncertainty is negatively correlated with the prediction confidence value. The image filtering submodule is used to filter out hard sample images from the multiple initial images based on the prediction confidence values ​​of each initial image.

[0008] In one possible implementation, the confidence value acquisition submodule includes a segmentation detection unit, a difficulty value acquisition unit, and a confidence value acquisition unit; the segmentation detection unit is used to perform segmentation detection on multiple validation samples in the labeled sample set using the student model to obtain a second segmentation detection result for each validation sample; the difficulty value acquisition unit is used to obtain the prediction difficulty value of the student model on multiple classification categories based on the second segmentation detection result and the segmentation detection result label of each validation sample; the confidence value acquisition unit is used to obtain the prediction confidence of the initial image based on the prediction uncertainty value of the initial image, the probability that the segmented object in the initial image belongs to multiple classification categories, and the prediction difficulty value of the student model on multiple classification categories.

[0009] In one possible implementation, the segmentation object includes segmentation detection boxes or pixels in the initial image; if the segmentation object is a pixel, the confidence value acquisition unit is further configured to, for each pixel in the initial image, multiply the probability of the pixel belonging to multiple classification categories by the prediction difficulty value of the student model in the multiple classification categories and then add them together to obtain the prediction difficulty value of each pixel in the initial image; obtain the prediction difficulty value of the initial image based on the prediction difficulty value of each pixel in the initial image; obtain the prediction confidence of the initial image based on the prediction difficulty value and the prediction uncertainty value, wherein the prediction difficulty value and the prediction uncertainty value are negatively correlated with the prediction confidence value.

[0010] In one possible implementation, if the segmentation object is a segmentation detection box, the confidence acquisition unit is further configured to perform segmentation detection on each initial image in the unlabeled image set using the student model trained by distillation, to obtain a segmentation detection result containing one or more segmentation detection boxes; the segmentation detection result includes the probability distribution of the objects contained in the detection boxes belonging to each category, and the confidence score of the detection boxes; based on the probability distribution of the objects contained in each detection box belonging to each category, the uncertainty value of each detection box is obtained; based on the uncertainty value and confidence score of each detection box in the initial image, the uncertainty value of the initial image is obtained; based on the prediction difficulty value and prediction uncertainty value of the initial image, the prediction difficulty value and prediction uncertainty value are negatively correlated with the prediction confidence value.

[0011] In one possible implementation, the confidence level acquisition unit is further configured to select N initial images with the highest prediction uncertainty values ​​and M images with the highest prediction difficulty values ​​from the plurality of initial images as difficult sample images, where M and N are positive integers; or to perform a weighted summation of the prediction difficulty value and prediction uncertainty value of each initial image to obtain a difficulty score for each initial image, and select K images with the highest difficulty scores as difficult sample images, where K is a positive integer.

[0012] In one possible implementation, the uncertainty value acquisition submodule includes a distribution entropy calculation unit and an uncertainty calculation unit; the distribution entropy calculation unit is used to obtain the probability distribution entropy of each segmented object in the initial image based on the probability that the segmented object indicated by the first segmentation detection result belongs to multiple classification categories; the uncertainty value calculation unit is used to calculate the prediction uncertainty value of the initial image based on the probability distribution entropy of each segmented object in the initial image.

[0013] In one possible implementation, the uncertainty value calculation unit is used to calculate the mean of the probability distribution entropy of each segmented object in the initial image to obtain the average entropy, and determine the prediction uncertainty of the initial image based on the average entropy, wherein the prediction uncertainty is positively correlated with the average entropy; or when the segmented object of the initial image is a pixel, the proportion of pixels in the initial image whose probability distribution entropy is greater than a preset entropy value is counted, and the prediction uncertainty of the initial image is determined according to the proportion of the pixels, wherein the prediction uncertainty is positively correlated with the proportion of the pixels.

[0014] In one possible implementation, the apparatus further includes an image deletion module for deleting the difficult sample images from the unlabeled image set.

[0015] Fourthly, embodiments of this application provide a segmentation detection device, the device including an image acquisition module for acquiring an image to be detected; and a segmentation detection module for using a segmentation detection model to segment and detect the image to be detected, thereby obtaining a segmentation detection result of the image to be detected. The segmentation detection model is obtained by training a student model trained with distillation using difficult sample images. The student model trained with distillation is based on a teacher model and a labeled sample set. The difficult sample images are obtained by filtering an unlabeled image set using the student model trained with distillation. The difficult sample images represent images whose prediction confidence is lower than that of other images in the unlabeled image set except for the difficult sample images when using the student model trained with distillation for prediction.

[0016] Fifthly, embodiments of this application provide an electronic device, including a processor and a memory; one or more programs are stored in the memory and configured to be executed by the processor to implement the above-described method.

[0017] Sixthly, embodiments of this application provide a computer-readable storage medium storing program code, wherein the above-described method is executed when the program code is run by a processor.

[0018] In a seventh aspect, embodiments of this application provide a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device retrieves the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method described above.

[0019] This application provides a training method, apparatus, and device for a segmentation detection model. The method includes: acquiring a labeled sample set and an unlabeled image set; performing distillation training on a student model using a teacher model and sample images from the labeled sample set to obtain a distilled-trained student model; using the distilled-trained student model to filter the unlabeled image set to obtain difficult sample images, where difficult sample images represent images whose prediction confidence is lower than other initial images in the unlabeled image set excluding difficult sample images when predicted using the distilled-trained student model; acquiring the annotation information corresponding to the difficult sample images; and using the difficult sample images and their corresponding annotation information to train the distilled-trained student model to obtain a trained segmentation detection model. By intelligently selecting and annotating the most challenging "difficult sample images" (i.e., images with high model prediction uncertainty) using the above method, the amount of data annotation can be effectively reduced. Simultaneously, in each iteration, the model can focus on learning difficult sample images, significantly improving the segmentation accuracy of the model in complex, edge cases, and ambiguous regions. This also avoids wasting annotation and computational resources on simple samples during training, thereby effectively improving model training efficiency and the accuracy of the trained model in segmentation detection. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1A flowchart illustrating a training method for a segmentation detection model provided in an embodiment of this application is shown. Figure 2 This illustration shows another flowchart of a training method for a segmentation detection model provided in an embodiment of this application; Figure 3 It shows Figure 1 A flowchart illustrating step S130; Figure 4 It shows Figure 3 A flowchart illustrating step S340; Figure 5 It shows Figure 4 A flowchart illustrating step S346; Figure 6 It shows Figure 4 Another flowchart of step S346; Figure 7 A flowchart illustrating a training method for a segmentation detection model provided in an embodiment of this application is shown. Figure 8 Another flowchart of a training method for a segmentation detection model provided in an embodiment of this application is shown; Figure 9 This illustration shows another flowchart of a training method for a segmentation detection model provided in an embodiment of this application; Figure 10 A flowchart illustrating a segmentation detection method provided in an embodiment of this application is shown; Figure 11 This paper shows a connection block diagram of a training device for a segmentation detection model according to an embodiment of this application; Figure 12 The diagram shows a connection block diagram of a segmentation detection device according to an embodiment of this application; Figure 13 A structural block diagram of an electronic device for performing the methods of embodiments of this application is shown. Detailed Implementation

[0022] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0023] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0024] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0025] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0026] It should be noted that "multiple" in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0027] It should also be noted that in this embodiment of the application, the collection, use, processing and storage of application information are all subject to the user's permission and must comply with the regulations of the region.

[0028] Figure 1 Specifically, this application illustrates a training method for a segmentation detection model, which can be applied to electronic devices, such as servers or terminals, that have data processing capabilities.

[0029] The methods include: Step S110: Obtain the labeled sample set and the unlabeled image set.

[0030] The labeled sample set includes multiple sample images and the segmentation detection result label corresponding to each sample image, while the unlabeled image set includes multiple initial images.

[0031] The segmentation and detection results labels for sample images in the labeled sample set can include pixel-level segmentation labels (i.e., each pixel is assigned a category label; the final label file is usually a mask with the same size as the original image, which can be filled with different color blocks, each color corresponding to a category), or segmentation information such as the position, size, and category of at least one bounding box (in this segmentation method, a rectangle is usually used to roughly outline the position and range of the target object). This segmentation method can divide an image into several non-overlapping regions, ensuring that pixels within each region have similar features. This task is commonly used in fields such as medical image analysis, autonomous driving, and remote sensing image processing.

[0032] Obtaining labeled sample sets can be achieved through various methods. These include acquiring sample images and obtaining labeled sample sets by professional annotation teams / companies using annotation tools; alternatively, they can be obtained from publicly available data. Data annotation refers to applying manual or semi-automated methods to a dataset to provide the label information needed for training machine learning models. In image segmentation tasks, data annotation typically involves assigning a label to each pixel in an image, or dividing the objects to be segmented into bounding boxes and labeling the categories, sizes, and positions of the objects within the boxes. This allows the model to learn how to perform effective segmentation. The methods described above for obtaining labeled sample sets are merely illustrative.

[0033] Unlabeled image sets can be obtained by acquiring images from business systems (vehicles in vehicle management systems, image acquisition equipment in medical institutions, or testing equipment in industrial testing systems, etc.); or by crawling publicly available images from the Internet. The above methods for obtaining unlabeled images are only illustrative and can be set according to actual needs.

[0034] Step S120: Distill the student model using the teacher model and the sample images in the labeled sample set to obtain the distilled student model.

[0035] The teacher model is typically a large, complex, high-performance but slow-running model (e.g., a deep CNN or Transformer). It has been thoroughly trained on a complete, high-quality set of labeled samples, and its predictions are highly accurate and reliable. The student model is a carefully designed or selected model that is more lightweight and efficient inference. Through knowledge distillation, it mimics the output behavior and internal features of the large teacher model, compressing the teacher model's "knowledge" into the lighter-weight student model. Ultimately, this allows the student model to provide high-performance prediction services in resource-constrained environments.

[0036] In one possible implementation, step S120 may specifically include: using the teacher model to segment and detect the sample image, obtaining a first sample segmentation and detection result; using the student model to segment and detect the sample image, obtaining a second sample segmentation and detection result; calculating a loss (e.g., cross-entropy loss, mean squared error loss, etc.) based on the second sample segmentation and detection result and the annotation information of the sample image to obtain a prediction loss; calculating a loss (e.g., contrast loss, similarity matching loss, and relative entropy, etc.) based on the first and second sample segmentation and detection results of the sample image to obtain a distillation loss; and weighting and summing the prediction loss and the distillation loss to obtain the model loss, thereby adjusting the model parameters of the student model based on the model loss (updating the model parameters of the student model through the backpropagation algorithm) to minimize the model loss, thus obtaining the student model after distillation training.

[0037] In another possible implementation, step S120 above can also be as follows: The teacher model performs forward propagation on the sample image, extracting intermediate feature maps of one or more layers in its network as teacher features; the student model performs forward propagation on the same sample image, extracting intermediate feature maps of the corresponding layers in its network as student features. Based on the student features and teacher features, loss calculations (e.g., mean squared error loss, cosine similarity loss, etc.) are performed to obtain feature matching loss; the student model performs segmentation detection on the sample image to obtain sample segmentation detection results, and based on these results and the annotation information of the sample image, loss calculations (e.g., cross-entropy loss calculation) are performed to obtain prediction loss; the feature matching loss and prediction loss are weighted and summed to obtain the total model loss, and the model parameters of the student model are adjusted based on the total model loss (updating the model parameters of the student model through the backpropagation algorithm) to minimize the total model loss, resulting in the student model after distillation training.

[0038] It is worth mentioning that in both of the above methods, the segmentation detection results output by the student model and the teacher model include the probability of the segmented object (such as a pixel or an object in the detection box) belonging to multiple classification categories. That is, the segmentation detection results output by the student model and the teacher model are soft labels.

[0039] The above process of distilling the student model to obtain the distilled student model is only illustrative. There are many other distillation training methods, which are not specifically limited here.

[0040] Step S130: Use the student model trained by distillation to filter the unlabeled image set to obtain difficult sample images. The difficult sample images indicate that the prediction confidence is lower than a preset confidence threshold when using the student model trained by distillation.

[0041] In one possible implementation, the student model trained by the distillation can be used to classify and predict the unlabeled image set to obtain the probability distribution of each unlabeled image belonging to each category. The prediction confidence is determined based on the probability distribution entropy of the unlabeled images, wherein the probability distribution entropy is negatively correlated with the prediction confidence.

[0042] The preset reliability threshold can be a fixed value or it can be dynamically changed.

[0043] If the preset reliability threshold is dynamically changed, it can start from a low initial value and gradually increase linearly or non-linearly with each iteration. In one implementation, the preset reliability threshold can be determined based on the initial threshold and the number of iterations of the student model, with the preset reliability decreasing as the number of iterations increases. In another implementation, the preset reliability threshold for the current iteration can also be determined based on the preset reliability threshold of the previous iteration, and the preset reliability threshold for the current iteration is lower than that of the previous iteration. By adopting this setting, the model can filter difficult sample images that challenge the current capability boundary of the model, ensuring that the selected difficult sample images always have high learning value.

[0044] Step S140: Obtain the annotation information corresponding to the difficult sample image.

[0045] In this context, step S140 can be: obtaining annotation information for the difficult sample image obtained by professional annotators manually annotating the difficult sample image; obtaining detection results from segmentation and detection of the difficult sample image based on the teacher model, followed by correction of the detection results by professional annotators to obtain annotation information for the difficult sample image; obtaining detection results from segmentation and detection of the difficult sample image based on the teacher model, followed by annotation information for the difficult sample image by professional annotators based on the detection results; or obtaining annotation information by segmentation and annotation using the SAM segmentation model (Segment Anything Model).

[0046] Step S150: Use the difficult sample images and corresponding annotation information to train the student model after distillation training to obtain the trained segmentation and detection model.

[0047] Specifically, the student model trained by distillation can be used to classify and predict difficult sample images to obtain prediction results. Based on the prediction results and the corresponding annotation information of the difficult sample images, loss is calculated. The student model trained by distillation is then adjusted based on the calculated loss to obtain the trained segmentation and detection model.

[0048] By employing the method described in this application, after distilling the student model, difficult sample images from the unlabeled image set are selected using the distilled student model. These difficult sample images are then used to train the distilled student model until the iteration training termination condition is met. It is evident that by adopting the method described in this application, only a small number of labeled samples are used initially, and only difficult sample images need to be selected and labeled subsequently, thus effectively reducing redundant labeling. Simultaneously, the introduction of model distillation technology, utilizing the soft labels of the teacher model to guide the student model training, further reduces the dependence on a large amount of labeled data, significantly reducing manual labeling costs while ensuring model performance. Furthermore, the joint optimization mechanism of distillation loss and true loss during distillation training allows the student model to acquire prior knowledge from the teacher model in the early stages of training, reducing the number of training iterations. At the same time, dynamically adjusting the uncertainty threshold of active learning allows the model to quickly adapt to complex data distributions as labeled data gradually accumulates, shortening the overall training cycle. In other words, the method described in this application can integrate active learning and model distillation to achieve integrated optimization of image segmentation model training and data labeling, reducing labeling costs and resulting in a more accurate image segmentation model.

[0049] In one possible implementation, such as Figure 2 The training method for the segmentation detection model includes: Step S210: Obtain a labeled sample set and an unlabeled image set. The labeled sample set includes multiple sample images and the segmentation detection result label corresponding to each sample image. The unlabeled image set includes multiple initial images.

[0050] Step S220: Use the teacher model and the sample images in the labeled sample set to perform distillation training on the student model to obtain the distilled student model.

[0051] For a detailed description of steps S210-S220, please refer to the previous description of steps S110 and S120, which will not be repeated here.

[0052] Step S230: Determine the prediction uncertainty value of each initial image based on the first segmentation detection result of each initial image obtained by segmenting and detecting multiple initial images in the unlabeled image set using the student model trained by the distillation.

[0053] In one possible implementation, step S130 can also involve running T predictions on the same initial image to obtain T probability distribution maps for each initial image (or the probability that the segmented object in each initial image belongs to each classification category). This allows for the calculation of the variance of the classification object in each category in the previous prediction. A larger variance indicates greater inconsistency in the model's multiple predictions and a higher uncertainty value. Finally, the mean of the variances for all categories and pixels is calculated, and the prediction uncertainty value is obtained based on the mean calculation result. The mean is positively correlated with the prediction uncertainty.

[0054] Step S240: Based on the prediction uncertainty value of each of the initial images, select difficult sample images from the multiple initial images.

[0055] In one possible implementation, the prediction confidence of the initial image is obtained based on the prediction uncertainty value of the initial image, wherein the prediction uncertainty is negatively correlated with the prediction confidence. Based on the prediction confidence of each initial image, difficult sample images are selected from the plurality of initial images.

[0056] In one possible implementation, the K initial images with the highest prediction uncertainty can be selected from multiple initial images as hard sample images.

[0057] In another possible implementation, the prediction difficulty value of the images can be used to filter samples. For example, the Intersection over Union (IoU) can be calculated using the prediction results obtained by the teacher model and the student model trained by distillation on each of the multiple initial images to obtain the prediction difficulty value (1-IoU) for each initial image. The prediction uncertainty value and prediction difficulty value of each image are then added together according to a preset weight to obtain its comprehensive screening score. Finally, the K initial images with the highest comprehensive screening scores are selected from the multiple initial images as difficult sample images.

[0058] Where K is an integer not less than 1. The Intersection over Union (IoU) ratio is a commonly used metric for evaluating image detection and segmentation performance, representing the degree of overlap between the detected or segmented results and the ground truth labeled regions. A higher IoU value indicates that the model's detection or segmentation results are closer to the actual labels, and the better the segmentation performance.

[0059] Step S250: Obtain the annotation information corresponding to the difficult sample image.

[0060] This annotation information is used to indicate the label of the segmentation detection result for difficult sample images.

[0061] In one possible implementation, the annotation information for the difficult sample images can be obtained by professional annotators manually annotating the difficult sample images. Alternatively, the detection results obtained from segmentation and detection of the difficult sample images based on the teacher model can be obtained, and then professional annotators can correct the detection results to obtain the annotation information for the difficult sample images. Another possibility is that after obtaining the detection results from segmentation and detection of the difficult sample images based on the teacher model, professional annotators annotate the difficult sample images based on the detection results. Finally, the annotation information can be obtained by segmenting and annotating using the SAM segmentation model (Segment Anything Model).

[0062] Step S260: Add the difficult sample image and its corresponding annotation information to the labeled sample set.

[0063] Then, it can be determined whether the conditions for ending the iterative training have been met.

[0064] The iteration termination condition includes at least one of the following: whether the number of iterations has reached a specified point, or whether the loss during the distillation training process is less than a preset loss.

[0065] If the iterative training termination condition is not met, return to step S220: use the teacher model and the sample images in the labeled sample set to perform distillation training on the student model; if the iterative training termination condition is met, use the student model after the last distillation training as the trained segmentation and detection model.

[0066] By employing the method described in this application, after distilling the student model, difficult sample images from the unlabeled image set are selected using the distilled student model. The annotation information of these difficult sample images is then obtained and added to the labeled sample set. The student model is then trained again using this method until the iteration training termination condition is met. It is evident that by using the method described in this application, only a small number of labeled samples are used initially, and only difficult sample images need to be selected and labeled subsequently, thus effectively reducing redundant annotations. Simultaneously, the introduction of model distillation technology utilizes the soft labels of the teacher model to guide the student model training, further reducing the dependence on a large amount of labeled data. This significantly reduces the cost of manual annotation while ensuring model performance. Furthermore, the joint optimization mechanism of distillation loss and true loss during distillation training allows the student model to acquire prior knowledge from the teacher model in the early stages of training, reducing the number of training iterations. Meanwhile, by dynamically adjusting the uncertainty threshold of active learning, the model can quickly adapt to complex data distributions as labeled data is gradually accumulated, thus shortening the overall training cycle. In other words, the above-mentioned method of this application can achieve the integration of active learning and model distillation to realize the integrated optimization of image segmentation model training and data labeling, reduce labeling costs, and make the trained image segmentation model more accurate.

[0067] It is worth mentioning that during distillation training, by using the soft labels output by the teacher model to supervise the training of the student model, model distillation can effectively guide the student model to learn the knowledge of the teacher model, especially for categories with less data distribution, significantly improving the model's predictive performance on these categories. The distillation process not only improves the accuracy of the student model but also effectively mitigates the bias caused by insufficient diversity in active learning, preventing oversampling or overfitting of certain categories, thereby improving the overall model's stability and accuracy. Simultaneously, combined with knowledge distillation, the teacher model provides more diverse predictive information, especially for categories with fewer or harder-to-label samples. The soft labels provided by distillation help the student model better learn difficult samples, avoiding bias in sample selection during training towards a particular class or feature, increasing the diversity of sample selection, and improving the efficiency of the training process and the model's generalization ability.

[0068] Furthermore, it can be combined with auxiliary annotation tools such as SAM to annotate difficult sample images, significantly improving annotation efficiency and making it widely applicable to fields with high precision requirements such as medical imaging and autonomous driving.

[0069] Please see Figure 3 In one possible implementation, the first segmentation detection result includes the probability that the segmented object in the initial image belongs to multiple classification categories, and step 130 (steps S230-S240) includes: Step S310: Use the student model trained by distillation to segment and detect the initial images in the unlabeled image set to obtain the first segmentation and detection result.

[0070] The segmentation detection results can include multiple pixels and their respective categories. In this case, the student model trained by distillation is a semantic segmentation detection model, with a model structure such as U-Net (Convolutional Networks for Biomedical Image Segmentation), a semantic segmentation network, or a convolutional neural network. Alternatively, the segmentation detection results can include the position and size of the detection boxes, the object category within the boxes, and the confidence level. In this case, the student model trained by distillation is an object segmentation detection model, with a model structure such as the YOLO series models (You Only Look Once, a first-order detection segmenter based on CNN), Mask DINO (Mask Detection with Interleaved Transformers, a first-order detection segmenter based on Transformer), or the SSD model (SingleShot MultiBox Detector).

[0071] Step S320: Based on the probability that the segmented object belongs to multiple classification categories indicated by the first segmentation detection result, obtain the probability distribution entropy of each segmented object in the initial image.

[0072] Among them, the probability distribution entropy of the segmented object is usually called information entropy or Shannon entropy. It is a core concept in information theory and is used to measure the uncertainty, disorder, or information content of a probability distribution.

[0073] The more uniform the distribution of the probability that a segmented object belongs to multiple categories (i.e., the probability that a segmented object belongs to all categories is about the same), the higher the entropy value and the greater the uncertainty.

[0074] Step S330: Calculate the prediction uncertainty value of the initial image based on the probability distribution entropy of each segmented object in the initial image.

[0075] Specifically, the mean entropy of the probability distribution of each segmented object in the initial image can be calculated to obtain the average entropy. The prediction uncertainty of the initial image can be determined based on the average entropy, and the prediction uncertainty is positively correlated with the average entropy. Alternatively, when the segmented object of the initial image is a pixel, the proportion of pixels in the initial image with a probability distribution entropy greater than a preset entropy value can be counted. The prediction uncertainty of the initial image can be determined based on the proportion of pixels, and the prediction uncertainty is positively correlated with the proportion of pixels.

[0076] It should be understood that the above method for calculating the prediction uncertainty of the image is only illustrative, and there are many other prediction methods, which are not specifically limited here.

[0077] Step S340: Based on the prediction uncertainty of the initial image, obtain the prediction confidence of the initial image, wherein the prediction uncertainty is negatively correlated with the prediction confidence.

[0078] Specifically, step S340 above can be to directly calculate the prediction confidence based on the prediction uncertainty value of the initial image.

[0079] Please see Figure 4 In another possible implementation, step S340 includes: Step S342: Use the student model to segment and detect multiple validation samples in the labeled sample set to obtain the second segmentation and detection results of each validation sample.

[0080] The principle of using the student model to segment and detect samples can be found in the detailed description of steps S120-S130 above, and will not be repeated here.

[0081] Step S344: Based on the second segmentation detection result and the segmentation detection result label of each of the verification samples, obtain the prediction difficulty value of the student model on multiple classification categories.

[0082] In one possible implementation, the T prediction results for each validation sample in the validation set are obtained, and the uncertainty of each segmented object is obtained by the average probability distribution of the T predictions for each segmented object of each validation sample. The mean of the uncertainties of multiple segmented objects that are indicated by the segmentation detection result label is calculated to obtain the mean uncertainty of each category. The prediction difficulty value of each category is obtained based on the mean uncertainty of each category, wherein the mean uncertainty of the category is positively correlated with its prediction difficulty.

[0083] In another possible implementation, for each validation sample, a pixel-level confusion matrix between its predicted result and the true label is calculated, and this matrix is ​​aggregated across the entire validation set to obtain a global confusion matrix. Based on this global confusion matrix, the intersection-over-union (IoU) value for each classification category is calculated; finally, the prediction difficulty value for each classification category is obtained based on its IoU value, where the IoU value for a classification category is negatively correlated with its prediction difficulty.

[0084] Step S346: Based on the prediction uncertainty value of the initial image, the probability that the segmented object in the initial image belongs to multiple classification categories, and the prediction difficulty value of the student model on multiple classification categories, obtain the prediction confidence of the initial image.

[0085] In one possible implementation, the segmentation object includes segmentation detection boxes or pixels in the initial image; see also Figure 5 If the segmentation object is a pixel, step S346 above includes: Step S346a: For each pixel in the initial image, multiply the probability of the pixel belonging to multiple classification categories by the prediction difficulty value of the student model in multiple classification categories, and then add them together to obtain the prediction difficulty value of each pixel in the initial image.

[0086] Step S346b: Obtain the prediction difficulty value of the initial image based on the prediction difficulty value of each pixel in the initial image.

[0087] In one possible implementation, the prediction difficulty value of each pixel can be averaged to obtain the prediction difficulty value of the initial image. In another possible implementation, the number of pixels within multiple preset difficulty ranges can be counted, and the prediction difficulty values ​​of the pixels in the S preset difficulty ranges with the largest number of pixels can be multiplied to obtain the prediction difficulty value. In yet another possible implementation, the proportion of pixels with prediction difficulty values ​​greater than a specified threshold in the initial image can be obtained, and the prediction difficulty value of the initial image can be determined based on this proportion, wherein the larger the proportion, the higher the prediction difficulty value of the initial image.

[0088] Step S346c: Based on the prediction difficulty value and prediction uncertainty value of the initial image, obtain the prediction confidence value of the initial image, wherein the prediction difficulty value and prediction uncertainty value are negatively correlated with the prediction confidence value.

[0089] In one possible implementation, step S346c may be: weighted summing of the prediction difficulty value and prediction uncertainty value of each initial image to obtain the difficulty score of each initial image, taking the reciprocal of the difficulty score as the prediction confidence, and selecting the K images with the lowest confidence (that is, selecting the K images with the highest difficulty scores) as difficult sample images, where K is a positive integer.

[0090] In one possible implementation, please refer to Figure 6 If the segmentation object is a segmentation detection box; step S346 above includes: Step S344e: For each initial image in the unlabeled image set, the student model trained by distillation is used to perform segmentation detection on the initial image to obtain a segmentation detection result containing one or more segmentation detection boxes.

[0091] The segmentation detection results include the probability distribution of objects within the detection box belonging to each category, and the confidence score of the detection box.

[0092] Step S344f: Based on the probability distribution of the objects contained in each detection box belonging to each category, obtain the uncertainty value of each detection box.

[0093] The specific implementation principle is similar to step S344b above. Please refer to the detailed description of step S344b above. It will not be repeated here.

[0094] Step S344g: Obtain the uncertainty value of the initial image based on the uncertainty value and confidence score of each detection box in the initial image.

[0095] Specifically, the reference uncertainty value of each detection box can be obtained by weighted summing of the uncertainty value and confidence score of each detection box; the mean of the reference uncertainty values ​​of each detection box in the initial image can be calculated to obtain the uncertainty value of the initial image; or the maximum reference uncertainty value can be selected as the uncertainty value of the initial image based on the reference uncertainty values ​​of each detection box in the initial image.

[0096] Step S344h: Based on the prediction difficulty value and prediction uncertainty value of the initial image, obtain the prediction confidence value of the initial image. The prediction difficulty value and prediction uncertainty value are negatively correlated with the prediction confidence value.

[0097] For a detailed description of step S344h, please refer to the detailed description of step S344c mentioned above; it will not be repeated here.

[0098] By adopting the above process, in the process of selecting difficult sample images, the selection of difficult sample images from multiple initial images is combined with the difficulty of category classification and the uncertainty of prediction. This ensures the balance of difficult sample image selection and avoids the problem of model overfitting on specific categories. In addition, by introducing the category difficulty value as prior knowledge, the uncertainty measurement can be improved, and the sample selection bias caused by the insufficient performance of the student model in the initial situation can be avoided. Through the intermediate features of the teacher model and knowledge transfer, the student model can effectively learn the segmentation features in complex scenes. When the segmentation object is a pixel, the pixel-level annotation requirement of compatible instance segmentation can be achieved.

[0099] Furthermore, introducing class difficulty as prior knowledge helps the model more accurately select challenging samples for labeling. This improvement effectively reduces redundant labeled samples, lowers the cost of manual labeling, and improves the quality and efficiency of labeling. Moreover, by introducing class difficulty as prior knowledge, it not only comprehensively considers the uncertainty of categories during active learning but also selects samples from various categories more evenly, avoiding overfitting on certain categories and improving the model's generalization ability and training performance.

[0100] For example, the overall training process is as follows: Figure 7 As shown in the diagram, the dashed box represents an iterative training process. The labels on the edges in the flowchart indicate the order of execution, where steps ② and ③ can be swapped or performed in parallel. First, the student model is trained using a teacher model and sample images from the labeled sample set, resulting in a distilled student model. After training, each sub-step in step S130 is executed to calculate the prediction difficulty value of the student model across multiple classification categories, and inference is performed using unlabeled data. Subsequently, the inference result and category difficulty are used as input to an uncertainty algorithm to calculate the prediction uncertainty of the initial images in the unlabeled image set. Based on the prediction uncertainty of multiple initial images, the probability that the segmented objects in the initial images belong to multiple classification categories, and the prediction difficulty value of the student model across multiple classification categories, a subset of difficult sample images are selected for manual annotation to expand the labeled sample set. Then, a new round of training is performed using the expanded labeled sample set.

[0101] See also Figure 8 and Figure 9 As shown, taking the teacher model as a YOLO-v8 segmentation network as an example, the input images to this segmentation network (e.g., sample images or initial images) can include... Three-dimensional images, among which This refers to the number of channels; for red-green-blue format images, it is usually 3. and These are the height and width of the input image, respectively. The dimension is... The image, after being processed by a feature extraction and fusion network, yields three intermediate features at different scales, with dimensions of [dimensions to be filled in]. , , The intermediate features are input to the detection head and segmentation head respectively, yielding the classification category of the object in the detection box, the position and size of the bounding box, etc., where the detection head outputs a dimension of [missing information]. , To preset the number of object categories, For probability distribution granularity, The segmentation head outputs dimensions (i.e., width and height). Based on these results, valid bounding boxes are then filtered (this can be done based on a confidence threshold for the detection boxes), ultimately yielding the number of valid detection boxes, their positions and sizes, their categories, and their confidence levels.

[0102] During the training phase, one or more features extracted from sample images by the teacher model, such as intermediate layer features, the probability of segmented objects belonging to multiple categories, and bounding box information (position and size), can be used to guide the student model in training. During training, the student model's loss is a weighted average of prediction loss and distillation loss. After one round of training, a subset of difficult sample images are selected based on the prediction uncertainty of multiple initial images, the probability of segmented objects belonging to multiple categories in the initial images, and the prediction difficulty values ​​of the student model across multiple categories. For these difficult sample images, SAM (Segment Anything Model) can be used for auxiliary annotation to obtain segmentation mask labels more efficiently. The difficult sample images and their corresponding annotation information are then added to the labeled sample set, and the expanded labeled sample set is used for the next round of training.

[0103] In one possible implementation, after performing step S150, the method further includes deleting the difficult sample images from the unlabeled image set.

[0104] By removing the difficult sample images from the unlabeled image set, the same images can be repeatedly selected in subsequent active learning loops, thus ensuring that each round of screening is performed on a new, unlabeled candidate set. This allows each round to explore new and different difficult regions in the dataset, thereby achieving efficient exploration and avoiding the problem of repeated labeling.

[0105] Please see Figure 10 As shown in the embodiments of this application, an image segmentation and detection method is also provided, the method comprising: Step S410: Obtain the image to be detected.

[0106] The image to be detected can be any image that needs to be segmented and detected. It can be obtained from the business system or uploaded by the user through the terminal. The above acquisition methods are only illustrative and are not specifically limited here.

[0107] Step S410: Use the segmentation detection model to segment and detect the image to be detected, and obtain the segmentation detection result of the image to be detected.

[0108] The segmentation detection model is obtained by training the student model after distillation training using difficult sample images. The student model after distillation training is obtained by training the student model based on the teacher model and the labeled sample set. The difficult sample images are obtained by filtering the unlabeled image set using the student model after distillation training. The difficult sample images represent the initial images whose prediction confidence is lower than a preset confidence threshold when using the student model after distillation training for prediction.

[0109] That is, the segmentation detection model is trained using the aforementioned training method for the segmentation detection model.

[0110] The principle of using the segmentation detection model to segment and detect the image to be detected is the same as the principle of using the student model to segment and detect the initial image, sample image and verification sample mentioned above, and will not be repeated here.

[0111] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0112] Please see Figure 10 Another embodiment of this application provides a training device 500 for a segmentation detection model. The device includes: a data acquisition module 510, used to acquire a labeled sample set and an unlabeled image set, wherein the labeled sample set includes multiple sample images and segmentation detection result labels corresponding to each sample image, and the unlabeled image set includes multiple initial images; a first training module 320, used to perform distillation training on a student model using a teacher model and the sample images in the labeled sample set to obtain a distilled-trained student model; an image filtering module 530, used to filter the unlabeled image set using the distilled-trained student model to obtain difficult sample images, wherein the difficult sample images represent initial images whose prediction confidence is lower than a preset confidence threshold when using the distilled-trained student model for prediction; a labeling information acquisition module 540, used to acquire labeling information for the difficult sample images; and a second training module 550, used to train the distilled-trained student model using the difficult sample images and the corresponding labeling information to obtain a trained segmentation detection model.

[0113] In one possible implementation, the first segmentation detection result includes the probability that the segmented object in the initial image belongs to multiple classification categories. The image filtering module 53 includes a first segmentation detection submodule, an uncertainty value acquisition submodule, a confidence value acquisition submodule, and an image filtering submodule. The first segmentation detection submodule is used to perform segmentation detection on multiple initial images in the unlabeled image set using the student model trained by distillation to obtain a first segmentation detection result for each initial image. The uncertainty value acquisition submodule is used to determine the prediction uncertainty value of the initial image based on the first segmentation detection result. The confidence value acquisition submodule is used to obtain the prediction confidence value of the initial image based on the prediction uncertainty value of the initial image, wherein the prediction uncertainty is negatively correlated with the prediction confidence value. The image filtering submodule is used to filter out difficult sample images from the multiple initial images based on the prediction confidence values ​​of each initial image.

[0114] In one possible implementation, the confidence value acquisition submodule includes a segmentation detection unit, a difficulty value acquisition unit, and a confidence value acquisition unit; the segmentation detection unit is used to perform segmentation detection on multiple validation samples in the labeled sample set using the student model to obtain a second segmentation detection result for each validation sample; the difficulty value acquisition unit is used to obtain the prediction difficulty value of the student model on multiple classification categories based on the second segmentation detection result and the segmentation detection result label of each validation sample; the confidence value acquisition unit is used to obtain the prediction confidence of the initial image based on the prediction uncertainty value of the initial image, the probability that the segmented object in the initial image belongs to multiple classification categories, and the prediction difficulty value of the student model on multiple classification categories.

[0115] In one possible implementation, the segmentation object includes segmentation detection boxes or pixels in the initial image; if the segmentation object is a pixel, the confidence value acquisition unit is further configured to, for each pixel in the initial image, multiply the probability of the pixel belonging to multiple classification categories by the prediction difficulty value of the student model in the multiple classification categories and then add them together to obtain the prediction difficulty value of each pixel in the initial image; obtain the prediction difficulty value of the initial image based on the prediction difficulty value of each pixel in the initial image; obtain the prediction confidence of the initial image based on the prediction difficulty value and the prediction uncertainty value, wherein the prediction difficulty value and the prediction uncertainty value are negatively correlated with the prediction confidence value.

[0116] In one possible implementation, if the segmentation object is a segmentation detection box, the confidence acquisition unit is further configured to perform segmentation detection on each initial image in the unlabeled image set using the student model trained by distillation, to obtain a segmentation detection result containing one or more segmentation detection boxes; the segmentation detection result includes the probability distribution of the objects contained in the detection boxes belonging to each category, and the confidence score of the detection boxes; based on the probability distribution of the objects contained in each detection box belonging to each category, the uncertainty value of each detection box is obtained; based on the uncertainty value and confidence score of each detection box in the initial image, the uncertainty value of the initial image is obtained; based on the prediction difficulty value and prediction uncertainty value of the initial image, the prediction difficulty value and prediction uncertainty value are negatively correlated with the prediction confidence value.

[0117] In one possible implementation, the confidence level acquisition unit is further configured to select N initial images with the highest prediction uncertainty values ​​and M images with the highest prediction difficulty values ​​from the plurality of initial images as difficult sample images, where M and N are positive integers; or to perform a weighted summation of the prediction difficulty value and prediction uncertainty value of each initial image to obtain a difficulty score for each initial image, and select K images with the highest difficulty scores as difficult sample images, where K is a positive integer.

[0118] In one possible implementation, the uncertainty value acquisition submodule includes a distribution entropy calculation unit and an uncertainty calculation unit; the distribution entropy calculation unit is used to obtain the probability distribution entropy of each segmented object in the initial image based on the probability that the segmented object indicated by the first segmentation detection result belongs to multiple classification categories; the uncertainty value calculation unit is used to calculate the prediction uncertainty value of the initial image based on the probability distribution entropy of each segmented object in the initial image.

[0119] In one possible implementation, the uncertainty value calculation unit is used to calculate the mean of the probability distribution entropy of each segmented object in the initial image to obtain the average entropy, and determine the prediction uncertainty of the initial image based on the average entropy, wherein the prediction uncertainty is positively correlated with the average entropy; or when the segmented object of the initial image is a pixel, the proportion of pixels in the initial image whose probability distribution entropy is greater than a preset entropy value is counted, and the prediction uncertainty of the initial image is determined according to the proportion of the pixels, wherein the prediction uncertainty is positively correlated with the proportion of the pixels.

[0120] In one possible implementation, the apparatus further includes an image deletion module for deleting the difficult sample images from the unlabeled image set.

[0121] Please see Figure 12As shown, another embodiment of this application provides a segmentation detection device 600, the device comprising: an image acquisition module 610 for acquiring an image to be detected; and a segmentation detection module 620 for using a segmentation detection model to segment and detect the image to be detected, thereby obtaining a segmentation detection result of the image to be detected. The segmentation detection model is obtained by training a student model trained with distillation using difficult sample images. The student model trained with distillation is based on a teacher model and a labeled sample set. The difficult sample images are obtained by filtering the unlabeled image set using the student model trained with distillation. The difficult sample images represent images whose prediction confidence is lower than that of other images in the unlabeled image set except for the difficult sample images when using the student model trained with distillation for prediction.

[0122] Each module in the above-described device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module. It should be noted that the device embodiments in this application correspond to the foregoing method embodiments. The specific principles of the device embodiments can be found in the foregoing method embodiments, and will not be repeated here.

[0123] The following will combine Figure 13 This application provides a description of an electronic device 100.

[0124] Please see Figure 13 Based on the methods provided in the above embodiments, this application also provides another electronic device 100 including a processor 102 capable of executing the aforementioned methods. The electronic device 100 may be a server or a terminal device.

[0125] The electronic device 100 also includes a memory 104. The memory 104 stores a program that can execute the contents of the foregoing embodiments, and the processor 102 can execute the program stored in the memory 104.

[0126] The processor 102 may include one or more cores for data processing and message matrix units. The processor 102 connects to various parts within the electronic device 100 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 104, and by calling data stored in the memory 104. Optionally, the processor 102 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 102 may integrate one or more of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 102 and may be implemented separately using a communication chip.

[0127] The memory 104 may include random access memory (RAM) or read-only memory (ROM). The memory 104 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 104 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing the various method embodiments described below, etc. The data storage area may also store data acquired by the electronic device 100 during use.

[0128] The electronic device 100 may also include a network module and a screen. The network module is used to receive and transmit electromagnetic waves, converting electromagnetic waves into electrical signals, thereby enabling communication with communication networks or other devices, such as audio playback devices. The network module may include various existing circuit elements used to perform these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, SIM cards, memory, etc. The network module can communicate with various networks such as the Internet, corporate intranets, and wireless networks, or communicate with other devices via wireless networks. The aforementioned wireless networks may include cellular telephone networks, wireless local area networks, or metropolitan area networks. The screen can display interface content and perform data interaction, such as displaying the aforementioned interface and triggering operations through the screen.

[0129] This application also provides a computer-readable storage medium. This computer-readable medium stores program code that can be called by a processor to execute the methods described in the above method embodiments.

[0130] Computer-readable storage media can be electronic storage devices such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, computer-readable storage media includes non-transitory computer-readable storage medium. The computer-readable storage medium has storage space for program code that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code can be compressed, for example, in a suitable form.

[0131] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods described in the various optional implementations above.

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A training method for a segmentation detection model, characterized in that, The method includes: Obtain a labeled sample set and an unlabeled image set. The labeled sample set includes multiple sample images and the segmentation detection result label corresponding to each sample image. The unlabeled image set includes multiple initial images. The student model is trained by distillation using the teacher model and sample images from the labeled sample set to obtain the distilled student model. The student model trained by distillation is used to filter the unlabeled image set to obtain difficult sample images. The difficult sample images are the initial images whose prediction confidence is lower than a preset confidence threshold when the student model trained by distillation is used for prediction. Obtain the annotation information corresponding to the difficult sample image; The student model trained by distillation is trained using the difficult sample images and their corresponding annotation information to obtain the trained segmentation and detection model.

2. The method according to claim 1, characterized in that, The step of using the student model trained by distillation to filter the unlabeled image set to obtain difficult sample images includes: The student model trained by distillation is used to segment and detect multiple initial images in the unlabeled image set to obtain the first segmentation and detection result for each initial image; Based on the first segmentation detection result, the prediction uncertainty value of the initial image is determined; The prediction confidence of the initial image is obtained based on the prediction uncertainty value of the initial image, and the prediction uncertainty is negatively correlated with the prediction confidence. Based on the prediction confidence of each initial image, difficult sample images are selected from the multiple initial images.

3. The method according to claim 2, characterized in that, The first segmentation detection result includes the probability that the segmented object in the initial image belongs to multiple classification categories; the prediction confidence of the initial image is obtained based on the prediction uncertainty value of the initial image, including: The student model is used to segment and detect multiple validation samples in the labeled sample set to obtain the second segmentation and detection results of each validation sample; Based on the second segmentation detection results and segmentation detection result labels of each of the aforementioned validation samples, the prediction difficulty value of the student model on multiple classification categories is obtained; The prediction confidence of the initial image is obtained based on the prediction uncertainty value of the initial image, the probability that the segmented object in the initial image belongs to multiple classification categories, and the prediction difficulty value of the student model on multiple classification categories.

4. The method according to claim 3, characterized in that, The segmentation object includes segmentation detection boxes or pixels in the initial image; If the segmentation object is a pixel, the prediction confidence of the initial image is obtained based on the prediction uncertainty value of each initial image, the probability that the segmentation object in the initial image belongs to multiple classification categories, and the prediction difficulty value of the student model on multiple classification categories, including: For each pixel in the initial image, the probability of that pixel belonging to multiple classification categories is multiplied by the prediction difficulty value of the student model in multiple classification categories, and then added together to obtain the prediction difficulty value of each pixel in the initial image. The prediction difficulty value of the initial image is obtained based on the prediction difficulty value of each pixel in the initial image; Based on the prediction difficulty value and prediction uncertainty value of the initial image, the prediction confidence value of the initial image is obtained, wherein the prediction difficulty value and prediction uncertainty value are negatively correlated with the prediction confidence value.

5. The method according to claim 3, characterized in that, If the segmentation object is a segmentation detection box; the prediction confidence of the initial image is obtained based on the prediction uncertainty value of each initial image, the probability that the segmentation object in the initial image belongs to multiple classification categories, and the prediction difficulty value of the student model on multiple classification categories, including: For each initial image in the unlabeled image set, the student model trained by distillation is used to segment and detect the initial image, resulting in a segmentation and detection result containing one or more segmentation detection boxes; the segmentation and detection result includes the probability distribution of the objects contained in the detection boxes belonging to each category, and the confidence score of the detection boxes; The uncertainty value of each detection box is obtained based on the probability distribution of the objects contained in each detection box belonging to each category. The uncertainty value of the initial image is obtained based on the uncertainty value and confidence score of each detection box in the initial image; Based on the prediction difficulty value and prediction uncertainty value of the initial image, the prediction confidence value of the initial image is obtained, wherein the prediction difficulty value and prediction uncertainty value are negatively correlated with the prediction confidence value.

6. The method according to claim 2, characterized in that, The step of determining the prediction uncertainty value of the initial image based on the first segmentation detection result includes: Based on the probability that the segmented object belongs to multiple classification categories indicated by the first segmentation detection result, the probability distribution entropy of each segmented object in the initial image is obtained; The prediction uncertainty value of the initial image is calculated based on the probability distribution entropy of each segmented object in the initial image.

7. An image segmentation and detection method, characterized in that, The method includes: Acquire the image to be detected; A segmentation detection model is used to segment and detect the image to be detected, and the segmentation detection result of the image to be detected is obtained. The segmentation detection model is trained on a student model after distillation training using difficult sample images. The student model after distillation training is trained on a teacher model and a labeled sample set. The difficult sample images are obtained by filtering the unlabeled image set using the student model after distillation training. The difficult sample images are those whose prediction confidence is lower than that of other images in the unlabeled image set except for the difficult sample images when the student model after distillation training makes a prediction.

8. A training device for a segmentation detection model, characterized in that, The device includes: The data acquisition module is used to acquire a labeled sample set and an unlabeled image set. The labeled sample set includes multiple sample images and the segmentation detection result label corresponding to each sample image. The unlabeled image set includes multiple initial images. The first training module is used to perform distillation training on the student model using the teacher model and sample images from the labeled sample set, to obtain the student model after distillation training. The image filtering module is used to filter the unlabeled image set using the student model trained by distillation to obtain difficult sample images. The difficult sample images are those whose prediction confidence is lower than that of other initial images in the unlabeled image set except for the difficult sample images when using the student model trained by distillation. The annotation information acquisition module is used to acquire annotation information for the difficult sample images; The second training module is used to train the student model trained by distillation using the difficult sample images and the corresponding annotation information to obtain the trained segmentation and detection model.

9. An image segmentation and detection device, characterized in that, The device includes: The image acquisition module is used to acquire the image to be detected; The segmentation detection module is used to segment and detect the image to be detected using a segmentation detection model, and obtain the segmentation detection result of the image to be detected. The segmentation detection model is obtained by training a student model after distillation training using difficult sample images. The student model after distillation training is based on a teacher model and a labeled sample set. The difficult sample images are obtained by filtering the unlabeled image set using the student model after distillation training. The difficult sample images are those whose prediction confidence is lower than that of other images in the unlabeled image set except for the difficult sample images when using the student model after distillation training for prediction.

10. An electronic device, characterized in that, include: One or more processors; Memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs being configured to perform the method as described in any one of claims 1-6 or 7.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code that can be invoked by a processor to perform the method as described in any one of claims 1-6 or 7.

12. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1-6 or 7.

Images