A semi-supervised image segmentation method based on adaptive pixel subdivision

Abstract

The application provides a kind of semi-supervised image segmentation method based on adaptive pixel subdivision, it is related to computer vision and machine learning technical field, including: first according to target scene acquisition image data and pre-processing, division training set and test set, and part of data is marked.The preliminary segmentation is carried out using the preset semantic segmentation model, the key pixel point area is screened by uncertainty analysis, and the segmentation precision is improved by combining multi-scale feature and iterative optimization.The teacher-student model framework is constructed, the pseudo label is generated using the teacher model, the high confidence sample is screened to optimize the student model, and the contrast learning strategy is introduced for the low confidence pixel point to enhance the model generalization ability.Finally, the image to be segmented is input into the optimized student model, and the category matrix is output to generate visual segmentation result, improve the segmentation precision and boundary detail performance, suitable for the scene with limited labeled data.

Description

Technical Field

[0001] This application belongs to the field of computer vision and machine learning technology, specifically relating to a semi-supervised image segmentation method based on adaptive pixel subdivision. Background Technology

[0002] Image segmentation is a key technology in computer vision, aiming to divide an image into multiple parts or regions, each corresponding to a specific object or part of an object. It has wide applications in various fields such as autonomous vehicles, medical diagnosis, satellite image analysis, and augmented reality. With the development of deep learning technology, especially the widespread application of convolutional neural networks (CNNs), image segmentation technology has made significant progress. However, current image segmentation methods mainly rely on fully supervised learning frameworks, which require a large amount of high-quality labeled data to train the model. Acquiring this data is not only time-consuming and labor-intensive but also requires the participation of professionals, resulting in high costs. Furthermore, existing methods often suffer from blurred or discontinuous edges when dealing with target boundaries in complex scenes, affecting the accuracy and visual quality of the segmentation results.

[0003] The inventors discovered that existing image segmentation methods typically require pixel-level precise annotations to train models, which poses a significant challenge for practical applications. Existing methods struggle to accurately capture object boundary details in complex scenarios, such as dynamic object segmentation or multi-scale object segmentation, leading to a decline in the quality of segmentation results. Summary of the Invention

[0004] This application provides a semi-supervised image segmentation method based on adaptive pixel subdivision to solve one of the aforementioned technical problems.

[0005] The technical solution adopted in this application is as follows:

[0006] This application provides a semi-supervised image segmentation method based on adaptive pixel subdivision, including:

[0007] Based on the target scene, the original image dataset is collected and preprocessed to generate training and test sets. The training set is divided into labeled and unlabeled data according to the proportion. The labeled data and test set are labeled to obtain labeled images.

[0008] The labeled image is initially segmented using a pre-defined semantic segmentation model to generate prediction results. Based on the uncertainty of the prediction results, key pixel regions are selected, multi-scale feature maps are extracted, and the key pixel regions are refined through iterative optimization to generate segmentation results.

[0009] Construct teacher and student models, train the teacher model using labeled data, generate pseudo-labels for unlabeled data using the teacher model, and select confidence pixels with confidence scores greater than a preset threshold for optimization of the student model.

[0010] Feature comparison learning is performed on pixels with confidence scores less than a preset threshold, and the parameters of the student model are optimized by constructing positive and negative samples and using a comparison loss function.

[0011] The image to be segmented is input into the optimized student model, which outputs a predicted category matrix and generates a visual segmentation result through color mapping.

[0012] According to one embodiment of this application, the acquisition of the original image dataset and its preprocessing to generate training and test sets include:

[0013] Image data is collected based on the target scene. The sources of image data include public datasets, internet resources, or custom-collected data.

[0014] Image data is sharpened by using Gaussian blur and sharpening algorithms to enhance image details;

[0015] Data augmentation of image data includes random rotation, flipping, and adding Gaussian noise;

[0016] Set the image size to the preset size.

[0017] Normalize pixel values ​​to the 0-1 range to eliminate differences in lighting or device;

[0018] The preprocessed image data is divided into training and test sets in an 8:2 ratio. The training set is further divided into labeled and unlabeled data in a 1:4 or 1:2 ratio.

[0019] According to one embodiment of this application, the preliminary segmentation of the labeled image using a preset semantic segmentation model to generate a prediction result includes:

[0020] The pre-defined DeepLabV3Plus model is used, with ResNet-101 as the encoder and ASPP module as the decoder;

[0021] Input the labeled image into the preset DeepLabV3Plus model, with the input size matching that of the preprocessed image;

[0022] The preset DeepLabV3Plus model outputs the class probability distribution of each pixel and generates prediction results.

[0023] According to one embodiment of this application, the step of calculating the uncertainty based on the prediction result and filtering out key pixel regions includes:

[0024] The probability entropy value is calculated for the prediction result of each pixel using the following formula:

[0025]

[0026] Where C represents the total number of categories. The probability of being class c;

[0027] Set an entropy threshold and select pixels with entropy values ​​higher than the threshold as key point regions.

[0028] According to one embodiment of this application, the step of extracting multi-scale feature maps and refining key pixel regions through iterative optimization includes:

[0029] Semantic segmentation models include the DeepLabV3Plus model;

[0030] Feature maps of different scales are extracted from the decoder of the DeepLabV3Plus model and enlarged to a uniform size using bilinear interpolation;

[0031] A small subnetwork is used to process the key point region in multiple rounds of iteration. In each round of iteration, the feature representation is updated based on the current prediction results, and the optimization strategy is dynamically adjusted.

[0032] According to one embodiment of this application, the construction of the teacher model and student model, and the training of the teacher model using labeled data, includes:

[0033] Both the teacher and student models employ an adaptive pixel-level subdivision feature extraction framework.

[0034] The teacher model is trained under supervised supervision using labeled data, and the loss function is cross-entropy loss.

[0035] During training, the teacher model parameters are kept constant, and only the student model parameters are optimized.

[0036] According to one embodiment of this application, the step of generating pseudo-labels for unlabeled data using a teacher model and selecting confidence pixels with a confidence level greater than a preset threshold for optimization of the student model includes:

[0037] The teacher model makes predictions on unlabeled data and outputs the class probability distribution for each pixel.

[0038] Calculate the entropy value of each pixel, and retain only pixels with entropy values ​​lower than a first preset threshold as confidence samples;

[0039] The prediction results of the confidence samples are used as pseudo-labels, and the student model is further optimized using the cross-entropy loss function.

[0040] According to one embodiment of this application, the feature comparison learning for confidence pixels with confidence levels less than a preset threshold includes:

[0041] Anchor features are extracted from confidence pixels with confidence scores greater than a preset threshold, and the average features of the corresponding categories are calculated as positive samples.

[0042] Eliminate category features with confidence scores exceeding a preset threshold, and treat the remaining category features as negative samples;

[0043] The model is trained by comparing positive and negative samples using a loss function, as shown in the formula:

[0044]

[0045] Where q represents the query feature. As a positive sample, For negative samples, This refers to the temperature parameter.

[0046] A second aspect of this application provides a computer-readable storage medium having a program stored thereon that, when executed by a processor, implements the steps described in the method.

[0047] A third aspect of this application provides an electronic device including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method as described.

[0048] Due to the adoption of the above technical solution, the beneficial effects achieved by this application are as follows:

[0049] This application collects raw image datasets based on the target application scenario and performs sharpening, data augmentation, resizing, and normalization operations on them to generate training and testing sets. The training set is further divided into labeled and unlabeled data proportionally, and both the labeled data and the testing set are annotated to obtain labeled images. This process ensures the quality and diversity of the data, laying the foundation for subsequent model training.

[0050] This application utilizes a pre-defined DeepLabV3Plus model to perform preliminary segmentation of labeled images and generate prediction results. Based on the uncertainty calculation of the prediction results, key pixel regions are selected. These regions are typically located at object boundaries and have high prediction uncertainty. By extracting multi-scale feature maps and refining the key pixel regions through iterative optimization, a refined segmentation result is finally generated. This method effectively improves the segmentation accuracy of object boundaries and solves the problems of blurred or discontinuous edges in traditional methods.

[0051] This application constructs teacher and student models. First, the teacher model is trained under supervision using labeled data. Then, the teacher model generates pseudo-labels for unlabeled data, and pixels with confidence scores greater than a preset threshold are selected for optimization of the student model. For pixels with confidence scores less than the preset threshold, a contrastive learning method is used to optimize the student model parameters through positive and negative sample construction and a contrastive loss function. This method fully utilizes unlabeled data, reduces reliance on high-quality labeled data, and improves the model's generalization ability.

[0052] This application inputs the image to be segmented into an optimized student model, outputs a predicted category matrix, and generates a visual segmentation result through color mapping. This intuitive presentation method facilitates user understanding and evaluation of the segmentation effect, enhancing the user experience.

[0053] This application introduces a low-confidence pixel reuse mechanism, enabling a large amount of unlabeled data to participate in the model training process, thereby reducing the need for high-quality labeled data and lowering the cost and difficulty of data collection. Adaptive pixel subdivision technology is employed to specifically optimize regions with high prediction uncertainty (usually object boundaries), significantly improving the accuracy of segmentation results, particularly when handling target boundaries in complex scenes. By performing feature contrastive learning on low-confidence pixels, the model maintains good performance even with unseen data, enhancing its generalization ability and robustness. The introduced small subnetworks and contrastive learning strategy reduce computational costs, making this method suitable for deployment on resource-constrained devices and improving the efficiency of model training. Attached Figure Description

[0054] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0055] Figure 1 A flowchart illustrating a semi-supervised image segmentation method based on adaptive pixel subdivision provided in this application embodiment;

[0056] Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0057] Figure 3 This diagram illustrates the iterative refinement process of a semi-supervised image segmentation method based on adaptive pixel subdivision, as provided in an embodiment of this application.

[0058] Figure label:

[0059] 810, Processor; 820, Communication interface; 830, Memory; 840, Communication bus. Detailed Implementation

[0060] To more clearly illustrate the overall concept of this application, a detailed explanation is provided below with reference to the accompanying drawings.

[0061] Many specific details are set forth in the following description to provide a thorough understanding of this application. However, this application may also be implemented in other ways different from those described herein. Therefore, the scope of protection of this application is not limited to the specific embodiments disclosed below. It should be noted that, unless otherwise specified, the embodiments of this application and the features thereof can be combined with each other.

[0062] In this application, unless otherwise expressly specified and limited, the "above" or "below" of the second feature can mean that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediate medium. In the description of this specification, references to terms such as "an embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples.

[0063] Example 1

[0064] like Figure 1 As shown, a semi-supervised image segmentation method based on adaptive pixel subdivision includes:

[0065] S100. Based on the target scene, collect the original image dataset, perform preprocessing to generate training and test sets, divide the training set into labeled and unlabeled data according to the proportion, and label the labeled data and test set to obtain labeled images.

[0066] As described above, depending on the target application scenario, the first step is to collect the original image dataset and perform a series of preprocessing operations on this data (such as sharpening, data augmentation, resizing, and normalization) to generate training and test sets. Next, the training set is divided into labeled and unlabeled data according to a certain ratio. Only the labeled data and the test set are subjected to detailed pixel-level annotations to obtain labeled images. This process ensures that the model can learn based on high-quality data and has good generalization ability in practical applications.

[0067] For example, in the application scenario of autonomous driving technology, it is necessary to extract image datasets from road traffic video clips recorded by vehicle-mounted cameras. These raw images are sharpened after Gaussian blurring, and then data augmentation is performed through rotation, flipping, etc., to ensure that all images are uniformly adjusted to a size suitable for the input model (such as 256×256 or 512×512), and pixel values ​​are normalized by calculating the mean and standard deviation. Assuming this dataset contains 1000 images, 200 of them can be selected as labeled data for detailed annotation, while the remaining 800 images are treated as unlabeled data and used directly for training. For each image in the selected labeled data and test set, the boundaries of different categories such as roads, vehicles, and pedestrians need to be manually delineated, and specific category numbers need to be assigned to them to form labeled images. This approach not only improves the quality of the data but also provides a solid foundation for subsequent model training.

[0068] It should be noted that, in specific implementation scenarios, in addition to traditional RGB images, other types of sensor data, such as infrared, ultrasonic, or LiDAR data, can be introduced based on the above solution. This can provide richer environmental information and help the model better understand complex scenes.

[0069] In specific implementation scenarios, generative adversarial networks (GANs) or other synthetic data generation techniques can be used to create new samples to expand the dataset, building upon the above approach. This method is particularly suitable for scenarios where obtaining real-world data is difficult, such as driving situations under extreme weather conditions.

[0070] In specific implementation scenarios, the ratio of labeled to unlabeled data can be dynamically adjusted based on the above approach and the characteristics of active learning or data distribution. For example, prioritizing the labeling of the most representative samples, rather than random selection, helps improve labeling efficiency and model performance.

[0071] In specific implementation scenarios, based on the above solutions, automated or semi-automated annotation tools and techniques can be developed to reduce the degree of manual intervention. Combining weakly supervised learning algorithms can significantly improve annotation speed and accuracy. Furthermore, designing a quality control system to ensure annotation accuracy is also crucial, thereby guaranteeing data quality and reliability.

[0072] S200. The labeled image is initially segmented using a preset semantic segmentation model to generate prediction results. Based on the uncertainty of the prediction results, key pixel regions are selected, multi-scale feature maps are extracted, and the key pixel regions are refined through iterative optimization to generate segmentation results.

[0073] As described above, firstly, the pre-defined DeepLabV3Plus model is used to perform preliminary segmentation on the labeled image. DeepLabV3Plus is an advanced semantic segmentation model that can classify each pixel, thus segmenting the image into different category regions. The result of the preliminary segmentation is usually a coarse prediction map, where each pixel is assigned a category label. Next, based on the uncertainty (such as entropy value) of the preliminary segmentation result, key pixel regions with high prediction inaccuracies are selected; these regions are often located on object boundaries. Then, information is extracted from feature maps at different scales, and these key pixel regions are iteratively optimized multiple times through a small sub-network to gradually improve their accuracy, ultimately generating a refined segmentation result.

[0074] For example, suppose we need to segment an image of a city street containing multiple vehicles and pedestrians. First, the image is fed into a pre-trained DeepLabV3Plus model, resulting in a preliminary segmentation that roughly separates vehicles, pedestrians, and background regions. However, there may be some blurring or misclassification at vehicle edges or pedestrian contours. To improve this, the entropy value of each pixel is calculated to measure its uncertainty. For example, if a pixel has a high entropy value, it indicates significant uncertainty in its classification and may be a keypoint located on an object boundary. Next, feature maps (e.g., H / 4 and H / 8 sizes) are extracted from different layers of the DeepLabV3Plus decoder and enlarged to a uniform size using bilinear interpolation. Finally, a small subnetwork is used to iteratively optimize these keypoint regions multiple times, refining boundary details in each iteration until a satisfactory segmentation result is achieved.

[0075] It should be noted that, in specific implementation scenarios, in addition to the existing multi-scale feature map extraction methods, more complex feature fusion strategies, such as attention mechanisms or adaptive weight adjustment, can be explored on the basis of the above scheme to better capture information at different levels and improve segmentation quality.

[0076] In specific implementation scenarios, the number of iterations can be dynamically adjusted based on the specific conditions of the key pixel regions, building upon the above approach. For example, for boundaries in complex scenes, the number of iterations can be increased to achieve higher accuracy; while for relatively simple regions, unnecessary computation can be reduced to improve efficiency.

[0077] In specific implementation scenarios, based on the above solutions, external knowledge bases (such as Geographic Information System (GIS), medical knowledge bases, etc.) can be combined to provide more contextual information for image segmentation in specific domains, helping the model make more accurate predictions. For example, in medical image analysis, known anatomical structure information can be used to guide the segmentation process.

[0078] In specific implementation scenarios, in addition to the currently used entropy value as an uncertainty metric, other indicators such as variance or Bayesian uncertainty estimation can be studied to more accurately identify key regions requiring further refinement. This helps to further improve the quality of segmentation results, especially when dealing with challenging boundaries.

[0079] S300. Construct teacher and student models. Train the teacher model using labeled data. Generate pseudo-labels for unlabeled data using the teacher model. Select confidence pixels with a confidence level greater than a preset threshold for optimization of the student model.

[0080] As described above, two deep learning models are constructed: one called the "teacher model" and the other the "student model." First, the teacher model is fully trained using a labeled dataset. Once trained, the teacher model can be used to process unlabeled datasets and generate pseudo-labels for these data. During this process, pixels with high confidence are selected as effective supervision signals by calculating the confidence level of each pixel (e.g., the probability output using the Softmax function). These high-confidence pseudo-labels, along with the original labeled data, are then used to train the student model to further optimize its performance.

[0081] For example, suppose we are developing a semi-supervised image segmentation system for urban street scene segmentation. First, a teacher model is trained using a set of labeled images (e.g., pedestrians, vehicles, and roads). Then, this trained teacher model is used to predict on a batch of unlabeled urban street images, generating pseudo-labels for each image. Next, for each pixel, its confidence level is evaluated based on the probability of it belonging to a certain category. For example, if a pixel has a 90% probability of belonging to the "pedestrian" category, it is considered a high-confidence pixel. Finally, all pixels with confidence levels above a preset threshold (e.g., 85%) are used as valid samples to train the student model. This not only increases the amount of training data but also improves the accuracy and robustness of the student model due to the use of high-quality pseudo-labels.

[0082] It should be noted that, in specific implementation scenarios, in addition to the existing simple teacher-student model, a more complex architecture, such as a bidirectional teacher-student framework, can be introduced based on the above scheme. In this framework, the student model can also guide the learning process of the teacher model, forming a mutually beneficial relationship in order to achieve better segmentation results.

[0083] In specific implementation scenarios, the threshold for selecting high-confidence pixels can be dynamically adjusted based on the characteristics of different tasks or datasets, building upon the above approach. For example, in cases with complex data distribution, the threshold can be appropriately lowered to allow more diverse data to participate in training; while in relatively simple scenarios, the threshold can be increased to ensure data quality.

[0084] In specific implementation scenarios, based on the above solutions, multi-view or multi-modal data (such as combining RGB images with depth information) can be combined to enhance the predictive ability of the teacher model by utilizing rich input information, thereby generating more accurate pseudo-labels and providing more powerful learning resources for the student model.

[0085] In specific implementation scenarios, while this approach primarily focuses on utilizing high-confidence pseudo-labels, it's also possible to explore how to effectively utilize low-confidence pixels. For example, contrastive learning can be employed, using low-confidence pixels as negative samples during training to help the model better understand the diversity of the data, thereby improving overall performance.

[0086] S400: Perform feature comparison learning on confidence pixels with confidence scores less than a preset threshold, and optimize student model parameters by constructing positive and negative samples and using a comparison loss function.

[0087] As mentioned above, in a semi-supervised learning framework, besides using high-confidence pseudo-labels to train the student model, feature contrastive learning can also be performed on pixels with confidence levels below a preset threshold. Specifically, these low-confidence pixels typically contain more uncertainty information, and can be incorporated into the model's learning process by constructing positive and negative samples. First, anchor features are extracted from high-confidence pixels, and the average feature of each category is calculated as a positive sample. For low-confidence pixels in unlabeled data, high-confidence category features are excluded based on their predicted probabilities, and the remaining category features are used as negative samples. Then, a contrastive loss function (such as InfoNCE loss) is used to measure the similarity between positive and negative samples, thereby optimizing the parameters of the student model. This method not only makes full use of unlabeled data but also enhances the model's ability to understand complex scenes.

[0088] For example, suppose in a city street image segmentation task, a teacher model has already generated pseudo-labels for a batch of unlabeled images, filtering out pixels with high and low confidence. For low-confidence pixels, firstly, pixels with high predicted probabilities in the ground truth label category are selected as anchor pixels, and the average of their feature vectors is calculated as positive samples. For example, in the "pedestrian" category, if a pixel has a predicted probability of 80%, it is considered an anchor pixel. Next, for low-confidence pixels in the unlabeled data, those with high predicted probabilities are excluded, and the remaining feature vectors are used as negative samples. Then, a contrastive loss function is used to evaluate the similarity between these positive and negative samples, and the parameters of the student model are adjusted accordingly. In this way, even in low-confidence regions, the model can learn more accurate feature representations, improving overall segmentation accuracy.

[0089] It should be noted that, in specific implementation scenarios, in addition to the existing positive sample construction methods based on high-confidence pixels, more diverse sample construction strategies can be explored based on the above solutions. For example, contextual information or neighborhood relationships can be introduced to construct more representative positive samples, thereby improving the model's ability to understand complex scenes.

[0090] In specific implementation scenarios, based on the above approach, the weight of the contrastive loss function within the overall loss function can be dynamically adjusted according to the training needs at different stages. For example, in the initial training phase, the weight of the contrastive loss can be increased to better guide the model to focus on low-confidence regions; while in later stages, its influence can be gradually reduced to focus on fine-tuning.

[0091] In specific implementation scenarios, based on the above approach, multi-scale feature maps (such as DeepLabV3Plus feature maps at different layers) can be combined to perform feature comparison learning at multiple levels. This helps to capture richer details, especially when dealing with challenging boundaries.

[0092] In specific implementation scenarios, based on the above solutions, contextual information provided by external knowledge bases (such as Geographic Information Systems (GIS), medical knowledge bases, etc.) can be used to assist in constructing positive and negative samples and optimizing the contrastive loss function. For example, in medical image analysis, known anatomical structure information can be combined to guide the feature contrastive learning process, helping the model make more accurate predictions.

[0093] S500: Input the image to be segmented into the optimized student model, output the predicted category matrix, and generate a visual segmentation result through color mapping.

[0094] As mentioned above, after training the student model, new images to be segmented can be input into this optimized student model. The student model classifies each pixel in each input image and outputs a category matrix, where each element represents the category label of the corresponding pixel (e.g., background is 0, and other categories such as pedestrians and vehicles increase sequentially). To facilitate user understanding and evaluation of the segmentation results, this category matrix is ​​usually further color-mapped, that is, each category is assigned a specific color, and these colors are applied to the original image to generate a visual segmentation result map. This visualization not only improves the intuitiveness of the segmentation results but also facilitates subsequent applications and analysis.

[0095] For example, suppose we are processing a set of images depicting city streets, with the goal of segmenting roads, vehicles, pedestrians, and background areas. First, an unlabeled city street image is input into a pre-trained student model. The student model classifies each pixel in the image, outputting a two-dimensional matrix where each value represents the probability of a pixel belonging to a certain category or is simply a category number. For example, a pixel with a value of 1 might represent "pedestrian," and a pixel with a value of 2 might represent "vehicle." Next, based on a predefined color map, the values ​​corresponding to each category are converted into corresponding colors (e.g., "pedestrian" is represented by red, and "vehicle" by blue). These colors are then overlaid on the original image to form the final visual segmentation result. This allows the user to clearly see the distribution of different objects in the image.

[0096] It should be noted that, in specific implementation scenarios, in addition to basic color mapping, more intelligent color allocation strategies can be introduced based on the above solutions. For example, the color scheme can be dynamically adjusted based on user preferences or application scenario requirements, or the most suitable color combination can be selected using the principles of color psychology to improve visual effects and user experience.

[0097] In specific implementation scenarios, based on the above solutions, multi-scale feature maps can be combined to generate visualization results at different resolutions, helping users to understand the segmentation results more comprehensively. For example, users can view detailed information at high resolution and the overall structure at low resolution, providing a multi-layered perspective.

[0098] In specific implementation scenarios, interactive visualization tools can be developed based on the above solutions, allowing users to manually adjust or mark the segmentation results while viewing them. For example, users can select specific object regions for reclassification or correct misclassified parts, further improving segmentation accuracy.

[0099] In specific implementation scenarios, based on the above solutions, other types of data sources, such as Geographic Information System (GIS) data and meteorological data, can be integrated when generating visual segmentation results to provide more contextual information. For example, in urban planning or traffic management applications, real-time traffic flow data can be combined with the segmentation results to generate richer analysis reports.

[0100] In specific implementation scenarios, an automatic evaluation system can be established based on the above solutions to assess the quality of the generated segmentation results and provide feedback suggestions based on the evaluation results. For example, if the segmentation effect in certain regions is found to be poor, the system can prompt the user to add more data from similar scenarios for retraining, or suggest adjusting certain parameter settings.

[0101] According to one embodiment of this application, the acquisition of the original image dataset and its preprocessing to generate training and test sets include:

[0102] Image data is collected based on the target scene. The sources of image data include public datasets, internet resources, or custom-collected data.

[0103] Image data is sharpened by using Gaussian blur and sharpening algorithms to enhance image details;

[0104] Data augmentation of image data includes random rotation, flipping, and adding Gaussian noise;

[0105] Set the image size to the preset size.

[0106] Normalize pixel values ​​to the 0-1 range to eliminate differences in lighting or device;

[0107] The preprocessed image data is divided into training and test sets in an 8:2 ratio. The training set is further divided into labeled and unlabeled data in a 1:4 or 1:2 ratio.

[0108] As mentioned above, image data is collected from multiple sources to meet the specific application requirements. These sources may include, but are not limited to, publicly available datasets, internet resources, or data collected specifically through customized methods. For example, in medical image analysis, it may be necessary to obtain X-ray or CT scan images of specific diseases from hospitals; while in the field of autonomous driving, it may be necessary to obtain images of the road environment from vehicle cameras or other sensors.

[0109] To enhance detail in the image, Gaussian blur and sharpening algorithms are used to process the original image. First, Gaussian blur is applied to smooth the image and reduce noise. Then, the original image is subtracted by a coefficient multiplied by the Gaussian blurred image to obtain the sharpened image. Finally, pixel values ​​are adjusted to the standard RGB range of 0 to 255 to ensure image quality while highlighting important features.

[0110] To increase the diversity of data during model training, a series of data augmentation techniques were employed to process the images. These included random rotation, flipping, and adding Gaussian noise. The aim was to simulate different situations that might occur in real-world application environments, such as changes in object position or varying lighting conditions, thereby improving the model's robustness and generalization ability.

[0111] Since images from different sources may have different sizes, all images need to be resized to a uniform size suitable for input into the image segmentation model. Typically, 256x256 or 512x512 pixels are chosen, but this can be adjusted according to specific model requirements and application scenarios.

[0112] To eliminate pixel value variations caused by differences in lighting conditions or imaging equipment, the pixel values ​​of the image are normalized by scaling them to between 0 and 1. This process helps to speed up model training and improve model performance.

[0113] After completing all the preprocessing steps described above, the dataset is randomly divided into training and test sets in an 8:2 ratio. The training set is used for model learning, while the test set is used to evaluate model performance. Furthermore, the data in the training set is further divided into labeled and unlabeled data in a 1:4 or 1:2 ratio to facilitate semi-supervised learning strategies. Labeled data is directly used in the supervised learning phase, while unlabeled data indirectly participates in the model optimization process through pseudo-labels or other methods. This approach not only makes full use of limited labeling resources but also effectively improves the accuracy and reliability of the model.

[0114] According to one embodiment of this application, the preliminary segmentation of the labeled image using a preset semantic segmentation model to generate a prediction result includes:

[0115] The pre-defined DeepLabV3Plus model is used, with ResNet-101 as the encoder and ASPP module as the decoder;

[0116] Input the labeled image into the preset DeepLabV3Plus model, with the input size matching that of the preprocessed image;

[0117] The preset DeepLabV3Plus model outputs the class probability distribution of each pixel and generates prediction results.

[0118] As mentioned above, DeepLabV3Plus is an advanced semantic segmentation model, particularly suitable for high-precision object recognition and segmentation in images. The model consists of an encoder and a decoder. The encoder uses the ResNet-101 architecture, a deep residual network capable of effectively learning complex feature representations of the input image. The decoder employs an Atrous Spatial Pyramid Pooling (ASPP) module, capturing multi-scale information through convolutional kernels of different scales to further improve segmentation accuracy.

[0119] After the preparation phase, the preprocessed labeled images are input into the preset DeepLabV3Plus model. It is important to note that the size of the input images should be consistent with the size set during preprocessing to ensure data format consistency and compatibility. This not only helps improve model running efficiency but also avoids data distortion or information loss caused by size mismatches.

[0120] When an labeled image is input into the DeepLabV3Plus model, the model processes the input image based on its trained parameters and calculates the probability distribution of each pixel's class. In other words, for each pixel in the image, the model provides a probability vector containing the probability values ​​of that pixel belonging to all possible classes (such as background, target object, etc.). Based on these probability distributions, the most likely class label for each pixel can be determined, thereby generating the predicted segmentation result for the entire image.

[0121] According to one embodiment of this application, the step of calculating the uncertainty based on the prediction result and filtering out key pixel regions includes:

[0122] The probability entropy value is calculated for the prediction result of each pixel using the following formula:

[0123]

[0124] Where C represents the total number of categories. The probability of being class c;

[0125] Set an entropy threshold and select pixels with entropy values ​​higher than the threshold as key point regions.

[0126] As mentioned above, after completing the initial segmentation, the DeepLabV3Plus model generates a class probability distribution for each pixel in the image. To quantify the uncertainty of these predictions, it is necessary to calculate the probability entropy value for each pixel. Probability entropy is a metric for measuring information uncertainty, and here it is used to evaluate the model's confidence in classifying each pixel.

[0127] The specific calculation formula is as follows:

[0128]

[0129] Where C represents the total number of categories, This represents the probability that the pixel belongs to class c. Using the above formula, we can obtain the entropy value H(p) for each pixel; a higher entropy value indicates greater uncertainty in the pixel's classification.

[0130] After calculating the entropy values ​​of all pixels, the next step is to set an entropy threshold. This threshold can be determined based on the actual application scenario, dataset characteristics, or experimental results. Generally, the entropy threshold should be chosen to effectively distinguish pixels with high classification uncertainty.

[0131] Then, pixels with entropy values ​​higher than a preset threshold are selected from all pixels; these pixels constitute the keypoint regions. Keypoint regions are usually located at object boundaries, category boundaries, or other complex structural areas, and are where model predictions are relatively blurry and prone to missegmentation.

[0132] According to one embodiment of this application, the step of extracting multi-scale feature maps and refining key pixel regions through iterative optimization includes:

[0133] Semantic segmentation models include the DeepLabV3Plus model;

[0134] Feature maps of different scales are extracted from the decoder of the DeepLabV3Plus model and enlarged to a uniform size using bilinear interpolation;

[0135] A small subnetwork is used to process the key point region in multiple rounds of iteration. In each round of iteration, the feature representation is updated based on the current prediction results, and the optimization strategy is dynamically adjusted.

[0136] As described above, firstly, based on the initial segmentation, multiple feature maps of different scales are extracted from the decoder part of the DeepLabV3Plus model. These feature maps contain information about the image at different levels of abstraction; for example, larger feature maps retain more detailed information, while smaller feature maps reflect the global structure more. By combining information from multiple scales, complex structures and edge information in the image can be captured more accurately.

[0137] Then, bilinear interpolation is used to scale these feature maps at different scales to a uniform size. This is done to ensure that all feature maps have the same resolution, facilitating subsequent processing and fusion. Bilinear interpolation is a commonly used image scaling technique that can effectively resize images while maintaining image quality.

[0138] Next, for the selected key pixel regions (i.e., regions with high prediction uncertainty), a specially designed small sub-network is used to perform multiple rounds of iterative optimization. Each iteration generates an updated feature representation based on the current key pixel region and its feature representation, and dynamically adjusts the optimization strategy according to the update results.

[0139] At the beginning of each iteration, the subnetwork receives the feature representation from the previous iteration (or, in the case of the first iteration, directly from the multi-scale feature map obtained in the previous step).

[0140] The subnetwork performs a series of transformations and processes on these feature representations, aiming to enhance the feature representation ability of key point regions and improve their classification accuracy.

[0141] Based on the processed feature representation, new predictions are calculated, and the differences between the predictions and the target are evaluated.

[0142] Based on the above evaluation results, the parameters of the sub-network or the optimization algorithm are dynamically adjusted to guide the direction and intensity of the next iteration.

[0143] According to one embodiment of this application, the construction of the teacher model and student model, and the training of the teacher model using labeled data, includes:

[0144] Both the teacher and student models employ an adaptive pixel-level subdivision feature extraction framework.

[0145] The teacher model is trained under supervised supervision using labeled data, and the loss function is cross-entropy loss.

[0146] During training, the teacher model parameters are kept constant, and only the student model parameters are optimized.

[0147] As described above, firstly, two image segmentation models are constructed: a teacher model and a student model. Both models employ the same adaptive pixel-level subdivision feature extraction framework, which enables refined processing of key regions in the image (typically boundary regions with high prediction uncertainty). This feature extraction framework includes steps such as initial segmentation network prediction, keypoint selection, multi-scale feature extraction, and iterative refinement to enhance the ability to capture complex structures and edge information.

[0148] The teacher model is initially trained using a labeled dataset. In this stage, the teacher model accepts input images and their corresponding labels as training data, and uses the cross-entropy loss function to measure the difference between the model's predictions and the actual labels, thereby optimizing the model parameters. The cross-entropy loss function is a widely used loss function in classification tasks; it effectively measures the difference between the probability distribution of the model's output and the true label distribution, thus guiding the model to learn a more accurate classification boundary.

[0149] After training the teacher model, its parameters are fixed and no longer updated. Next, the student model is trained using a mixed dataset containing both labeled and unlabeled data. During this process, while the teacher model's parameters remain unchanged, its knowledge is indirectly transferred to the student model. Specifically, for unlabeled data, the teacher model can generate pseudo-labels, which are then used as one of the learning objectives for the student model. Simultaneously, for labeled data, it is directly used for supervised learning of the student model, further optimizing its parameters. This design aims to enable the student model to learn not only from limited labeled data but also from useful information gained from unlabeled data, thereby improving the model's overall performance and generalization ability.

[0150] According to one embodiment of this application, the step of generating pseudo-labels for unlabeled data using a teacher model and selecting confidence pixels with a confidence level greater than a preset threshold for optimization of the student model includes:

[0151] The teacher model makes predictions on unlabeled data and outputs the class probability distribution for each pixel.

[0152] Calculate the entropy value of each pixel, and retain only pixels with entropy values ​​lower than a first preset threshold as confidence samples;

[0153] The prediction results of the confidence samples are used as pseudo-labels, and the student model is further optimized using the cross-entropy loss function.

[0154] As described above, firstly, the images in the unlabeled dataset are processed using a pre-trained teacher model. The teacher model generates a class probability distribution map for each input image, representing the probability that each pixel belongs to each class. This step provides the basis for subsequently determining which pixels have higher confidence levels.

[0155] Next, based on the probability distribution output by the teacher model, the entropy value of each pixel is calculated. The entropy value reflects the uncertainty of the classification result of that pixel: the lower the entropy value, the more certain the classification result of that pixel; conversely, the higher the entropy value, the more uncertain the classification result. Based on the calculated entropy values, a first preset threshold is set, and only pixels with entropy values ​​below this threshold are retained as confidence samples. This means that only pixels with relatively certain classification results are selected for the next step.

[0156] For selected confidence samples, their corresponding class predictions are treated as pseudo-labels. These pseudo-labels represent the class assignments that the teacher model believes are most likely to be correct, and thus can be used to guide the student model's learning process. These pseudo-labels, along with the corresponding input images, are used as supervision signals to further optimize the student model using the cross-entropy loss function. In this process, the student model attempts to minimize the difference between its predictions and the pseudo-labels, thereby learning more accurate and robust segmentation capabilities.

[0157] According to one embodiment of this application, the feature comparison learning for confidence pixels with confidence levels less than a preset threshold includes:

[0158] Anchor features are extracted from confidence pixels with confidence scores greater than a preset threshold, and the average features of the corresponding categories are calculated as positive samples.

[0159] Eliminate category features with confidence scores exceeding a preset threshold, and treat the remaining category features as negative samples;

[0160] The model is trained by comparing positive and negative samples using a loss function, as shown in the formula:

[0161]

[0162] Where q represents the query feature. As a positive sample, For negative samples, This refers to the temperature parameter.

[0163] As described above, firstly, for pixels with confidence scores higher than a preset threshold, anchor features are extracted from them. These anchor features represent relatively certain classification results from the model. Then, for each category, the average of all anchor features belonging to that category is calculated, and this average feature is used as the positive sample for that category. The purpose of this is to obtain a relatively stable feature representation that can represent the category.

[0164] Next, the category features corresponding to the positive samples calculated above are excluded, and the remaining category features are treated as negative samples. These negative samples are used for comparative learning with the positive samples, helping the model better understand the differences and relationships between different categories.

[0165] Finally, the model is trained using a contrastive loss function to optimize its representation ability in the feature space. The specific formula for the contrastive loss function is as follows:

[0166]

[0167] Where q is the query feature, that is, the feature of the pixel that needs to be classified; A positive sample is a feature that belongs to the same category as the query feature. A negative sample is a feature that does not belong to the same category as the query feature. The temperature parameter is used to control the smoothness of the contrast loss function.

[0168] By minimizing the contrastive loss function described above, the model can learn more robust and discriminative feature representations, making features of the same class closer together in the feature space, while features of different classes are further apart. This helps improve the model's classification accuracy when dealing with pixels with low confidence, thereby improving the overall segmentation performance.

[0169] A second aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any of the embodiments of the first aspect above.

[0170] Figure 2 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 2 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 may call logical instructions in the memory 830 to execute the method in any of the embodiments of the first aspect described above, the method including:

[0171] Based on the target scene, the original image dataset is collected and preprocessed to generate training and test sets. The training set is divided into labeled and unlabeled data according to the proportion. The labeled data and test set are labeled to obtain labeled images.

[0172] The pre-set DeepLabV3Plus model is used to perform preliminary segmentation of the labeled image and generate prediction results. Based on the uncertainty calculation of the prediction results, key pixel regions are selected, multi-scale feature maps are extracted, and the key pixel regions are refined through iterative optimization to generate segmentation results.

[0173] Construct teacher and student models, train the teacher model using labeled data, generate pseudo-labels for unlabeled data using the teacher model, and select confidence pixels with confidence scores greater than a preset threshold for optimization of the student model.

[0174] Feature comparison learning is performed on pixels with confidence scores less than a preset threshold, and the parameters of the student model are optimized by constructing positive and negative samples and using a comparison loss function.

[0175] The image to be segmented is input into the optimized student model, which outputs a predicted category matrix and generates a visual segmentation result through color mapping.

[0176] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0177] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program, the computer program being able to be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer being able to perform the methods provided by the above methods, the method comprising:

[0178] Based on the target scene, the original image dataset is collected and preprocessed to generate training and test sets. The training set is divided into labeled and unlabeled data according to the proportion. The labeled data and test set are labeled to obtain labeled images.

[0179] The pre-set DeepLabV3Plus model is used to perform preliminary segmentation of the labeled image and generate prediction results. Based on the uncertainty calculation of the prediction results, key pixel regions are selected, multi-scale feature maps are extracted, and the key pixel regions are refined through iterative optimization to generate segmentation results.

[0180] Construct teacher and student models, train the teacher model using labeled data, generate pseudo-labels for unlabeled data using the teacher model, and select confidence pixels with confidence scores greater than a preset threshold for optimization of the student model.

[0181] Feature comparison learning is performed on pixels with confidence scores less than a preset threshold, and the parameters of the student model are optimized by constructing positive and negative samples and using a comparison loss function.

[0182] The image to be segmented is input into the optimized student model, which outputs a predicted category matrix and generates a visual segmentation result through color mapping.

[0183] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the cigarette box image recognition method provided by the methods described above, the method comprising:

[0184] Based on the target scene, the original image dataset is collected and preprocessed to generate training and test sets. The training set is divided into labeled and unlabeled data according to the proportion. The labeled data and test set are labeled to obtain labeled images.

[0185] The pre-set DeepLabV3Plus model is used to perform preliminary segmentation of the labeled image and generate prediction results. Based on the uncertainty calculation of the prediction results, key pixel regions are selected, multi-scale feature maps are extracted, and the key pixel regions are refined through iterative optimization to generate segmentation results.

[0186] Construct teacher and student models, train the teacher model using labeled data, generate pseudo-labels for unlabeled data using the teacher model, and select confidence pixels with confidence scores greater than a preset threshold for optimization of the student model.

[0187] Feature comparison learning is performed on pixels with confidence scores less than a preset threshold, and the parameters of the student model are optimized by constructing positive and negative samples and using a comparison loss function.

[0188] The image to be segmented is input into the optimized student model, which outputs a predicted category matrix and generates a visual segmentation result through color mapping.

[0189] Example 2

[0190] S101: Collect image data and perform preprocessing on the images, while dividing the data into training and test sets according to the proportion.

[0191] Specifically, for a specific domain task, a comprehensive collection of image datasets is conducted from various reliable sources. These datasets include, but are not limited to, publicly available datasets, internet crawling, specialized acquisition equipment (such as medical imaging equipment, surveillance cameras, etc.), or customized data collection activities tailored to the specific needs of the domain task.

[0192] After collecting enough raw image data, these images undergo preprocessing, including the following steps:

[0193] Image sharpening: First, apply a Gaussian blur to the original image. Then, subtract the Gaussian-blurred image from the original image by a coefficient. Finally, normalize the pixel values ​​of this image to the RGB pixel value range of 0-255. The specific processing formula is as follows:

[0194]

[0195] Data augmentation: Applying a series of techniques (such as rotation, flipping, adding noise, etc.) to enhance sharpened images and improve data diversity.

[0196] Resizing: Adjust all images to a size suitable for input into the image segmentation model. Common input image sizes are 256*256 or 512*512 pixels.

[0197] Normalization: Normalize the pixel values ​​of an image to between 0 and 1 using the following method:

[0198] ,

[0199] in, Represents the raw pixel values ​​of the image. This represents the average value of the image pixels. The standard deviation of image pixel values. This represents the normalized pixel value.

[0200] After completing the preprocessing steps for the image data, the entire dataset is randomly divided according to a certain ratio to create a training set and a test set. The training set data is used by the image segmentation model to learn the patterns in the data, while the test set data is not visible during the image segmentation model training process. When the model training is complete, the test set data is used to verify the model's capabilities. Commonly used training to test set ratios include 8:2 and 7:3.

[0201] S102: Divide the training set into labeled and unlabeled data according to the proportion, and then label the labeled data and test set data for the image segmentation task.

[0202] Specifically, since this application adopts a semi-supervised image segmentation training paradigm as the basic framework of the neural network, it is not necessary to label all training set images; only the labeled data and test set data in the training set need to be labeled. Commonly used ratios for labeling and unlabeling data are 1:4, 1:2, and 1:1. Generally, as the ratio increases, the image segmentation effect improves, but the workload of labeling data also increases. Therefore, a trade-off needs to be struck between model accuracy and labeling workload.

[0203] After dividing the training set into labeled and unlabeled data, data labeling for the image segmentation task is performed. This includes the following steps:

[0204] Choose the appropriate tool. First, select a suitable image segmentation and annotation tool. Commonly used annotation tools include LabelMe, VGG Image Annotator (VIA), RectLabel, Labelbox, and Supervisely.

[0205] Determine category labels. Before starting the formal labeling work, it is first necessary to define the categories to be identified based on the task scenario and actual intent, and assign a label to each category. For example, in an autonomous driving scenario, the categories that may need to be labeled include pedestrians, vehicles, and roads. Usually, after determining the categories of objects to be identified, an additional background class is also needed as a default category, with a label of 0. The labels for other categories start from 1 and are assigned sequentially.

[0206] Creating annotation guidelines: Annotation work often requires multiple people working simultaneously. To ensure consistency and accuracy, it's essential to first develop detailed annotation guidelines to standardize and confirm the annotation of each category of objects. Furthermore, these guidelines should include handling issues such as blurred boundaries, occlusion, and annotation rules for objects from different viewpoints.

[0207] The annotation process involves assigning a category label to each pixel in the image. Specifically, an image segmentation annotation tool is used to precisely outline the contour of each category; pixels within the outline are considered as one category and assigned the same category label.

[0208] Inspection and correction. After completing the initial image annotation, conduct one or more rounds of annotation quality checks to correct any incorrect or inaccurate annotations.

[0209] S103: An adaptive pixel-level subdivision feature extraction framework is constructed based on the DeepLabV3Plus semantic segmentation model. This feature extraction framework mainly includes the following steps:

[0210] Basic segmentation network prediction: First, a pre-trained basic image segmentation network is used to process the input image, generating preliminary coarse image segmentation results. This step provides initial class predictions for the image. Let the initial class predictions be denoted as... ,in The size is , For the height of the image, For the width of the image, The number of categories is the preset number.

[0211] Keypoint selection: Based on the preliminary coarse image segmentation results, pixels are selected as keypoints using an adaptive point sampling strategy. These selected pixels represent regions with high prediction uncertainty, typically concentrated at object boundaries, requiring further refinement of the predictions for these pixels. Specifically, initial category prediction is first used... Calculate the entropy of each pixel. The calculation formula is as follows:

[0212]

[0213] in It is the pixel pair of the first The predicted probability for each category is calculated. A higher entropy value indicates greater prediction uncertainty for that pixel. Then, a threshold is set, and pixels with entropy values ​​greater than the threshold are selected as keypoints.

[0214] Multi-scale feature extraction: DeepLabV3Plus is an encoder-decoder image segmentation model where the image size gradually decreases in the encoder and gradually returns to its original size in the decoder. This application uses DeepLabV3Plus to extract features at various scales in the decoder. and size Features at two scales. (See attached image.) Figure 2 As shown, bilinear interpolation is used to amplify the features at both scales. .

[0215] Iterative refinement: Figure 3 It shows the use of size The process of iteratively refining the features. After enlarging the feature size, a small subnetwork is used to refine the selected key points ( Figure 3 The features extracted from the location of the blue dots are processed, and after a series of iterations, more refined features can be obtained. The size is... The features are processed in the same way, and then the features of the two scales after refinement are concatenated to obtain the final output feature of the adaptive pixel subdivision feature extraction framework.

[0216] S104: Construct a semi-supervised image segmentation training paradigm based on a teacher-student model that reuses low-confidence pixels, and train the model using the dataset. Specifically, this includes the following steps:

[0217] Two adaptive pixel-level feature extraction frameworks built in S103 were used as the teacher model and the student model, respectively. For the teacher model, all labeled images were first input into the teacher model for multi-round supervised learning training, and the teacher model was optimized using the standard cross-entropy loss function.

[0218] After multiple rounds of training and optimization of the teacher model, semi-supervised image segmentation training was performed. Batch training was used, with each batch containing both labeled and unlabeled images. Labeled images were directly input into the student model, which was also optimized using the standard cross-entropy loss function, denoted as [missing value]. For unlabeled images, they are simultaneously input into both the teacher and student models. For the output features of the teacher model, the Softmax function is first used to calculate the predicted probability of each pixel belonging to each category, and then the entropy of each pixel is calculated. A threshold is set; pixels with entropy values ​​less than the threshold are considered high-confidence pixels, and pixels with entropy values ​​greater than the threshold are considered low-confidence pixels.

[0219] For high-confidence pixels, the class with the highest predicted probability is considered the true class of that pixel. These pixels serve as pseudo-labels for the student model, acting as supervision signals to monitor the model's training. The standard cross-entropy loss function is also used to optimize the student model, denoted as... .

[0220] For low-confidence pixels, typical semi-supervised image segmentation models usually discard them directly, preventing most unlabeled pixels from participating in model training and resulting in insufficient model training. This application uses the idea of ​​contrastive learning to reuse low-confidence pixels, enabling a large number of low-confidence unlabeled samples to participate in the model optimization process. Specifically, positive and negative samples for contrastive learning are first constructed, and positive and negative samples need to be constructed for each preset category. The method for constructing positive samples is as follows: In a batch of data, a series of anchor pixels are sampled for each category. For labeled data, pixels whose predicted probability in the ground truth label category is greater than a certain threshold are selected as anchor pixels. Specifically, the selection formula is as follows:

[0221]

[0222] in, This represents the anchor pixel features of labeled samples. Features representing pixels The truth label category for a pixel is And in category The predicted probability is greater than the threshold. .

[0223] For unlabeled data, pixels whose predicted probability for the pseudo-label category is greater than a certain threshold are selected as anchor pixels. Specifically, the selection formula is as follows:

[0224]

[0225] in, Anchor pixel features representing unlabeled samples The pseudo-label category representing pixels is And in category The predicted probability is greater than the threshold. .

[0226] The average value of the features of the anchor pixels mentioned above is used as a positive sample for one category. Specifically, the category... The positive sample features are denoted as The calculation formula is as follows:

[0227]

[0228] The method for constructing negative samples is as follows: For labeled samples, since the category to which each pixel belongs is known, all categories except the ground truth label can be used as negative samples for that pixel; for unlabeled samples, due to the low confidence of the prediction, pseudo-labels may be incorrect, and the truthfulness of pseudo-labels cannot be completely trusted. This application sets a threshold to filter out categories with predicted probabilities greater than this threshold, treating those pixels as negative samples for the remaining few categories. low The features of each negative sample pixel are denoted as... .

[0229] After the positive and negative samples of each category are determined, the model is optimized using the following loss function:

[0230]

[0231] In summary, the loss function for the semi-supervised image segmentation training paradigm that reuses low-confidence pixels is as follows:

[0232]

[0233] in and The coefficient is used. Based on this loss function, the model is trained and optimized through multiple rounds using backpropagation.

[0234] S105: Input the image to be segmented into the trained semi-supervised image segmentation model and try to predict it to obtain the segmented image.

[0235] Specifically, the fully trained student model in S104 is first used as the image segmentation prediction model. The image is input into this model to obtain the image segmentation prediction result, which is typically a value of [size missing]. A two-dimensional matrix is ​​generated, where each value represents the category label of an object pixel. A color map is then set up, assigning a fixed color to each category. Based on the color map, the predicted category matrix is ​​converted into a color mask, with each pixel assigned a corresponding color according to its category. Finally, the color mask is overlaid onto the original image with a certain degree of transparency, clearly showing the regions of different categories.

[0236] For any parts not mentioned in this application, existing technologies may be used or referenced.

[0237] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0238] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A semi-supervised image segmentation method based on adaptive pixel subdivision, characterized in that, include: Based on the target scene, the original image dataset is collected and preprocessed to generate training and test sets. The training set is divided into labeled and unlabeled data according to the proportion. The labeled data and test set are labeled to obtain labeled images. The labeled image is initially segmented using a pre-defined semantic segmentation model to generate prediction results. Based on the uncertainty of the prediction results, key pixel regions are selected, multi-scale feature maps are extracted, and the key pixel regions are refined through iterative optimization to generate segmentation results. Construct teacher and student models, train the teacher model using labeled data, generate pseudo-labels for unlabeled data using the teacher model, and select confidence pixels with confidence scores greater than a preset threshold for optimization of the student model. Feature comparison learning is performed on pixels with confidence scores less than a preset threshold, and the parameters of the student model are optimized by constructing positive and negative samples and using a comparison loss function. The image to be segmented is input into the optimized student model, which outputs a predicted category matrix and generates a visual segmentation result through color mapping. Both the teacher model and the student model employ an adaptive pixel-level subdivision feature extraction framework. The teacher model is trained under supervised training using labeled data, and the loss function is cross-entropy loss. During training, the teacher model parameters are kept constant, and only the student model parameters are optimized.

2. The method according to claim 1, characterized in that, The process of acquiring the original image dataset and preprocessing it to generate training and test sets includes: Image data is collected based on the target scene. The sources of image data include public datasets, internet resources, or custom-collected data. Image data is sharpened by using Gaussian blur and sharpening algorithms to enhance image details; Data augmentation of image data includes random rotation, flipping, and adding Gaussian noise; Set the image size to the preset size. Normalize pixel values ​​to the 0-1 range to eliminate differences in lighting or device; The preprocessed image data is divided into training and test sets in an 8:2 ratio. The training set is further divided into labeled and unlabeled data in a 1:4 or 1:2 ratio.

3. The method according to claim 1, characterized in that, The process of using a preset semantic segmentation model to perform preliminary segmentation of the labeled image and generate prediction results includes: The pre-defined DeepLabV3Plus model is used, with ResNet-101 as the encoder and ASPP module as the decoder; Input the labeled image into the preset DeepLabV3Plus model, with the input size matching that of the preprocessed image; The preset DeepLabV3Plus model outputs the class probability distribution of each pixel and generates prediction results.

4. The method according to claim 1, characterized in that, The uncertainty calculation based on the prediction results is used to filter out key pixel regions, including: The probability entropy value is calculated for the prediction result of each pixel using the following formula: Where C represents the total number of categories. The probability of being class c; Set an entropy threshold and select pixels with entropy values ​​higher than the threshold as key point regions.

5. The method according to claim 1, characterized in that, The extraction of multi-scale feature maps and the refinement of key pixel regions through iterative optimization include: Semantic segmentation models include the DeepLabV3Plus model; Feature maps of different scales are extracted from the decoder of the DeepLabV3Plus model and enlarged to a uniform size using bilinear interpolation; A small subnetwork is used to process the key point region in multiple rounds of iteration. In each round of iteration, the feature representation is updated based on the current prediction results, and the optimization strategy is dynamically adjusted.

6. The method according to claim 1, characterized in that, The step of generating pseudo-labels for unlabeled data using a teacher model and selecting pixels with confidence scores greater than a preset threshold for optimization of the student model includes: The teacher model makes predictions on unlabeled data and outputs the class probability distribution for each pixel. Calculate the entropy value of each pixel, and retain only pixels with entropy values ​​lower than a first preset threshold as confidence samples; The prediction results of the confidence samples are used as pseudo-labels, and the student model is further optimized using the cross-entropy loss function.

7. The method according to claim 1, characterized in that, The feature comparison learning for confidence pixels with confidence scores less than a preset threshold includes: Anchor features are extracted from confidence pixels with confidence scores greater than a preset threshold, and the average features of the corresponding categories are calculated as positive samples. Eliminate category features with confidence scores exceeding a preset threshold, and treat the remaining category features as negative samples; The model is trained by comparing positive and negative samples using a loss function, as shown in the formula: Where q represents the query feature. As a positive sample, For negative samples, This refers to the temperature parameter.

8. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in any one of claims 1-7.

9. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method as described in any one of claims 1-7.

Images