Training methods, apparatus, and computer equipment for image segmentation models

By generating a highlighted foreground map and an edge contour map, and combining it with a segmentation quality diagnostic model for error attribution and subset training, the shortcomings of traditional image segmentation models in complex scenes are addressed, achieving more efficient training and performance improvement.

CN123118665APending Publication Date: 2026-11-17XIAMEN MEITUZHIJIA TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611236325.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-14
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Traditional image segmentation models perform poorly in complex scenarios, mainly due to low training sample quality, reliance on human experience leading to high data collection costs, blind selection of difficult samples, and imprecise error type attribution, resulting in insufficient model performance in long-tailed complex scenarios.

Method used

By generating a highlighted foreground image and an edge contour image, an error attribution is performed using a pre-trained segmentation quality diagnostic model. Error sample subsets are then divided, and these subsets are used to train specific task sub-models. Finally, the weights are fused to obtain the trained image segmentation model.

Benefits of technology

It improves the quality and utilization efficiency of training samples, enhances the model's performance in long-tailed complex scenarios, reduces reliance on human experience, and shortens the iteration cycle.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123118665A_ABST
    Figure CN123118665A_ABST
Patent Text Reader

Abstract

This application relates to a training method, apparatus, computer device, readable storage medium, and computer program product for an image segmentation model. The method includes: segmenting a target object in a sample image using an image segmentation model to be trained, obtaining an image segmentation mask; generating a highlight foreground image and an edge contour image based on the image segmentation mask; inputting the sample image, highlight foreground image, and edge contour image into a pre-trained segmentation quality diagnostic model to obtain error attribution results for erroneous samples; dividing the erroneous samples according to the error attribution results to obtain multiple erroneous sample subsets; training a corresponding image segmentation model to be trained using each erroneous sample subset to obtain a sub-model for each segmentation error type; and fusing the weights of each sub-model to obtain a trained image segmentation model. This method can improve the performance of image segmentation models in long-tailed complex scenes.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a training method, apparatus, computer device, readable storage medium, and computer program product for an image segmentation model. Background Technology

[0002] Image segmentation is a fundamental task in computer vision, aiming to distinguish objects, regions, or background in an image based on pixel-level boundaries. Depending on the task, image segmentation typically includes semantic segmentation, instance segmentation, and panoptic segmentation.

[0003] Traditional image segmentation training frameworks typically consist of data annotation, model training, result evaluation, and error correction. However, traditional training methods often rely on manual experience to collect and annotate training samples for long-tail scenes in images, such as human hair, clothing edges, transparent objects, shadows, complex backgrounds, and small objects. This manual approach is time-consuming and labor-intensive, resulting in high data collection costs. Furthermore, sample selection methods that rely on online user feedback or manual quality checks are somewhat indiscriminate, with insufficiently refined attribution of error types for difficult samples. The proportion of truly valuable difficult samples is unstable, and the training data is not effectively utilized. Consequently, the collected training samples lack high training value, leading to poor performance of the final trained image segmentation model in long-tail problems such as subject integrity, edge details, semi-transparent areas, and shadow processing.

[0004] It is evident that traditional image segmentation model training schemes suffer from low-quality training samples, making it difficult to improve model performance in complex scenarios. Summary of the Invention

[0005] Therefore, it is necessary to provide a training method, apparatus, computer device, readable storage medium, and computer program product for an image segmentation model to address the aforementioned technical problems, in order to improve the quality of training samples and thus enhance the performance of the image segmentation model in long-tailed complex scenarios.

[0006] In a first aspect, this application provides a method for training an image segmentation model, the method comprising the following steps:

[0007] The target object in the sample image is segmented using the image segmentation model to be trained, and the image segmentation mask is obtained.

[0008] Based on the image segmentation mask, a highlight foreground image and an edge contour image of the sample image are generated; the highlight foreground image is used to identify the region of the target object in the sample image; the edge contour image is used to identify the edges of the target object in the sample image.

[0009] The sample image, the highlighted foreground image, and the edge contour image are input into a pre-trained segmentation quality diagnostic model to obtain the error attribution results of the erroneous samples; the erroneous samples are the sample images corresponding to the segmentation results of images with segmentation errors.

[0010] Based on the error attribution results, the error samples are divided into multiple error sample subsets; each error sample subset corresponds to a segmentation error type.

[0011] Each of the aforementioned error sample subsets is used to train a corresponding image segmentation model to be trained, thereby obtaining a specific task sub-model corresponding to each of the aforementioned segmentation error types; the weights of each specific task sub-model are fused to obtain the trained image segmentation model.

[0012] In one embodiment, the step of inputting the sample image, the highlighted foreground image, and the edge contour image into a pre-trained segmentation quality diagnostic model to obtain the error attribution result of the erroneous sample includes:

[0013] Obtain a large target visual language model and training samples for the target visual language model, and label the training samples according to the segmentation error type to obtain training sample labels;

[0014] The training parameters for the target visual language large model are configured as a low-rank matrix. The target visual language large model is then fine-tuned using the low-rank matrix and the training sample labels to obtain the segmentation quality diagnostic model.

[0015] The sample image, the highlighted foreground image, the edge contour image, and the pre-defined prompt words are input into the segmentation quality diagnostic model, and the error attribution result is output; wherein, the prompt words are used to instruct the segmentation quality diagnostic model to output the error attribution result according to the set structured standard format.

[0016] In one embodiment, the step of dividing the error samples according to the error attribution result to obtain multiple error sample subsets includes:

[0017] Based on the error attribution results, the error samples corresponding to each segmentation error type are assigned to the initial error sample subset corresponding to each segmentation error type;

[0018] According to the segmentation error type corresponding to the initial subset of error samples, the labels of the error samples are re-labeled to obtain error sample labels;

[0019] Based on the error sample labels, the sample images, and the image segmentation mask, the error sample subsets corresponding to each segmentation error type are obtained.

[0020] In one embodiment, fusing the weights of each of the specific task sub-models to obtain the trained image segmentation model includes:

[0021] Obtain the performance metrics of each of the special task sub-models, and determine the parameter fusion weights corresponding to each of the special task sub-models based on the performance metrics;

[0022] Based on the parameter fusion weights, the trainable parameters in the image segmentation model to be trained and the trainable parameters in each of the specific task sub-models are fused to obtain the target trainable parameters of the trained image segmentation model.

[0023] The trained image segmentation model is obtained based on the target trainable parameters.

[0024] In one embodiment, generating the highlight foreground image and edge contour image of the sample image based on the image segmentation mask includes:

[0025] Within the target object area identified by the image segmentation mask, the pixels of the sample image are weighted and fused with a preset highlight color according to a preset transparency to generate the highlight foreground image;

[0026] Based on the image segmentation mask, edge detection is performed on the sample image to obtain the edge band of the sample image; the edge band is superimposed on the sample image in the form of a preset line to generate the edge contour map.

[0027] In one embodiment, the method further includes:

[0028] The trained image segmentation model is used as the new image segmentation model to be trained.

[0029] Return to the step of segmenting the target object in the sample image using the image segmentation model to be trained, and obtaining the image segmentation result, until the trained image segmentation model meets the preset training termination condition.

[0030] Secondly, this application provides a training apparatus for an image segmentation model, the apparatus comprising:

[0031] The image segmentation module is used to segment target objects in sample images using the image segmentation model to be trained, and obtain image segmentation masks.

[0032] The segmentation result generation module is used to generate a highlight foreground image and an edge contour image of the sample image based on the image segmentation mask; the highlight foreground image is used to identify the region of the target object in the sample image; the edge contour image is used to identify the edge of the target object in the sample image;

[0033] The segmentation quality diagnosis module is used to input the sample image, the highlighted foreground image, and the edge contour image into a pre-trained segmentation quality diagnosis model to obtain the error attribution result of the erroneous sample; the erroneous sample is the sample image corresponding to the segmentation result of the image with segmentation error;

[0034] The error sample segmentation module is used to segment the error samples according to the error attribution results, obtaining multiple error sample subsets; each error sample subset corresponds to a segmentation error type;

[0035] The subdomain training and fusion module is used to train a corresponding image segmentation model to be trained using each of the error sample subsets to obtain a special task sub-model corresponding to each segmentation error type; and to fuse the weights of each special task sub-model to obtain a trained image segmentation model.

[0036] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described image segmentation model training method.

[0037] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0038] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0039] The aforementioned image segmentation model training method, apparatus, computer equipment, readable storage medium, and computer program product segment the target object in the sample image using the image segmentation model to be trained, obtaining an initial model segmentation prediction image segmentation mask. Based on the image segmentation mask, a highlight foreground image and edge contour image of the sample image are generated, providing various visual auxiliary images for subsequent segmentation quality diagnosis. Then, the sample image, highlight foreground image, and edge contour image are input into a pre-trained segmentation quality diagnosis model to obtain the error attribution results of erroneous samples. Based on the automatic analysis and verification of the segmentation quality diagnosis model, it is possible to perform refined attribution of erroneous samples for various long-tailed complex scene problems in image segmentation, and to discover segmentation error samples more efficiently and accurately. This approach reduces reliance on human experience. Based on error attribution results, error samples are divided into subsets corresponding to each segmentation error type. This standardization and structured data binning based on error attribution results improves the quality of training samples. Furthermore, each error sample subset is used to train a corresponding image segmentation model, resulting in a specialized task sub-model for each segmentation error type. This allows each error sample subset to focus more clearly on specific task objectives, improving the efficiency of training data utilization. Finally, the weights of each specialized task sub-model are fused to obtain the trained image segmentation model. This results in the trained image segmentation model incorporating specialized capabilities for different long-tailed complex error types, improving the performance of the image segmentation model in long-tailed complex scenarios. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is an application environment diagram of a training method for an image segmentation model in one embodiment;

[0042] Figure 2 This is a flowchart illustrating a training method for an image segmentation model in one embodiment.

[0043] Figure 3 This is a schematic diagram of an iterative training method for an image segmentation model based on visual language quality diagnosis and error subdomain enhancement in one embodiment;

[0044] Figure 4This is a structural block diagram of a training device for an image segmentation model in one embodiment;

[0045] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0047] Image segmentation is a fundamental task in computer vision, aiming to distinguish objects, regions, or background in an image based on pixel-level boundaries. The core objective of image segmentation is to perform pixel-level recognition of objects, regions, or semantic categories in an image, enabling the model to output an accurate mask. Compared to classification and detection, segmentation results directly affect object boundaries, detailed structures, transparent regions, shadow regions, and foreground-background relationships; therefore, it demands higher requirements for data quality, annotation accuracy, and model generalization ability.

[0048] However, in the actual training of image segmentation models, the first problem encountered is the high cost of obtaining high-quality labeled samples. Image segmentation requires drawing masks along the target boundaries. When encountering long-tail problems such as hair, clothing edges, handheld objects, glass, thin gauze, shadows, thin rods, and small objects, data collection is highly dependent on human experience, the collection direction is prone to subjectivity, and the labeling difficulty will increase significantly. Relying entirely on manual labeling from scratch is time-consuming, labor-intensive, and costly.

[0049] The second problem is the difficulty in collecting valuable samples. The samples that truly improve model capabilities in online or offline image libraries are often the difficult samples that the model is prone to segmenting incorrectly, such as missing subjects, jagged edges, hardened transparent areas, cropped shadows, misidentified backgrounds, confusion of similar colors, and missed small objects. Traditional methods struggle to systematically identify fine-grained long-tail problems like rough edges, unnatural semi-transparent areas, and misidentified shadows in massive datasets. Furthermore, traditional methods that automatically filter difficult samples using metrics like model confidence, prediction entropy, mask area anomalies, and connected component anomalies often only reveal statistical anomalies, making it difficult to determine which specialized training set a sample should be included in, and easily overlooking the differences in annotation difficulty between different semantic regions such as small objects and slender structures.

[0050] The third problem is the difficulty in automatically attributing error types. Traditional hard sample collection schemes, because they do not perform structured error attribution on the erroneous samples, mix all the collected misclassified samples together and add them to the training set for retraining. This leads to unclear training objectives, unstable model improvement, and makes it difficult for a unified model to simultaneously take into account all long-tail scenarios.

[0051] The fourth problem is the persistent issues of boundary imbalance, class imbalance, and pixel importance. Traditional segmentation models often suffer from class imbalance, incorrect or incomplete boundaries, and differences in pixel importance during training. Traditional methods, based on manual rule-based binning, divide misclassified samples into multiple specialized datasets such as hair, transparent objects, complex backgrounds, shadows, and small objects, and then train or enhance them separately. However, when the data volume reaches hundreds of thousands or even millions, manual binning becomes almost unsustainable, and the boundaries of incorrect classes are unclear. A single image may simultaneously exhibit multiple problems such as missing subjects, rough edges, and misclassified shadows.

[0052] The fifth issue is the increasingly higher requirements for data scale and data closed-loop capabilities. The capabilities of traditional image segmentation models come not only from network structure but also from large-scale data engines. This fact indicates that the competitive focus of modern segmentation systems has shifted from "training a model once" to long-term data engineering that involves "continuously discovering data, generating annotations, validating quality, and feeding back into the model."

[0053] The sixth issue is the long model iteration cycle. A typical iteration often involves multiple stages, including online problem discovery, sample collection, manual screening, relabeling, training, evaluation, regression validation, and online phase-out. Each stage can become a bottleneck. Without automated error diagnosis and sample binning mechanisms, the model iteration cycle can expand from an "algorithm training cycle" to a "data discovery and quality verification cycle."

[0054] Overall, traditional technical solutions suffer from several common problems: First, they rely on human experience, resulting in high costs for data collection and error attribution; second, sample selection is prone to being blind, and the proportion of truly valuable difficult samples is unstable; third, error types lack structured representation, making it impossible to directly drive data binning and specialized training; fourth, the model iteration chain is long, requiring multiple stages such as manual discovery, collection, selection, annotation, training, and evaluation; and fifth, simple mixed training can easily lead to capability conflicts, making it difficult to stably improve long-tail capabilities such as transparent areas, shadows, edges, and small objects.

[0055] Therefore, it is evident that the core difficulty in image segmentation training is not "whether a model is available," but rather how to find truly valuable error samples at low cost and continuously, and transform these error samples into trainable, evaluable, and iterative data assets. This is also the practical significance of the inventors' proposed "customized VLM (Vision Language Model) quality diagnostic tool + structured error attribution + error subdomain training + weight fusion closed loop": the technical solution of this application attempts to automate the originally fragmented processes of manual quality inspection, error attribution, data binning, and model iteration, thereby shortening the iteration cycle, improving the efficiency of long-tail problem repair, and enabling the segmentation model to have continuous evolution capabilities.

[0056] The image segmentation model training method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. Server 104 can segment the target object in the sample image using a training image segmentation model to obtain an image segmentation mask. Server 104 can generate a highlight foreground image and an edge contour image of the sample image based on the image segmentation mask. Server 104 can input the sample image, highlight foreground image, and edge contour image into a pre-trained segmentation quality diagnostic model to obtain the error attribution results for the erroneous samples. Server 104 can divide the erroneous samples according to the error attribution results to obtain multiple subsets of erroneous samples corresponding to each segmentation error type. Server 104 can use each subset of erroneous samples to train a corresponding training image segmentation model to obtain a sub-model for each segmentation error type. The weights of each sub-model for each sub-model are fused to obtain the trained image segmentation model. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, etc. Server 104 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0057] In one exemplary embodiment, such as Figure 2 As shown, a training method for an image segmentation model is provided. This embodiment applies this method to... Figure 1 The method is illustrated using server 104 as an example. In this embodiment, the method includes the following steps S202 to S210. Wherein:

[0058] Step S202: The target object in the sample image is segmented using the image segmentation model to be trained, and an image segmentation mask is obtained.

[0059] The image segmentation model to be trained (which can be called Model A) can be a mainstream image segmentation model. Its specific type can depend on the business needs in the actual application, such as a human portrait segmentation model, a commodity subject segmentation model, a general foreground segmentation model, a semantic segmentation model, or an instance segmentation model.

[0060] Sample images can come from business image libraries (business scenario data), online user data, test sets (test image libraries), historical failure sample libraries, etc., or they can come from manually collected data to expand the sample coverage and enable the model to face real and complex scenarios.

[0061] Understandably, to ensure stable inference in subsequent models, sample images can be images that have undergone basic preprocessing of the original input images. This preprocessing can include operations such as image format standardization, size scaling, orientation correction, color space conversion, and outlier filtering. For example, images from different sources can be uniformly converted to RGB format and then resized to a fixed size or proportionally scaled to a specified long side range according to the input requirements of Model A.

[0062] The target object can be any object, region, or background present in the image. Therefore, an image segmentation mask is a tool for precisely locating the target object. The mask indicates which regions belong to the foreground and which belong to the background. An image segmentation mask can be a binary mask or a probability mask image with confidence levels. If the model outputs a probability mask image, it can be converted to a binary mask based on a preset threshold, while retaining the original probability mask image for subsequent identification of uncertain regions in the model.

[0063] In practical applications, server 104 can use the existing segmentation model Model A to perform batch forward inference on a large number of input sample images img. It can use Model A to perform pixel-level recognition of target objects in the input sample images img, segment out the boundaries, detailed structures, transparent areas, shadow areas and foreground-background relationships of target objects in the image, and generate corresponding prediction masks, i.e., image segmentation masks.

[0064] Step S204: Generate a highlight foreground image and an edge contour image of the sample image based on the image segmentation mask.

[0065] Among them, the highlight foreground map, or highlight blending map, can be used to identify the area of ​​the target object in the sample image to show the spatial coverage relationship between the mask and the original image, so that it can be seen intuitively whether the segmented area is covered correctly; the edge contour map, or edge visualization map, can be used to identify the edge of the target object in the sample image to show the matching relationship between the mask boundary and the real contour of the target object in the original image, so that it can be checked whether the contour is accurate and whether there are jagged edges, missing edges, or background missampling.

[0066] In the specific implementation, server 104 can use the image segmentation mask and sample image img to mark the foreground region of the target object represented by the mask with a highlight color and generate a highlight fusion image merge_img; and can extract the edge contour of the target object and generate an edge contour image edge_img. In this way, the mask output by Model A is converted into a variety of visualization results that can be understood and judged by the subsequent segmentation quality diagnosis model. That is, the image segmentation mask, the highlight foreground image merge_img, and the edge contour image edge_img can be used as the visualization of the segmentation results.

[0067] Step S206: Input the sample image, the highlighted foreground image, and the edge contour image into the pre-trained segmentation quality diagnosis model to obtain the error attribution results of the erroneous samples.

[0068] Among them, the segmentation quality diagnostic model, also known as the quality diagnostic tool, can be a large visual language model (VLM) that has been customized and trained for the task of judging image segmentation quality.

[0069] Error samples are sample images corresponding to segmentation results containing segmentation errors. Understandably, the segmentation quality diagnostic model can combine the original sample image (img), the highlight foreground image (merge_img), and the edge contour image (edge_img) to determine whether there are problems with the segmentation results of Model A, thus identifying error sample images containing segmentation errors.

[0070] For example, VLM can mainly determine the following key issues: whether the subject is complete, whether there are any omissions in the classification of areas such as body, clothing, product parts, and handheld objects; whether the edge accuracy meets the standards, whether there are rough edges, jagged edges, excessive shrinkage, or background misidentification; whether the semi-transparent areas are natural, such as whether areas such as gauze, glass, and hair edges are hard-cut; and whether the shadow areas are handled properly, such as whether natural shadows are incorrectly deleted, or whether background shadows are mistakenly treated as foreground.

[0071] In practical applications, server 104 can input sample images, highlight foreground images, and edge contour images into a pre-trained segmentation quality diagnostic model (VLM). The segmentation quality diagnostic model analyzes and judges the segmentation quality of the image segmentation results, diagnoses sample images with segmentation errors, and transforms the diagnosed error samples into structured error attribution results, such as outputting structured JSON (JavaScript Object Notation, a lightweight data exchange format).

[0072] Step S208: Based on the error attribution results, the error samples are divided to obtain multiple error sample subsets.

[0073] Each subset of error samples (or sub-training set, subdomain dataset) corresponds to a major segmentation error type or a similar segmentation error type.

[0074] Segmentation error types, also known as error attribution dimensions, can be used to characterize different long-tailed complex scene problems in image segmentation tasks, including core dimensions such as subject integrity, edge accuracy, background mis-absorption, semi-transparent regions, shadow regions, and small structures. In addition to specifying the exact segmentation error type, the error attribution results also include sample routing, i.e., recommended subsets for inclusion in the training dataset. This is used to determine which training subset a sample should enter based on the error type, such as subsets for incomplete subjects, edge errors, semi-transparent areas, shadows, background mis-absorption, and small structures, to facilitate more efficient data binning in subsequent steps.

[0075] Among these criteria, "Subject Integrity" determines whether the target subject is fully preserved. For example, in a portrait, does it omit hair, arms, clothing, or held objects? In a product, does it omit accessories, handles, shoulder straps, or small structures? The corresponding error sample subset is the "Subject Incompleteness" subset. "Edge Precision" determines whether the mask boundaries conform to the true contours of the target. For example, does it check for jagged edges, outward or inward borders, hardened contours, or excessively smooth edges? The corresponding error sample subset is the "Edge Error" subset. "Background Mis-Inclusion" determines whether the background is incorrectly included in the foreground. For example, complex backgrounds, similar-colored backgrounds, shadowed backgrounds, or reflective backgrounds are incorrectly included in the mask. The corresponding error sample subset is the "Background Mis-Inclusion" subset. "Semi-Transparent Areas" determines whether the segmentation of areas such as gauze, glass, hair, smoke, and transparent plastic is natural. For example, does it involve hard cutting, loss of transparency, or white / black edges? The corresponding error sample subset is the "Semi-Transparent Areas" subset. The shadow area is used to determine whether natural shadows have been properly preserved or excluded. For example, whether the subject's projection has been incorrectly deleted, whether background shadows have been mistakenly treated as foreground, and whether contact shadows are broken. The corresponding subset of incorrect samples is the shadow subset. The fine structure is used to determine whether small structures such as thin rods, hair strands, ropes, ornaments, and product accessories have been missed or misclassified. The corresponding subset of incorrect samples is the fine structure omission subset.

[0076] In the specific implementation, server 104 can allocate the error samples corresponding to a certain segmentation error type to the corresponding error sample subset according to the sample routing recommendation field in the error attribution result, so as to obtain the error sample subset corresponding to each segmentation error type.

[0077] Step S210: Train a corresponding image segmentation model using each subset of error samples to obtain a sub-model for each segmentation error type; fuse the weights of each sub-model for each sub-model to obtain the trained image segmentation model.

[0078] Understandably, a special sub-model can be a model that performs reinforcement learning for one or more segmentation error types, and each sub-model can provide specialized image analysis capabilities for a specific type of difficult scenario.

[0079] In practical applications, specialized training for error sample subsets corresponding to each segmentation error type can start from the checkpoint of the same base model Model A (i.e., the image segmentation model to be trained), and fine-tune all parameters using error sample subsets from different subdomains. This ensures that the specialized sub-models and Model A have the same model structure and number of parameters, guaranteeing that the sub-models for each specialized task are in similar parameter spaces, which facilitates subsequent weight fusion.

[0080] Server 104 can train a segmentation model for each image under test using each subset of error samples, resulting in a special sub-model for each segmentation error type. For example, training with a subset of incomplete subjects yields a model for complete subjects (Model_subject), which focuses on improving the ability to preserve subjects and appendages; training with a subset of edge errors (edge ​​refinement subset) yields a model for edge refinement (Model_boundary), which focuses on improving contour fitting and edge details; training with a subset of semi-transparent data yields a model for semi-transparent data processing (Model_transparent), which focuses on improving the ability to handle transparent areas and soft boundaries; training with a subset of shadow data yields a model for shadow processing (Model_shadow), which focuses on improving the ability to identify and preserve natural shadows; and training with a subset of background misclassification data yields a model for background misclassification (Model_background), which focuses on reducing misclassification of complex backgrounds.

[0081] Furthermore, the weights of the various task-specific sub-models are fused to obtain the trained image segmentation model (referred to as Model B), thus integrating the specialized capabilities of multiple special sub-models into a unified model. For example, the edge precision sub-model helps Model B learn more refined edge contours of target objects, the semi-transparency / shadow sub-model helps Model B learn more natural transparency transitions and shadow processing, and the subject integrity sub-model helps Model B improve the integrity of target objects and reduce region omissions. This allows Model B to absorb the specialized advantages of each sub-model while maintaining its overall generalization ability.

[0082] The training method for the aforementioned image segmentation model involves segmenting the target object in a sample image using the image segmentation model to be trained, obtaining an initial model segmentation prediction image segmentation mask. Based on the image segmentation mask, a highlight foreground image and an edge contour image of the sample image are generated, providing various visual auxiliary images for subsequent segmentation quality diagnosis. Then, the sample image, highlight foreground image, and edge contour image are input into a pre-trained segmentation quality diagnosis model to obtain the error attribution results for erroneous samples. Based on the automatic analysis and verification of the segmentation quality diagnosis model, it is possible to perform refined attribution of erroneous samples for various long-tailed complex scene problems in image segmentation, enabling more efficient and accurate detection of segmentation error samples and reducing reliance on human experience. Based on the error attribution results, the error samples are divided into subsets corresponding to each segmentation error type. This achieves standardized and structured data binning of error samples based on the error attribution results, improving the quality of training samples. Then, each subset of error samples is used to train a corresponding image segmentation model, resulting in a sub-model for each segmentation error type. This allows each subset of error samples to focus more on a specific task objective, improving the efficiency of training data utilization. The weights of each sub-model are fused to obtain the trained image segmentation model. This results in the trained image segmentation model incorporating specialized capabilities for different long-tailed complex error types, improving the performance of the image segmentation model in long-tailed complex scenarios.

[0083] In an exemplary embodiment, generating a highlight foreground image and an edge contour image of a sample image based on an image segmentation mask includes: within the region of the target object identified by the image segmentation mask, weightedly fusing the pixels of the sample image with a preset highlight color according to a preset transparency to generate a highlight foreground image; performing edge detection on the sample image based on the image segmentation mask to obtain an edge band of the sample image; and superimposing the edge band onto the sample image in the form of a preset line to generate an edge contour image.

[0084] In practical implementation, since the prediction mask is often not intuitive enough, it is not easy for visual language models or humans to quickly determine whether the edge, transparent area and shadow processing is reasonable if only the image segmentation mask is used. Therefore, the highlight foreground map and edge contour map can be used as auxiliary maps for subsequent quality diagnosis.

[0085] Server 104 can perform weighted fusion of pixels from the original sample image with a preset highlight color (a specified highlight color) within the target object region identified by the image segmentation mask, according to a preset transparency, to cover the mask region with a semi-transparent color and generate a highlight foreground image, i.e., the highlight fusion image merge_img. For regions that are not the target object identified by the image segmentation mask, i.e., non-mask regions, the original image remains unchanged. This image is a visualization of the predicted mask superimposed on the original image, allowing the subsequent segmentation quality diagnostic model (VLM) to intuitively see whether the segmented region covers the correct target object. This allows VLM to simultaneously see the original image content and the mask region predicted by Model A, facilitating the determination of whether the subject is missed, whether the background is mistakenly identified, or whether the target boundary is offset.

[0086] Simultaneously, server 104 can perform edge detection (e.g., Canny edge detection) on the sample image based on the image segmentation mask, obtaining the edge bands of the sample image. Specifically, the mask boundaries, i.e., the edge bands of the sample image, can be obtained through morphological gradients, boundary extraction, or mask dilation and erosion. For example, the mask can be dilated once and eroded once, and the difference between the two can yield the edge bands. Then, the edge bands are superimposed onto the sample image with pre-defined prominent lines to form an edge visualization result, generating an edge contour map edge_img. The edge contour map edge_img is mainly used to highlight the contour boundaries of the mask, and is particularly important for details such as hair, clothing edges, product outlines, and transparent object boundaries, helping subsequent VLM determine whether the edge accuracy meets the standards.

[0087] It should be noted that after obtaining the image segmentation mask, but before generating the highlight foreground image and edge contour image of the sample image, necessary post-processing is performed on the mask. This includes operations such as removing small noise areas, filling holes inside the main body, preserving the largest connected components, smoothing edges, and correcting abnormal fragments. These post-processing operations remove obvious invalid noise from the mask, ensuring that the output segmentation result has basic diagnostic value. For scenarios requiring edge analysis, both the pre- and post-processed masks can be retained to prevent post-processing from masking errors in the original model.

[0088] Furthermore, for image segmentation tasks sensitive to semi-transparent and shadowed areas, auxiliary visualizations can be generated. For example, the segmented foreground can be pasted onto white, black, gray, or checkerboard backgrounds to create composite images under different backgrounds, which can be used to observe whether the transparency transitions are natural, whether the edges are white or black, and whether the shadows are hard-cut.

[0089] The technical solution of this embodiment generates a highlight foreground map and an edge contour map based on the image segmentation mask. This can intuitively identify the relationship between the segmentation mask and the foreground region and edge contour corresponding to the target in the image, thereby providing an intuitive and visual auxiliary map for subsequent quality diagnosis of the segmentation results.

[0090] In an exemplary embodiment, the sample image, the highlighted foreground image, and the edge contour image are input into a pre-trained segmentation quality diagnostic model to obtain the error attribution result of the erroneous sample. This includes: obtaining a target visual language large model and training samples for the target visual language large model; labeling the training samples according to the segmentation error type to obtain training sample labels; configuring the training parameters for the target visual language large model as a low-rank matrix; fine-tuning the target visual language large model using the low-rank matrix and the training sample labels to obtain the segmentation quality diagnostic model; and inputting the sample image, the highlighted foreground image, the edge contour image, and pre-defined prompt words into the segmentation quality diagnostic model to output the error attribution result.

[0091] The prompt word is used to instruct the Virtual Segmentation Quality Diagnostic Model (VLM) to output error attribution results according to a predefined structured standard format. An example of a prompt word for VLM is shown below:

[0092] Please perform a quality check on the current segmentation results.

[0093] Input image description:

[0094] The first image is the original image (img); the second image is the mask highlight merge image (merge_img), where the highlighted areas represent the foreground predicted by the model; the third image is the edge image (edge_img), where the edge lines represent the contours of the predicted mask.

[0095] Please assess the segmentation quality based on the three images and output structured JSON. Please focus on checking the following issues:

[0096] 1. Is the main body complete? Are there any obvious omissions in the target subject? Are there any missing hair, limbs, clothing, product parts, handheld items, accessories, etc.?

[0097] 2. Does the edge accuracy meet the standard? Does the edge conform to the actual target contour? Are there jagged edges, outward expansion, inward contraction, excessive smoothing, or edge breakage?

[0098] 3. Background mis-inclusion: Whether the background, clutter, similar color areas, or complex textures are mistakenly included in the foreground.

[0099] 4. Whether the semi-transparent areas are natural: Are there any hard cuts, gaps, or unnatural transitions in areas such as gauze, glass, hair, transparent plastic, or smoke?

[0100] 5. Are the shadow areas natural? Have the subject's projection and contact shadows been mistakenly deleted, or have the background shadows been mistakenly treated as the foreground?

[0101] 6. Are small details preserved? Have any small details such as thin ropes, hair strands, ornaments, or small parts of the product been omitted?

[0102] 7. Please determine whether this sample needs to be relabeled and recommend its inclusion in one or more error sub-training sets.

[0103] In a possible implementation, the pre-trained segmentation quality diagnostic model, i.e. the quality diagnosticer, can be fine-tuned for the target visual language large model (VLM) based on a lightweight training method of LoRA (Low-Rank Adaptation).

[0104] A mainstream Visual Language Model (VLM) can be selected as the target VLM, and multiple sets of training samples can be obtained for the target VLM. Understandably, each set of training samples still needs to include the original sample image (img), the highlighted foreground image (merge_img), and the edge contour image (edge_img) used for VLM training. The training samples for VLM can cover qualified samples, slightly erroneous samples, moderately erroneous samples, and severely erroneous samples for image segmentation quality inspection tasks, to obtain customized segmentation quality inspection data so that VLM can learn the judgment criteria under different quality levels.

[0105] Then, labeling tools or manual annotation can be used to label the training samples with segmentation error types, and the error types of the segmentation results can be clearly identified, such as missing subject, rough edges, background mis-absorption, hard cutting of semi-transparent areas, incorrect deletion of shadows, omission of handheld objects, omission of accessories, etc., to obtain training sample labels.

[0106] Based on the LoRA training method, training parameters such as the rank dimension, training resolution, LoRA scaling factor (alpha coefficient), learning rate (LR), and number of training steps of the target visual language large model can be set, and these training parameters can be configured as a low-rank matrix (low-rank adaptation matrix). The target visual language large model can be fine-tuned using this low-rank matrix to obtain a segmentation quality diagnostic model.

[0107] Furthermore, server 104 can input the sample image `img`, the highlighted foreground image `merge_img`, and the edge contour image `edge_img` into the segmentation quality diagnostic model in a fixed order. The pre-trained segmentation quality diagnostic model can judge the image segmentation quality according to pre-set prompts and training sample labels. The segmentation quality diagnostic model outputs error attribution results in a strictly formatted JSON format according to fixed standards. The error attribution results in the structured JSON format can include information such as the overall quality conclusion, scores for each quality dimension, error type, error severity, description of the error region, whether re-labeling is needed, recommended sub-training set, and diagnostic confidence.

[0108] The technical solution of this embodiment trains a general visual language model (VLM) using LoRA to obtain a segmentation quality diagnostic model specifically for segmentation quality inspection. By analyzing and diagnosing the segmentation results through the segmentation quality diagnostic model, problems such as missing subjects, rough edges, abnormal transparent areas, and incorrect shadow cutting can be exposed and quickly discovered in large-scale data in advance. This enables efficient and accurate identification of segmentation failure samples, and the diagnosed failure sample data is more targeted. Furthermore, the segmentation quality diagnostic model outputs structured error attribution results, making the quality diagnosis results more resolvable, thereby providing standardized data support for subsequent error sample binning and specialized training data organization.

[0109] In an exemplary embodiment, the error samples are divided into multiple error sample subsets based on the error attribution results, including: assigning the error samples corresponding to each segmentation error type to the initial error sample subset corresponding to each segmentation error type based on the error attribution results; re-labeling the error samples according to the segmentation error type corresponding to the initial error sample subset to obtain error sample labels; and obtaining the error sample subset corresponding to each segmentation error type based on the error sample labels, sample images, and image segmentation masks.

[0110] As an example, server 104 can divide error samples into different initial sub-training sets, i.e., the initial error sample subsets, based on the recommended bucket field (i.e., the field of the recommended sub-training set, such as the recommended_buckets field) in the structured JSON error attribution results output by VLM.

[0111] Furthermore, the labels of the error samples can be re-labeled according to the segmentation error type corresponding to the initial subset of error samples, resulting in error sample labels (error type labels). For semi-transparent or shadow tasks, auxiliary labels such as alpha mask, trimap, boundary region mask, and shadow region mask can also be added. Then, for each segmentation error type, the error sample labels, sample images, and image segmentation masks can be organized into a corresponding trainable subdomain dataset, i.e., the error sample subset. The error sample subset corresponding to each segmentation error type should at least include the original sample image (img), the predicted mask, the highlighted foreground image (merge_img), the edge contour image (edge_img), the error sample labels, and the training weights. Thus, each sub-training set corresponds to a major segmentation error type or a similar segmentation error type, used to train task-specific sub-models corresponding to each segmentation error type.

[0112] For example, the `subject_completeness_set` is used to collect samples of incomplete subjects, including missing hands, hair, clothing, accessories, and objects being held. The `boundary_refinement_set` is used to collect samples of inaccurate edges, including jagged edges, outward-expanding edges, inward-expanding edges, and overly smooth boundaries. The `transparent_region_set` is used to collect issues with semi-transparent regions, including areas like sheer fabric, glass, hair, and transparent plastic that are harshly cut or have unnatural transitions between transparent areas. The `shadow_quality_set` is used to collect shadow issues, including the accidental deletion of natural contact shadows, broken subject projections, and the accidental inclusion of background shadows. The `background_false_positive_set` is used to collect background mis-inclusion issues, including complex backgrounds, backgrounds with similar colors, and reflective areas being incorrectly included in the foreground.

[0113] For example, in terms of data organization, each subdomain dataset can be divided into a training set, a validation set, and a regression test set. The training set is used for learning the special task sub-model, the validation set is used to select the best checkpoint and fusion weights, and the regression test set is used to determine whether capability degradation has occurred.

[0114] Furthermore, the quality and quantity of the subdomain dataset can be controlled; not all erroneous samples diagnosed by VLM are directly used for training. Low-confidence samples, samples with unclear attributions, and samples with disputed annotations can enter a manual review pool. Samples with serious errors and high diagnostic confidence are given priority for annotation and training. This ensures the effective sample ratio of the subdomain dataset.

[0115] The technical solution of this embodiment can decompose the segmentation error samples into multiple error type dimensions based on the structured error attribution results output by VLM, and assign the error samples to the error sample subsets corresponding to the error types. Thus, training based on the error sample subsets corresponding to each segmentation error type improves the effective utilization of training data, thereby making the capability objectives of various special task sub-models clearer and improving the special capabilities of the image segmentation model.

[0116] In an exemplary embodiment, the weights of each specific task sub-model are fused to obtain a trained image segmentation model, including: obtaining the performance index of each specific task sub-model; determining the parameter fusion weights corresponding to each specific task sub-model based on the performance index; fusing the trainable parameters in the image segmentation model to be trained and the trainable parameters in each specific task sub-model based on the parameter fusion weights to obtain the target trainable parameters of the trained image segmentation model; and obtaining the trained image segmentation model based on the target trainable parameters.

[0117] As one possible implementation, for each type of segmentation error, the performance metrics (e.g., accuracy, precision, recall, IoU, etc.) of the special sub-model in the corresponding sub-validation set or business scenario can be obtained for each special sub-model. Based on these performance metrics, the parameter fusion weights for each special sub-model can be determined. These fusion weights can be normalized based on the number of erroneous sample subsets and then inverted to ensure balance, or they can be manually set for better practical results.

[0118] Based on the parameter fusion weights, the trainable parameters in the image segmentation model Model A to be trained are... and the trainable parameters in each special sub-model By fusing the data, we obtain the target trainable parameters of the trained image segmentation model Model B. For example, the target trainable parameters can be obtained using the following formula. :

[0119]

[0120] in, This represents the parameters of the fused Model B, i.e., the target trainable parameters. This represents the parameters of the original Model A. , , , , … , etc. represent the parameters of the special sub-model corresponding to different segmentation error types; This represents the parameter fusion weights corresponding to the sub-models of each specific task, and the sum of all weights is 1. Therefore, the trainable parameters can be determined based on the target. The trained image segmentation model, Model B, is obtained.

[0121] Subsequently, the fused Model B can be trained uniformly. The training data can include a general high-quality dataset, re-labeled datasets for each subdomain (i.e., subsets of error samples), and key samples from historical regression tests. The general high-quality dataset is used to maintain the model's basic capabilities and prevent excessive bias towards difficult samples; the re-labeled datasets for each subdomain are used to absorb specialized capabilities; and historical regression samples are used to prevent problems already solved in older versions from regressing. Since the subsets of error samples in each subdomain are usually small in number but high in value, a weighted sampling strategy can be used. A fixed proportion of ordinary samples and difficult subdomain samples can be added to each training batch, for example, 50% ordinary samples, 20% marginal samples, 10% semi-transparent samples, 10% shaded samples, and 10% samples with complete subject integrity. The specific proportions can be adjusted based on the distribution of business problems in the actual application and the performance on the validation set.

[0122] The technical solution of this embodiment can integrate the specialized capabilities of multiple special sub-models into a unified model, Model B, without increasing inference costs. Compared to Model A, Model B is not simply trained on a larger batch of data, but has undergone structured error correction. As a new generation segmentation model obtained through error sample diagnosis, subdomain training, and weight fusion, Model B outperforms Model A in key areas such as subject integrity, edge accuracy, semi-transparent areas, and shadow naturalness, while maintaining stable performance in typical scenarios.

[0123] In an exemplary embodiment, the method further includes: using the trained image segmentation model as a new image segmentation model to be trained; returning to the step of segmenting the target object in the sample image using the image segmentation model to be trained to obtain the image segmentation result, until the trained image segmentation model meets the preset training termination condition.

[0124] In the specific implementation, after the trained image segmentation model Model B is generated and verified, it can return to the front-end segmentation inference stage. Model B is used as a new image segmentation model Model A to be trained, and the new image data is automatically segmented again to generate img, mask, merge_img and edge_img. The model is then sent to the VLM quality diagnostic tool for quality diagnosis and error attribution. After dividing the subdomain error sample subsets corresponding to each error type, it enters the specialized training and model fusion again to iteratively optimize Model B.

[0125] During the iterative optimization of Model B, it can replace or run in parallel with Model A in the next round of data mining. When there are multiple Model A images to be trained, Model B can be iteratively optimized in parallel with Model A; when there is only one Model A images to be trained, Model B can replace Model A in iterative optimization.

[0126] Therefore, during the iterative optimization process of Model B, the Quality Diagnostic Tool (VLM) can identify errors that still exist in Model B. These errors may include old problems that have not been fully fixed, or new problems introduced by the new model. For example, Model B may improve edge accuracy, but it may increase false positives in complex backgrounds; or it may improve shadow preservation, but in some scenarios, it may mistake background shadows for foreground. The system can rebuild the subdomain dataset based on the new error attribution results, re-label, retrain the special sub-model, and fuse them to generate the next version of the model. This enables the trained image segmentation model Model B to continuously evolve and fix problems, continuously discover new long-tail problems, and continuously optimize for real-world failure scenarios, forming a stable data loop and a continuous evolution path for the model.

[0127] For the convenience of those skilled in the art, Figure 3 This document provides an example of an iterative training method for an image segmentation model based on visual language quality diagnosis and error subdomain enhancement, thereby providing a detailed description of the training method for the image segmentation model using a specific embodiment. It is important to understand that the following description is merely illustrative and not intended to limit the scope of the application.

[0128] The technical solution provided in this application is an iterative training method for image segmentation models based on customized Visual Language Model (VLM) quality diagnosis. The overall process can be divided into three stages: automatic segmentation and result generation, VLM diagnosis and error attribution, and subdomain training and fusion iteration. Its core idea is not simply to "judge" the segmentation results as "good or bad," but rather to use a customized visual language model to perform structured analysis on the segmentation results, clearly identifying where the model goes wrong, and automatically assigning erroneous samples to the corresponding sub-training sets, thereby driving subsequent specialized training and continuous model optimization.

[0129] The core objective of this approach is not simply to train a segmentation model, but to establish a closed-loop system capable of continuously identifying, analyzing, and fixing problems, and then iterating again. In simple terms, the system first uses the current segmentation model, i.e., the image segmentation model to be trained (Model A), to automatically segment a large number of input images, obtaining preliminary mask results. Then, the original image, the highlighted foreground image, and the edge contour image are fed into a customized pre-trained VLM quality diagnostic tool. The VLM judges the quality and problems of the segmentation results, obtaining the error attribution results for the segmentation error samples. Based on the diagnosed error attribution results, the error samples are then assigned to sub-training sets (i.e., error sample subsets) corresponding to different segmentation error types for re-labeling and training, resulting in task-specific sub-models corresponding to different segmentation error types. Finally, the capabilities of multiple task-specific sub-models are merged to obtain a better-performing Model B, which then continues into the next iteration.

[0130] The key value of this framework lies in its transformation of the past process of "identifying problems by examining images, manually classifying them, and retraining," which relied on human experience, into a semi-automated, structured, and sustainably operating data loop. The system doesn't just judge a segmentation result as "good" or "bad," but further determines the type of error it falls into, such as whether the subject is missing, whether the edges are rough, whether the semi-transparent areas are natural, or whether shadows have been incorrectly handled. Based on the error type, it then determines the subsequent training method.

[0131] Specifically, as shown in the figure, in the first stage, image segmentation and result generation occur. Based on the current segmentation model, Model A, the system automatically performs forward inference on a large amount of raw image data (img) to generate a corresponding image segmentation mask, i.e., a prediction mask. After obtaining the prediction mask, necessary post-processing is performed to further generate a highlight foreground image (merge_img) and an edge contour image (edge_img). These results collectively constitute the input for subsequent quality diagnosis, enabling the quality assessment model to not only rely on a single mask but also simultaneously observe the original image, segmented regions, and edge details.

[0132] In the second stage, VLM diagnosis and error attribution are performed. The system inputs `img`, `merge_img`, and `edge_img` into a customized VLM quality diagnostic tool. This diagnostic tool is a large-scale visual language model trained on customized labeled data (LoRA). It can judge whether the sample has segmentation errors based on dimensions such as subject integrity, edge accuracy, naturalness of semi-transparent region segmentation, and naturalness of shadow region segmentation. It further provides the error type, severity, confidence level, and recommended subset for inclusion in the training set, outputting structured JSON error attribution results to organize the VLM diagnostic results into a machine-readable and executable format. In this way, segmentation failure samples can be transformed from disordered problem data into manageable, labeled, and trainable data assets.

[0133] In the third stage, error sample segmentation, subdomain model training, and fusion are performed. Based on the structured error attribution results, the system automatically segments and relabels error samples to organize training data according to error sample type. Different types of error samples are assigned to different data pools, resulting in subsets of error samples corresponding to different segmentation error types. Subsequently, each subset of error samples is used to train a corresponding image segmentation model, obtaining a special sub-model for each segmentation error type. This allows each specialized task sub-model to perform reinforcement learning for a specific error type, forming specialized repair capabilities. Finally, through weight fusion and sub-model guided training, the specialized capabilities of multiple special sub-models are integrated into a unified model, resulting in the final trained image segmentation model, Model B. The core idea of ​​this stage is that different segmentation errors should not be trained together; instead, they should first be split into multiple subdomains according to error type, then reinforced separately, and finally fused together.

[0134] The solution also incorporates an iterative optimization loop. After Model B is generated, the training process doesn't end there; instead, it returns to the front-end segmentation and inference stage. The system uses Model B to segment, diagnose, attribute, re-label, and train sub-models on a new large-scale image dataset, generating diagnostic samples and training data for the next round of model optimization. These are then fused together to generate the next version of the model. Through this cyclical mechanism, the model can continuously discover new long-tail problems and continuously optimize for real-world failure scenarios, forming a stable data loop and a continuous model evolution path. This ensures that the entire system doesn't just complete one model training iteration, but continuously runs, discovers, and fixes problems.

[0135] As can be seen, the core feature of this approach is that the training framework does not blindly collect data or simply perform mixed training. Instead, it first identifies the error types through VLM (Visual Model), and then organizes the data and trains the model around these error types. This reduces the cost of manual screening, increases the utilization rate of high-value samples, shortens the model iteration cycle, and more effectively improves the performance of the segmentation model in complex scenarios such as subject integrity, edge details, semi-transparent areas, and shadow processing.

[0136] Therefore, the technical solution of this application has at least the following technical effects:

[0137] The superiority of this solution lies primarily in its more efficient, accurate, less experience-dependent, and proactive approach to error sample detection. Traditional methods typically rely on manual image review, user feedback, or simple confidence level screening to identify failed samples. This approach suffers from delayed and inefficient problem detection, limited coverage of manual quality checks, and a tendency to miss long-tail issues in large-scale data aggregation. This solution, through customized VLM automatic analysis and diagnostic segmentation results, can proactively expose and quickly identify issues such as missing subjects, rough edges, abnormal transparent areas, and incorrect shadow segmentation in large-scale data. The resulting diagnosed failed samples are more targeted, reducing invalid data collection and annotation, increasing the model's benefit per unit annotation cost, and significantly reducing manual screening costs. This shifts the model's approach from "passively fixing problems after deployment" to "proactively identifying problems before deployment," thus mitigating online experience risks.

[0138] Meanwhile, compared to manual rule-based binning solutions, this solution offers advantages in scalability, sustainability, and greater standardization. By using a customized VLM quality diagnostic tool to perform standardized, structured judgments on misclassified sample data, this solution maintains relatively consistent binning logic across large-scale data sets. This reduces data operation costs and upgrades the management of challenging samples from a manual, experience-based process to an automated data engineering workflow.

[0139] Secondly, the advantage of this approach lies in its more refined and interpretable error attribution. Traditional model iterations often only indicate that a particular sample segmentation is poor, but it's difficult to automatically identify and explain the specific reasons for the error. This approach, through VLM outputting structured error attribution JSON, can not only select difficult samples but also decompose error samples into multiple dimensions such as main body integrity, edge precision, semi-transparent areas, and shadow areas. It automatically assigns error samples to corresponding sub-training sets, thereby reducing manual secondary screening and ensuring that data flow directly serves the training objective. Labeling, training, and evaluation all revolve around clearly defined error subdomains, preventing the indiscriminate use of high-value samples. This provides a clear direction for subsequent data processing and model training, avoiding blind collection and training, and realizing a model learning approach that extends from "sample selection" to "sample understanding and training routing."

[0140] Third, this approach improves the effective utilization of training data, making training objectives clearer and model improvements more stable. A common practice in the past was to mix all misclassified samples and add them to the training set for retraining. However, the optimization objectives for different error types are not consistent, easily leading to mixed training goals and potentially conflicting optimization directions for different error types. This approach divides the misclassified samples into multiple sub-domain datasets and trains specialized sub-models for each, then merges or guides training. This ensures that each type of data serves a clear capability improvement objective, enhancing specialized capabilities while reducing the risk of mutual interference between capabilities, thereby improving the utilization efficiency of labeled data and training resources. Furthermore, compared to direct fine-tuning of a single model, this approach can accumulate multiple specialized capabilities, allowing each sub-model to correspond to a clear capability direction. The model capability structure is clearer, facilitating subsequent weight fusion, model distillation, scene routing, or specialized regression analysis.

[0141] Fourth, this solution is beneficial for addressing long-tail complex scene problems. The difficulties in image segmentation are typically concentrated in scenes such as hair edges, transparent objects, shadows, complex backgrounds, small objects, and attachments. These problems have a low percentage in the ordinary training set but a significant impact on user experience. This solution, through VLM diagnosis and error subdomain binning, further breaks down the segmentation results from "whether it's good or bad" to "where the error is, how serious the error is, what type of error it is, and which training subset it should be included in." This is combined with weighted evaluation based on the performance of each error subdomain validation set, resulting in a more balanced improvement in Model B not only in overall metrics but also in key sub-scenes such as subject integrity, edge accuracy, semi-transparent areas, and shadow areas. This allows for the continuous accumulation of these high-value, difficult samples and targeted improvement in the model's performance in long-tail scenes, alleviating the problem of "improved average metrics but poor user experience in key scenes." This solution is more suitable for solving the long-tail problem in the long-term iteration of industrial segmentation models.

[0142] Fifth, this solution can shorten the model iteration cycle and has continuous iteration capabilities. Traditional processes require manual problem identification, classification, labeling, and training evaluation, resulting in a lengthy process and a lack of a unified error understanding and training scheduling mechanism. This solution feeds Model B back to the front-end inference stage, allowing the new model to continue participating in large-scale graph runs, VLM diagnostics, error attribution, and data updates. This connects segmentation result generation, quality diagnosis, error attribution, sample binning, specialized training, and model fusion into an automated closed loop. This links "problem identification, error attribution, data binning, specialized training, model fusion, and continuous iteration" into a long-term model iteration closed loop, reducing manual intervention and allowing the model to continuously evolve with changes in business data and user scenarios, thus significantly shortening the cycle from problem identification to model update.

[0143] Overall, the core value of this solution lies in transforming segmentation model optimization from traditional "human experience-driven" to "VLM diagnosis-driven data closed-loop optimization." It can not only detect segmentation errors but also interpret them, organize data, guide training, and continuously iterate. It is suitable for business scenarios with high segmentation quality requirements, such as intelligent image matting, portrait segmentation, product subject segmentation, image editing foreground extraction, and video segmentation.

[0144] It should be noted that the specific limitations of the above steps can be found in the specific limitations of the training method for an image segmentation model described above.

[0145] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0146] Based on the same inventive concept, this application also provides a training apparatus for an image segmentation model to implement the training method for the image segmentation model described above. The solution provided by this apparatus is similar to the implementation described in the above method. Therefore, the specific limitations of one or more image segmentation model training apparatus embodiments provided below can be found in the limitations of the image segmentation model training method described above, and will not be repeated here.

[0147] In one exemplary embodiment, such as Figure 4 As shown, a training apparatus for an image segmentation model is provided, the apparatus comprising:

[0148] The image segmentation module 410 is used to segment the target object in the sample image using the image segmentation model to be trained, and obtain the image segmentation mask.

[0149] The segmentation result generation module 420 is used to generate a highlight foreground map and an edge contour map of the sample image based on the image segmentation mask; the highlight foreground map is used to identify the region of the target object in the sample image; the edge contour map is used to identify the edge of the target object in the sample image.

[0150] The segmentation quality diagnosis module 430 is used to input the sample image, the highlight foreground image, and the edge contour image into the pre-trained segmentation quality diagnosis model to obtain the error attribution result of the erroneous sample; the erroneous sample is the sample image corresponding to the segmentation result of the image with segmentation error.

[0151] The error sample segmentation module 440 is used to segment the error samples according to the error attribution results, resulting in multiple error sample subsets; each error sample subset corresponds to a segmentation error type.

[0152] The subdomain training and fusion module 450 is used to train a corresponding image segmentation model to be trained using each subset of error samples to obtain a sub-model for each segmentation error type; the weights of each sub-model for each sub-model for each sub-model for each sub-model for each sub-model for each sub-model are fused to obtain the trained image segmentation model.

[0153] In an exemplary embodiment, the device further includes an iterative optimization module 460, which is used to take the trained image segmentation model as a new image segmentation model to be trained; return to the step of segmenting the target object in the sample image using the image segmentation model to be trained to obtain the image segmentation result, until the trained image segmentation model meets the preset training termination condition.

[0154] The modules in the training device of the aforementioned image segmentation model can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0155] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores training data for an image segmentation model. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a training method for an image segmentation model.

[0156] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0157] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the various embodiments of the image segmentation model training method described above.

[0158] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the training method for the image segmentation model described above.

[0159] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the training method for the image segmentation model described above.

[0160] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic resistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence processors, etc., and are not limited to these.

[0161] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0162] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A training method for an image segmentation model, characterized in that, The method includes: The target object in the sample image is segmented using the image segmentation model to be trained, and the image segmentation mask is obtained. Based on the image segmentation mask, a highlight foreground image and an edge contour image of the sample image are generated; the highlight foreground image is used to identify the region of the target object in the sample image; the edge contour image is used to identify the edges of the target object in the sample image. The sample image, the highlighted foreground image, and the edge contour image are input into a pre-trained segmentation quality diagnostic model to obtain the error attribution results of the erroneous samples; the erroneous samples are the sample images corresponding to the segmentation results of images with segmentation errors. Based on the error attribution results, the error samples are divided into multiple error sample subsets; each error sample subset corresponds to a segmentation error type. Each of the aforementioned error sample subsets is used to train a corresponding image segmentation model to be trained, thereby obtaining a specific task sub-model corresponding to each of the aforementioned segmentation error types; the weights of each specific task sub-model are fused to obtain the trained image segmentation model.

2. The method according to claim 1, characterized in that, The step of inputting the sample image, the highlighted foreground image, and the edge contour image into a pre-trained segmentation quality diagnostic model to obtain the error attribution results for erroneous samples includes: Obtain a large target visual language model and training samples for the target visual language model, and label the training samples according to the segmentation error type to obtain training sample labels; The training parameters for the target visual language large model are configured as a low-rank matrix. The target visual language large model is then fine-tuned using the low-rank matrix and the training sample labels to obtain the segmentation quality diagnostic model. The sample image, the highlighted foreground image, the edge contour image, and the pre-defined prompt words are input into the segmentation quality diagnostic model, and the error attribution result is output; wherein, the prompt words are used to instruct the segmentation quality diagnostic model to output the error attribution result according to the set structured standard format.

3. The method according to claim 1, characterized in that, The error samples are divided according to the error attribution results to obtain multiple error sample subsets, including: Based on the error attribution results, the error samples corresponding to each segmentation error type are assigned to the initial error sample subset corresponding to each segmentation error type; According to the segmentation error type corresponding to the initial subset of error samples, the labels of the error samples are re-labeled to obtain error sample labels; Based on the error sample labels, the sample images, and the image segmentation mask, the error sample subsets corresponding to each segmentation error type are obtained.

4. The method according to claim 1, characterized in that, The process of fusing the weights of each of the specific task sub-models to obtain the trained image segmentation model includes: Obtain the performance metrics of each of the special task sub-models, and determine the parameter fusion weights corresponding to each of the special task sub-models based on the performance metrics; Based on the parameter fusion weights, the trainable parameters in the image segmentation model to be trained and the trainable parameters in each of the specific task sub-models are fused to obtain the target trainable parameters of the trained image segmentation model. The trained image segmentation model is obtained based on the target trainable parameters.

5. The method according to claim 1, characterized in that, The step of generating the highlight foreground image and edge contour image of the sample image based on the image segmentation mask includes: Within the target object area identified by the image segmentation mask, the pixels of the sample image are weighted and fused with a preset highlight color according to a preset transparency to generate the highlight foreground image; Based on the image segmentation mask, edge detection is performed on the sample image to obtain the edge band of the sample image; the edge band is superimposed on the sample image in the form of a preset line to generate the edge contour map.

6. The method according to claim 1, characterized in that, The method further includes: The trained image segmentation model is used as the new image segmentation model to be trained. Return to the step of segmenting the target object in the sample image using the image segmentation model to be trained, and obtaining the image segmentation result, until the trained image segmentation model meets the preset training termination condition.

7. A training device for an image segmentation model, characterized in that, The device includes: The image segmentation module is used to segment target objects in sample images using the image segmentation model to be trained, and obtain image segmentation masks. The segmentation result generation module is used to generate a highlight foreground image and an edge contour image of the sample image based on the image segmentation mask; the highlight foreground image is used to identify the region of the target object in the sample image; the edge contour image is used to identify the edge of the target object in the sample image; The segmentation quality diagnosis module is used to input the sample image, the highlighted foreground image, and the edge contour image into a pre-trained segmentation quality diagnosis model to obtain the error attribution result of the erroneous sample; the erroneous sample is the sample image corresponding to the image segmentation result with segmentation error; The error sample segmentation module is used to segment the error samples according to the error attribution results, obtaining multiple error sample subsets; each error sample subset corresponds to a segmentation error type; The subdomain training and fusion module is used to train a corresponding image segmentation model to be trained using each of the error sample subsets to obtain a special task sub-model corresponding to each segmentation error type; and to fuse the weights of each special task sub-model to obtain a trained image segmentation model.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.