Method, device and storage medium for training matting model and method and device for matting

Abstract

The application discloses a matting model training method, a matting method, a matting device, equipment and a storage medium, wherein the method comprises the following steps: acquiring a first sample image with a label and a second sample image without a label, the first sample image and the second sample image both comprise a target object, and the label is a matting result of the first sample image; and training a student model in a manner of knowledge distillation of a joint multi-teacher model and semi-supervised learning based on the first sample image and the second sample image, to obtain a matting model. Through the above manner, the semantic representation and information extraction capability and the generalization capability of the matting small model can be improved simultaneously.

Description

Training methods, matting techniques, devices, equipment, and storage media for image matting models Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method for training a matting model, a matting method, an apparatus, a device, and a storage medium. Background Technology

[0002] Image segmentation is a classic challenge in computer vision research and has become a hot topic in the field of image understanding. It is the first step in image analysis, the foundation of computer vision, and an important component of image understanding; it is also one of the most difficult problems in image processing. Image segmentation refers to dividing an image into several non-overlapping regions based on features such as grayscale, color, spatial texture, and geometric shape, ensuring that these features exhibit consistency or similarity within the same region, while showing significant differences between different regions. Simply put, it's about separating the target from the background in an image, commonly known as image matting.

[0003] Subject matting algorithms are algorithms that segment the main subject from an input image based on saliency matting techniques. However, small subject matting models are often limited by their number of parameters, making it difficult to achieve the semantic representation and information extraction capabilities of larger models, and they also cannot achieve the same level of generalization ability. Summary of the Invention

[0004] The first aspect of this application provides a training method for a matting model, comprising: acquiring a labeled first sample image and an unlabeled second sample image, both the first and second sample images including the target object, and the label being the matting result of the first sample image; and training a student model based on the first and second sample images using a combination of knowledge distillation from a multi-teacher model and semi-supervised learning to obtain a matting model.

[0005] A second aspect of this application provides a training apparatus for a matting model, comprising: an acquisition module for acquiring a labeled first sample image and an unlabeled second sample image, both the first and second sample images including a target object, and the label being the matting result of the first sample image; and a training module for training a student model based on the first and second sample images using a combination of knowledge distillation from a multi-teacher model and semi-supervised learning to obtain a matting model.

[0006] A third aspect of this application provides a matting method, comprising: acquiring an image to be processed, the image to be processed containing a target object; and processing the image to be processed using a matting model trained by the aforementioned matting model training method to obtain a matted image.

[0007] The fourth aspect of this application provides an electronic device, an image matting apparatus, comprising: an acquisition module for acquiring an image to be processed, the image to be processed containing a target object; and a matting module for processing the image to be processed using a matting model trained by the matting model training method described above, to obtain a matted image.

[0008] The fifth aspect of this application provides an electronic device including a memory and a processor coupled to each other, the memory being used to store program data and the processor being used to execute the program data to implement the aforementioned method.

[0009] A sixth aspect of this application provides a computer-readable storage medium storing program data, which, when executed by a processor, is used to implement the aforementioned method.

[0010] The beneficial effects of this application are as follows: Unlike the prior art, this application obtains a labeled first sample image and an unlabeled second sample image, both of which include the target object, and the label is the matting result of the first sample image; based on the first and second sample images, the student model is trained by a combination of knowledge distillation of a multi-teacher model and semi-supervised learning to obtain a matting model, which can not only improve the semantic representation and information extraction capabilities of the small matting model, but also improve its generalization ability. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in this application, the accompanying drawings required in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Among them:

[0012] Figure 1 is a flowchart illustrating an embodiment of the training method for the matting model of this application;

[0013] Figure 2 is a flowchart illustrating another embodiment of step S12 in Figure 1;

[0014] Figure 3 is a flowchart illustrating another embodiment of step S122 in Figure 2;

[0015] Figure 4 is another flowchart illustrating an embodiment of the training method for the matting model of this application;

[0016] Figure 5 is a flowchart illustrating an embodiment of the image matting method of this application;

[0017] Figure 6 is a schematic block diagram of an embodiment of the training device for the matting model of this application;

[0018] Figure 7 is a schematic block diagram of an embodiment of the image matting device of this application;

[0019] Figure 8 is a schematic block diagram of the structure of an embodiment of the electronic device of this application;

[0020] Figure 9 is a schematic block diagram of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0021] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0022] The terms "first" and "second" in this application are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0024] Please refer to Figure 1, which is a flowchart illustrating an embodiment of the training method for the image matting model of this application. In this embodiment, the executing entity is an electronic device, specifically an electronic device with computing capabilities, such as a computer, mobile phone, or server.

[0025] The method may include the following steps:

[0026] Step S11: Obtain a labeled first sample image and an unlabeled second sample image. Both the first and second sample images include the target object, and the label is the cutout result of the first sample image.

[0027] The target object can include, but is not limited to: human figures (including any part of a person), animals (such as pets like cats and dogs), still life (such as inanimate objects like cups and benches), and plants (such as flowers, grass, and trees). The purpose of image cutout is to separate the target object from the background in an image and extract the target object separately.

[0028] In this embodiment, the sample image includes two parts: a first sample image with a label and a second sample image without a label.

[0029] In some implementations, sample images can be obtained manually, specifically using an image acquisition device, such as a camera capturing the target object. In other implementations, sample images can be obtained from the internet or from public sample databases using web crawlers. It is understood that the above implementations can be combined, meaning that some sample images are obtained manually, while others are obtained from the internet or from public sample databases using web crawlers. The labels on the first sample image can be manually or machine-labeled; this is not limited here.

[0030] Step S12: Based on the first sample image and the second sample image, train the student model using a combination of knowledge distillation from a multi-teacher model and semi-supervised learning to obtain the image matting model.

[0031] Specifically, when training the student model using the knowledge distillation method based on a multi-teacher model, it is based on the first sample image; when training the student model using the semi-supervised learning method, it is based on the first sample image and the second sample image. It is understood that there are various specific implementation methods for jointly training the student model using multi-teacher model knowledge distillation and semi-supervised learning, and this embodiment does not limit the specific method of combination.

[0032] Generally, the teacher model is a large model, and the student model is a small model. Both teacher and student models are deep learning models. Compared to small models, large models have more complex and larger network structures, more parameters, and deeper layers, enabling them to handle and learn more complex and advanced patterns and rules. Small models, on the other hand, are often limited by their number of parameters and have lower capabilities in semantic representation and information extraction compared to large models.

[0033] In this embodiment, the student model can be obtained by trimming the teacher model, for example, by reducing the network structure and parameters.

[0034] The multi-teacher model knowledge distillation training method is based on multiple teacher models. Distilling multiple teacher models into a single student model can transfer more diverse knowledge. In this embodiment, the multiple teacher models are good at matting in different dimensions; that is, the matting effect varies for images of different dimensions. In some implementations, the image dimension can be determined according to the type of target object. For example, teacher model A is good at matting human portraits, while teacher model B is good at matting animal portraits. Therefore, the output results of each teacher model for a given image may be different. However, through the multi-teacher model knowledge distillation training method, the student model can learn the knowledge of the teacher model with the best matting effect in different dimensions, thereby improving the student model's semantic representation and information extraction capabilities in different dimensions, enabling the student model to achieve optimal matting results in different dimensions of the image.

[0035] Semi-supervised learning involves training the model using both labeled and unlabeled sample data. Semi-supervised learning algorithms can include, but are not limited to: self-training, graph-based semi-supervised learning, and semi-supervised supported vector machine (S3VM).

[0036] In this embodiment, semi-supervised learning involves training the student model using a first sample image and an unlabeled second sample image. The specific training method is not limited here. Semi-supervised training can improve the generalization ability of the student model.

[0037] The above scheme obtains a labeled first sample image and an unlabeled second sample image, both of which include the target object. The label is the matting result of the first sample image. Based on the first and second sample images, the student model is trained using a combination of knowledge distillation from a multi-teacher model and semi-supervised learning to obtain a matting model. This not only improves the semantic representation and information extraction capabilities of the small matting model, but also enhances its generalization ability.

[0038] Please refer to Figures 2 to 4. Figure 2 is a flowchart of another embodiment of step S12 in Figure 1, Figure 3 is a flowchart of another embodiment of step S122 in Figure 2, and Figure 4 is a flowchart of another embodiment of the training method of the matting model of this application.

[0039] Based on the above embodiments, in this embodiment, step S12 may include sub-steps S121 to S127:

[0040] Step S121: Use at least two pre-trained teacher models to process the same first sample image to obtain at least two first matting results, wherein different teacher models are trained based on sample images of different dimensions.

[0041] Optionally, the dimensions of the fourth sample image include at least one of the following: human portrait dimension, animal dimension, still life dimension, and general dimension. Furthermore, the dimensions of the fourth sample image may also include plant dimension, etc., and the specific selection can be made according to actual needs; no limitation is made here. The general dimension means that the dimension is not limited and can include any dimension, such as one or more of the following: human portrait dimension, animal dimension, and still life dimension.

[0042] In some implementations, at least two teacher models are trained using the following method: a base model is pre-trained using labeled third sample images to obtain a pre-trained model; the pre-trained model is then trained using fourth sample images of different dimensions to obtain at least two pre-trained teacher models. The base model is a large model.

[0043] The basic models are neural network models, including but not limited to: U2net, Unet, VGGNet, FCN (Fully Convolutional Network), etc.

[0044] In some implementations, the base model can be the U2net model, which is a new network structure based on UET. It adopts a two-layer nested U-shaped structure, which has better image segmentation effect and higher real-time performance.

[0045] It is understood that any two or more of the first, second, third, and fourth sample images used in this application may be the same or different. For example, the third sample image used for pre-training the base model may also be used as the fourth sample image for training the pre-trained model.

[0046] Understandably, in some implementations, the actual model is trained based on multiple sample images, such as multiple first sample images, multiple second sample images, etc.

[0047] In some specific implementations, at least two teacher models may include a human teacher model, an animal teacher model, a still life teacher model, and a general teacher model. Specifically, after obtaining a pre-trained model, the pre-trained model can be trained using fourth sample images of the human dimension to obtain a human teacher model; the pre-trained model can be trained using fourth sample images of the animal dimension to obtain an animal teacher model; the pre-trained model can be trained using fourth sample images of the still life dimension to obtain a still life teacher model; and the pre-trained model can be trained using fourth sample images of the general dimension to obtain a general teacher model. This results in three large models that perform better in human, animal, and still life dimensions, as well as a general large model that may perform better in other dimensions. Specifically, training the pre-trained model using fourth sample images of the general dimension can involve training the pre-trained model using fourth sample images of multiple dimensions, such as human, animal, and still life dimensions.

[0048] In some implementations, each first sample image can be input into a portrait teacher model, an animal teacher model, a still life teacher model, and a general teacher model, and then four first matting results can be output.

[0049] Step S122: Based on at least two first matting results, determine the optimal teacher model from at least two teacher models.

[0050] The best cutout result is determined from at least two first cutout results, and the teacher model that obtains the first cutout result is then taken as the optimal teacher model.

[0051] In some implementations, step S122 may include sub-steps S1221 to S1222:

[0052] Step S1221: For each teacher model, calculate the evaluation index based on the first matting result and the label.

[0053] In some implementations, the matting effect can be reflected by objective metrics for subject matting. Evaluation metrics can be objective metrics for subject matting. Objective metrics for subject matting are quantitative standards used to evaluate the performance and effect of matting algorithms. Different metrics may focus on different aspects, such as the segmentation accuracy of foreground and background, edge smoothness, and the degree of detail preservation. Objective metrics for subject matting may include, but are not limited to: SAD (Sum of absolute difference), MAD (Mean absolute difference), MSE (Mean squared error), Gradient error, Connectivity error, etc. The smaller these metrics are, the better, indicating that the matting result is closer to the reality.

[0054] Here, SAD is the sum of the absolute differences between the predicted and actual alpha values. MAD is the mean of the absolute differences between the predicted and actual alpha values. MSE is the mean of the squared differences between the predicted and actual alpha values. Gradient error is the sum of the squared differences between the normalized gradients of the predicted and actual alpha values. Connectivity error is the sum of the differences in connectivity between the predicted and actual alpha values ​​within the largest connected component. The specific calculation formulas for the above evaluation metrics will not be elaborated further.

[0055] The first matting result is the predicted matte, and the label is the groundtruth matte.

[0056] The preset alpha value is the transparency of one pixel in the first cutout result, while the actual alpha value is the transparency of one pixel in the label. Both the first cutout result and the label are grayscale images.

[0057] The alpha value refers to the transparency of an image, usually represented as a floating-point number between 0 and 1. When the alpha value is 1, the image is completely opaque, meaning no background color is visible; when the alpha value is 0, the image is completely transparent, meaning the background color is fully displayed and the foreground image is not visible.

[0058] Step S1222: Determine the optimal teacher model based on the evaluation index corresponding to each teacher model.

[0059] In some implementations, the teacher model corresponding to the lowest value among the same evaluation metrics can be determined as the optimal teacher model. For example, the teacher model corresponding to the lowest value in SAD for each teacher model can be determined as the optimal teacher model. The smaller the value, the better the image matting effect of that teacher model.

[0060] Step S123: Use a pre-trained preset model to process the second sample image to obtain the second matting result.

[0061] The second sample image is unlabeled. By processing the second sample image using a pre-trained model, the model's predicted matting result can be obtained. This second matting result can then be used as a pseudo-label for the second sample image, which can be used for semi-supervised training of the student model.

[0062] Optionally, the preset model is a neural network model, and the specific model structure and parameters can be selected according to the actual situation. In some implementations, the preset model is a general teacher model trained using a fourth sample image set with common dimensions. For unlabeled second sample images, the dimensions involved are rich and diverse, so using the general teacher model to process the second sample images can improve the matting effect of the second matting result, thereby improving the accuracy of pseudo-labels. Furthermore, using pseudo-labels to constrain the student model can improve the matting ability of the student model.

[0063] Step S124: Process the first sample image using the student model to obtain the third matting result, and process the second sample image using the student model to obtain the fourth matting result.

[0064] Step S125: Calculate the first loss value based on the first and third matting results of the optimal teacher model, and calculate the second loss value based on the third matting result and the label of the first sample image.

[0065] Specifically, for the first labeled sample image, a first loss value (loss1) is calculated based on the first and third matting results of the optimal teacher model, and a second loss value (loss2) is calculated based on the third matting result and the label of the first sample image.

[0066] Step S126: Calculate the third loss value based on the second and fourth matting results.

[0067] Specifically, for the unlabeled second sample image, a third loss value (loss2) is calculated based on the second matting result (pseudo-label) and the fourth matting result. Here, semi-supervised learning involves training the student model simultaneously using the labeled first sample image and the second matting result (i.e., pseudo-label) generated by the pre-defined model.

[0068] Step S127: Adjust the parameters of the student model based on the first loss value, the second loss value, and the third loss value.

[0069] Specifically, the total loss value can be calculated based on the first loss value, the second loss value, and the third loss value, and then the parameters of the student model can be adjusted based on the total loss value. In some implementations, the parameters of the student model are adjusted until the total loss value is less than a preset loss threshold or the number of training iterations reaches a preset threshold, at which point training of the student model is stopped. Optionally, the preset loss threshold and the preset number of training iterations threshold can be selected according to the actual situation, and are not limited here.

[0070] In some implementations, the total loss value can be the sum of the first loss value, the second loss value, and the third loss value. In other implementations, weight values ​​can be assigned to the first loss value, the second loss value, and the third loss value, and the total loss value can be the weighted sum of the first loss value, the second loss value, and the third loss value.

[0071] In some implementations, the total loss value can also be calculated based on the original loss value, the first loss value, the second loss value, and the third loss value. Specifically, the total loss value can be calculated by summing the original loss value, the first loss value, the second loss value, and the third loss value. The original loss value is the loss value calculated using the original loss function of the student model.

[0072] In some implementations, before obtaining the matting model by training the student model using a combination of knowledge distillation from a multi-teacher model and semi-supervised learning based on the first and second sample images, the process includes: first training the student model using semi-supervised learning based on the first and second sample images; and then second training the student model using knowledge distillation from a multi-teacher model based on the first sample images. That is, the preset model and the student model can be semi-supervised first to improve their generalization ability. Then, the parameters of the preset model are fixed, and the student model undergoes knowledge distillation. At this point, the student model may lose some generalization ability. Finally, knowledge distillation from both semi-supervised and multi-teacher models is combined to simultaneously improve both generalization and matting capabilities.

[0073] In some implementations, when the student model is trained for the first time using semi-supervised learning, the loss function of the student model can be the student model's native loss function plus the semi-supervised loss function. When the student model is trained for the second time using knowledge distillation from a multi-teacher model, the loss function of the student model can be the student model's native loss function plus the knowledge distillation loss function (i.e., the KD loss function). When semi-supervised learning and knowledge distillation are combined, the loss function of the student model can be the student model's native loss function plus the semi-supervised loss function plus the knowledge distillation loss function. Optionally, the specific types of the native loss function, semi-supervised loss function, and knowledge distillation loss function can be selected according to the actual situation, and are not limited here.

[0074] The above scheme addresses the issue that teacher models of different dimensions have varying strengths and weaknesses in their performance on different types of input images. Using the first matting result of the optimal teacher model as a constraint on the student model can effectively leverage the strengths of each model while mitigating their weaknesses. Different input images can be distilled using different large models to distill knowledge from the smaller models. This multi-teacher model distillation approach can further enhance the matting capabilities of the smaller models.

[0075] This application aims to optimize a smaller model by using multiple large models that have undergone category-specific optimization and scoring them based on evaluation metrics. The specific knowledge distillation method used is not limited here. Furthermore, the specific semi-supervised learning method used is also not limited. The training method provided in this application can empower the subject matting edge-side small model with stronger semantic representation and extraction capabilities, improving the small model's generalization ability in open worlds.

[0076] Please refer to Figure 5, which is a flowchart illustrating an embodiment of the image matting method of this application. In this embodiment, the executing entity is an electronic device, specifically an electronic device with computing capabilities, such as a computer, mobile phone, or server.

[0077] Step S21: Obtain the image to be processed, which contains the target object.

[0078] The target object can include, but is not limited to: human figures (including any part of a person), animals (such as pets like cats and dogs), still life (such as inanimate objects like cups and benches), and plants (such as flowers, grass, and trees). The image to be processed can be a single captured image or an image from a video frame, etc., without any limitations.

[0079] Step S22: The matting model trained using the matting model training method in any of the above embodiments of this application is used to process the image to be processed to obtain a matted image.

[0080] The matting model trained using the training method of the matting model in any of the above embodiments can take into account both generalization ability and matting ability. Therefore, using this matting model to process the image to be processed can yield a matted image with better matting effect.

[0081] Please refer to Figure 6, which is a schematic block diagram of an embodiment of the training device for the image matting model of this application.

[0082] The training device 100 for the matting model includes an acquisition module 110 and a training module 120. The acquisition module 110 acquires a labeled first sample image and an unlabeled second sample image, both of which include the target object, and the label is the matting result of the first sample image. The training module 120 trains a student model based on the first and second sample images using a combination of knowledge distillation from a multi-teacher model and semi-supervised learning to obtain the matting model.

[0083] In some implementations, the training module 120 is specifically used to process the same first sample image using at least two pre-trained teacher models to obtain at least two first matting results, wherein different teacher models are trained based on sample images of different dimensions; based on the at least two first matting results, determine the optimal teacher model from the at least two teacher models; process the second sample image using a pre-trained model to obtain a second matting result; process the first sample image using a student model to obtain a third matting result, and process the second sample image using a student model to obtain a fourth matting result; calculate a first loss value based on the first and third matting results of the optimal teacher model, and calculate a second loss value based on the third matting result and the label of the first sample image; calculate a third loss value based on the second and fourth matting results; and adjust the parameters of the student model based on the first, second, and third loss values.

[0084] In some implementations, the training module 120 is specifically used to calculate an evaluation metric for each teacher model based on the first matting result and the label; and to determine the optimal teacher model based on the evaluation metric corresponding to each teacher model.

[0085] In some implementations, the two teacher models are trained as follows: a base model is pre-trained using labeled third sample images to obtain a pre-trained model; the pre-trained model is then trained using fourth sample images of different dimensions to obtain at least two pre-trained teacher models.

[0086] In some implementations, the dimensions of the fourth sample image include at least one of human portrait dimension, animal dimension, still life dimension and general dimension, and / or the preset model is a general teacher model trained using the fourth sample image set with general dimension.

[0087] In some implementations, the training module 120 is further configured to train the student model using a combination of knowledge distillation from a multi-teacher model and semi-supervised learning based on the first sample image and the second sample image, before obtaining the matting model, to perform a first training on the student model using semi-supervised learning based on the first sample image and the second sample image; and to perform a second training on the student model using knowledge distillation from a multi-teacher model based on the first sample image.

[0088] For a description of the execution steps of each of the above modules, please refer to the aforementioned implementation examples; they will not be repeated here.

[0089] Please refer to Figure 7, which is a schematic block diagram of an embodiment of the image cutout device of this application.

[0090] The image matting device 200 includes an acquisition module 210 and a matting module 220. The acquisition module 210 is used to acquire an image to be processed, which contains a target object; the matting module 220 is used to process the image to be processed using a matting model trained by the matting model training method in any of the above embodiments of this application, to obtain a matted image.

[0091] For a description of the execution steps of each of the above modules, please refer to the aforementioned implementation examples; they will not be repeated here.

[0092] Please refer to Figure 8, which is a schematic block diagram of the structure of an embodiment of the electronic device of this application.

[0093] The electronic device 300 includes a memory 310 and a processor 320 coupled to each other. The memory 310 is used to store program data, and the processor 320 is used to execute the program data to implement the steps in any of the above method embodiments.

[0094] Electronic device 300 may include, but is not limited to: personal computers (e.g., desktop computers, laptop computers, tablet computers, handheld computers, etc.), mobile phones, servers, wearable devices, as well as augmented reality (AR) and virtual reality (VR) devices, televisions, etc., without limitation.

[0095] Specifically, processor 320 controls itself and memory 310 to implement the steps in any of the above method embodiments. Processor 320 may also be referred to as a Central Processing Unit (CPU). Processor 320 may be an integrated circuit chip with signal processing capabilities. Processor 320 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor. Furthermore, processor 320 may be implemented by multiple integrated circuit chips.

[0096] Please refer to Figure 9, which is a schematic block diagram of a computer-readable storage medium according to an embodiment of the present application.

[0097] The computer-readable storage medium 400 stores program data 410, which, when executed by a processor, is used to implement the steps in any of the above method embodiments.

[0098] The computer-readable storage medium 400 can be a medium capable of storing computer programs, such as a USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk. It can also be a server storing the computer program, which can send the stored computer program to other devices for execution or run the stored computer program itself.

[0099] In this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, the term "at least one" in this application means any combination of at least two of any one or more of a plurality of elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0100] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0101] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0102] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0103] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0104] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A training method for an image matting model, characterized in that, include: A first sample image with labels and a second sample image without labels are obtained. Both the first and second sample images include the target object, and the labels are the matting results of the first sample image. Based on the first and second sample images, a student model is trained using a combination of knowledge distillation from a multi-teacher model and semi-supervised learning to obtain a matting model. The process of training the student model using a combination of knowledge distillation from a multi-teacher model and semi-supervised learning to obtain the matting model includes: processing the same first sample image using at least two pre-trained teacher models to obtain at least two first matting results, wherein different teacher models are trained based on sample images of different dimensions. Based on at least two of the first matting results, an optimal teacher model is determined from at least two teacher models; a pre-trained preset model is used to process the second sample image to obtain a second matting result; the student model is used to process the first sample image to obtain a third matting result, and the student model is used to process the second sample image to obtain a fourth matting result; a first loss value is calculated based on the first and third matting results of the optimal teacher model, and a second loss value is calculated based on the third matting result and the label of the first sample image; a third loss value is calculated based on the second and fourth matting results; and the parameters of the student model are adjusted based on the first, second, and third loss values.

2. The method according to claim 1, characterized in that, The step of determining the optimal teacher model from at least two teacher models based on at least two of the first matting results includes: for each teacher model, calculating an evaluation index based on the first matting result and the label; and determining the optimal teacher model based on the evaluation index corresponding to each teacher model.

3. The method according to claim 1, characterized in that, At least two of the teacher models are trained using the following method: a base model is pre-trained using labeled third sample images to obtain a pre-trained model; the pre-trained model is then trained using fourth sample images of different dimensions to obtain at least two pre-trained teacher models.

4. The method according to claim 3, characterized in that, The dimensions of the fourth sample image include at least one of human portrait dimension, animal dimension, still life dimension and general dimension, and / or the preset model is a general teacher model trained using the fourth sample image set with general dimensions.

5. The method according to claim 1, characterized in that, Before training the student model using a combination of multi-teacher model knowledge distillation and semi-supervised learning based on the first sample image and the second sample image to obtain the matting model, the method further includes: performing a first training on the student model using semi-supervised learning based on the first sample image and the second sample image; and performing a second training on the student model using multi-teacher model knowledge distillation based on the first sample image.

6. A training device for an image matting model, characterized in that, include: The acquisition module is used to acquire a first sample image with a label and a second sample image without a label. Both the first sample image and the second sample image include the target object, and the label is the cutout result of the first sample image. The training module is used to train a student model based on the first sample image and the second sample image using a combination of knowledge distillation from a multi-teacher model and semi-supervised learning to obtain a matting model. Specifically, the training module is used to: process the same first sample image using at least two pre-trained teacher models to obtain at least two first matting results, wherein different teacher models are trained on sample images of different dimensions; determine the optimal teacher model from the at least two teacher models based on the at least two first matting results; process the second sample image using a pre-trained model to obtain a second matting result; process the first sample image using the student model to obtain a third matting result, and process the second sample image using the student model to obtain a fourth matting result; calculate a first loss value based on the first and third matting results of the optimal teacher model, and calculate a second loss value based on the third matting result and the label of the first sample image; calculate a third loss value based on the second and fourth matting results; and adjust the parameters of the student model based on the first, second, and third loss values.

7. A method for image cutout, characterized in that, include: Obtain an image to be processed, wherein the image to be processed contains a target object; The matting model trained using the matting model training method described in any one of claims 1-5 is used to process the image to be processed to obtain a matted image.

8. A background removal device, characterized in that, include: The acquisition module is used to acquire an image to be processed, wherein the image to be processed contains a target object; The image matting module is used to process the image to be processed using the image matting model trained by the training method of the image matting model as described in any one of claims 1-5, so as to obtain a matted image.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor coupled to each other, the memory being used to store program data and the processor being used to execute the program data to implement the method as described in any one of claims 1-5 and 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program data, which, when executed by a processor, is used to implement the method as described in any one of claims 1-5 and 7.

Images