Image segmentation model training method and device, equipment and storage medium

Abstract

The invention provides an image segmentation model training method and device, equipment and a storage medium, and the method comprises the steps: carrying out the merging of a foreground image, mask information and a background image, obtaining a first annotation image, generating annotation information for an original image according to a target detector and a first segmentation model, obtaining a second annotation image, generating a high-confidence first training dataset and a low-confidence second training dataset according to the first annotation image and the second annotation image, training based on the first training dataset to obtain a target teacher model, and then segmenting the second training dataset through the target teacher model to obtain a third training dataset; and finally training a student model according to the third training data set to obtain a target image segmentation model. According to the method and the device, the utilization rate of the training data is effectively improved in a low-resource scene, and high-reliability labeling information is automatically added to the image.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image segmentation model training method, apparatus, device and storage medium. Background Art

[0002] Binary image segmentation is a basic task in computer vision, which aims to assign a label of one of two categories to each pixel in an image. In recent years, with the rapid development of deep learning, binary image segmentation methods based on deep learning have made significant progress.

[0003] In the prior art, the binary image segmentation method usually requires a large amount of high-quality training data, a segmentation model is trained based on the training data, and then the image is segmented through the segmentation model.

[0004] However, in low-resource scenarios, there are problems with image acquisition and high labeling costs. The existing automatic labeling technology also has the problem of poor reliability. In addition, a large amount of unlabeled data and low-confidence data are discarded during model training in the existing technology, and there is also the problem of low data utilization. Summary of the invention

[0005] The purpose of this application is to provide an image segmentation model training method, device, equipment and storage medium to address the deficiencies in the above-mentioned prior art, so as to solve the problems of low data utilization and poor labeling reliability in low-resource scenarios in the prior art.

[0006] To achieve the above objectives, the technical solutions adopted in this application are as follows:

[0007] In a first aspect, the present application provides an image segmentation model training method, the method comprising:

[0008] Generate a plurality of first annotated images according to a plurality of target foreground images, a plurality of target mask information, and a plurality of preset background images, wherein the first annotated images include annotation information, and the annotation information is used to indicate an area corresponding to the target foreground image in the first annotated images;

[0009] Performing enhanced detection processing according to a plurality of original images, a pre-trained object detector, and a pre-trained first segmentation model to obtain a second annotated image of each of the original images, wherein the second annotated image includes annotation information of the original image;

[0010] Generate a first training data set and a second training data set according to the plurality of first annotated images and the plurality of second annotated images, and train an initial teacher model according to the first training data set to obtain a target teacher model, wherein the confidence of each annotated image in the first training data set is greater than the confidence of each annotated image in the second training data set;

[0011] Predicting the second training data set according to the target teacher model to obtain a third training data set;

[0012] The initial student model is trained according to the third training data set to obtain a target image segmentation model.

[0013] In a second aspect, the present application provides an image segmentation model training device, the device comprising:

[0014] A generating module, configured to generate a plurality of first annotated images according to a plurality of target foreground images, a plurality of target mask information and a plurality of preset background images, wherein the first annotated images include annotation information, and the annotation information is used to indicate an area corresponding to the target foreground image in the first annotated images;

[0015] A detection module, configured to perform enhanced detection processing according to a plurality of original images, a pre-trained target detector, and a pre-trained first segmentation model, to obtain a second annotated image of each of the original images, wherein the second annotated image includes annotation information of the original image;

[0016] A first training module, used to generate a first training data set and a second training data set according to the plurality of first annotated images and the plurality of second annotated images, and train an initial teacher model according to the first training data set to obtain a target teacher model, wherein the confidence of each annotated image in the first training data set is greater than the confidence of each annotated image in the second training data set;

[0017] A prediction module, used to predict the second training data set according to the target teacher model to obtain a third training data set;

[0018] The second training module is used to train the initial student model according to the third training data set to obtain a target image segmentation model.

[0019] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of the image segmentation model training method as described in any one of the first aspects.

[0020] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the image segmentation model training method as described in any one of the first aspects are executed.

[0021] The beneficial effects of the present application are: by merging the target foreground image with the background image based on the mask information, a large number of usable annotated images can be generated more quickly when the number of foreground images is limited. By performing enhanced detection processing on the original image based on the target detector and the first segmentation model, more reliable annotation information can be automatically generated for the original image to obtain an annotated image with a high degree of reliability. By training the initial teacher model with an annotated image with high reliability to obtain a target teacher model, and predicting the second training data set based on the target teacher model, an annotated image with a higher degree of reliability can be generated, and the annotated image can be expanded, so that a large number of usable, highly reliable annotated images can be used as training samples when training based on the student model. The present application effectively realizes the expansion of training data in low-resource scenarios. Through the teacher-student model training method, the reuse of unlabeled data and low-confidence data can be realized, the utilization rate of training data is improved, and the efficiency of model training in low-resource scenarios and the reliability of the trained model are improved.

[0022] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 A flowchart of an image segmentation model training method provided in an embodiment of the present application is shown;

[0025] Figure 2 A schematic diagram of annotating an image provided by an embodiment of the present application is shown;

[0026] Figure 3 A flowchart of obtaining a first annotated image provided by an embodiment of the present application is shown;

[0027] Figure 4A flow chart of obtaining a second annotated image provided by an embodiment of the present application is shown;

[0028] Figure 5 A flowchart of another method of obtaining a second annotated image provided by an embodiment of the present application is shown;

[0029] Figure 6 A flowchart of obtaining a processed image and a processed bounding box provided by an embodiment of the present application is shown;

[0030] Figure 7 A flow chart of obtaining a segmentation mask result provided by an embodiment of the present application is shown;

[0031] Figure 8 A flowchart of another method of obtaining a second annotated image provided by an embodiment of the present application is shown;

[0032] Fig. 9 A flowchart of a training method for obtaining a target teacher model provided in an embodiment of the present application is shown;

[0033] Fig.10 A flow chart for determining a loss value provided by an embodiment of the present application is shown;

[0034] Fig.11 A flowchart of image segmentation based on a target teacher model provided in an embodiment of the present application is shown;

[0035] Fig.12 A schematic diagram of the structure of an image segmentation model training device provided in an embodiment of the present application is shown;

[0036] Fig.13 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0037] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0038] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0039] In the existing technical solutions, in order to improve the accuracy and efficiency of binary image segmentation, researchers have designed a variety of network architectures, such as fully convolutional network models, encoding-decoding models, coarse-to-fine models, prediction-optimization models and visual Transformer models.

[0040] In order to improve the accuracy of model segmentation, a large amount of high-quality training data is generally required for model training. High quality means that the annotation information in the sample image needs to have a high confidence level. Before model training, a large number of sample images need to be collected, and then annotation information is added to the sample images. The model is then trained based on the sample images and the annotation information of the sample images to obtain the binary image segmentation model used in the end.

[0041] Among them, the methods of adding annotation information to sample images include manual addition and model addition. When the model adds annotation information, it is necessary to manually input prompt information first, and the model can generate annotation information for the sample image based on the prompt information. However, the annotation information generated by the model when there is less prompt information still has the problem of low reliability.

[0042] Low-resource scenarios refer to scenarios where sample images are difficult to obtain or the number of sample images obtained is small. Most of the binary classification image segmentation schemes in the existing technology are not designed for low-resource scenarios. Therefore, there are many problems in directly applying the existing binary classification image segmentation schemes to low-resource scenarios.

[0043] In low-resource scenarios, the number of sample images is small. When adding annotations to training data, the reliability of the annotation information added by the model means that the amount of high-quality training data that can be used is even more limited. A large amount of unlabeled data and data with low reliability of annotation information are discarded, resulting in low utilization of training data and low reliability of annotation information.

[0044] Based on this, this application proposes an image segmentation model training method. Under the premise of no or only a small number of annotated samples, the foreground image and background image of the target of interest are used for data synthesis, and the annotated data is expanded in a low-cost and automated manner through an external general detection segmentation model, and combined with a self-training learning method to achieve binary classification image segmentation in low-resource scenarios. For low-resource scenarios, the method of this application can effectively expand the training data, improve the reliability of the annotation information, and improve the utilization rate of the training data.

[0045] Next, combine Figure 1The image segmentation model training method of the present application is described. The execution subject of the method can be an electronic device, wherein the image segmentation model can be a binary image segmentation model, such as Figure 1 As shown, the method includes:

[0046] S101, generating a plurality of first annotated images according to a plurality of target foreground images, a plurality of target mask information and a plurality of preset background images, wherein the first annotated images include annotation information, and the annotation information is used to indicate a region corresponding to the target foreground image in the first annotated image.

[0047] The target foreground image may be a target area image, such as a transparent background image of at least one target to be detected in image segmentation. The target mask information is a mask image of the target foreground image, which is used to indicate the area in the target foreground image where the target to be detected in the target foreground image is located. The background image may be a non-target area in the image, that is, an image area that does not include the target to be detected and does not require image segmentation.

[0048] The target foreground image and the background image are superimposed based on the target mask information to obtain a first annotated image. As a possible implementation method, when superimposing, 0-K target foreground images can be randomly superimposed on the background image, and the target mask information of each target foreground image can be superimposed. After the image synthesis is completed, the annotation information of the superimposed image is generated based on the superimposed target mask information. The value of K can be 2, for example.

[0049] like Figure 2 FIG. 1 is a schematic diagram of a first annotated image provided in the present application, referring to FIG. Figure 2 , the targets to be detected in the image include target A and target B. When generating the first annotated image, the target foreground images of target A and target B are superimposed on the background image to obtain the superimposed image, and then the annotation information of the superimposed image is generated based on the target mask information of target A and target B. Among them, the area where target A and target B are located, that is, the black area in the figure, the annotation information of each pixel is 1, and the annotation information of the rest of the positions in the image is 0.

[0050] The annotation information may indicate the area corresponding to the target foreground image in the first annotated image, and the annotation information may be the union of the mask annotations of the target foreground image in the first annotated image. After the target foreground image is superimposed on the background image, if there are multiple target foreground images, the target mask information of the target foreground image may also be superimposed according to the position of the target foreground image, and the union of the area occupied by the target mask information in the superimposed image is used as the annotation information of the first annotated image.

[0051] S102, performing enhanced detection processing according to a plurality of original images, a pre-trained target detector, and a pre-trained first segmentation model to obtain a second annotated image of each original image, wherein the second annotated image includes annotation information of the original image.

[0052] The original image may be a natural image without annotations, and the original image may include at least one target to be detected. The second annotated image may be an original image with annotation information added, and the annotation information is used to indicate the area occupied by the detection target in the original image.

[0053] In one possible implementation, target detection may be performed on the original image to obtain the detected targets included in the original image, and then image segmentation may be performed on the detected targets in the original image to obtain the areas occupied by each detected target in the original image, and a classification label "1" is added to the pixels in the area occupied by the detected targets in the original image, and a classification label "0" is added to the pixels in the remaining areas. The classification labels "1" and "0" are merely examples given in this application, and other distinguishing labels may also be added, and this application does not limit this.

[0054] The object detector can detect objects in the original image and generate bounding boxes for the detected objects in the original image. The object detector can be an open set object detector, which can detect and identify object categories that do not appear in the training set and has stronger generalization capabilities. The architecture of the open set object detector mainly includes a feature extraction module, a feature enhancement module, a language-guided query selection module, and a cross-modal decoder module. Through cross-modal feature fusion, clause-level text expression, and end-to-end training, efficient object detection in an open world environment is achieved. Exemplarily, the object detector can be Grounding DINO.

[0055] The first segmentation model can be a general image segmentation model, such as a Segment Anything Model (SAM) model. After the bounding box and the detected target obtained by the target detector are input into the first segmentation model, the first segmentation model can use the bounding box as prompt information to perform image segmentation on the detected target in the original image to obtain a segmentation mask of the detected target in the original image.

[0056] It is worth noting that in the process of detecting the original image, in order to further improve the reliability and accuracy of the generated annotation information, the present application can also enhance the image and bounding box of the original image after detection by the target detector, and then use the first segmentation model to segment the enhanced image and bounding box, and finally fuse the enhanced image and the segmentation mask to obtain a second annotated image of the original image.

[0057] S103. Generate a first training data set and a second training data set based on multiple first annotated images and multiple second annotated images, and train an initial teacher model based on the first training data set to obtain a target teacher model, wherein the confidence of each annotated image in the first training data set is greater than the confidence of each annotated image in the second training data set.

[0058] Among them, each pixel in the annotated image includes an information entropy, which can characterize the reliability of the pixel. The confidence of the annotated image can be calculated according to the reliability of each pixel.

[0059] In a first implementation, multiple first annotated images and multiple second annotated images can be traversed, and annotated images with confidence greater than a preset threshold are added to the first training data set, and annotated images with confidence less than the preset threshold are added to the second training data set.

[0060] In the second implementation, all first annotated images can be added to the first training data set, and the second annotated images are traversed, and the second annotated images with confidence greater than a preset threshold are added to the first training data set, and the second annotated images with confidence less than the preset threshold are added to the second training data set. It should be understood that the first annotated images are obtained by merging the foreground image, the mask information, and the background image, and the reliability of the annotation information is relatively high, so the first annotated images can be directly added to the first training data set.

[0061] In a third implementation, the first annotated images and the second annotated images may be sorted from high to low according to confidence, the first N annotated images may be added to the first training data set, and the remaining annotated images may be added to the second training data set.

[0062] Optionally, the initial teacher model can be an initial segmentation model. After obtaining the target teacher model by training the initial teacher model according to the first training data set, the target teacher model can realize the mapping from the image domain to the segmentation result domain, that is, complete the image segmentation of the input image and obtain the segmentation mask of the input image.

[0063] S104. Predict the second training data set according to the target teacher model to obtain a third training data set.

[0064] The first training dataset contains annotated images with higher confidence, and the images in the second training dataset include unannotated images and annotated images with lower confidence. In low-resource scenarios, the number of annotated images in the first training dataset may be small, so the images in the second training dataset can be re-segmented and predicted based on the target teacher model to obtain the segmentation mask of the image, and the annotation information of the image is generated based on the segmentation mask to obtain annotated images with higher confidence.

[0065] After predicting the annotated images in the second training data set based on the target teacher model to obtain annotated images with higher confidence, each annotated image can be added to the first training data set to obtain a third training data set.

[0066] It is worth noting that the step of predicting the second training data set based on the target teacher model to obtain labeled images can be performed multiple times to obtain a larger number of highly confident labeled images, thereby improving the utilization rate of a limited number of training samples in low-resource scenarios.

[0067] S105. Train the initial student model according to the third training data set to obtain a target image segmentation model.

[0068] Among them, the student model can have the same network structure and loss function as the teacher model. The annotated images in the third training data set are input into the initial student model, and the initial student model performs image segmentation on each annotated image to obtain the segmentation results of each annotated image. Finally, the initial student model is optimized based on the segmentation results of each annotated image and the annotation information of the annotated image to obtain the target image segmentation model.

[0069] In an embodiment of the present application, by merging the target foreground image with the background image based on mask information, a large number of usable annotated images can be generated more quickly when the number of foreground images is limited. By performing enhanced detection processing on the original image based on the target detector and the first segmentation model, more reliable annotation information can be automatically generated for the original image to obtain an annotated image with a high degree of reliability. The initial teacher model is trained with an annotated image with high reliability to obtain a target teacher model, and the second training data set is predicted based on the target teacher model, so that an annotated image with a higher degree of reliability can be generated, and the annotated image can be expanded, so that a large number of usable, highly reliable annotated images can be used as training samples when training based on the student model. The present application effectively realizes the expansion of training data in low-resource scenarios. Through the teacher-student model training method, the reuse of unlabeled data and low-confidence data can be realized, the utilization rate of training data is improved, and the efficiency of model training in low-resource scenarios and the reliability of the trained model are improved.

[0070] The following is a further description of generating a plurality of first annotated images according to a plurality of target foreground images, a plurality of target mask information and a plurality of preset background images. Figure 3 As shown, the above step S101 includes:

[0071] S301 , transform multiple target foreground images and multiple target mask information to obtain transformed foreground images and transformed mask information.

[0072] After obtaining the target foreground image and background image, the images can be cleaned and filtered to remove non-four-channel foreground images.

[0073] Each target foreground image corresponds to a target mask information, and the target foreground image and the target mask information corresponding to the target foreground image are transformed as a combination. The target foreground image and the target mask information can be scaled, transformed and rotated according to a random scaling ratio, affine transformation and rotation matrix.

[0074] S302: performing superposition processing on each transformed foreground image, each transformed mask information and each background image to obtain a plurality of superimposed images.

[0075] After completing the random change of the target foreground image and the mask information, the target foreground image can be superimposed on the background image according to the random superposition position to obtain the superimposed image, and the transformed target mask information is used as the mask information corresponding to the superimposed image.

[0076] When performing image superposition, one target foreground image may be superimposed on the background image, or multiple target foreground images may be superimposed on the background image. When superimposing multiple target foreground images on the background image, the target mask information of each target foreground image may also be superimposed according to the position of each target foreground image to obtain a superimposed mask of the superimposed image, and the superimposed mask is used as the mask image of the superimposed image.

[0077] It should be noted that the mask image can represent the area that needs to be segmented in the overlay image. Therefore, when generating annotation information of the overlay image based on the mask image, different labels can be added to the pixels covered by the mask image and the pixels not covered in the overlay image, and the labels of all pixels are used as the annotation information of the overlay image.

[0078] S303: Perform enhancement processing on the multiple superimposed images to obtain multiple first annotated images.

[0079] Among them, the enhancement processing can be to scale, transform and rotate the superimposed image and the mask of the superimposed image according to a random scaling ratio, affine transformation and rotation matrix, and compress the superimposed image according to a random compression quality to obtain the final first annotated image.

[0080] In the embodiment of the present application, by merging the foreground image and the background image, simulation data containing the target with higher reliability can be generated, and the user does not need to perform manual labeling, thereby reducing labor costs.

[0081] When the number of foreground images is large, the first annotated image can be obtained by performing image synthesis based on the above step 101. When the number of foreground images and mask information is small, the second annotated image can also be generated based on the acquired natural image. The following is a further description of the above-mentioned enhanced detection processing based on multiple original images, pre-trained object detectors and pre-trained first segmentation models to obtain the second annotated image of each original image, such as Figure 4 As shown, the above step S102 includes:

[0082] S401. Input a plurality of original images into an object detector to obtain at least one bounding box of each original image, where the bounding box is used to indicate a position of a detection object in the original image.

[0083] The original image is sequentially input into the target detector, which can detect a plurality of pre-set targets, obtain at least one detected target contained in the original image, and generate a bounding box for each detected target.

[0084] The bounding box may be a maximum rectangular enclosing box of the detected object, and is used to indicate the position of the detected object in the original image.

[0085] S402: Perform enhancement processing on each original image and at least one bounding box corresponding to each original image to obtain a plurality of processed images and at least one processed bounding box corresponding to each processed image.

[0086] In a possible implementation, the original image and the bounding box corresponding to each original image may be subjected to random enhancement processing N times to obtain N processed images and processed bounding boxes corresponding to each processed image. The enhancement method may be horizontal flipping of the image, random scaling, right rotation, etc., and one of the multiple enhancement methods may be randomly selected each time the enhancement processing is performed.

[0087] S403: Input each processed image and at least one processed bounding box corresponding to each processed image into a pre-trained first segmentation model, and perform image segmentation processing by the first segmentation model to obtain multiple instance segmentation masks of each processed image.

[0088] The first segmentation model may be a general target segmentation model. The first segmentation model may use the processed bounding box as prompt information to perform image segmentation on the detection target indicated by the processed border in the processed image to obtain an instance segmentation mask of the detection target indicated by the processed border.

[0089] The instance segmentation mask can be a mask image of a detection target. A processed image can include multiple detection targets, each detection target has a processed bounding box, and after segmentation, each detection target corresponds to an instance segmentation mask.

[0090] S404 , performing fusion screening processing according to multiple instance segmentation masks of each processed image to obtain a second annotated image of each original image.

[0091] By fusing all instance segmentation masks of each processed image, a mask image of the processed image can be obtained, and annotation information of the processed image can be generated based on the mask image of the processed image. For example, different classification labels can be added to pixels covered by the mask image in the processed image and pixels not covered by the mask image to obtain a second annotated image of each original image.

[0092] It should be noted that the general image segmentation model may not be able to obtain accurate segmentation results for some detection targets with complex structures and low significance. Based on this, in the embodiment of the present application, the confidence of the processed image can also be calculated according to the reliability of each pixel point of the processed image, and the processed image can be screened based on the confidence of the processed image to add the second annotated image with higher confidence to the above-mentioned first training data set, and the second annotated image with lower confidence to the above-mentioned second training data set.

[0093] The following is a further description of the above-mentioned fusion screening process based on multiple instance segmentation masks of each processed image to obtain the second annotated image of each original image, such as Figure 5 As shown, the above step S404 includes:

[0094] S501 , performing fusion processing on the instance segmentation masks of each processed image to obtain the segmentation mask results of the original images corresponding to each processed image.

[0095] The processed image includes multiple detection targets, each detection target corresponds to an instance segmentation mask. After merging the instance segmentation masks of each detection target, the segmentation mask results of each processed image can be obtained. By fusing the segmentation mask results of the processed images obtained by enhancing the same original image, the segmentation mask results of the original image can be obtained, that is, the mask image of the original image.

[0096] Specifically, after the original image is enhanced, N processed images are obtained, and each processed image can include at least one detected target. Therefore, after segmentation based on the bounding box of the detected target, each processed image can include at least one instance segmentation mask. The instance segmentation masks of each processed image are fused, which can be done by first fusion of the instance segmentation masks of the same processed image to obtain the segmentation mask result of the processed image, and then fusion of the segmentation mask results of the N processed images to obtain the segmentation mask result of the original image.

[0097] Optionally, the segmentation mask result of the original image may be a mask image of the original image, which is composed of the union of masks of all detected objects in the original image.

[0098] S502: Screen each original image based on the confidence of each segmentation mask result and a preset first confidence threshold to obtain a second annotated image, wherein the confidence of the segmentation mask result is used to characterize the confidence of each pixel in the segmentation mask result.

[0099] The segmentation mask result can characterize the pixel area occupied by the detection target in the original image, and the confidence of the segmentation mask result can characterize the confidence of each pixel in the segmentation mask result. Therefore, based on the confidence of each pixel, the prediction reliability of the segmentation of the detection target in the original image can be calculated. The higher the prediction reliability, the closer the segmentation mask result is to the actual shape and actual position of the detection target. Based on the confidence of the segmentation mask result and the preset first confidence threshold, the original image with high prediction reliability can be screened out, and the screened original image with high reliability and the segmentation mask result of the original image are annotated with the left taillight festival image.

[0100] The following is a further description of the above-mentioned enhancement processing of each original image and at least one bounding box corresponding to each original image to obtain multiple processed images and at least one processed bounding box corresponding to each processed image. Figure 6 As shown, the above step S402 includes:

[0101] S601. Perform image enhancement processing on each original image based on a preset enhancement strategy to obtain a processed image and at least one enhanced bounding box corresponding to each processed image. The enhancement strategy includes at least one of the following: image horizontal flipping, image scaling, and image rotation.

[0102] After the original image is detected based on the object detector to obtain at least one bounding box of the original image, image enhancement processing can be performed on the original image and at least one bounding box corresponding to the original image. During the enhancement processing, N enhancement processing can be performed to obtain N processed images and enhanced bounding boxes of each processed image.

[0103] Exemplarily, the original image can be horizontally flipped, and at least one bounding box of the original image can also be horizontally flipped to obtain a processed image 1 and at least one enhanced bounding box of the processed image 1; the original image can be scaled by 0.5 times, and at least one bounding box of the original image can also be scaled by 0.5 times to obtain a processed image 2 and at least one enhanced bounding box of the processed image 2.

[0104] S602 : performing offset processing and scaling processing on each enhanced bounding box according to a preset offset interval and scaling coefficient interval to obtain a processed bounding box corresponding to each enhanced bounding box.

[0105] In order to further enhance the image, the enhanced bounding box can be randomly transformed, for example, the enhanced bounding box is randomly offset and length scaled to obtain the processed bounding box of each enhanced bounding box. The horizontal axis offset is a random integer in the range of [-W*σ, W*σ], the vertical axis offset is a random integer in the range of [-H*σ, H*σ], and the offset is controlled not to exceed the image boundary. The length scaling coefficient of the bounding box ranges from [1-μ1, 1+μ2], and the length and width of the bounding box are scaled and rounded using different random scaling coefficients. W and H represent the width and height of the enhanced image. σ, μ1 and μ2 are values ​​between 0 and 1. σ can generally be 0.05, and μ1 and μ2 can generally be 0.05 and 0.1 or more.

[0106] In one possible implementation, Figure 7 As shown, the above-mentioned process of fusing the instance segmentation masks of each processed image to obtain the segmentation mask results of the original image corresponding to each processed image includes:

[0107] S701 , fusing instance segmentation masks of each processed image to obtain segmentation mask results of each processed image.

[0108] The instance segmentation masks are fused by taking a union of at least one instance segmentation mask of the processed image as a segmentation mask result of the processed image.

[0109] S702 , averaging and fusing the segmentation mask results of each processed image to obtain the segmentation mask results of the original image corresponding to each processed image.

[0110] Optionally, average fusion can be performed using the following formula (1): where N represents the number of processed images, S i Represents the segmentation mask result of the i-th processed image.

[0111]

[0112] As another possible implementation, the segmentation mask results of each processed image may be subjected to weight fusion processing, maximum fusion processing or minimum fusion processing. This application only uses average fusion processing as an example for illustration and should not be used as a limitation.

[0113] After obtaining the segmentation mask results, each original image can be screened based on the confidence of each segmentation mask result and a preset first confidence threshold to obtain a second annotated image, such as Figure 8 As shown, the above step S502 includes:

[0114] S801: Compare the confidence of each pixel in the segmentation mask result with a first confidence threshold, and determine the number of pixels in the segmentation mask result whose confidence is greater than the first confidence threshold.

[0115] S802: Determine the ratio of the number of pixels to the total number of pixels in the original image.

[0116] The confidence of each pixel can be determined based on the information entropy of each pixel in the segmentation mask result. As an example, the difference between 1 and the information entropy of the pixel can be used as the confidence of the pixel. The following formula (2) is an implementation method of information entropy calculation given in this application, and the following formula (3) is a method for calculating the confidence of a pixel.

[0117]

[0118]

[0119] in, Represents the confidence of each pixel in the segmentation mask result, represents the segmentation mask result, After calculating the confidence of all pixels, the confidence of each pixel can be compared with the first confidence threshold to obtain the number of pixels in the segmentation mask result whose confidence is greater than the first confidence threshold.

[0120] S803: If the ratio is greater than a preset second confidence threshold, the original image indicated by the ratio and the segmentation mask result are combined to obtain a second annotated image.

[0121] The original image indicated by the ratio may be the original image to which the segmentation mask result corresponding to the ratio belongs.

[0122] Exemplarily, if the number of pixels whose confidence is greater than a first confidence threshold is a, and the total number of pixels is b, the ratio can be expressed as a / b, and the ratio a / b is compared with the second confidence threshold. If the ratio is greater than the second confidence threshold, the original image indicated by the ratio and the segmentation mask result can be used as the second annotated image.

[0123] In the embodiment of the present application, by screening the segmentation mask results based on the confidence of the pixel points, a more reliable annotated image can be obtained, thereby improving the reliability of the segmentation model obtained by subsequent training.

[0124] As a possible implementation manner, the process of generating the first training data set and the second training data set according to the plurality of first annotated images and the plurality of second annotated images includes:

[0125] Traverse multiple first annotated images and second annotated images, and for the traversed current annotated image, if the confidence of the current annotated image is greater than or equal to a preset threshold, add the current annotated image to the first training data set, otherwise, add the current annotated image to the second training data set.

[0126] When traversing the first annotated image and the second annotated image, the first annotated image and the second annotated image may be traversed separately, or the first annotated image and the second annotated image may be combined into one set and traversed on the set.

[0127] The current annotated image may be the first annotated image or the second annotated image.

[0128] In another implementation, all first annotated images may be added to the first training data set, and then all second annotated images may be traversed, and second annotated images with confidence greater than a preset threshold may be added to the first training data set, and second annotated images with confidence less than the preset threshold may be added to the second training data set. At this time, the current annotated image is the second annotated image.

[0129] Next, the process of training the initial teacher model according to the first training data set to obtain the target teacher model is described. Fig. 9 As shown, the above step S103 includes:

[0130] S901. Input each annotated image in the first training data set into an initial teacher model, and use the initial teacher model to perform image segmentation processing on each annotated image to obtain an initial segmentation result of each annotated image.

[0131] Among them, the annotated images in the first training data set have a higher confidence level, so the data in the first training data set can be used as seed data, and the target teacher model can be trained based on the annotated images in the first training data set to obtain a target teacher model with accurate segmentation effect.

[0132] The annotated image in the first training data set includes an image and the annotation information of the image, and the annotation information of the image is the mask image of the image. After the image in the first training data set is input into the initial teacher model, the initial teacher model can perform segmentation processing on the image to obtain an initial segmentation result of the image, and the initial segmentation result can be a predicted mask image of the image by the initial teacher model.

[0133] Optionally, the initial teacher model can be F implemented from the image domain To the segmentation result domain Mapping: Y = F(θ, I), θ is the learnable weight parameter of the segmentation model, and the initial teacher model can use U2-Net or other network architectures such as Transformer as the segmentation backbone.

[0134] S902: Determine the loss value of the initial teacher model according to the initial segmentation results of each annotated image, the annotation information of each annotated image, and the confidence of each annotated image.

[0135] In the first implementation, the initial segmentation result may be a predicted mask image of the image, and the annotation information of the annotated image is used to indicate the annotated mask image of the image. Therefore, the loss value of the initial teacher model may be calculated based on the initial segmentation result and the annotation information of each annotated image.

[0136] In the second implementation method, in order to further improve the accuracy of loss value calculation, the confidence of the annotated image can be used as a weight, and the loss value of the initial teacher model can be calculated based on the initial segmentation result, the annotation information of each annotated image, and the confidence of the annotated image.

[0137] S903. Optimize the initial teacher model according to the loss value to obtain the target teacher model.

[0138] It should be understood that the process of training the target teacher model may include multiple training rounds, that is, the initial teacher model may be trained for multiple rounds based on the first training data set to obtain the target teacher model.

[0139] In an embodiment of the present application, the model parameters of the initial teacher model can be adjusted according to the loss value to obtain a new initial teacher model, and then the annotated image in the first training data set is input into the new initial teacher model to obtain a new predicted segmentation result, and then the loss value is calculated based on the new predicted segmentation result and the annotation information. If the loss value meets the preset requirements, or the current iteration round is equal to the preset value, the teacher model at the end of this round can be used as the target teacher model.

[0140] The following is a further explanation of determining the loss value of the initial teacher model based on the initial segmentation results of each annotated image, the annotation information of each annotated image, and the confidence of each annotated image. Fig.10 As shown, the second implementation method in the above step S902 specifically includes:

[0141] S1001. The initial segmentation result of the annotated image, the confidence level of each annotated image and the annotation information are used as input parameters of the loss function, and the loss function determines the loss value of each pixel point according to the initial segmentation result and the annotation information.

[0142] Among them, the loss function can be a cross entropy loss function, a focal loss function, a weighted cross entropy loss function, etc., and this application does not limit it here.

[0143] The confidence of the annotated image includes the annotation confidence of each pixel. According to the annotation confidence of each pixel in the annotated image and the predicted confidence of each pixel in the initial segmentation result, the loss value of each pixel can be calculated based on the loss function.

[0144] S1002. The loss function performs weighted calculation on the loss value of each pixel according to the confidence of each labeled image to obtain the loss value of the initial teacher model.

[0145] In an embodiment of the present application, the loss value of the model can be calculated based on the loss value of the pixel with a higher degree of reliability. In a possible implementation, the confidence of each pixel can be judged based on a preset threshold, and the filtered pixels greater than the threshold can be screened out, and the labeled confidence of each filtered pixel is used as the loss weight, and the loss value of the filtered pixel is weightedly calculated, and the calculation result is used as the loss value of the model.

[0146] In another possible implementation, a preset first loss weight may be assigned to the filtered pixels whose confidence is greater than a threshold, and a preset second loss weight may be assigned to the remaining pixels, and a weighted calculation may be performed on the loss values ​​of all pixels, and the calculation result may be used as the loss value of the model. The first loss weight may be a value greater than 1, and the second loss weight may be 1.

[0147] After obtaining the target teacher model, the number of annotated images can be expanded based on the target teacher model. Specifically, the target teacher model can be used as an image segmentation model to perform image segmentation on low-confidence and unannotated images, obtain image mask results, and generate image annotation information. Fig.11 As shown, the process of predicting the second training data set according to the target teacher model to obtain the third training data set includes:

[0148] S1101 , performing enhancement processing on each annotated image to obtain a plurality of processed images.

[0149] The processed image is an image after the annotated image is enhanced, and each annotated image may correspond to at least one processed image. The process of enhancing the annotated image in the second training data set is the same as the process of enhancing the original image, and this application will not repeat it here.

[0150] S1102, input each processed image into the target teacher model, and the target teacher model performs image segmentation processing to obtain a segmentation mask of each processed image.

[0151] In an embodiment of the present application, the target teacher model can be used as a segmentation model, and each processed image can be input into the target teacher model in sequence. The target teacher model performs image segmentation processing to obtain a segmentation mask for each processed image.

[0152] S1103 , performing fusion processing on the instance segmentation masks of each processed image to obtain the segmentation mask results of the annotated images corresponding to each processed image.

[0153] S1104 , screening each annotated image based on the confidence of each segmentation mask result and a preset first confidence threshold to obtain a screened annotated image.

[0154] The process of fusing the instance segmentation masks of the processed image to obtain the segmentation mask result and screening based on the confidence of the segmentation mask result and the preset first confidence threshold can be the same as the above-mentioned steps S501-S502, and this application will not repeat them here.

[0155] The screened annotated image may be an annotated image whose confidence of the segmentation mask is greater than a first confidence threshold.

[0156] S1105 . Generate a third training data set according to the screened annotated images and the first training data set.

[0157] After obtaining the screened annotated images, the screened annotated images may be merged with the annotated images in the first training data set to obtain a third training data set.

[0158] Furthermore, in the first implementation, a third training data set may be generated according to the round currently used by the target teacher model. In this case, the process of generating the third training data set according to the filtered annotated images and the first training data set includes:

[0159] If the round number of the third training data set obtained by predicting the second training data set according to the target teacher model is equal to the preset value, all the obtained filtered and annotated images are added to the first training data set to obtain the third training data set.

[0160] Exemplarily, assuming that the preset value is 5, after five predictions are made based on the target teacher model, all the filtered and annotated images obtained from the five predictions can be added to the first training data set to obtain a third training data set.

[0161] In the second implementation, the number of the third training data set can be preset, and the third training data set is generated based on a preset number threshold. In this case, the process of generating the third training data set based on the screened annotated images and the first training data set includes:

[0162] If the sum of the number of the filtered labeled images and the number of the third labeled images in the first training data set is equal to or greater than a preset number threshold, the filtered labeled images and the first training data set are merged into a third training data set.

[0163] If the sum of the number of annotated images after screening and the number of third annotated images in the first training data set is less than a preset threshold, the second training data set is re-determined, and the step of predicting the second training data set according to the target teacher model is performed until the current number of annotated images after screening and the sum of the number of third annotated images in the first training data set is equal to or greater than the preset threshold, so as to obtain the third training data set.

[0164] Exemplarily, assuming that the sum of the number of filtered annotated images and the number of annotated images in the first training data set is m, and m is greater than a preset number threshold, all the currently obtained filtered annotated images can be added to the first training data set to obtain a third training data set.

[0165] If m is less than a preset threshold, all annotated images whose confidence levels are less than the first confidence threshold can be used as the second training data set, and the images in the second training data set can be re-predicted based on the target teacher model until the sum of the number of all currently screened annotated images and the number of annotated images in the first training data set is greater than or equal to m.

[0166] In another possible implementation, the number of filtered annotated images and the total number of annotated images in the first training data set can also be judged. Under the premise that time continues to increase, if the total number does not increase, or the increase in the total number is less than x, the currently obtained filtered annotated images and all the annotated images in the first training data set can be merged as the third training data set.

[0167] After obtaining the third training data set, the initial student model can be trained based on the third training data set, and the student model obtained after the training is completed is used as the target image segmentation model, wherein the student model has the same network structure and loss function as the teacher model.

[0168] Based on the same inventive concept, an image segmentation model training device corresponding to the image segmentation model training method is also provided in the embodiment of the present application. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned image segmentation model training method in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0169] Fig.12 A structural schematic diagram of an image segmentation model training device provided in an embodiment of the present application is shown.

[0170] A generating module 1201 is used to generate a plurality of first annotated images according to a plurality of target foreground images, a plurality of target mask information and a plurality of preset background images, wherein the first annotated images include annotation information, and the annotation information is used to indicate an area corresponding to the target foreground image in the first annotated image;

[0171] A detection module 1202 is used to perform enhanced detection processing according to the multiple original images, the pre-trained object detector and the pre-trained first segmentation model to obtain a second annotated image of each original image, wherein the second annotated image includes annotation information of the original image;

[0172] A first training module 1203 is used to generate a first training data set and a second training data set according to the plurality of first annotated images and the plurality of second annotated images, and train the initial teacher model according to the first training data set to obtain a target teacher model, wherein the confidence of each annotated image in the first training data set is greater than the confidence of each annotated image in the second training data set;

[0173] A prediction module 1204 is used to predict the second training data set according to the target teacher model to obtain a third training data set;

[0174] The second training module 1205 is used to train the initial student model according to the third training data set to obtain a target image segmentation model.

[0175] In a feasible implementation scheme, the generation module 1201 is specifically used for:

[0176] Transforming a plurality of target foreground images and a plurality of target mask information to obtain transformed foreground images and transformed mask information;

[0177] Performing superposition processing on each transformed foreground image, each transformed mask information and each background image to obtain a plurality of superimposed images;

[0178] The multiple superimposed images are enhanced to obtain multiple first annotated images.

[0179] In a feasible implementation scheme, the detection module 1202 is specifically used for:

[0180] Inputting a plurality of original images into an object detector to obtain at least one bounding box of each original image, wherein the bounding box is used to indicate a position of a detection object in the original image;

[0181] Performing enhancement processing on each original image and at least one bounding box corresponding to each original image to obtain a plurality of processed images and at least one processed bounding box corresponding to each processed image;

[0182] Inputting each processed image and at least one processed bounding box corresponding to each processed image into a pre-trained first segmentation model, performing image segmentation processing by the first segmentation model, and obtaining a plurality of instance segmentation masks of each processed image;

[0183] A fusion screening process is performed according to multiple instance segmentation masks of each processed image to obtain a second annotated image of each original image.

[0184] In a feasible implementation scheme, the detection module 1202 is specifically used for:

[0185] The instance segmentation masks of each processed image are fused to obtain the segmentation mask results of the original image corresponding to each processed image;

[0186] Based on the confidence of each segmentation mask result and a preset first confidence threshold, each original image is screened to obtain a second annotated image, and the confidence of the segmentation mask result is used to characterize the confidence of each pixel in the segmentation mask result.

[0187] In a feasible implementation scheme, the detection module 1202 is specifically used for:

[0188] Performing image enhancement processing on each original image based on a preset enhancement strategy to obtain a processed image and at least one enhanced bounding box corresponding to each processed image, wherein the enhancement strategy includes at least one of the following: image horizontal flipping, image scaling, and image rotation;

[0189] An offset process and a scaling process are performed on each enhanced bounding box according to a preset offset range and a scaling coefficient range to obtain a processed bounding box corresponding to each enhanced bounding box.

[0190] In a feasible implementation scheme, the detection module 1202 is specifically used for:

[0191] The instance segmentation masks of each processed image are fused to obtain the segmentation mask results of each processed image;

[0192] The segmentation mask results of each processed image are averaged and fused to obtain the segmentation mask results of the original image corresponding to each processed image.

[0193] In a feasible implementation scheme, the detection module 1202 is specifically used for:

[0194] Compare the confidence of each pixel in the segmentation mask result with the first confidence threshold, and determine the number of pixels in the segmentation mask result whose confidence is greater than the first confidence threshold;

[0195] Determine the ratio of the number of pixels to the total number of pixels in the original image;

[0196] If the ratio is greater than a preset second confidence threshold, the original image indicated by the ratio and the segmentation mask result are combined to obtain a second annotated image.

[0197] In a feasible implementation scheme, the first training module 1203 is specifically used for:

[0198] Traverse multiple first annotated images and second annotated images, and for the traversed current annotated image, if the confidence of the current annotated image is greater than or equal to a preset threshold, add the current annotated image to the first training data set, otherwise, add the current annotated image to the second training data set.

[0199] In a feasible implementation scheme, the first training module 1203 is specifically used for:

[0200] Inputting each annotated image in the first training data set into the initial teacher model, and performing image segmentation processing on each annotated image by the initial teacher model to obtain an initial segmentation result of each annotated image;

[0201] Determine the loss value of the initial teacher model according to the initial segmentation result of each annotated image, the annotation information of each annotated image, and the confidence of each annotated image;

[0202] The initial teacher model is optimized according to the loss value to obtain the target teacher model.

[0203] In a feasible implementation scheme, the first training module 1203 is specifically used for:

[0204] The initial segmentation result of the annotated image, the confidence level of each annotated image and the annotation information are used as input parameters of the loss function, and the loss function determines the loss value of each pixel based on the initial segmentation result and the annotation information;

[0205] The loss function performs weighted calculation on the loss value of each pixel according to the confidence of each labeled image to obtain the loss value of the initial teacher model.

[0206] In a feasible implementation, the prediction module 1204 is specifically used to:

[0207] Performing enhancement processing on each labeled image to obtain multiple processed images;

[0208] Input each processed image into the target teacher model, and the target teacher model performs image segmentation processing to obtain the segmentation mask of each processed image;

[0209] The segmentation masks of each processed image are fused to obtain the segmentation mask results of the annotated image corresponding to each processed image;

[0210] Filtering each annotated image based on the confidence of each segmentation mask result and a preset first confidence threshold to obtain a filtered annotated image;

[0211] A third training data set is generated according to the filtered labeled images and the first training data set.

[0212] In a feasible implementation, the prediction module 1204 is specifically used to:

[0213] If the round number of the third training data set obtained by predicting the second training data set according to the target teacher model is equal to the preset value, all the obtained filtered and annotated images are added to the first training data set to obtain the third training data set.

[0214] In a feasible implementation, the prediction module 1204 is specifically used to:

[0215] If the sum of the number of the filtered annotated images and the number of the third annotated images in the first training data set is equal to or greater than a preset number threshold, merging the filtered annotated images and the first training data set into a third training data set;

[0216] If the sum of the number of annotated images after screening and the number of third annotated images in the first training data set is less than a preset threshold, the second training data set is re-determined, and the step of predicting the second training data set according to the target teacher model is performed until the current number of annotated images after screening and the sum of the number of third annotated images in the first training data set is equal to or greater than the preset threshold, so as to obtain the third training data set.

[0217] The embodiment of the present application can generate a large number of usable annotated images more quickly when the number of foreground images is limited by merging the target foreground image with the background image based on mask information. By performing enhanced detection processing on the original image based on the target detector and the first segmentation model, more reliable annotation information can be automatically generated for the original image to obtain an annotated image with a high degree of reliability. The initial teacher model is trained with an annotated image with high reliability to obtain a target teacher model, and the second training data set is predicted based on the target teacher model to generate an annotated image with a higher degree of reliability, thereby expanding the annotated image, so that a large number of usable, highly reliable annotated images can be used as training samples when training based on the student model. The present application effectively realizes the expansion of training data in low-resource scenarios. Through the teacher-student model training method, the reuse of unlabeled data and low-confidence data can be realized, the utilization rate of training data is improved, and the efficiency of model training in low-resource scenarios and the reliability of the trained model are improved.

[0218] Fig.13 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown, including: a processor 1301, a storage medium 1302 and a bus 1303, wherein the storage medium 1302 stores machine-readable instructions executable by the processor 1301. When the electronic device runs an image segmentation model training method in the embodiment, the processor 1301 communicates with the storage medium 1302 via the bus 1303, and the processor 1301 executes the machine-readable instructions. The processor 1301 performs the preamble of the method item to perform the following steps:

[0219] Generate a plurality of first annotated images according to a plurality of target foreground images, a plurality of target mask information and a plurality of preset background images, wherein the first annotated images include annotation information, and the annotation information is used to indicate an area corresponding to the target foreground image in the first annotated image;

[0220] Performing enhanced detection processing according to the plurality of original images, the pre-trained target detector, and the pre-trained first segmentation model to obtain a second annotated image of each original image, wherein the second annotated image includes annotation information of the original image;

[0221] Generate a first training data set and a second training data set according to the plurality of first annotated images and the plurality of second annotated images, and train the initial teacher model according to the first training data set to obtain a target teacher model, wherein the confidence of each annotated image in the first training data set is greater than the confidence of each annotated image in the second training data set;

[0222] Predict the second training data set according to the target teacher model to obtain a third training data set;

[0223] The initial student model is trained according to the third training data set to obtain a target image segmentation model.

[0224] In a feasible implementation manner, when the processor 1301 generates a plurality of first annotated images according to a plurality of target foreground images, a plurality of target mask information and a plurality of preset background images, it is specifically configured to:

[0225] Transforming a plurality of target foreground images and a plurality of target mask information to obtain transformed foreground images and transformed mask information;

[0226] Performing superposition processing on each transformed foreground image, each transformed mask information and each background image to obtain a plurality of superimposed images;

[0227] The multiple superimposed images are enhanced to obtain multiple first annotated images.

[0228] In a feasible implementation, the processor 1301, when performing enhanced detection processing according to multiple original images, a pre-trained target detector, and a pre-trained first segmentation model to obtain a second annotated image of each original image, is specifically used to:

[0229] Inputting a plurality of original images into an object detector to obtain at least one bounding box of each original image, wherein the bounding box is used to indicate a position of a detection object in the original image;

[0230] Performing enhancement processing on each original image and at least one bounding box corresponding to each original image to obtain a plurality of processed images and at least one processed bounding box corresponding to each processed image;

[0231] Inputting each processed image and at least one processed bounding box corresponding to each processed image into a pre-trained first segmentation model, performing image segmentation processing by the first segmentation model, and obtaining a plurality of instance segmentation masks for each processed image;

[0232] A fusion screening process is performed according to multiple instance segmentation masks of each processed image to obtain a second annotated image of each original image.

[0233] In a feasible implementation, when the processor 1301 performs fusion screening processing according to multiple instance segmentation masks of each processed image to obtain the second annotated image of each original image, it is specifically used to:

[0234] The instance segmentation masks of each processed image are fused to obtain the segmentation mask results of the original image corresponding to each processed image;

[0235] Based on the confidence of each segmentation mask result and a preset first confidence threshold, each original image is screened to obtain a second annotated image, and the confidence of the segmentation mask result is used to characterize the confidence of each pixel in the segmentation mask result.

[0236] In a feasible implementation manner, when the processor 1301 performs enhancement processing on each original image and at least one bounding box corresponding to each original image to obtain multiple processed images and at least one processed bounding box corresponding to each processed image, it is specifically configured to:

[0237] Performing image enhancement processing on each original image based on a preset enhancement strategy to obtain a processed image and at least one enhanced bounding box corresponding to each processed image, wherein the enhancement strategy includes at least one of the following: image horizontal flipping, image scaling, and image rotation;

[0238] An offset process and a scaling process are performed on each enhanced bounding box according to a preset offset range and a scaling coefficient range to obtain a processed bounding box corresponding to each enhanced bounding box.

[0239] In a feasible implementation manner, when the processor 1301 performs fusion processing on the instance segmentation masks of each processed image to obtain the segmentation mask result of the original image corresponding to each processed image, it is specifically used to:

[0240] The instance segmentation masks of each processed image are fused to obtain the segmentation mask results of each processed image;

[0241] The segmentation mask results of each processed image are averaged and fused to obtain the segmentation mask results of the original image corresponding to each processed image.

[0242] In a feasible implementation manner, when the processor 1301 performs screening of each original image based on the confidence of each segmentation mask result and a preset first confidence threshold to obtain the second annotated image, it is specifically used to:

[0243] Compare the confidence of each pixel in the segmentation mask result with the first confidence threshold, and determine the number of pixels in the segmentation mask result whose confidence is greater than the first confidence threshold;

[0244] Determine the ratio of the number of pixels to the total number of pixels in the original image;

[0245] If the ratio is greater than a preset second confidence threshold, the original image indicated by the ratio and the segmentation mask result are combined to obtain a second annotated image.

[0246] In a feasible implementation manner, when the processor 1301 generates a first training data set and a second training data set according to a plurality of first annotated images and a plurality of second annotated images, the processor 1301 is specifically configured to:

[0247] Traverse multiple first annotated images and second annotated images, and for the traversed current annotated image, if the confidence of the current annotated image is greater than or equal to a preset threshold, add the current annotated image to the first training data set, otherwise, add the current annotated image to the second training data set.

[0248] In a feasible implementation manner, when the processor 1301 performs training on the initial teacher model according to the first training data set to obtain the target teacher model, it is specifically used to:

[0249] Inputting each annotated image in the first training data set into the initial teacher model, and performing image segmentation processing on each annotated image by the initial teacher model to obtain an initial segmentation result of each annotated image;

[0250] Determine the loss value of the initial teacher model according to the initial segmentation result of each annotated image, the annotation information of each annotated image, and the confidence of each annotated image;

[0251] The initial teacher model is optimized according to the loss value to obtain the target teacher model.

[0252] In a feasible implementation manner, when the processor 1301 determines the loss value of the initial teacher model according to the initial segmentation results of each annotated image, the annotation information of each annotated image, and the confidence of each annotated image, it is specifically used to:

[0253] The initial segmentation result of the annotated image, the confidence level of each annotated image and the annotation information are used as input parameters of the loss function, and the loss function determines the loss value of each pixel based on the initial segmentation result and the annotation information;

[0254] The loss function performs weighted calculation on the loss value of each pixel according to the confidence of each labeled image to obtain the loss value of the initial teacher model.

[0255] In a feasible implementation manner, when the processor 1301 predicts the second training data set according to the target teacher model to obtain the third training data set, it is specifically used to:

[0256] Performing enhancement processing on each labeled image to obtain multiple processed images;

[0257] Input each processed image into the target teacher model, and the target teacher model performs image segmentation processing to obtain the segmentation mask of each processed image;

[0258] The segmentation masks of each processed image are fused to obtain the segmentation mask results of the annotated image corresponding to each processed image;

[0259] Filtering each annotated image based on the confidence of each segmentation mask result and a preset first confidence threshold to obtain a filtered annotated image;

[0260] A third training data set is generated according to the filtered labeled images and the first training data set.

[0261] In a feasible implementation manner, when the processor 1301 generates the third training data set according to the screened labeled image and the first training data set, it is specifically used to:

[0262] If the round number of the third training data set obtained by predicting the second training data set according to the target teacher model is equal to the preset value, all the obtained filtered and annotated images are added to the first training data set to obtain the third training data set.

[0263] In a feasible implementation manner, when the processor 1301 generates the third training data set according to the screened labeled image and the first training data set, it is specifically used to:

[0264] If the sum of the number of the filtered annotated images and the number of the third annotated images in the first training data set is equal to or greater than a preset number threshold, merging the filtered annotated images and the first training data set into a third training data set;

[0265] If the sum of the number of annotated images after screening and the number of third annotated images in the first training data set is less than a preset threshold, the second training data set is re-determined, and the step of predicting the second training data set according to the target teacher model is performed until the current number of annotated images after screening and the sum of the number of third annotated images in the first training data set is equal to or greater than the preset threshold, so as to obtain the third training data set.

[0266] The embodiment of the present application can generate a large number of usable annotated images more quickly when the number of foreground images is limited by merging the target foreground image with the background image based on mask information. By performing enhanced detection processing on the original image based on the target detector and the first segmentation model, more reliable annotation information can be automatically generated for the original image to obtain an annotated image with a high degree of reliability. The initial teacher model is trained with an annotated image with high reliability to obtain a target teacher model, and the second training data set is predicted based on the target teacher model to generate an annotated image with a higher degree of reliability, thereby expanding the annotated image, so that a large number of usable, highly reliable annotated images can be used as training samples when training based on the student model. The present application effectively realizes the expansion of training data in low-resource scenarios. Through the teacher-student model training method, the reuse of unlabeled data and low-confidence data can be realized, the utilization rate of training data is improved, and the efficiency of model training in low-resource scenarios and the reliability of the trained model are improved.

[0267] The embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. The computer program is executed when a processor is running, and the processor performs the following steps:

[0268] Generate a plurality of first annotated images according to a plurality of target foreground images, a plurality of target mask information and a plurality of preset background images, wherein the first annotated images include annotation information, and the annotation information is used to indicate an area corresponding to the target foreground image in the first annotated image;

[0269] Performing enhanced detection processing according to the plurality of original images, the pre-trained target detector, and the pre-trained first segmentation model to obtain a second annotated image of each original image, wherein the second annotated image includes annotation information of the original image;

[0270] Generate a first training data set and a second training data set according to the plurality of first annotated images and the plurality of second annotated images, and train the initial teacher model according to the first training data set to obtain a target teacher model, wherein the confidence of each annotated image in the first training data set is greater than the confidence of each annotated image in the second training data set;

[0271] Predict the second training data set according to the target teacher model to obtain a third training data set;

[0272] The initial student model is trained according to the third training data set to obtain a target image segmentation model.

[0273] In a feasible implementation manner, when the processor generates a plurality of first annotated images according to a plurality of target foreground images, a plurality of target mask information and a plurality of preset background images, the processor is specifically configured to:

[0274] Transforming a plurality of target foreground images and a plurality of target mask information to obtain transformed foreground images and transformed mask information;

[0275] Performing superposition processing on each transformed foreground image, each transformed mask information and each background image to obtain a plurality of superimposed images;

[0276] The multiple superimposed images are enhanced to obtain multiple first annotated images.

[0277] In a feasible implementation, when the processor performs enhanced detection processing according to multiple original images, a pre-trained target detector, and a pre-trained first segmentation model to obtain a second annotated image of each original image, it is specifically used to:

[0278] Inputting a plurality of original images into an object detector to obtain at least one bounding box of each original image, wherein the bounding box is used to indicate a position of a detection object in the original image;

[0279] Performing enhancement processing on each original image and at least one bounding box corresponding to each original image to obtain a plurality of processed images and at least one processed bounding box corresponding to each processed image;

[0280] Inputting each processed image and at least one processed bounding box corresponding to each processed image into a pre-trained first segmentation model, performing image segmentation processing by the first segmentation model, and obtaining a plurality of instance segmentation masks of each processed image;

[0281] A fusion screening process is performed according to multiple instance segmentation masks of each processed image to obtain a second annotated image of each original image.

[0282] In a feasible implementation, when the processor performs fusion screening processing according to multiple instance segmentation masks of each processed image to obtain the second annotated image of each original image, it is specifically used to:

[0283] The instance segmentation masks of each processed image are fused to obtain the segmentation mask results of the original image corresponding to each processed image;

[0284] Based on the confidence of each segmentation mask result and a preset first confidence threshold, each original image is screened to obtain a second annotated image, and the confidence of the segmentation mask result is used to characterize the confidence of each pixel in the segmentation mask result.

[0285] In a feasible implementation, when the processor performs enhancement processing on each original image and at least one bounding box corresponding to each original image to obtain multiple processed images and at least one processed bounding box corresponding to each processed image, it is specifically used to:

[0286] Performing image enhancement processing on each original image based on a preset enhancement strategy to obtain a processed image and at least one enhanced bounding box corresponding to each processed image, wherein the enhancement strategy includes at least one of the following: image horizontal flipping, image scaling, and image rotation;

[0287] An offset process and a scaling process are performed on each enhanced bounding box according to a preset offset range and a scaling coefficient range to obtain a processed bounding box corresponding to each enhanced bounding box.

[0288] In a feasible implementation manner, when the processor performs fusion processing on the instance segmentation masks of each processed image to obtain the segmentation mask results of the original image corresponding to each processed image, it is specifically used to:

[0289] The instance segmentation masks of each processed image are fused to obtain the segmentation mask results of each processed image;

[0290] The segmentation mask results of each processed image are averaged and fused to obtain the segmentation mask results of the original image corresponding to each processed image.

[0291] In a feasible implementation manner, when the processor performs screening of each original image based on the confidence of each segmentation mask result and a preset first confidence threshold to obtain the second annotated image, the processor is specifically used to:

[0292] Compare the confidence of each pixel in the segmentation mask result with the first confidence threshold, and determine the number of pixels in the segmentation mask result whose confidence is greater than the first confidence threshold;

[0293] Determine the ratio of the number of pixels to the total number of pixels in the original image;

[0294] If the ratio is greater than a preset second confidence threshold, the original image indicated by the ratio and the segmentation mask result are combined to obtain a second annotated image.

[0295] In a feasible implementation manner, when the processor generates a first training data set and a second training data set according to a plurality of first annotated images and a plurality of second annotated images, the processor is specifically configured to:

[0296] Traverse multiple first annotated images and second annotated images, and for the traversed current annotated image, if the confidence of the current annotated image is greater than or equal to a preset threshold, add the current annotated image to the first training data set, otherwise, add the current annotated image to the second training data set.

[0297] In a feasible implementation manner, when the processor performs training on the initial teacher model according to the first training data set to obtain the target teacher model, it is specifically used to:

[0298] Inputting each annotated image in the first training data set into the initial teacher model, and performing image segmentation processing on each annotated image by the initial teacher model to obtain an initial segmentation result of each annotated image;

[0299] Determine the loss value of the initial teacher model according to the initial segmentation result of each annotated image, the annotation information of each annotated image, and the confidence of each annotated image;

[0300] The initial teacher model is optimized according to the loss value to obtain the target teacher model.

[0301] In a feasible implementation, when the processor determines the loss value of the initial teacher model according to the initial segmentation results of each labeled image, the labeling information of each labeled image, and the confidence of each labeled image, it is specifically used to:

[0302] The initial segmentation result of the annotated image, the confidence level of each annotated image and the annotation information are used as input parameters of the loss function, and the loss function determines the loss value of each pixel based on the initial segmentation result and the annotation information;

[0303] The loss function performs weighted calculation on the loss value of each pixel according to the confidence of each labeled image to obtain the loss value of the initial teacher model.

[0304] In a feasible implementation manner, when the processor predicts the second training data set according to the target teacher model to obtain the third training data set, it is specifically used to:

[0305] Performing enhancement processing on each labeled image to obtain multiple processed images;

[0306] Input each processed image into the target teacher model, and the target teacher model performs image segmentation processing to obtain the segmentation mask of each processed image;

[0307] The segmentation masks of each processed image are fused to obtain the segmentation mask results of the annotated image corresponding to each processed image;

[0308] Filtering each annotated image based on the confidence of each segmentation mask result and a preset first confidence threshold to obtain a filtered annotated image;

[0309] A third training data set is generated according to the filtered labeled images and the first training data set.

[0310] In a feasible implementation manner, when the processor generates the third training data set according to the screened labeled image and the first training data set, the processor is specifically configured to:

[0311] If the round number of the third training data set obtained by predicting the second training data set according to the target teacher model is equal to the preset value, all the obtained filtered and annotated images are added to the first training data set to obtain the third training data set.

[0312] In a feasible implementation manner, when the processor generates the third training data set according to the screened labeled image and the first training data set, the processor is specifically configured to:

[0313] If the sum of the number of the filtered annotated images and the number of the third annotated images in the first training data set is equal to or greater than a preset number threshold, merging the filtered annotated images and the first training data set into a third training data set;

[0314] If the sum of the number of annotated images after screening and the number of third annotated images in the first training data set is less than a preset threshold, the second training data set is re-determined, and the step of predicting the second training data set according to the target teacher model is performed until the current number of annotated images after screening and the sum of the number of third annotated images in the first training data set is equal to or greater than the preset threshold, so as to obtain the third training data set.

[0315] The embodiment of the present application can generate a large number of usable annotated images more quickly when the number of foreground images is limited by merging the target foreground image with the background image based on mask information. By performing enhanced detection processing on the original image based on the target detector and the first segmentation model, more reliable annotation information can be automatically generated for the original image to obtain an annotated image with a high degree of reliability. The initial teacher model is trained with an annotated image with high reliability to obtain a target teacher model, and the second training data set is predicted based on the target teacher model to generate an annotated image with a higher degree of reliability, thereby expanding the annotated image, so that a large number of usable, highly reliable annotated images can be used as training samples when training based on the student model. The present application effectively realizes the expansion of training data in low-resource scenarios. Through the teacher-student model training method, the reuse of unlabeled data and low-confidence data can be realized, the utilization rate of training data is improved, and the efficiency of model training in low-resource scenarios and the reliability of the trained model are improved.

[0316] In the embodiment of the present application, the computer program can also execute other machine-readable instructions when run by the processor to execute other methods described in the embodiment. For the specific execution method steps and principles, please refer to the description of the embodiment, which will not be repeated here.

[0317] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0318] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0319] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0320] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0321] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0322] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. They should all be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for training an image segmentation model, characterized in that: include: Generate a plurality of first annotated images according to a plurality of target foreground images, a plurality of target mask information, and a plurality of preset background images, wherein the first annotated images include annotation information, and the annotation information is used to indicate an area corresponding to the target foreground image in the first annotated images; Performing enhanced detection processing according to a plurality of original images, a pre-trained object detector, and a pre-trained first segmentation model to obtain a second annotated image of each of the original images, wherein the second annotated image includes annotation information of the original image; Generate a first training data set and a second training data set according to the plurality of first annotated images and the plurality of second annotated images, and train an initial teacher model according to the first training data set to obtain a target teacher model, wherein the confidence of each annotated image in the first training data set is greater than the confidence of each annotated image in the second training data set; Predicting the second training data set according to the target teacher model to obtain a third training data set; The initial student model is trained according to the third training data set to obtain a target image segmentation model.

2. The method according to claim 1, characterized in that The step of generating a plurality of first annotated images according to a plurality of target foreground images, a plurality of target mask information and a plurality of preset background images includes: Transforming the multiple target foreground images and the multiple target mask information to obtain transformed foreground images and transformed mask information; Performing a superposition process on each of the transformed foreground images, each of the transformed mask information, and each of the background images to obtain a plurality of superimposed images; The multiple superimposed images are enhanced to obtain the multiple first annotated images.

3. The method according to claim 1, characterized in that The method of performing enhanced detection processing according to the plurality of original images, the pre-trained target detector and the pre-trained first segmentation model to obtain a second annotated image of each of the original images includes: Inputting the plurality of original images into an object detector to obtain at least one bounding box of each of the original images, wherein the bounding box is used to indicate a position of a detection object in the original image; Performing enhancement processing on each of the original images and at least one bounding box corresponding to each of the original images to obtain a plurality of processed images and at least one processed bounding box corresponding to each of the processed images; Inputting each of the processed images and at least one processed bounding box corresponding to each of the processed images into a pre-trained first segmentation model, and performing image segmentation processing by the first segmentation model to obtain a plurality of instance segmentation masks for each of the processed images; A fusion screening process is performed according to multiple instance segmentation masks of each of the processed images to obtain a second annotated image of each of the original images.

4. The method according to claim 3, characterized in that The step of performing fusion screening processing according to the multiple instance segmentation masks of each processed image to obtain a second annotated image of each original image includes: Performing fusion processing on the instance segmentation masks of each processed image to obtain the segmentation mask results of the original images corresponding to each processed image; The original images are screened based on the confidence of each segmentation mask result and a preset first confidence threshold to obtain the second annotated image, wherein the confidence of the segmentation mask result is used to characterize the confidence of each pixel in the segmentation mask result.

5. The method according to claim 3, characterized in that: The step of performing enhancement processing on each of the original images and at least one bounding box corresponding to each of the original images to obtain a plurality of processed images and at least one processed bounding box corresponding to each of the processed images includes: Performing image enhancement processing on each of the original images based on a preset enhancement strategy to obtain the processed image and at least one enhanced bounding box corresponding to each of the processed images, wherein the enhancement strategy includes at least one of the following: image horizontal flipping, image scaling, and image rotation; The enhanced bounding boxes are subjected to offset processing and scaling processing according to a preset offset interval and scaling coefficient interval to obtain processed bounding boxes corresponding to the enhanced bounding boxes.

6. The method according to claim 4, characterized in that The step of fusing the instance segmentation masks of the processed images to obtain the segmentation mask results of the original images corresponding to the processed images includes: Fusing the instance segmentation masks of the processed images to obtain the segmentation mask results of the processed images; The segmentation mask results of each processed image are averaged and fused to obtain the segmentation mask results of the original image corresponding to each processed image.

7. The method according to claim 4, characterized in that The step of screening each of the original images based on the confidence of each of the segmentation mask results and a preset first confidence threshold to obtain the second annotated image includes: Comparing the confidence of each pixel in the segmentation mask result with the first confidence threshold, and determining the number of pixels in the segmentation mask result whose confidence is greater than the first confidence threshold; Determine a ratio of the number of pixels to the total number of pixels of the original image; If the ratio is greater than a preset second confidence threshold, the original image indicated by the ratio and the segmentation mask result are combined to obtain the second annotated image.

8. The method according to claim 1, characterized in that The step of generating a first training data set and a second training data set according to the plurality of first annotated images and the plurality of second annotated images comprises: The plurality of first annotated images and the second annotated images are traversed, and for a current annotated image traversed, if a confidence level of the current annotated image is greater than or equal to a preset threshold, the current annotated image is added to the first training data set; otherwise, the current annotated image is added to the second training data set.

9. The method according to claim 1, characterized in that: The step of training the initial teacher model according to the first training data set to obtain a target teacher model includes: Inputting each annotated image in the first training data set into an initial teacher model, and performing image segmentation processing on each annotated image by the initial teacher model to obtain an initial segmentation result of each annotated image; Determining the loss value of the initial teacher model according to the initial segmentation result of each annotated image, the annotation information of each annotated image, and the confidence of each annotated image; The initial teacher model is optimized according to the loss value to obtain the target teacher model.

10. The method according to claim 9, characterized in that Determining the loss value of the initial teacher model according to the initial segmentation results of each annotated image, the annotation information of each annotated image, and the confidence of each annotated image includes: The initial segmentation result of the annotated image, the confidence level of each annotated image and the annotation information are used as input parameters of a loss function, and the loss function determines the loss value of each pixel according to the initial segmentation result and the annotation information; The loss function performs weighted calculation on the loss value of each pixel point according to the confidence of each labeled image to obtain the loss value of the initial teacher model.

11. The method according to claim 1, characterized in that: The step of predicting the second training data set according to the target teacher model to obtain a third training data set includes: Performing enhancement processing on each labeled image to obtain multiple processed images; Inputting each of the processed images into the target teacher model, and performing image segmentation processing by the target teacher model to obtain a segmentation mask for each of the processed images; Performing fusion processing on the segmentation masks of each processed image to obtain the segmentation mask results of the annotated images corresponding to each processed image; Screening each of the annotated images based on the confidence of each of the segmentation mask results and a preset first confidence threshold to obtain a screened annotated image; The third training data set is generated according to the filtered annotated images and the first training data set.

12. The method according to claim 11, characterized in that The step of generating the third training data set according to the filtered annotated image and the first training data set includes: If the round number of the third training data set obtained by predicting the second training data set according to the target teacher model is equal to a preset value, all the obtained filtered and annotated images are added to the first training data set to obtain the third training data set.

13. The method according to claim 11, characterized in that The step of generating the third training data set according to the filtered annotated image and the first training data set includes: If the sum of the number of the filtered annotated images and the number of the third annotated images in the first training data set is equal to or greater than a preset number threshold, merging the filtered annotated images and the first training data set into the third training data set; If the sum of the number of the filtered annotated images and the number of the third annotated images in the first training data set is less than a preset threshold, the second training data set is re-determined, and the step of predicting the second training data set according to the target teacher model is performed until the current number of filtered annotated images and the sum of the number of the third annotated images in the first training data set is equal to or greater than the preset threshold, so as to obtain the third training data set.

14. An image segmentation model training device, characterized in that: include: A generating module, configured to generate a plurality of first annotated images according to a plurality of target foreground images, a plurality of target mask information and a plurality of preset background images, wherein the first annotated images include annotation information, and the annotation information is used to indicate an area corresponding to the target foreground image in the first annotated images; A detection module, configured to perform enhanced detection processing according to a plurality of original images, a pre-trained target detector, and a pre-trained first segmentation model, to obtain a second annotated image of each of the original images, wherein the second annotated image includes annotation information of the original image; A first training module, used to generate a first training data set and a second training data set according to the plurality of first annotated images and the plurality of second annotated images, and train an initial teacher model according to the first training data set to obtain a target teacher model, wherein the confidence of each annotated image in the first training data set is greater than the confidence of each annotated image in the second training data set; A prediction module, used to predict the second training data set according to the target teacher model to obtain a third training data set; The second training module is used to train the initial student model according to the third training data set to obtain a target image segmentation model.

15. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the image segmentation model training method as described in any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image segmentation model training method according to any one of claims 1 to 13 are executed.

Images