Target segmentation model training method and device, readable storage medium and electronic equipment

Abstract

The application belongs to the technical field of target segmentation, and particularly relates to a target segmentation model training method and device, a computer readable storage medium and an electronic device. The method comprises: inputting a sample image into a target segmentation model, and obtaining a first predicted target segmentation mask of the sample image output by the target segmentation model; the resolution of the first predicted target segmentation mask is less than that of a real target segmentation mask of the sample image; performing up-sampling processing on the first predicted target segmentation mask to obtain an up-sampled target segmentation mask of the sample image; the resolution of the up-sampled target segmentation mask is equal to that of the real target segmentation mask; determining a first target segmentation loss of the target segmentation model according to the up-sampled target segmentation mask and the real target segmentation mask; and training the target segmentation model based on the first target segmentation loss to obtain a trained target segmentation model.

Description

Technical Field

[0001] This application belongs to the field of target segmentation technology, and in particular relates to a target segmentation model training method, apparatus, computer-readable storage medium and electronic device. Background Technology

[0002] With the rapid advancement of embodied intelligence technology and deep learning visual perception algorithms, humanoid robots are playing an increasingly important role in tasks such as handling, stacking, sorting, and warehousing logistics. To achieve stable grasping and precise manipulation in complex environments, robots must possess high-precision perception and segmentation capabilities for boxes (such as bins and cartons). Through accurate contour extraction and region separation of the target box using segmentation algorithms, the robot can further perform key operations such as grasping and positioning, posture estimation, and obstacle avoidance, thereby ensuring the reliability and safety of the overall handling task.

[0003] In practical applications, due to limitations in computing resources and real-time requirements of humanoid robot end-devices, lightweight target segmentation models have become the mainstream deployment choice. These target segmentation models extract features and perform multi-scale predictions on the input image, outputting bounding boxes (bboxes) and corresponding target segmentation masks to describe the spatial region of the box in the image.

[0004] However, in box segmentation tasks, especially for fine edge structures on the top surface of the box (such as box borders, narrow edge strips, etc.), since they only occupy a small number of pixels in the original image, they are prone to edge breakage or structure disappearance when performing mask downsampling during the training phase, resulting in poor segmentation performance of the trained target segmentation model in fine structure regions. Summary of the Invention

[0005] In view of this, embodiments of this application provide a target segmentation model training method, apparatus, computer-readable storage medium, and electronic device to solve the problem of poor segmentation performance in fine-structure regions in the prior art.

[0006] A first aspect of this application provides a method for training a target segmentation model, which may include: A sample image is input into a target segmentation model, and a first predicted target segmentation mask of the sample image is obtained from the output of the target segmentation model; wherein the resolution of the first predicted target segmentation mask is smaller than the resolution of the true target segmentation mask of the sample image; The first predicted target segmentation mask is upsampled to obtain an upsampled target segmentation mask of the sample image; wherein the resolution of the upsampled target segmentation mask is equal to the resolution of the real target segmentation mask; Based on the upsampled target segmentation mask and the real target segmentation mask, the first target segmentation loss of the target segmentation model is determined; The target segmentation model is trained based on the first target segmentation loss to obtain the trained target segmentation model.

[0007] In one specific implementation of the first aspect, the upsampling process performed on the first predicted target segmentation mask to obtain the upsampled target segmentation mask of the sample image may include: Based on a preset upsampling factor, the first predicted target segmentation mask is upsampled using a bilinear interpolation algorithm to obtain the upsampled target segmentation mask. Wherein, the upsampling factor is the resolution ratio between the real target segmentation mask and the first predicted target segmentation mask.

[0008] In one specific implementation of the first aspect, determining the first target segmentation loss of the target segmentation model based on the upsampled target segmentation mask and the real target segmentation mask may include: Determine the binary cross-entropy loss between the upsampled target segmentation mask and the real target segmentation mask; The binary cross-entropy loss is determined as the first target segmentation loss of the target segmentation model.

[0009] In one specific implementation of the first aspect, before inputting the sample image into the target segmentation model and obtaining the first predicted target segmentation mask of the sample image output by the target segmentation model, the method may further include: The sample image is input into an initial target segmentation model, and a second predicted target segmentation mask of the sample image is obtained from the output of the initial target segmentation model; wherein the resolution of the second predicted target segmentation mask is smaller than the resolution of the real target segmentation mask; The real target segmentation mask is downsampled to obtain a downsampled target segmentation mask of the sample image; wherein the resolution of the downsampled target segmentation mask is equal to the resolution of the second predicted target segmentation mask; Based on the second predicted target segmentation mask and the downsampled target segmentation mask, determine the second target segmentation loss of the initial target segmentation model; The initial target segmentation model is trained based on the second target segmentation loss to obtain the target segmentation model.

[0010] In one specific implementation of the first aspect, the downsampling process performed on the real target segmentation mask to obtain the downsampled target segmentation mask of the sample image may include: Based on a preset downsampling factor, a bilinear interpolation algorithm is used to downsample the real target segmentation mask to obtain the downsampled target segmentation mask. Wherein, the downsampling factor is the resolution ratio between the real target segmentation mask and the second predicted target segmentation mask.

[0011] In one specific implementation of the first aspect, determining the second target segmentation loss of the initial target segmentation model based on the second predicted target segmentation mask and the downsampled target segmentation mask may include: Determine the binary cross-entropy loss between the second predicted target segmentation mask and the downsampled target segmentation mask; The binary cross-entropy loss is determined as the second target segmentation loss of the initial target segmentation model.

[0012] In one specific implementation of the first aspect, after training the target segmentation model based on the first target segmentation loss to obtain the trained target segmentation model, it may further include: Obtain the image to be segmented; The trained target segmentation model is used to perform target segmentation processing on the image to be segmented, and the target segmentation result of the image to be segmented is obtained.

[0013] A second aspect of this application provides a target segmentation model training apparatus, which may include: The first mask acquisition module is used to input a sample image into a target segmentation model and acquire a first predicted target segmentation mask of the sample image output by the target segmentation model; wherein the resolution of the first predicted target segmentation mask is smaller than the resolution of the real target segmentation mask of the sample image; A mask upsampling processing module is used to perform upsampling processing on the first predicted target segmentation mask to obtain an upsampled target segmentation mask of the sample image; wherein, the resolution of the upsampled target segmentation mask is equal to the resolution of the real target segmentation mask; The first loss determination module is used to determine the first target segmentation loss of the target segmentation model based on the upsampled target segmentation mask and the real target segmentation mask. The first model training module is used to train the target segmentation model based on the first target segmentation loss to obtain the trained target segmentation model.

[0014] In one specific implementation of the second aspect, the mask upsampling processing module can be specifically used to: perform upsampling processing on the first predicted target segmentation mask using a bilinear interpolation algorithm based on a preset upsampling factor to obtain the upsampled target segmentation mask; wherein, the upsampling factor is the resolution factor between the real target segmentation mask and the first predicted target segmentation mask.

[0015] In one specific implementation of the second aspect, the first loss determination module may be specifically used to: determine the binary cross-entropy loss between the upsampled target segmentation mask and the real target segmentation mask; and determine the binary cross-entropy loss as the first target segmentation loss of the target segmentation model.

[0016] In one specific implementation of the second aspect, the target segmentation model training device may further include: The second mask acquisition module is used to input the sample image into the initial target segmentation model and acquire the second predicted target segmentation mask of the sample image output by the initial target segmentation model; wherein the resolution of the second predicted target segmentation mask is smaller than the resolution of the real target segmentation mask; A mask downsampling processing module is used to downsample the real target segmentation mask to obtain a downsampled target segmentation mask of the sample image; wherein the resolution of the downsampled target segmentation mask is equal to the resolution of the second predicted target segmentation mask; The second loss determination module is used to determine the second target segmentation loss of the initial target segmentation model based on the second predicted target segmentation mask and the downsampled target segmentation mask. The second model training module is used to train the initial target segmentation model based on the second target segmentation loss to obtain the target segmentation model.

[0017] In one specific implementation of the second aspect, the mask downsampling processing module can be specifically used to: perform downsampling processing on the real target segmentation mask using a bilinear interpolation algorithm based on a preset downsampling ratio to obtain the downsampled target segmentation mask; wherein, the downsampling ratio is the resolution ratio between the real target segmentation mask and the second predicted target segmentation mask.

[0018] In one specific implementation of the second aspect, the second loss determination module may be specifically used to: determine the binary cross-entropy loss between the second predicted target segmentation mask and the downsampled target segmentation mask; and determine the binary cross-entropy loss as the second target segmentation loss of the initial target segmentation model.

[0019] In one specific implementation of the second aspect, the target segmentation model training device may further include: The image to be segmented acquisition module is used to acquire the image to be segmented; The target segmentation processing module is used to perform target segmentation processing on the image to be segmented using the trained target segmentation model, so as to obtain the target segmentation result of the image to be segmented.

[0020] A third aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above-described target segmentation model training methods.

[0021] A fourth aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above-described target segmentation model training methods.

[0022] The fifth aspect of this application provides a computer program product that, when run on an electronic device, causes the electronic device to execute the steps of any of the above-described target segmentation model training methods.

[0023] The beneficial effects of this application embodiment compared with the prior art are as follows: This application embodiment inputs a sample image into a target segmentation model and obtains a first predicted target segmentation mask of the sample image output by the target segmentation model; wherein, the resolution of the first predicted target segmentation mask is less than the resolution of the real target segmentation mask of the sample image; the first predicted target segmentation mask is upsampled to obtain an upsampled target segmentation mask of the sample image; wherein, the resolution of the upsampled target segmentation mask is equal to the resolution of the real target segmentation mask; a first target segmentation loss of the target segmentation model is determined based on the upsampled target segmentation mask and the real target segmentation mask; the target segmentation model is trained based on the first target segmentation loss to obtain a trained target segmentation model. Through this application embodiment, the predicted target segmentation mask output by the target segmentation model can be upsampled to the same resolution as the real target segmentation mask before the target segmentation loss is calculated with the real target segmentation mask. This avoids the destruction of fine edge structures due to downsampling, thereby effectively improving the segmentation effect of the trained target segmentation model in fine structure regions. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart of one embodiment of a target segmentation model training method in this application. Figure 2 This is a flowchart illustrating the first phase of the training process; Figure 3 Example image of the target segmentation result; Figure 4 Example diagrams comparing the target segmentation results of existing technologies and embodiments of this application; Figure 5 This is a structural diagram of one embodiment of a target segmentation model training device according to the present application. Figure 6 This is a schematic block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0026] To make the inventive objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0028] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0029] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0030] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [the described condition or event] is detected," or "in response to detection of [the described condition or event]."

[0031] Furthermore, in the description of this application, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0032] With the rapid advancement of embodied intelligence technology and deep learning visual perception algorithms, humanoid robots are playing an increasingly important role in tasks such as handling, stacking, sorting, and warehousing logistics. To achieve stable grasping and precise manipulation in complex environments, robots must possess high-precision perception and segmentation capabilities for boxes (such as bins and cartons). Through accurate contour extraction and region separation of the target box using segmentation algorithms, the robot can further perform key operations such as grasping and positioning, posture estimation, and obstacle avoidance, thereby ensuring the reliability and safety of the overall handling task.

[0033] In practical applications, due to limitations in computing resources and real-time requirements of humanoid robot end-devices, lightweight target segmentation models have become the mainstream deployment choice. These target segmentation models extract features and perform multi-scale predictions on the input image, outputting bounding boxes (bboxes) and corresponding target segmentation masks to describe the spatial region of the box in the image.

[0034] However, in box segmentation tasks, especially for fine edge structures on the top surface of the box (such as box borders, narrow edge strips, etc.), since they only occupy a small number of pixels in the original image, they are prone to edge breakage or structure disappearance when performing mask downsampling during the training phase, resulting in poor segmentation performance of the trained target segmentation model in fine structure regions.

[0035] In view of this, embodiments of this application provide a method, apparatus, computer-readable storage medium, and electronic device for training a target segmentation model, to solve the problem of poor segmentation performance in fine-structure regions in the prior art. In this embodiment, the predicted target segmentation mask output by the target segmentation model can be upsampled to the same resolution as the real target segmentation mask before calculating the target segmentation loss with the real target segmentation mask. This avoids the destruction of fine edge structures due to downsampling, thereby effectively improving the segmentation performance of the trained target segmentation model in fine-structure regions.

[0036] The execution subject of this application embodiment can be an electronic device, including but not limited to mobile phones, tablets, desktop computers, laptops, PDAs, robot vacuums, robots, and servers. It should be noted that the electronic device for model training and the electronic device for object segmentation can be the same electronic device or different electronic devices. That is, the model can be trained in one electronic device, and then the trained model can be deployed to another electronic device for object segmentation.

[0037] Please see Figure 1 One embodiment of a target segmentation model training method in this application may include: Step S101: Input the sample image into the target segmentation model and obtain the first predicted target segmentation mask of the sample image output by the target segmentation model.

[0038] The target segmentation model can be flexibly configured according to actual conditions, and may include, but is not limited to, YOLO, EfficientDet, RetinaNet, or other models. Considering the limited computing resources and real-time response requirements of humanoid robots in actual deployment environments, in one specific implementation of this application, the lightweight YOLOv8-S-Seg model can be selected as the target segmentation model. By fully training on a multi-scenario training dataset covering boxes such as tote boxes and cartons, the model is ensured to have high accuracy and high robustness in box detection and segmentation tasks.

[0039] For each sample image in the training dataset, the sample image can be input into the target segmentation model, and the predicted target segmentation mask of the sample image output by the target segmentation model can be obtained. For ease of distinction, this is referred to as the first predicted target segmentation mask. The resolution of the first predicted target segmentation mask is smaller than the resolution of the ground truth (GT) target segmentation mask of the sample image, and the resolution of the ground truth target segmentation mask is the same as the resolution of the sample image.

[0040] Step S102: Upsample the first predicted target segmentation mask to obtain the upsampled target segmentation mask of the sample image.

[0041] In one specific implementation of this application, a bilinear interpolation algorithm can be used to upsample the first predicted target segmentation mask based on a preset upsampling factor, thereby obtaining an upsampled target segmentation mask of the sample image.

[0042] The upsampling factor is the resolution ratio between the ground truth segmentation mask and the first predicted segmentation mask. When using the YOLOv8-S-Seg model, the upsampling factor is 4, meaning the resolution of the ground truth segmentation mask is four times the resolution of the first predicted segmentation mask.

[0043] After upsampling, the resolution of the upsampled target segmentation mask is equal to the resolution of the real target segmentation mask, and both are consistent with the resolution of the sample image.

[0044] Step S103: Determine the first target segmentation loss of the target segmentation model based on the upsampled target segmentation mask and the real target segmentation mask.

[0045] In this embodiment, the original resolution real target segmentation mask can be retained and used as a supervision target, so that the upsampled target segmentation mask and the real target segmentation mask are aligned in the same resolution space.

[0046] Based on this, the binary cross entropy (BCE) loss between the upsampled target segmentation mask and the real target segmentation mask can be determined, and this binary cross entropy loss can be used as the first target segmentation loss of the target segmentation model. As shown in the following equation: hr_loss = BCE(pred_mask_up, gt_mask) Where pred_mask_up is the upsampled target segmentation mask, gt_mask is the real target segmentation mask, BCE is the binary cross-entropy loss calculation function, and hr_loss is the first target segmentation loss of the target segmentation model.

[0047] Step S104: Train the target segmentation model based on the first target segmentation loss to obtain the trained target segmentation model.

[0048] By calculating the target segmentation loss in the original resolution space, the fine edge region of the box top surface can participate in the optimization in a complete and continuous form, avoiding the edge breakage problem caused by downsampling, thereby effectively improving the segmentation effect of the trained target segmentation model in fine structure regions.

[0049] It should be noted that target segmentation models generally perform classification prediction, bounding box regression prediction, and segmentation mask prediction simultaneously. The total training loss of the target segmentation model is obtained by summing the training losses of these three branches. The embodiments of this application improve the calculation method of the training loss for segmentation mask prediction. For details on classification prediction and bounding box regression prediction, please refer to the prior art, which will not be repeated in this embodiment.

[0050] Considering that supervising segmentation loss at the original resolution would increase training time, a phased training strategy can be adopted to balance training efficiency and segmentation accuracy: Phase 1: Training the model using downsampling resolution supervision with the initial default model settings to quickly learn the overall structural features of the box; Phase 2: After completing Phase 1 training, training the model again using original resolution supervision to correct and optimize the fine edge regions of the prediction mask, allowing the model to focus on correcting the breakage problem of the box's top edge structure. The training epochs in Phase 1 can be longer than those in Phase 2. For example, 200 epochs can be trained in Phase 1, and at least one epoch in Phase 2.

[0051] Please see Figure 2 The first phase of the training process may include the following steps: Step S201: Input the sample image into the initial target segmentation model and obtain the second predicted target segmentation mask of the sample image output by the initial target segmentation model.

[0052] The initial target segmentation model can be flexibly set according to the actual situation, and may include, but is not limited to, YOLO, EfficientDet, RetinaNet, or other models. Considering the limited computing resources and real-time response requirements of humanoid robots in actual deployment environments, in one specific implementation of this application embodiment, a lightweight YOLOv8-S-Seg model can be selected as the initial target segmentation model. By fully training on a multi-scenario training dataset covering boxes such as tote boxes and cartons, the model is ensured to have high accuracy and high robustness in box detection and segmentation tasks.

[0053] For each sample image in the training dataset, the sample image can be input into the initial target segmentation model, and the predicted target segmentation mask of the sample image output by the initial target segmentation model can be obtained. For ease of distinction, this is referred to as the second predicted target segmentation mask. The resolution of the second predicted target segmentation mask is smaller than the resolution of the true target segmentation mask of the sample image.

[0054] Step S202: Perform downsampling processing on the real target segmentation mask to obtain the downsampled target segmentation mask of the sample image.

[0055] In one specific implementation of this application, a bilinear interpolation algorithm can be used to downsample the real target segmentation mask based on a preset downsampling ratio, thereby obtaining a downsampled target segmentation mask of the sample image.

[0056] The downsampling factor is the resolution ratio between the ground truth segmentation mask and the second predicted segmentation mask. When using the YOLOv8-S-Seg model, the downsampling factor is 4, meaning the resolution of the ground truth segmentation mask is four times the resolution of the second predicted segmentation mask.

[0057] After downsampling, the resolution of the downsampled target segmentation mask is equal to the resolution of the second predicted target segmentation mask.

[0058] Step S203: Determine the second target segmentation loss of the initial target segmentation model based on the second predicted target segmentation mask and the downsampled target segmentation mask.

[0059] In this embodiment, the downsampled target segmentation mask can be used as a supervision target, so that the second predicted target segmentation mask and the downsampled target segmentation mask are aligned in the same resolution space.

[0060] Based on this, the binary cross-entropy loss between the second predicted target segmentation mask and the downsampled target segmentation mask can be determined, and this binary cross-entropy loss can be used as the second target segmentation loss of the initial target segmentation model. As shown in the following equation: ori_loss = BCE(pred_mask_ds, gt_mask_ds) Where pred_mask_ds is the second predicted target segmentation mask, gt_mask_ds is the downsampled target segmentation mask, and ori_loss is the second target segmentation loss of the target segmentation model.

[0061] Step S204: Train the initial target segmentation model based on the second target segmentation loss to obtain the target segmentation model.

[0062] After completing the first stage of training through steps S201 to S204, the second stage of training can continue following steps S101 to S104. It is important to note that in this case, the target segmentation model in step S101 refers to the target segmentation model obtained in the first stage of training; that is, the second stage of training is based on the results of the first stage. After completing the second stage of training, the trained target segmentation model is finally obtained.

[0063] After obtaining the trained target segmentation model, it can be used for actual target segmentation. Specifically, the image to be segmented can be acquired, and the trained target segmentation model can be used to perform target segmentation processing on the image to be segmented, thereby obtaining the target segmentation result of the image to be segmented. Figure 3 The image shown is an example of the target segmentation result. The target segmentation result may include, but is not limited to, box category (such as box_top, box_front, etc.), confidence score, detection box position, and corresponding target segmentation mask.

[0064] Figure 4 The diagram shows a comparison of the target segmentation results of the prior art and the embodiments of this application. The left diagram shows the target segmentation result of the prior art, and the right diagram shows the target segmentation result of the embodiments of this application. It can be seen that the fine edge mask in the left diagram has obvious breaks, while the optimized mask edge in the right diagram remains continuous and complete. That is, the target segmentation result of the embodiments of this application is more accurate.

[0065] In summary, through the embodiments of this application, the predicted target segmentation mask output by the target segmentation model can be upsampled to the same resolution as the real target segmentation mask, and then the target segmentation loss can be calculated by comparing it with the real target segmentation mask. This can avoid the damage to fine edge structures due to downsampling, thereby effectively improving the segmentation effect of the trained target segmentation model in fine structure regions.

[0066] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0067] Corresponding to the target segmentation model training method described in the above embodiments, Figure 5 This diagram illustrates a structural diagram of an embodiment of a target segmentation model training device provided in this application.

[0068] In this embodiment, a target segmentation model training device may include: The first mask acquisition module 501 is used to input a sample image into a target segmentation model and acquire a first predicted target segmentation mask of the sample image output by the target segmentation model; wherein the resolution of the first predicted target segmentation mask is smaller than the resolution of the real target segmentation mask of the sample image; The mask upsampling processing module 502 is used to perform upsampling processing on the first predicted target segmentation mask to obtain an upsampled target segmentation mask of the sample image; wherein, the resolution of the upsampled target segmentation mask is equal to the resolution of the real target segmentation mask; The first loss determination module 503 is used to determine the first target segmentation loss of the target segmentation model based on the upsampled target segmentation mask and the real target segmentation mask; The first model training module 504 is used to train the target segmentation model based on the first target segmentation loss to obtain the trained target segmentation model.

[0069] In one specific implementation of this application, the mask upsampling processing module can be specifically used to: perform upsampling processing on the first predicted target segmentation mask using a bilinear interpolation algorithm based on a preset upsampling factor to obtain the upsampled target segmentation mask; wherein, the upsampling factor is the resolution factor between the real target segmentation mask and the first predicted target segmentation mask.

[0070] In one specific implementation of this application, the first loss determination module may be specifically used to: determine the binary cross-entropy loss between the upsampled target segmentation mask and the real target segmentation mask; and determine the binary cross-entropy loss as the first target segmentation loss of the target segmentation model.

[0071] In one specific implementation of this application embodiment, the target segmentation model training device may further include: The second mask acquisition module is used to input the sample image into the initial target segmentation model and acquire the second predicted target segmentation mask of the sample image output by the initial target segmentation model; wherein the resolution of the second predicted target segmentation mask is smaller than the resolution of the real target segmentation mask; A mask downsampling processing module is used to downsample the real target segmentation mask to obtain a downsampled target segmentation mask of the sample image; wherein the resolution of the downsampled target segmentation mask is equal to the resolution of the second predicted target segmentation mask; The second loss determination module is used to determine the second target segmentation loss of the initial target segmentation model based on the second predicted target segmentation mask and the downsampled target segmentation mask. The second model training module is used to train the initial target segmentation model based on the second target segmentation loss to obtain the target segmentation model.

[0072] In one specific implementation of this application, the mask downsampling processing module can be specifically used to: perform downsampling processing on the real target segmentation mask using a bilinear interpolation algorithm based on a preset downsampling factor to obtain the downsampled target segmentation mask; wherein, the downsampling factor is the resolution factor between the real target segmentation mask and the second predicted target segmentation mask.

[0073] In one specific implementation of this application, the second loss determination module may be specifically used to: determine the binary cross-entropy loss between the second predicted target segmentation mask and the downsampled target segmentation mask; and determine the binary cross-entropy loss as the second target segmentation loss of the initial target segmentation model.

[0074] In one specific implementation of this application embodiment, the target segmentation model training device may further include: The image to be segmented acquisition module is used to acquire the image to be segmented; The target segmentation processing module is used to perform target segmentation processing on the image to be segmented using the trained target segmentation model, so as to obtain the target segmentation result of the image to be segmented.

[0075] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0076] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0077] Figure 6 A schematic block diagram of an electronic device provided in an embodiment of this application is shown. For ease of explanation, only the parts related to the embodiment of this application are shown.

[0078] like Figure 6 As shown, the electronic device 6 in this embodiment includes: a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and executable on the processor 60. When the processor 60 executes the computer program 62, it implements the steps in the various target segmentation model training method embodiments described above, for example... Figure 1 Steps S101 to S104 are shown. Alternatively, when the processor 60 executes the computer program 62, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 5 The functions of modules 501 to 504 are shown.

[0079] For example, the computer program 62 may be divided into one or more modules / units, which are stored in the memory 61 and executed by the processor 60 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 62 in the electronic device 6.

[0080] The electronic device 6 may include, but is not limited to, computing devices such as mobile phones, tablets, desktop computers, laptops, handheld computers, robots, and servers. Those skilled in the art will understand that... Figure 6 This is merely an example of electronic device 6 and does not constitute a limitation on electronic device 6. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device 6 may also include input / output devices, network access devices, buses, etc.

[0081] The processor 60 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0082] The memory 61 can be an internal storage unit of the electronic device 6, such as a hard disk or memory. The memory 61 can also be an external storage device of the electronic device 6, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 6. Furthermore, the memory 61 can include both internal and external storage units of the electronic device 6. The memory 61 is used to store the computer program and other programs and data required by the electronic device 6. The memory 61 can also be used to temporarily store data that has been output or will be output.

[0083] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0084] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0085] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0086] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0087] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0088] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0089] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.

[0090] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for training a target segmentation model, characterized in that, include: A sample image is input into a target segmentation model, and a first predicted target segmentation mask of the sample image is obtained from the output of the target segmentation model; wherein the resolution of the first predicted target segmentation mask is smaller than the resolution of the true target segmentation mask of the sample image; The first predicted target segmentation mask is upsampled to obtain an upsampled target segmentation mask of the sample image; wherein the resolution of the upsampled target segmentation mask is equal to the resolution of the real target segmentation mask; Based on the upsampled target segmentation mask and the real target segmentation mask, the first target segmentation loss of the target segmentation model is determined; The target segmentation model is trained based on the first target segmentation loss to obtain the trained target segmentation model.

2. The target segmentation model training method according to claim 1, characterized in that, The upsampling process of the first predicted target segmentation mask to obtain the upsampled target segmentation mask of the sample image includes: Based on a preset upsampling factor, the first predicted target segmentation mask is upsampled using a bilinear interpolation algorithm to obtain the upsampled target segmentation mask. Wherein, the upsampling factor is the resolution ratio between the real target segmentation mask and the first predicted target segmentation mask.

3. The target segmentation model training method according to claim 1, characterized in that, The step of determining the first target segmentation loss of the target segmentation model based on the upsampled target segmentation mask and the real target segmentation mask includes: Determine the binary cross-entropy loss between the upsampled target segmentation mask and the real target segmentation mask; The binary cross-entropy loss is determined as the first target segmentation loss of the target segmentation model.

4. The target segmentation model training method according to claim 1, characterized in that, Before inputting the sample image into the target segmentation model and obtaining the first predicted target segmentation mask of the sample image output by the target segmentation model, the method further includes: The sample image is input into an initial target segmentation model, and a second predicted target segmentation mask of the sample image is obtained from the output of the initial target segmentation model; wherein the resolution of the second predicted target segmentation mask is smaller than the resolution of the real target segmentation mask; The real target segmentation mask is downsampled to obtain a downsampled target segmentation mask of the sample image; wherein the resolution of the downsampled target segmentation mask is equal to the resolution of the second predicted target segmentation mask; Based on the second predicted target segmentation mask and the downsampled target segmentation mask, determine the second target segmentation loss of the initial target segmentation model; The initial target segmentation model is trained based on the second target segmentation loss to obtain the target segmentation model.

5. The target segmentation model training method according to claim 4, characterized in that, The step of downsampling the real target segmentation mask to obtain the downsampled target segmentation mask of the sample image includes: Based on a preset downsampling factor, a bilinear interpolation algorithm is used to downsample the real target segmentation mask to obtain the downsampled target segmentation mask. Wherein, the downsampling factor is the resolution ratio between the real target segmentation mask and the second predicted target segmentation mask.

6. The target segmentation model training method according to claim 4, characterized in that, The step of determining the second target segmentation loss of the initial target segmentation model based on the second predicted target segmentation mask and the downsampled target segmentation mask includes: Determine the binary cross-entropy loss between the second predicted target segmentation mask and the downsampled target segmentation mask; The binary cross-entropy loss is determined as the second target segmentation loss of the initial target segmentation model.

7. The target segmentation model training method according to any one of claims 1 to 6, characterized in that, After training the target segmentation model based on the first target segmentation loss to obtain the trained target segmentation model, the method further includes: Obtain the image to be segmented; The trained target segmentation model is used to perform target segmentation processing on the image to be segmented, and the target segmentation result of the image to be segmented is obtained.

8. A target segmentation model training device, characterized in that, include: The first mask acquisition module is used to input a sample image into a target segmentation model and acquire a first predicted target segmentation mask of the sample image output by the target segmentation model; wherein the resolution of the first predicted target segmentation mask is smaller than the resolution of the real target segmentation mask of the sample image; A mask upsampling processing module is used to perform upsampling processing on the first predicted target segmentation mask to obtain an upsampled target segmentation mask of the sample image; wherein, the resolution of the upsampled target segmentation mask is equal to the resolution of the real target segmentation mask; The first loss determination module is used to determine the first target segmentation loss of the target segmentation model based on the upsampled target segmentation mask and the real target segmentation mask. The first model training module is used to train the target segmentation model based on the first target segmentation loss to obtain the trained target segmentation model.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the target segmentation model training method as described in any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the target segmentation model training method as described in any one of claims 1 to 7.

Images