Image segmentation model training method and related equipment

Abstract

The disclosed embodiments provide a method for training an image segmentation model and related equipment. The method includes: obtaining an image to be segmented and an interactive representation image; determining a current segmentation mode in a foreground segmentation mode and an interactive segmentation mode based on a random selection method, wherein the foreground segmentation mode is used to perform stage training on the image segmentation model using the image to be segmented, and the interactive segmentation mode is used to perform stage training on the image segmentation model using the image to be segmented and the interactive representation image; performing stage training on the image segmentation model using the current segmentation mode; upon completion of the stage training, updating the current segmentation mode based on a random selection method, so as to train the image segmentation model using the updated current segmentation mode until the training end condition is met. This method can save memory space and avoid the problem of lag caused by the trained image segmentation model during application; at the same time, it can improve the generalization and robustness of the two tasks.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a training method for an image segmentation model, an image segmentation method, a training device for an image segmentation model, an image segmentation device, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the rapid development of computer vision technology, image object segmentation technology, as an important computer vision task, has many applications in image retrieval, image editing, and film and television production.

[0003] In related technologies, foreground segmentation or interactive segmentation is used alone for image target segmentation. Foreground segmentation and interactive segmentation are tested and deployed as two independent tasks. This method ignores the commonalities between the two tasks. In business deployment, their respective model files need to be deployed separately. The two tasks require a large amount of memory space, which can easily cause problems such as lag or memory explosion, affecting the actual product experience.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute prior art known to ordinary technicians in the field. Summary of the Invention

[0005] The embodiments of the present disclosure provide a training method for an image segmentation model, an image segmentation method, a training device for an image segmentation model, an image segmentation device, an electronic device, and a computer-readable storage medium. The method can save memory space and avoid the problem of lag caused by the trained image segmentation model during application; at the same time, it can improve the generalization and robustness of the two tasks.

[0006] An embodiment of the present disclosure provides a method for training an image segmentation model, the method comprising: acquiring an image to be segmented and an interactive representation image; determining a current segmentation mode in a foreground segmentation mode and an interactive segmentation mode based on a random selection method, wherein the foreground segmentation mode is used to perform stage training on the image segmentation model using the image to be segmented, and the interactive segmentation mode is used to perform stage training on the image segmentation model using the image to be segmented and the interactive representation image; performing stage training on the image segmentation model using the current segmentation mode; and upon completion of the stage training, updating the current segmentation mode based on the random selection method, so as to train the image segmentation model using the updated current segmentation mode until a training end condition is met.

[0007] In some exemplary embodiments of the present disclosure, when the current segmentation mode is the foreground segmentation mode, the image segmentation model is trained in stages using the current segmentation mode, including: obtaining a preset mask image, a preset positive interaction representation image, and a preset negative interaction representation image, wherein the pixel values ​​of the preset mask image, the preset positive interaction representation image, and the preset negative interaction representation image are all specified pixel values; merging the image to be segmented, the preset mask image, the preset positive interaction representation image, and the preset negative interaction representation image to obtain a first input image; and inputting the first input image into the image segmentation model to perform stage training on the image segmentation model.

[0008] In some exemplary embodiments of the present disclosure, the image segmentation model includes an encoding layer and a decoding layer; inputting the first input image into the image segmentation model and performing stage training on the image segmentation model includes: inputting the first input image into the encoding layer for encoding processing to obtain a first feature vector; inputting the first feature vector into the decoding layer for decoding processing to obtain a first predicted mask image; and performing stage training on the image segmentation model based on the first predicted mask image.

[0009] In some exemplary embodiments of the present disclosure, when the current segmentation mode is the interactive segmentation mode, the interactive representation image includes a first mask image, a first positive interaction representation image, and a first negative interaction representation image; the use of the current segmentation mode to perform stage training on the image segmentation model includes: merging the image to be segmented, the first mask image, the first positive interaction representation image, and the first negative interaction representation image to obtain a second input image; inputting the second input image into the image segmentation model to perform stage training on the image segmentation model.

[0010] In some exemplary embodiments of the present disclosure, the interactive representation image also includes a second positive interaction representation image and a second negative interaction representation image; inputting the second input image into the image segmentation model and performing stage training on the image segmentation model includes: inputting the second input image into the image segmentation model to obtain a second mask image; merging the image to be segmented, the second mask image, the second positive interaction representation image and the second negative interaction representation image to obtain a third input image; inputting the third input image into the image segmentation model and performing stage training on the image segmentation model.

[0011] In some exemplary embodiments of the present disclosure, the image segmentation model includes an encoding layer and a decoding layer; the second input image is input into the image segmentation model, and the image segmentation model is subjected to stage training, including: inputting the second input image into the encoding layer for encoding processing to obtain a second feature vector; inputting the second feature vector into the decoding layer for decoding processing to obtain a second predicted mask image; and the image segmentation model is subjected to stage training according to the second predicted mask image.

[0012] In some exemplary embodiments of the present disclosure, the random selection method is to randomly select between the foreground segmentation mode and the interactive segmentation mode with a preset probability.

[0013] An embodiment of the present disclosure provides an image segmentation method, comprising: acquiring an image to be segmented and an interactive representation image; when the segmentation mode is a foreground segmentation mode, inputting the image to be segmented into an image segmentation model trained according to any of the above methods to obtain a first target mask image; when the segmentation mode is an interactive segmentation mode, inputting the image to be segmented and the interactive representation image into the image segmentation model trained according to any of the above methods to obtain a second target mask image.

[0014] An embodiment of the present disclosure provides a training device for an image segmentation model, comprising: an acquisition module, configured to acquire an image to be segmented and an interactive representation image; a selection module, configured to determine a current segmentation mode in a foreground segmentation mode and an interactive segmentation mode based on a random selection method, wherein the foreground segmentation mode is used to perform stage training on the image segmentation model using the image to be segmented, and the interactive segmentation mode is used to perform stage training on the image segmentation model using the image to be segmented and the interactive representation image; a training module, configured to perform stage training on the image segmentation model using the current segmentation mode; and an update module, configured to update the current segmentation mode based on the random selection method when the stage training is completed, so as to train the image segmentation model using the updated current segmentation mode until a training end condition is met.

[0015] An embodiment of the present disclosure provides an image segmentation device, comprising: an acquisition module, configured to acquire an image to be segmented and an interactive representation image; the acquisition module, configured to, when the segmentation mode is a foreground segmentation mode, input the image to be segmented into an image segmentation model trained according to any of the above methods, to obtain a first target mask image; the acquisition module is further configured to, when the segmentation mode is an interactive segmentation mode, input the image to be segmented and the interactive representation image into the image segmentation model trained according to any of the above methods, to obtain a second target mask image.

[0016] An embodiment of the present disclosure provides an electronic device, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the executable instructions to implement a training method for an image segmentation model as described above or an image segmentation method as described above.

[0017] An embodiment of the present disclosure provides a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute any of the above-mentioned image segmentation model training methods or the above-mentioned image segmentation methods.

[0018] An embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned image segmentation model training methods or the above-mentioned image segmentation methods.

[0019] The training method of the image segmentation model provided by the embodiment of the present disclosure, during the model training process, determines the current segmentation mode in the foreground segmentation mode and the interactive segmentation mode based on a random selection method, uses the current segmentation mode to perform stage training on the image segmentation model, and after the stage training of the image segmentation model using the current segmentation mode is completed, randomly selects a segmentation mode in the foreground segmentation mode and the interactive segmentation mode again based on the random selection method, updates the current segmentation mode, and uses the updated current segmentation mode to perform stage training on the image segmentation model, thereby achieving the sharing of one image segmentation model by the two tasks of foreground segmentation and interactive segmentation. On the one hand, it can save memory space and computer resources, thereby avoiding the problem that the trained image segmentation model is prone to lag when applied; on the other hand, training the network in a multi-task manner can improve the generalization and robustness of the two tasks.

[0020] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0022] Figure 1 A schematic diagram of an exemplary system architecture of an image segmentation model training method or an image segmentation method to which the embodiments of the present disclosure can be applied is shown.

[0023] Figure 2 The figure is a flowchart of a method for training an image segmentation model according to an exemplary embodiment.

[0024] Figure 3 is a schematic diagram of training an image segmentation model according to an exemplary embodiment.

[0025] Figure 4 The figure is a flowchart of another method for training an image segmentation model according to an exemplary embodiment.

[0026] Figure 5 The figure is a flowchart of another method for training an image segmentation model according to an exemplary embodiment.

[0027] Figure 6 The figure is a flowchart of another method for training an image segmentation model according to an exemplary embodiment.

[0028] Figure 7 The figure is a flowchart of an image segmentation method according to an exemplary embodiment.

[0029] Figure 8 The figure is a block diagram of a training device for an image segmentation model according to an exemplary embodiment.

[0030] Figure 9 The figure is a block diagram of an image segmentation apparatus according to an exemplary embodiment.

[0031] Figure 10 It is a schematic structural diagram showing an electronic device suitable for implementing the exemplary embodiments of the present disclosure according to an exemplary embodiment. DETAILED DESCRIPTION

[0032] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be embodied in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art. Like reference numerals in the drawings represent like or similar parts, and thus repetitive description thereof will be omitted.

[0033] The features, structures or characteristics described in the present disclosure may be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0034] The accompanying drawings are merely schematic illustrations of the present disclosure. Identical reference numerals in the drawings denote identical or similar components, and thus their repeated descriptions will be omitted. Some of the block diagrams shown in the accompanying drawings do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in at least one hardware module or integrated circuit, or in different networks and / or processor devices and / or microcontroller devices.

[0035] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all content and steps, nor must they be executed in the order described. For example, some steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0036] In this specification, the terms "a", "an", "the", "said" and "at least one" are used to indicate the presence of at least one element / component / etc.; the terms "comprising", "including" and "having" are used to express open-ended inclusion and mean that additional elements / components / etc. may exist in addition to the listed elements / components / etc.; the terms "first", "second" and "third" etc. are used only as labels and are not intended to limit the quantity of their objects.

[0037] Figure 1 A schematic diagram of an exemplary system architecture of an image segmentation model training method or an image segmentation method to which the embodiments of the present disclosure can be applied is shown.

[0038] like Figure 1 As shown, the system architecture may include a server 101, a network 102, a terminal device 103, a terminal device 104, and a terminal device 105. The network 102 is used as a medium for providing a communication link between the terminal device 103, the terminal device 104, or the terminal device 105 and the server 101. The network 102 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0039] Server 101 may be a server that provides various services, such as a background management server that supports devices operated by users using terminal device 103, terminal device 104, or terminal device 105. The background management server may analyze and process received data such as requests, and feed back the processing results to terminal device 103, terminal device 104, or terminal device 105.

[0040] Terminal device 103, terminal device 104 and terminal device 105 can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a wearable smart device, a virtual reality device, an augmented reality device, etc., but are not limited thereto.

[0041] In an embodiment of the present disclosure, the server 101 may: obtain an image to be segmented and an interactive representation image; determine a current segmentation mode in a foreground segmentation mode and an interactive segmentation mode based on a random selection method, wherein the foreground segmentation mode is used to perform stage training on an image segmentation model using the image to be segmented, and the interactive segmentation mode is used to perform stage training on an image segmentation model using the image to be segmented and the interactive representation image; perform stage training on the image segmentation model using the current segmentation mode; and when the stage training is completed, update the current segmentation mode based on a random selection method to train the image segmentation model using the updated current segmentation mode until the training end condition is met.

[0042] In the embodiment of the present disclosure, the server 101 can obtain the image to be segmented and the interactive representation image from the terminal device; when the segmentation mode is the foreground segmentation mode, the image to be segmented is input into the image segmentation model trained according to the above method to obtain a first target mask image; when the segmentation mode is the interactive segmentation mode, the image to be segmented and the interactive representation image are input into the image segmentation model trained according to the above method to obtain a second target mask image; the server 101 can return the obtained first target mask image or second target mask image to the terminal device.

[0043] It should be understood that Figure 1 The number of terminal devices 103, terminal devices 104, terminal devices 105, networks 102 and servers 101 is merely illustrative. Server 101 may be a physical server, a server cluster consisting of multiple servers, or a cloud server. Depending on actual needs, it may have any number of terminal devices, networks and servers.

[0044] Below, the steps of the training method of the image segmentation model in the exemplary embodiment of the present disclosure will be described in more detail with reference to the accompanying drawings and embodiments. The method provided in the exemplary embodiment of the present disclosure can be executed by any electronic device, such as the above-mentioned Figure 1 The server and / or terminal device in the present disclosure is not limited to this.

[0045] Figure 2 The figure is a flowchart of a method for training an image segmentation model according to an exemplary embodiment.

[0046] like Figure 2 As shown, the method provided by the embodiment of the present disclosure may include the following steps.

[0047] In step S210 , an image to be segmented and an interactive representation image are obtained.

[0048] In the embodiments of the present disclosure, the image to be segmented refers to an image that requires image instance segmentation to obtain a target mask. The image to be segmented may include one or at least two image instances. Specifically, the server may obtain the image to be segmented from a terminal device; the server may also obtain the image to be segmented from a service server; the server may also obtain the image to be segmented from the internet; or the server may directly obtain the image to be segmented from a database, although this disclosure does not limit this.

[0049] In the embodiment of the present disclosure, an interactive representation image refers to an image generated based on the user's interactive behavior, which is used to represent the user's interactive behavior; the user's interactive behavior may include but is not limited to clicking and scribbles, and each interaction mode can be divided into positive interaction and negative interaction (the corresponding interactive representation images can be called positive interaction representation images and negative interaction representation images), positive interaction refers to scribbles (or clicks) performed in the user's area of ​​interest, and negative interaction refers to scribbles (or clicks) performed in the background mis-segmentation area.

[0050] In the embodiments of the present disclosure, the interactive representation image can be generated directly based on user interaction behavior or automatically generated by a machine. Specifically, the server can obtain the interactive representation image from a terminal device; the server can also obtain the interactive representation image from a business server; the server can also obtain the interactive representation image from the internet; the server can also obtain the interactive representation image directly from a database, and this disclosure is not limited to this.

[0051] In the embodiment of the present disclosure, the image segmentation model can be trained respectively through the foreground segmentation mode and the interactive segmentation mode. In the foreground segmentation mode, the training set can include the image to be segmented (the foreground segmentation mode can be considered to be non-interactive, and the corresponding interactive representation image can be a preset image); in the interactive segmentation mode, the training set can include the image to be segmented and the interactive representation image.

[0052] In step S220 , a current segmentation mode is determined between the foreground segmentation mode and the interactive segmentation mode based on a random selection method.

[0053] Among them, the foreground segmentation mode is used to perform stage training on the image segmentation model using the image to be segmented, and the interactive segmentation mode is used to perform stage training on the image segmentation model using the image to be segmented and the interactive representation image.

[0054] Foreground segmentation (also known as foreground object segmentation) uses intelligent algorithms to simulate human visual characteristics and extract salient areas (i.e., object masks) from an image pixel by pixel. Interactive segmentation (also known as interactive object segmentation) distinguishes objects of interest from the background based on user interaction, effectively adjusting the current segmentation results based on user guidance. Both image foreground segmentation and interactive image segmentation are computer vision tasks with diverse applications in image retrieval, visual tracking, image editing, and film and television production.

[0055] In related technologies, foreground segmentation and interactive segmentation are trained, tested, and deployed as two independent tasks; however, in the present application, the current segmentation mode can be randomly selected between the foreground segmentation mode and the interactive segmentation mode, and can be randomly switched between the foreground segmentation mode and the interactive segmentation mode, so that the two tasks of foreground segmentation and interactive segmentation share one image segmentation model, thereby saving computer resources.

[0056] In an embodiment of the present disclosure, during the training process of the image segmentation model, the current segmentation mode can be randomly selected from the foreground segmentation mode and the interactive segmentation mode based on a random selection method, and the current segmentation mode (one of the foreground segmentation mode and the interactive segmentation mode) is used for stage training, and then a mode is randomly selected from the foreground segmentation mode and the interactive segmentation mode based on a random selection method to update the current segmentation mode, and the updated current segmentation mode (one of the foreground segmentation mode and the interactive segmentation mode) is used to continue stage training, and so on, until the model training is completed. Among them, the training end condition of the image segmentation model can be the number of training times, and the number of training times can be set according to actual conditions. For example, the number of training times can be directly set (for example, the number of training times can be set to N, where N is an integer greater than or equal to 2); the training completion condition can also be set (the training completion condition can be, for example, the number of training times corresponding to a certain mode, or the loss function in the image segmentation model converges, or a parameter in the image segmentation model reaches a preset condition, etc.), and the present disclosure does not limit this.

[0057] For example, during the training process of the image segmentation model, the foreground segmentation model is first selected as the current segmentation mode, and the foreground segmentation mode is used to perform stage training on the image segmentation model; after the training of this stage is completed, a segmentation mode is randomly selected to update the current segmentation mode. For example, the interactive segmentation mode is selected to update the current segmentation mode, and the interactive segmentation mode is used to perform stage training on the same image segmentation model; after the training of this stage is completed, a segmentation mode is randomly selected to update the current segmentation mode, and so on. Each stage training randomly selects a segmentation mode from the foreground segmentation mode and the interactive segmentation mode, and the two segmentation modes train the same image segmentation model until the model training is completed.

[0058] In the embodiments of the present disclosure, an image segmentation model is used to segment an image to obtain a target mask. It can be a deep neural network model or other models, which are not limited in this disclosure. Deep neural networks can be applied to image segmentation. The high-level semantic features extracted from deep networks can more accurately distinguish target objects and backgrounds in complex scenes, thereby improving the image segmentation effect.

[0059] In an exemplary embodiment, the random selection method is to perform random selection in the foreground segmentation mode and the interactive segmentation mode with a preset probability.

[0060] The preset probability may be the probability of randomly selecting one of the foreground segmentation mode and the interactive segmentation mode. For example, the preset probability may be the probability of selecting the foreground segmentation mode, or the preset probability may be the probability of selecting the interactive segmentation mode, or the probability of selecting the foreground segmentation mode may be set to the first preset probability and the probability of selecting the interactive segmentation mode may be set to the second preset probability.

[0061] The preset probability can be set according to actual conditions, for example, it can be set to 0.3, 0.5, 0.8, etc., and this disclosure does not limit this.

[0062] In the embodiment of the present disclosure, the preset probability of selecting the foreground segmentation mode (or interactive segmentation mode) may be determined based on the usage frequency of the foreground segmentation mode (or interactive segmentation mode) when applying the image segmentation model.

[0063] For example, when applying the image segmentation model, the foreground segmentation mode is used more frequently. Then, when training the image segmentation model, the preset probability of selecting the foreground segmentation mode for training can be set to a larger value (for example, greater than 0.5), and the foreground segmentation mode is used for training more times. This can make the image segmentation model more accurate when using the foreground segmentation mode for segmentation during application, thereby making the overall accuracy of the image segmentation model higher during application.

[0064] For example, when applying the image segmentation model, the interactive segmentation mode is used more frequently. Then, when training the image segmentation model, the preset probability of selecting the interactive segmentation mode for training can be set to a larger value (for example, greater than 0.5), and the interactive segmentation mode is used for training more times. This can make the image segmentation model more accurate when using the interactive segmentation mode for segmentation during application, thereby making the overall accuracy of the image segmentation model higher when applied.

[0065] For example, when applying an image segmentation model, the frequencies of use of the foreground segmentation mode and the interactive segmentation mode are basically the same. Therefore, when training the image segmentation model, the preset probability of selecting the foreground segmentation mode (or interactive segmentation mode) for training can be set to 0.5, and the number of times the foreground segmentation mode and the interactive segmentation mode are used for training is basically the same, which can make the overall accuracy of the image segmentation model higher when applied.

[0066] In step S230, the image segmentation model is stage-trained using the current segmentation mode.

[0067] In the embodiment of the present disclosure, if in step S220, the foreground segmentation mode is selected as the current segmentation mode, the image segmentation model is trained in stages using the foreground segmentation mode; if in step S220, the interactive segmentation mode is selected as the current segmentation mode, the image segmentation model is trained in stages using the interactive segmentation mode.

[0068] The following describes the stages of training the image segmentation model using the foreground segmentation mode and the interactive segmentation mode.

[0069] In the disclosed embodiment, when the current segmentation mode is the foreground segmentation mode, the image segmentation model is trained in stages using the image to be segmented.

[0070] When the current segmentation mode is the foreground segmentation mode, the image to be segmented and the target mask image of the image to be segmented can be used as training sets, and the image to be segmented can be input into the image segmentation model for training to obtain a predicted mask image; the loss function is determined according to the target mask image and the predicted mask image, and the image segmentation model is trained according to the loss function to complete the training of this stage.

[0071] In the embodiment of the present disclosure, when the current segmentation mode is the interactive segmentation mode, the image segmentation model is trained in stages using the image to be segmented and the interactive representation image.

[0072] When the current segmentation mode is the foreground segmentation mode, the image to be segmented, the interactive representation image and the target mask image of the image to be segmented can be used as training sets, and the image to be segmented and the interactive representation image can be input into the image segmentation model for training to obtain a predicted mask image; the loss function is determined according to the target mask image and the predicted mask image, and the image segmentation model is trained according to the loss function to complete the training of this stage.

[0073] Specifically, the image segmentation model may include an encoding layer (encoder) and a decoding layer (decoder).

[0074] The following combination Figure 3This section explains the training process of the image segmentation model.

[0075] refer to Figure 3 When the current segmentation mode is the foreground segmentation mode, the input data corresponding to the foreground segmentation mode (shown in the dotted box) can be input into the encoding layer 309 for encoding processing to obtain a first eigenvector; the first eigenvector is input into the decoding layer 310 for decoding processing to obtain a first predicted mask image 311; the image segmentation model is trained according to the first predicted mask image 311 and the target mask image.

[0076] When the current segmentation mode is the interactive segmentation mode, the input data corresponding to the interactive segmentation mode (shown in the dotted box) can be input into the encoding layer 309 for encoding processing to obtain a second eigenvector; the second eigenvector is input into the decoding layer 310 for decoding processing to obtain a second predicted mask image 311; and the image segmentation model is trained based on the second predicted mask image 311 and the target mask image.

[0077] Specifically, when the current segmentation mode is the foreground segmentation mode, the input data corresponding to the foreground segmentation mode may include an image to be segmented 301, a preset mask image 302, a preset positive interaction representation image 303, and a preset negative interaction representation image 304, wherein the pixel values ​​of the preset mask image 302, the preset positive interaction representation image 303, and the preset negative interaction representation image 304 are all specified pixel values, that is, when the current segmentation mode is the foreground segmentation mode, it is not necessary to use the interactive representation image as the input image. At this time, these three images can be set to an inactive state (set to an active state when the current segmentation mode is the interactive segmentation mode), that is, they can be set to preset images. Among them, the preset image can be a binary image, and the specified pixel value can be 0 (that is, the pixel point is displayed as black). That is, when the current segmentation mode is the foreground segmentation mode, a completely black image can be used as the preset mask image, the preset positive interaction representation image, and the preset negative interaction representation image.

[0078] Specifically, when the current segmentation mode is the interactive segmentation mode, the input data corresponding to the interactive segmentation mode may include the image to be segmented 305, the first mask image 306, the first positive interaction representation image 307 and the first negative interaction representation image 308, wherein, if this training is the first training of the interactive segmentation mode, the first mask image 306 is the initial mask image; if this training is the second or Nth training of the interactive segmentation, the first mask image 306 is the predicted mask image M obtained after the last interactive segmentation. prevThe first mask image 306 can be a binary image, and its pixel value can be in the range of [0, 1]. The larger the value, the greater the probability of representing the foreground, and the smaller the value, the greater the probability of representing the background. The first positive interaction representation image 307 is the positive representation image generated by the user smearing (or clicking) on ​​the image to be segmented 305 according to the first mask image 306 during this interaction. (The user smears or clicks on the point or area that should belong to the mask but does not belong to the first mask image 306 on the image to be segmented 305). The first positive interaction representation image can be a binary image, and its pixel value range can be [0, 1]. The part smeared by the user has a value of 1, and the other parts have a value of 0. The first negative interaction representation image 308 is a negative representation image generated by the user smearing (or clicking) based on the first mask image 306 during this interaction. (The user paints or clicks on points or areas on the image to be segmented 305 that should not belong to the mask but belong to the first mask image 306). The second orthogonal interactive representation image can be a binary image, and its pixel value can range from [0, 1]. The part painted by the user takes the value of 1, and the other parts take the value of 0.

[0079] The image to be segmented 305, the first mask image 306, the first positive interaction representation image 307 and the first negative interaction representation image 308 are input into the encoding layer 309 and the decoding layer 310 of the image segmentation model for processing to obtain a second mask image; the user then smears or clicks according to the image to be segmented 305 and the second mask image to generate a second positive interaction representation image and a second negative interaction representation image; the image to be segmented 305, the second mask image, the second positive interaction representation image and the second negative interaction representation image are input into the encoding layer 309 and the decoding layer 310 of the image segmentation model for processing, and so on to obtain a predicted mask image (i.e., the above-mentioned second predicted mask image).

[0080] In step S240 , when the stage training is completed, the current segmentation mode is updated based on a random selection method, so as to train the image segmentation model using the updated current segmentation mode until the training end condition is met.

[0081] After the stage training is performed using the foreground segmentation mode or the interactive segmentation model, a mode is randomly selected again in the foreground segmentation mode and the interactive segmentation mode to update the current segmentation mode, and the updated current segmentation mode (one of the foreground segmentation mode and the interactive segmentation mode) is used to continue the stage training, and so on, until the model training is completed.

[0082] In an exemplary embodiment, a random selection may be made between the foreground segmentation mode and the interactive segmentation mode with a preset probability to update the current segmentation mode.

[0083] Random selection is performed again in the foreground segmentation mode and the interactive segmentation mode with a preset probability. The steps of random selection in the foreground segmentation mode and the interactive segmentation mode with a preset probability are similar to the steps of the first random selection in the foreground segmentation mode and the interactive segmentation mode with a preset probability. Please refer to the above text description, and the present disclosure will not repeat them here.

[0084] The following combination Figure 4 This section describes the process of randomly selecting a split pattern twice. Figure 4 In step S410, the image to be segmented and the interactive representation image are obtained; in step S420, the current segmentation mode is randomly selected from the foreground segmentation model and the interactive segmentation mode. For example, if the foreground segmentation mode is selected as the current segmentation mode, step S430 is executed, i.e., the image segmentation model is stage-trained in the foreground segmentation mode; after the image segmentation model is trained in the foreground segmentation mode, in step S450, a segmentation mode is randomly selected again from the foreground segmentation mode and the interactive segmentation mode to update the current segmentation mode. For example, if the interactive segmentation mode is selected as the updated current segmentation mode, step S470 is executed (when the updated current segmentation mode is the interactive segmentation mode, the image segmentation model is stage-trained using the image to be segmented and the interactive representation image), i.e., the image segmentation model is stage-trained in the interactive segmentation mode; after the image segmentation model is trained in the interactive segmentation mode, segmentation modes are continuously randomly selected from the foreground segmentation mode and the interactive segmentation mode in step S280 to update the current segmentation model again, so as to train the image segmentation model using the continuously selected segmentation mode until the model training is completed. In this case, the execution steps of the method are: S410→S420→S430→S450→S470→S480.

[0085] Alternatively, refer to Figure 4In step S420, a segmentation mode is randomly selected as the current segmentation mode. For example, if the interactive segmentation mode is selected as the current segmentation mode, step S440 is executed, i.e., the image segmentation model is stage-trained in the interactive segmentation mode. After the stage-training of the image segmentation model in the interactive segmentation mode is completed, a segmentation mode is randomly selected again in step S450 to update the current segmentation mode. For example, if the foreground segmentation mode is selected as the updated current segmentation mode, step S460 is executed (when the updated current segmentation mode is the foreground segmentation mode, the image segmentation model is stage-trained using the image to be segmented), i.e., the image segmentation model is stage-trained in the foreground mode. After the stage-training of the image segmentation model in the foreground segmentation mode is completed, in step S480, segmentation modes are continuously randomly selected in the foreground segmentation mode and the interactive segmentation mode to train the image segmentation model using the continuously selected segmentation mode until the model training is completed. In this case, the execution steps of the method are: S410→S420→S440→S450→S460→S480.

[0086] Similarly, the execution steps of the method may also be: S410 → S420 → S430 → S450 → S460 → S480, or, S410 → S420 → S440 → S450 → S470 → S480.

[0087] Similarly, the process of randomly selecting a segmentation mode multiple times is similar to the process of randomly selecting a segmentation mode twice, and will not be described in detail in this disclosure.

[0088] The training method of the image segmentation model provided by the embodiment of the present disclosure, during the model training process, determines the current segmentation mode in the foreground segmentation mode and the interactive segmentation mode based on a random selection method, uses the current segmentation mode to perform stage training on the image segmentation model, and after the stage training of the image segmentation model using the current segmentation mode is completed, randomly selects a segmentation mode in the foreground segmentation mode and the interactive segmentation mode again based on the random selection method, updates the current segmentation mode, and uses the updated current segmentation mode to perform stage training on the image segmentation model, thereby achieving the sharing of one image segmentation model by the two tasks of foreground segmentation and interactive segmentation. On the one hand, it can save memory space and computer resources, thereby avoiding the problem that the trained image segmentation model is prone to lag when applied; on the other hand, training the network in a multi-task manner can improve the generalization and robustness of the two tasks.

[0089] In addition, compared with related technologies, in which foreground segmentation and interactive segmentation are trained, tested, and deployed as two independent tasks, the image segmentation model trained by this method can save about half of the memory space during training, testing, and deployment.

[0090] Figure 5 The figure is a flowchart of another method for training an image segmentation model according to an exemplary embodiment. Figure 5 The specific steps of performing stage training on the image segmentation model using the current segmentation mode are shown when the current segmentation mode is the foreground segmentation mode.

[0091] exist Figure 5 In the embodiment, the above Figure 2 Step S230 in the embodiment may further include the following steps.

[0092] In step S231 , a preset mask image, a preset positive interaction representation image, and a preset negative interaction representation image are acquired.

[0093] The pixel values ​​of the preset mask image, the preset positive interaction representation image, and the preset negative interaction representation image are all specified pixel values. That is, when the current segmentation mode is the foreground segmentation mode, the interactive representation image is not required as the input image. In this case, these three images can be set to an inactive state (set to an active state when the current segmentation mode is the interactive segmentation mode), that is, they can be set as preset images. The preset image can be a binary image, and the specified pixel value can be 0 (that is, the pixel point is displayed as black). That is, when the current segmentation mode is the foreground segmentation mode, a completely black image can be used as the preset mask image, the preset positive interaction representation image, and the preset negative interaction representation image.

[0094] In step S232 , the image to be segmented, the preset mask image, the preset positive interaction representation image, and the preset negative interaction representation image are merged to obtain a first input image.

[0095] In the embodiment of the present disclosure, the image to be segmented I RGB ∈R H×W×3 , preset mask image M prev ∈R H×W×1 , preset positive interaction representation image and preset negative interaction representation image Merge (concat) in the channel dimension to obtain the first input image I input ∈R H×W×5 , where the H in the superscript of R represents the height of the image, W represents the width of the image, and the number represents the number of channels.

[0096] In step S233, the first input image is input into the image segmentation model, and the image segmentation model is trained in stages.

[0097] In the embodiment of the present disclosure, a first input image can be input into the image segmentation model to obtain a first predicted mask image; a loss function is determined based on the first predicted mask image and the target mask image (label) of the image to be segmented, and the model parameters of the image segmentation model are adjusted so that the error between the first predicted mask image and the target mask image of the image to be segmented meets the preset conditions, thereby completing the stage training of the image segmentation model.

[0098] In an exemplary embodiment, an image segmentation model includes an encoding layer and a decoding layer; a first input image is input into the image segmentation model, and the image segmentation model is subjected to stage training, including: inputting the first input image into the encoding layer for encoding processing to obtain a first eigenvector; inputting the first eigenvector into the decoding layer for decoding processing to obtain a first predicted mask image; and conducting stage training on the image segmentation model according to the first predicted mask image.

[0099] Combine Figure 3 When the current segmentation mode is the foreground segmentation mode, the image to be segmented 301, the preset mask image 302, the preset positive interaction representation image 303 and the preset negative interaction representation image 304 can be merged to obtain a first input image; the first input image is input to the encoding layer 309 for encoding processing to obtain a first eigenvector; the first eigenvector is input to the decoding layer 310 for decoding processing to obtain a first predicted mask image 311; the image segmentation model is stage-trained according to the first predicted mask image 311 and the target mask image, the loss function is determined according to the first predicted mask image 311 and the target mask image of the image to be segmented, and the model parameters of the image segmentation model are adjusted so that the error between the first predicted mask image 311 and the target mask image of the image to be segmented meets the preset conditions, thereby completing the stage training of the image segmentation model.

[0100] Figure 6 The figure is a flowchart of another method for training an image segmentation model according to an exemplary embodiment. Figure 6 The specific steps of performing stage training on the image segmentation model using the current segmentation mode are shown when the current segmentation mode is the interactive segmentation mode.

[0101] exist Figure 6 In the embodiment, the above Figure 2 Step S230 in the embodiment may further include the following steps.

[0102] In an exemplary embodiment, the interactive representation image includes a first mask image, a first positive interaction representation image, and a first negative interaction representation image. If this training is the first training of the interactive segmentation mode, the first mask image is the initial mask image; if this training is the second or Nth training of the interactive segmentation mode, the first mask image is the predicted mask image obtained after the previous interaction; the first positive interaction representation image is a positive representation image generated by the user smearing (or clicking) on ​​the image to be segmented based on the first mask image during this interaction (the user smears or clicks on points or areas on the image to be segmented that should belong to the mask but do not belong to the first mask image); the first negative interaction representation image is a negative representation image generated by the user smearing (or clicking) on ​​the first mask image during this interaction (the user smears or clicks on points or areas on the image to be segmented that should belong to the mask but do belong to the first mask image).

[0103] In step S234 , the image to be segmented, the first mask image, the first positive interaction representation image, and the first negative interaction representation image are merged to obtain a second input image.

[0104] In the embodiment of the present disclosure, the image to be segmented I RGB ∈R H×W×3 , the first mask image M prev ∈R H×W×1 , the first orthogonal interaction representation image and the first negative interaction representation image Merge (concat) on the channel dimension to obtain the second input image I input ∈R H×W×5 , where the H in the superscript of R represents the height of the image, W represents the width of the image, and the number represents the number of channels.

[0105] In step S235 , the second input image is input into the image segmentation model, and the image segmentation model is trained in stages.

[0106] In the embodiment of the present disclosure, a second input image can be input into the image segmentation model to obtain a second predicted mask image; the loss function is determined based on the second predicted mask image and the target mask image (label) of the image to be segmented, and the model parameters of the image segmentation model are adjusted so that the error between the second predicted mask image and the target mask image of the image to be segmented meets the preset conditions, thereby completing the stage training of the image segmentation model.

[0107] In an exemplary embodiment, the interactive representation image further includes a second positive interactive representation image and a second negative interactive representation image.

[0108] In an exemplary embodiment, inputting the second input image into the image segmentation model and training the image segmentation model may include: inputting the second input image into the image segmentation model to obtain a second mask image; merging the image to be segmented, the second mask image, the second positive interaction representation image, and the second negative interaction representation image to obtain a third input image; inputting the third input image into the image segmentation model to perform stage training on the image segmentation model.

[0109] Specifically, the image to be segmented, the first mask image, the first positive interaction representation image and the first negative interaction representation image are input into the image segmentation model for processing to obtain the second mask image; the user then paints or clicks according to the image to be segmented and the second mask image to generate a second positive interaction representation image and a second negative interaction representation image; the image to be segmented, the second mask image, the second positive interaction representation image and the second negative interaction representation image are merged to obtain a third input image; the third input image is input into the image segmentation model for processing, and so on to obtain a predicted mask image (that is, the above-mentioned second predicted mask image).

[0110] In an exemplary embodiment, the image segmentation model includes an encoding layer and a decoding layer; a second input image is input into the image segmentation model, and the image segmentation model is subjected to stage training, including: inputting the second input image into the encoding layer for encoding processing to obtain a second eigenvector; inputting the second eigenvector into the decoding layer for decoding processing to obtain a second predicted mask image; and conducting stage training on the image segmentation model according to the second predicted mask image.

[0111] Combine Figure 3 When the current segmentation mode is the interactive segmentation mode, the image to be segmented 305, the first mask image 306, the first positive interaction representation image 307 and the first negative interaction representation image 308 are merged to obtain a second input image; the second input image is input into the encoding layer 309 of the image segmentation model for processing to obtain a second eigenvector; the second eigenvector is input into the decoding layer 310 for processing to obtain a second mask image; the loss function is determined according to the second predicted mask image and the target mask image (label) of the image to be segmented, and the model parameters of the image segmentation model are adjusted so that the error between the second predicted mask image and the target mask image of the image to be segmented meets the preset conditions, thereby completing the stage training of the image segmentation model.

[0112] Figure 7 The figure is a flowchart of an image segmentation method according to an exemplary embodiment. Figure 7 The application process of the image segmentation model after the image segmentation model is obtained by training using the method provided in the above embodiment is shown.

[0113] In step S710, an image to be segmented and an interactive representation image are obtained.

[0114] In step S720, when the segmentation mode is the foreground segmentation mode, the image to be segmented is input into the image segmentation model trained according to the method provided in any of the above embodiments to obtain a first target mask image.

[0115] In step S730, when the segmentation mode is the interactive segmentation mode, the image to be segmented and the interactive representation image are input into the image segmentation model trained according to the method provided in any of the above embodiments to obtain a second target mask image.

[0116] In practical applications, a segmentation model can be selected according to actual needs; when the selected segmentation mode is the foreground segmentation mode, the image to be segmented is input into the image segmentation model (the interactive representation image can be set to an inactive state at this time), and the image segmentation model automatically processes the image to be segmented to obtain a first target mask image; when the selected segmentation mode is the interactive segmentation mode, the image to be segmented and the interactive representation image are input into the image segmentation model (the interactive representation image can be set to an active state at this time), and the image segmentation model automatically processes the image to be segmented and the interactive representation image to obtain a second target mask image.

[0117] Therefore, the image segmentation method provided by the embodiment of the present disclosure can use one image segmentation model to handle the two tasks of foreground segmentation and interactive segmentation respectively during the model application process. On the one hand, it can save memory space and computer resources, thereby avoiding the problem of image segmentation model easily causing lag when applied; on the other hand, it can improve the generalization and robustness of the two tasks.

[0118] In addition, compared with related technologies, in which foreground segmentation and interactive segmentation are tested and deployed as two independent tasks, the image segmentation model trained by this method can save about half of the memory space during testing and deployment.

[0119] It should also be understood that the above is merely intended to help those skilled in the art better understand the embodiments of the present disclosure, and is not intended to limit the scope of the embodiments of the present disclosure. Based on the above examples, those skilled in the art can obviously make various equivalent modifications or variations. For example, certain steps in the above method may be unnecessary, or certain new steps may be added. Or any combination of any two or more of the above embodiments. Such modifications, variations, or combinations also fall within the scope of the embodiments of the present disclosure.

[0120] It should also be understood that the above description of the embodiments of the present disclosure focuses on emphasizing the differences between the various embodiments. The same or similar points that are not mentioned can be referenced to each other. For the sake of brevity, they will not be repeated here.

[0121] It should also be understood that the size of the sequence numbers of the above processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present disclosure.

[0122] It should also be understood that in the various embodiments of the present disclosure, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments based on their internal logical relationships.

[0123] The above describes in detail an example of a training method for an image segmentation model provided by the present disclosure. It is understandable that, in order to implement the above functions, the computer device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software driven hardware manner depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present disclosure.

[0124] The following are embodiments of the apparatus disclosed herein, which can be used to implement the method embodiments disclosed herein. For details not disclosed in the apparatus embodiments disclosed herein, please refer to the method embodiments disclosed herein.

[0125] Figure 8 FIG. 1 is a block diagram of a training device for an image segmentation model according to an exemplary embodiment. Figure 8 , the apparatus 800 may include an acquisition module 810 , a selection module 820 , a training module 830 and an update module 840 .

[0126] Among them, the acquisition module 810 is configured to execute acquisition of the image to be segmented and the interactive representation image; the selection module 820 is configured to execute determination of the current segmentation mode in the foreground segmentation mode and the interactive segmentation mode based on a random selection method, the foreground segmentation mode is used to perform stage training on the image segmentation model using the image to be segmented, and the interactive segmentation mode is used to perform stage training on the image segmentation model using the image to be segmented and the interactive representation image; the training module 830 is configured to execute stage training of the image segmentation model using the current segmentation mode; the update module 840 is configured to execute, when the stage training is completed, updating the current segmentation mode based on the random selection method, so as to train the image segmentation model using the updated current segmentation mode until the training end condition is met.

[0127] In some exemplary embodiments of the present disclosure, when the current segmentation mode is the foreground segmentation mode, the training module 830 is further configured to execute acquisition of a preset mask image, a preset positive interaction representation image, and a preset negative interaction representation image, wherein the pixel values ​​of the preset mask image, the preset positive interaction representation image, and the preset negative interaction representation image are all specified pixel values; the image to be segmented, the preset mask image, the preset positive interaction representation image, and the preset negative interaction representation image are merged to obtain a first input image; the first input image is input into the image segmentation model to perform stage training on the image segmentation model.

[0128] In some exemplary embodiments of the present disclosure, the image segmentation model includes an encoding layer and a decoding layer; the training module 830 is also configured to input the first input image into the encoding layer for encoding processing to obtain a first feature vector; input the first feature vector into the decoding layer for decoding processing to obtain a first predicted mask image; and perform stage training on the image segmentation model according to the first predicted mask image.

[0129] In some exemplary embodiments of the present disclosure, when the current segmentation mode is the interactive segmentation mode, the interactive representation image includes a first mask image, a first positive interaction representation image and a first negative interaction representation image; wherein, the training module 830 is further configured to execute merging the image to be segmented, the first mask image, the first positive interaction representation image and the first negative interaction representation image to obtain a second input image; input the second input image into the image segmentation model, and perform stage training on the image segmentation model.

[0130] In some exemplary embodiments of the present disclosure, the interactive representation image also includes a second positive interaction representation image and a second negative interaction representation image; wherein, the training module 830 is further configured to input the second input image into the image segmentation model to obtain a second mask image; merge the image to be segmented, the second mask image, the second positive interaction representation image and the second negative interaction representation image to obtain a third input image; input the third input image into the image segmentation model to perform stage training on the image segmentation model.

[0131] In some exemplary embodiments of the present disclosure, the image segmentation model includes an encoding layer and a decoding layer; wherein the training module 830 is further configured to input the second input image into the encoding layer for encoding processing to obtain a second feature vector; input the second feature vector into the decoding layer for decoding processing to obtain a second predicted mask image; and perform stage training on the image segmentation model according to the second predicted mask image.

[0132] In some exemplary embodiments of the present disclosure, the random selection method is to randomly select between the foreground segmentation mode and the interactive segmentation mode with a preset probability.

[0133] Figure 9 FIG. 1 is a block diagram of an image segmentation apparatus according to an exemplary embodiment. Figure 9 , the device 900 may include an acquisition module 910 and an obtaining module 920.

[0134] Among them, the acquisition module 910 is configured to execute acquisition of the image to be segmented and the interactive representation image; the acquisition module 920 is configured to execute, when the segmentation mode is the foreground segmentation mode, input the image to be segmented into the image segmentation model trained according to the method described in any of the above embodiments, to obtain a first target mask image; the acquisition module 920 is also configured to execute, when the segmentation mode is the interactive segmentation mode, input the image to be segmented and the interactive representation image into the image segmentation model trained according to the method described in any of the above embodiments, to obtain a second target mask image.

[0135] It should be noted that the block diagrams shown in the above figures are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor terminal devices and / or microcontroller terminal devices.

[0136] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0137] Refer to the following Figure 10 1000 according to this embodiment of the present disclosure will be described. Figure 10 The electronic device 1000 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0138] like Figure 10 As shown, electronic device 1000 is implemented as a general-purpose computing device. Components of electronic device 1000 may include, but are not limited to, the aforementioned at least one processing unit 1010, the aforementioned at least one storage unit 1020, a bus 1030 connecting various system components (including storage unit 1020 and processing unit 1010), and a display unit 1040.

[0139] The storage unit stores program codes, which can be executed by the processing unit 1010, so that the processing unit 1010 performs the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processing unit 1010 can perform the following steps: Figure 2 The steps shown in .

[0140] For example, electronic devices can achieve Figure 2 The steps shown.

[0141] The storage unit 1020 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 1021 and / or a cache memory unit 1022 , and may further include a read-only memory unit (ROM) 1023 .

[0142] The storage unit 1020 may also include a program / utility 1024 having a set (at least one) of program modules 1025, such program modules 1025 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0143] Bus 1030 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0144] The electronic device 1000 can also communicate with one or more external devices 1070 (e.g., a keyboard, a pointing device, a Bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device 1000, and / or any device that enables the electronic device 1000 to communicate with one or more other computing devices (e.g., a router, a modem, etc.). Such communication can occur via an input / output (I / O) interface 1050. Furthermore, the electronic device 1000 can also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter 1060. As shown, the network adapter 1060 communicates with other modules of the electronic device 1000 via a bus 1030. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 1000, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0145] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0146] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions, wherein the instructions are executable by a processor of the device to perform the above method. Alternatively, the computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, or the like.

[0147] In an exemplary embodiment, a computer program product is also provided, including a computer program / instruction, which implements the training method of the image segmentation model in the above embodiment when the computer program / instruction is executed by a processor.

[0148] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0149] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A training method for an image segmentation model, characterized in that: include: Obtaining an image to be segmented and an interactive representation image; Determining a current segmentation mode based on a random selection method between a foreground segmentation mode and an interactive segmentation mode, wherein the foreground segmentation mode is used to perform stage training on an image segmentation model using the image to be segmented, and the interactive segmentation mode is used to perform stage training on the image segmentation model using the image to be segmented and the interactive representation image; Performing stage training on the image segmentation model using the current segmentation mode; When the stage training is completed, the current segmentation mode is updated based on the random selection method, so as to train the image segmentation model using the updated current segmentation mode until a training end condition is met.

2. The image segmentation model training method according to claim 1, characterized in that: When the current segmentation mode is the foreground segmentation mode, performing stage training on the image segmentation model using the current segmentation mode includes: Acquire a preset mask image, a preset positive interaction representation image, and a preset negative interaction representation image, wherein pixel values ​​of the preset mask image, the preset positive interaction representation image, and the preset negative interaction representation image are all specified pixel values; Merging the image to be segmented, the preset mask image, the preset positive interaction representation image, and the preset negative interaction representation image to obtain a first input image; The first input image is input into the image segmentation model, and the image segmentation model is trained in stages.

3. The image segmentation model training method according to claim 2, characterized in that: The image segmentation model includes a coding layer and a decoding layer; Inputting the first input image into the image segmentation model and performing stage training on the image segmentation model includes: Inputting the first input image into the encoding layer for encoding processing to obtain a first eigenvector; Inputting the first feature vector into the decoding layer for decoding processing to obtain a first predicted mask image; The image segmentation model is trained in stages according to the first predicted mask image.

4. The image segmentation model training method according to claim 1 or 2, characterized in that: When the current segmentation mode is the interactive segmentation mode, the interactive representation image includes a first mask image, a first positive interaction representation image, and a first negative interaction representation image; The performing stage training on the image segmentation model using the current segmentation mode includes: Merging the image to be segmented, the first mask image, the first positive interaction representation image, and the first negative interaction representation image to obtain a second input image; The second input image is input into the image segmentation model, and the image segmentation model is trained in stages.

5. The image segmentation model training method according to claim 4, characterized in that: The interactive representation image further includes a second positive interactive representation image and a second negative interactive representation image; Inputting the second input image into the image segmentation model and performing stage training on the image segmentation model includes: Inputting the second input image into the image segmentation model to obtain a second mask image; Merging the image to be segmented, the second mask image, the second positive interaction representation image, and the second negative interaction representation image to obtain a third input image; The third input image is input into the image segmentation model, and stage training is performed on the image segmentation model.

6. The image segmentation model training method according to claim 4, characterized in that: The image segmentation model includes a coding layer and a decoding layer; Inputting the second input image into the image segmentation model and performing stage training on the image segmentation model includes: Inputting the second input image into the encoding layer for encoding processing to obtain a second eigenvector; Inputting the second feature vector into the decoding layer for decoding processing to obtain a second predicted mask image; The image segmentation model is trained in stages according to the second predicted mask image.

7. The image segmentation model training method according to claim 1, characterized in that: The random selection method is to perform random selection in the foreground segmentation mode and the interactive segmentation mode with a preset probability.

8. An image segmentation method, characterized in that: include: Obtaining an image to be segmented and an interactive representation image; When the segmentation mode is a foreground segmentation mode, inputting the image to be segmented into an image segmentation model trained according to the method according to any one of claims 1 to 7 to obtain a first target mask image; When the segmentation mode is the interactive segmentation mode, the image to be segmented and the interactive representation image are input into an image segmentation model trained according to the method according to any one of claims 1 to 7 to obtain a second target mask image.

9. A training device for an image segmentation model, characterized in that: include: An acquisition module is configured to acquire an image to be segmented and an interactive representation image; A selection module is configured to determine a current segmentation mode based on a random selection method in a foreground segmentation mode and an interactive segmentation mode, wherein the foreground segmentation mode is used to perform stage training on the image segmentation model using the image to be segmented, and the interactive segmentation mode is used to perform stage training on the image segmentation model using the image to be segmented and the interactive representation image; A training module, configured to perform stage training on the image segmentation model using the current segmentation mode; The updating module is configured to update the current segmentation mode based on the random selection method when the stage training is completed, so as to train the image segmentation model using the updated current segmentation mode until the training end condition is met.

10. An image segmentation device, characterized in that: include: An acquisition module is configured to acquire an image to be segmented and an interactive representation image; an obtaining module configured to, when the segmentation mode is a foreground segmentation mode, input the image to be segmented into an image segmentation model trained according to the method according to any one of claims 1 to 7 to obtain a first target mask image; The acquisition module is further configured to, when the segmentation mode is the interactive segmentation mode, input the image to be segmented and the interactive representation image into an image segmentation model trained according to the method according to any one of claims 1 to 7 to obtain a second target mask image.

11. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the executable instructions to implement the image segmentation model training method according to any one of claims 1 to 7 or the image segmentation method according to claim 8.

12. A computer-readable storage medium, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to execute the training method of the image segmentation model as described in any one of claims 1 to 7 or the image segmentation method as described in claim 8.

Images